While Getting started with Condor and EC2, it is useful to display the EC2 specific attributes on jobs. This is a script that mirrors condor_q output, using its formatting parameters, and adds details for EC2 jobs.
condor_ec2_q:
#!/bin/sh
# NOTE:
# . Requires condor_q >= 7.5.2, old classads do not
# have %
# . When running, jobs show RUN_TIME of their current
# run, not accumulated, which would require adding
# in RemoteWallClockTime
# . See condor_utils/condor_q.cpp:encode_status for
# JobStatus map
# TODO:
# . Remove extra ShadowBday==0 test,
# condor_gridmanager < 7.7.5 (3a896d01) did not
# delete ShadowBday when a job was not running.
# RUN_TIME of held EC2 jobs would be wrong.
echo ' ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD'
condor_q \
-format '%4d.' ClusterId \
-format '%-3d ' ProcId \
-format '%-14s ' Owner \
-format '%-11s ' 'formatTime(QDate,"%m/%d %H:%M")' \
-format '%3d+' 'ifThenElse(ShadowBday =!= UNDEFINED, ifThenElse(ShadowBday != 0, time() - ShadowBday, int(RemoteWallClockTime)), int(RemoteWallClockTime)) / (60*60*24)' \
-format '%02d:' '(ifThenElse(ShadowBday =!= UNDEFINED, ifThenElse(ShadowBday != 0, time() - ShadowBday, int(RemoteWallClockTime)), int(RemoteWallClockTime)) % (60*60*24)) / (60*60)' \
-format '%02d:' '(ifThenElse(ShadowBday =!= UNDEFINED, ifThenElse(ShadowBday != 0, time() - ShadowBday, int(RemoteWallClockTime)), int(RemoteWallClockTime)) % (60*60)) / 60' \
-format '%02d ' 'ifThenElse(ShadowBday =!= UNDEFINED, ifThenElse(ShadowBday != 0, time() - ShadowBday, int(RemoteWallClockTime)), int(RemoteWallClockTime)) % 60' \
-format '%-2s ' 'substr("?IRXCH>S", JobStatus, 1)' \
-format '%-3d ' JobPrio \
-format '%-4.1f ' ImageSize/1024.0 \
-format '%-18.18s' 'strcat(Cmd," ",Arguments)' \
-format '\n' Owner \
-format ' Instance name: %s\n' EC2InstanceName \
-format ' Hostname: %s\n' EC2RemoteVirtualMachineName \
-format ' Keypair file: %s\n' EC2KeyPairFile \
-format ' User data: %s\n' EC2UserData \
-format ' User data file: %s\n' EC2UserDataFile \
-format ' AMI id: %s\n' EC2AmiID \
-format ' Instance type: %s\n' EC2InstanceType \
"$@" | awk 'BEGIN {St["I"]=0;St["R"]=0;St["H"]=0} \
{St[$6]++; print} \
END {for (i=0;i<=7;i++) jobs+=St[substr("?IRXCH>S",i,1)]; \
print jobs, "jobs;", \
St["I"], "idle,", St["R"], "running,", St["H"], "held"}'
In action,
$ condor_q -- Submitter: eeyore.local : : eeyore.local ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD 1728.0 matt 10/31 23:09 0+00:04:15 H 0 0.0 EC2_Instance-ami-6 1732.0 matt 11/1 01:43 0+05:16:46 R 0 0.0 EC2_Instance-ami-6 5 jobs; 0 idle, 4 running, 1 held $ ./condor_ec2_q ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD 1728.0 matt 10/31 23:09 0+00:04:15 H 0 0.0 EC2_Instance-ami-6 Instance name: i-31855752 Hostname: ec2-50-19-175-62.compute-1.amazonaws.com Keypair file: /home/matt/Documents/AWS/EC2_Instance-ami-60bd4609.1728.pem User data: Hello EC2_Instance-ami-60bd4609! AMI id: ami-60bd4609 Instance type: m1.small 1732.0 matt 11/01 01:43 0+05:16:48 R 0 0.0 EC2_Instance-ami-6 Instance name: i-a90edcca Hostname: ec2-107-20-6-83.compute-1.amazonaws.com Keypair file: /home/matt/Documents/AWS/EC2_Instance-ami-60bd4609.1732.pem User data: Hello EC2_Instance-ami-60bd4609! AMI id: ami-60bd4609 Instance type: m1.small 5 jobs; 0 idle, 4 running, 1 held
Advertisement
November 10, 2011 at 7:53 am |
[...] have been over starting and managing instances from Condor, using condor_ec2_q to help, and importing existing instances. Here we will cover extending an existing pool using [...]
January 24, 2012 at 10:27 am |
[...] VNC_VIA_CMD=’/usr/bin/ssh -i KEYPAIR.pem -l ec2-user -f -L “$L”:”$H”:”$R” “$G” sleep 20′ vncviewer localhost:1 -via INSTANCE_ADDRESS What’s going on here? vncviewer allows for a proxy host when connecting to the vncserver. That is the -via argument. The VNC_VIA_CMD is an environment variable that specifies the command used to connect to the proxy. Here it is modified to provide the keypair needed to access the instance, and the user ec2-user, which is the default user on Fedora AMIs. The INSTANCE_ADDRESS is the Hostname from condor_ec2_q. [...]