Submitting a workflow to Condor via SOAP using Java

In Condor, a workflow is called a DAG. DAG stand for directed acyclic graph. DAGs in Condor are managed by processes called condor_dagman. condor_dagman is a program that takes a description of a DAG as input and walks it; submitting jobs and monitoring their progress. The condor_dagman process itself is often submitted to Condor and managed by the Schedd like any other job. The only thing special about condor_dagman jobs is that they are run on the same machine as the Schedd. Typically in Condor’s Scheduler Universe. Historical note: the first job submitted to Condor via SOAP was a DAG.

Here’s a recipe for submitting a DAG to Condor via SOAP:

0) Get your favorite SOAP library. Here we’ll use Apache Axis: http://www.apache.org/dyn/closer.cgi/ws/axis/1_4

You’ll want to untar it:

$ tar zxf axis-bin-1_4.tar.gz

1) Generate and compile your SOAP library code

# To avoid passing -classpath to java and javac, set CLASSPATH in your env
$ export CLASSPATH=.:$(echo axis-1_4/lib/*.jar | tr ' ' ':')

# Have Axis generate Java stubs
$ java org.apache.axis.wsdl.WSDL2Java file:///usr/share/condor/webservice/condorSchedd.wsdl

# Compile the stubs
$ javac condor/*.java

2) Write a DAG. This one is pretty simple, it’s a diamond: D depends on B and C, B and C depend on A:

~/dagman/diamond.dag:
Job  A  A.submit 
Job  B  B.submit 
Job  C  C.submit        
Job  D  D.submit
PARENT A CHILD B C
PARENT B C CHILD D

~/dagman/A.submit:
executable = /bin/sleep
arguments = 112
output = A.out
log = diamond.log
queue

~/dagman/B.submit:
executable = /bin/sleep
arguments = 358
output = B.out
log = diamond.log
queue

~/dagman/C.submit:
executable = /bin/sleep
arguments = 132
output = C.out
log = diamond.log
queue

~/dagman/D.submit:
executable = /bin/sleep
arguments = 134
output = D.out
log = diamond.log
queue

3) Write a program that will submit your DAG via SOAP.

The Schedd’s Submit() function takes a ClassAd representing a job. What condor_submit does is take a submit file and convert it into a ClassAd. To look at a submit file for a DAG run condor_submit_dag -no_submit diamond.dag and read diamond.dag.condor.sub. That will give you a hint at what kind of environment condor_dagman wants to run in. Note: remove_kill_sig, arguments, environment and on_exit_remove.

However, diamond.dag.condor.sub is not a ClassAd. To see the ClassAd you can run diamond.dag.condor.sub through condor_submit. Do so with condor_submit -dump diamond.dag.condor.sub.ad diamond.dag.condor.sub. Have a look at diamond.dag.condor.sub.ad and note RemoveKillSig, Arguments, Env and OnExitRemove.

Now you have the basis for what a DAG job needs to run. In the example code I used a little extra knowledge to generate the ClassAd, for brevity. You can use the Schedd’s CreateJobTemplate function to help generate the ClassAd for you instead. Extending arrays is kinda annoying in Java, but cake in python.

Start with this example: CondorSubmitDAG.java

4) Configure your condor_schedd to accept SOAP requests. The basic configuration you will need is:

ENABLE_SOAP = TRUE
ALLOW_SOAP = *
QUEUE_ALL_USERS_TRUSTED = TRUE
SCHEDD_ARGS = -p 1984

This configuration lets anyone talk to your Schedd and submit jobs as any user. For deployment, you should restrict this access by narrowing the ALLOW_SOAP and by setting QUEUE_ALL_USERS_TRUSTED to FALSE. Note: changing QUEUE_ALL_USERS_TRUSTED requires that clients can authenticate themselves via SSL.

This configuration also gives you a fixed port, 1984, for the condor_schedd. Otherwise the port is ephemeral, and you’ll have to query the Collector to find it.

5) Compile your submission program and submit your DAG.

$ javac CondorSubmitDAG.java

$ java CondorSubmitDAG http://localhost:1984 soapmonkey /some/shared/space/where/you/put/dagman/diamond.dag

You can watch the DAG run with condor_q, and condor_q -dag. Don’t be afraid of the —????— for the DAG itself, that’s just because I pruned the job ad to the bare minimum to run. condor_q expects a few extra attributes to be present. If you use CreateJobTemplate you’ll get all the attributes condor_q wants.

Tags: Condor, Java, SOAP, Workfloaw