How To Run One Command On Each Machine

Running a single command on every machine in the pool can be accomplished by submitting a set of HTCondor jobs using a single submit description file of the following form:

executable = X
requirements = TARGET.name == MY.TargetSlot

+TargetSlot = "slot1@machine1"
queue

+TargetSlot = "slot1@machine2"
queue

+TargetSlot = "slot1@machine3"
queue

 .
 .
 .

There will be one +TargetSlot and one queue command for each machine in the pool.

A list of machine names for all machines in the pool may be obtained using the condor_status command:

condor_status -constraint 'SlotID==1' -format "%s\n" Name

And, you can have HTCondor do most of the work for you in generating the above submit file:

> cat <<EOF > runit.sub
universe = vanilla
executable = X
requirements = TARGET.name == MY.TargetSlot

EOF

> condor_status -constraint 'SlotID==1' -format '+TargetSlot = "%s"\nqueue\n\n' Name >> runit.sub

--

If you are ok with different clusters for each job -

$ cat do_cmd.sub
cmd = da_cmd
requirements = TARGET.SlotId == 1 && TARGET.Machine == "$(machine)"
queue
$ for m in $(condor_status -master); do condor_submit -a machine=$m do_cmd.sub; done

If you want it in a single cluster -

#!/bin/sh

cmd=$1; shift
args=$@

(
 echo "cmd = $cmd"
 echo "args = $args"
 echo "requirements = TARGET.SlotId == 1 && TARGET.Machine == My.TargetMachine"
 for machine in $(condor_status -master); do
    echo "+TargetMachine = \"$machine\""
    echo "queue"
 done
) | condor_submit