1just-man-pages/condor_submit_Gdeange(r1a)l CommandsjuMsatn-umaaln-pages/condor_submit_dag(1)
2
6 condor_submit_dag Manage and queue jobs within a specified DAG for exe‐
7 cution on remote machines
8
10 condor_submit_dag [ -help | -version ]
11 condor_submit_dag [ -no_submit ] [ -verbose ] [ -force ] [ -maxidle
13 NumberOfProcs ] [ -maxjobs NumberOfClusters ] [ -dagman DagmanExe‐
14 cutable ] [ -maxpre NumberOfPreScripts ] [ -maxpost NumberOfPostScripts
15 ] [ -notification value ] [ -noeventchecks ] [ -allowlogerror ] [ -r
16 schedd_name ] [ -debug level ] [ -usedagdir ] [ -outfile_dir directory
17 ] [ -config ConfigFileName ] [ -insert_sub_file FileName ] [ -append
18 Command ] [ -batch-name batch_name ] [ -autorescue 0|1 ] [ -dorescue‐
19 from number ] [ -allowversionmismatch ] [ -no_recurse ] [ -do_recurse ]
20 [ -update_submit ] [ -import_env ] [ -DumpRescue ] [ -valgrind ] [
21 -DontAlwaysRunPost ] [ -AlwaysRunPost ] [ -priority number ] [
22 -dont_use_default_node_log ] [ -schedd-daemon-ad-file FileName ] [
23 -schedd-address-file FileName ] [ -suppress_notification ] [ -dont_sup‐
24 press_notification ] [ -DoRecovery ] DAGInputFile1 [ DAGInputFile2
25 ...DAGInputFileN ]
26
28 condor_submit_dag is the program for submitting a DAG (directed acyclic
29 graph) of jobs for execution under HTCondor. The program enforces the
30 job dependencies defined in one or more DAGInputFile s. Each DAGInput‐
31 File contains commands to direct the submission of jobs implied by the
32 nodes of a DAG to HTCondor. Extensive documentation is in the HTCondor
33 User Manual section on DAGMan.
34 Some options may be specified on the command line or in the configura‐
36 tion or in a node job's submit description file. Precedence is given to
37 command line options or configuration over settings from a submit
38 description file. An example is e-mail notifications. When configura‐
39 tion variable DAGMAN_SUPPRESS_NOTIFICATION is its default value of
40 True , and a node job's submit description file contains
41 notification = Complete
43 e-mail will not be sent upon completion, as the value of DAGMAN_SUP‐
45 PRESS_NOTIFICATION is enforced.
46
48 -help
49 Display usage information and exit.
51 -version
55 Display version information and exit.
57 -no_submit
61 Produce the HTCondor submit description file for DAGMan, but do not
63 submit DAGMan as an HTCondor job.
64 -verbose
68 Cause condor_submit_dag to give verbose error messages.
70 -force
74 Require condor_submit_dag to overwrite the files that it produces,
76 if the files already exist. Note that dagman.out will be appended
77 to, not overwritten. If new-style rescue DAG mode is in effect, and
78 any new-style rescue DAGs exist, the -force flag will cause them to
79 be renamed, and the original DAG will be run. If old-style rescue
80 DAG mode is in effect, any existing old-style rescue DAGs will be
81 deleted, and the original DAG will be run.
82 -maxidle NumberOfProcs
86 Sets the maximum number of idle procs allowed before condor_dagman
88 stops submitting more node jobs. Note that for this argument, each
89 individual proc within a cluster counts as a towards the limit,
90 which is inconsistent with -maxjobs . Once idle procs start to run,
91 condor_dagman will resume submitting jobs once the number of idle
92 procs falls below the specified limit. NumberOfProcs is a non-nega‐
93 tive integer. If this option is omitted, the number of idle procs is
94 limited by the configuration variable DAGMAN_MAX_JOBS_IDLE (see ),
95 which defaults to 1000. To disable this limit, set NumberOfProcs to
96 0. Note that submit description files that queue multiple procs can
97 cause the NumberOfProcs limit to be exceeded. Setting queue 5000 in
98 the submit description file, where -maxidle is set to 250 will
99 result in a cluster of 5000 new procs being submitted to the con‐
100 dor_schedd , not 250. In this case, condor_dagman will resume sub‐
101 mitting jobs when the number of idle procs falls below 250.
102 -maxjobs NumberOfClusters
106 Sets the maximum number of clusters within the DAG that will be sub‐
108 mitted to HTCondor at one time. Note that for this argument, each
109 cluster counts as one job, no matter how many individual procs are
110 in the cluster. NumberOfClusters is a non-negative integer. If this
111 option is omitted, the number of clusters is limited by the configu‐
112 ration variable DAGMAN_MAX_JOBS_SUBMITTED (see ), which defaults to
113 0 (unlimited).
114 -dagman DagmanExecutable
118 Allows the specification of an alternate condor_dagman executable to
120 be used instead of the one found in the user's path. This must be a
121 fully qualified path.
122 -maxpre NumberOfPreScripts
126 Sets the maximum number of PRE scripts within the DAG that may be
128 running at one time. NumberOfPreScripts is a non-negative integer.
129 If this option is omitted, the number of PRE scripts is limited by
130 the configuration variable DAGMAN_MAX_PRE_SCRIPTS (see ), which
131 defaults to 20.
132 -maxpost NumberOfPostScripts
136 Sets the maximum number of POST scripts within the DAG that may be
138 running at one time. NumberOfPostScripts is a non-negative integer.
139 If this option is omitted, the number of POST scripts is limited by
140 the configuration variable DAGMAN_MAX_POST_SCRIPTS (see ), which
141 defaults to 20.
142 -notification value
146 Sets the e-mail notification for DAGMan itself. This information
148 will be used within the HTCondor submit description file for DAGMan.
149 This file is produced by condor_submit_dag . See the description of
150 notification within condor_submit manual page for a specification of
151 value .
152 -noeventchecks
156 This argument is no longer used; it is now ignored. Its functional‐
158 ity is now implemented by the DAGMAN_ALLOW_EVENTS configuration
159 variable.
160 -allowlogerror
164 As of verson 8.5.5 this argument is no longer supported, and setting
166 it will generate a warning.
167 -r schedd_name
171 Submit condor_dagman to a remote machine, specifically the con‐
173 dor_schedd daemon on that machine. The condor_dagman job will not
174 run on the local condor_schedd (the submit machine), but on the
175 specified one. This is implemented using the -remote option to con‐
176 dor_submit . Note that this option does not currently specify input
177 files for condor_dagman , nor the individual nodes to be taken
178 along! It is assumed that any necessary files will be present on the
179 remote computer, possibly via a shared file system between the local
180 computer and the remote computer. It is also necessary that the user
181 has appropriate permissions to submit a job to the remote machine;
182 the permissions are the same as those required to use condor_submit
183 -remote option. If other options are desired, including transfer of
184 other input files, consider using the -no_submit option, modifying
185 the resulting submit file for specific needs, and then using con‐
186 dor_submit on that.
187 -debug level
191 Passes the the level of debugging output desired to condor_dagman .
193 level is an integer, with values of 0-7 inclusive, where 7 is the
194 most verbose output. See the condor_dagman manual page for detailed
195 descriptions of these values. If not specified, no -debug v alue is
196 passed to condor_dagman .
197 -usedagdir
201 This optional argument causes condor_dagman to run each specified
203 DAG as if condor_submit_dag had been run in the directory containing
204 that DAG file. This option is most useful when running multiple DAGs
205 in a single condor_dagman . Note that the -usedagdir flag must not
206 be used when running an old-style Rescue DAG.
207 -outfile_dir directory
211 Specifies the directory in which the .dagman.out file will be writ‐
213 ten. The directory may be specified relative to the current working
214 directory as condor_submit_dag is executed, or specified with an
215 absolute path. Without this option, the .dagman.out file is placed
216 in the same directory as the first DAG input file listed on the com‐
217 mand line.
218 -config ConfigFileName
222 Specifies a configuration file to be used for this DAGMan run. Note
224 that the options specified in the configuration file apply to all
225 DAGs if multiple DAGs are specified. Further note that it is a fatal
226 error if the configuration file specified by this option conflicts
227 with a configuration file specified in any of the DAG files, if they
228 specify one.
229 -insert_sub_file FileName
233 Specifies a file to insert into the .condor.sub file created by
235 condor_submit_dag . The specified file must contain only legal sub‐
236 mit file commands. Only one file can be inserted. (If both the DAG‐
237 MAN_INSERT_SUB_FILE configuration variable and -insert_sub_file are
238 specified, -insert_sub_file overrides DAGMAN_INSERT_SUB_FILE.) The
239 specified file is inserted into the .condor.sub file before the
240 Queue command and before any commands specified with the -append
241 option.
242 -append Command
246 Specifies a command to append to the .condor.sub file created by
248 condor_submit_dag . The specified command is appended to the .con‐
249 dor.sub file immediately before the Queue command. Multiple commands
250 are specified by using the -append option multiple times. Each new
251 command is given in a separate -append option. Commands with spaces
252 in them must be enclosed in double quotes. Commands specified with
253 the -append option are appended to the .condor.sub file after com‐
254 mands inserted from a file specified by the -insert_sub_file option
255 or the DAGMAN_INSERT_SUB_FILE configuration variable, so the -append
256 command(s) will override commands from the inserted file if the com‐
257 mands conflict.
258 -batch-name batch_name
262 Set the batch name for this DAG/workflow. The batch name is dis‐
264 played by condor_q -batch . It is intended for use by users to give
265 meaningful names to their workflows and to influence how condor_q
266 groups jobs for display. As of version 8.5.5, the batch name set
267 with this argument is propagated to all node jobs of the given DAG
268 (including sub-DAGs), overriding any batch names set in the individ‐
269 ual submit files. Note: set the batch name to ' ' (space) to avoid
270 overriding batch names specified in node job submit files. If no
271 batch name is set, the batch name defaults to DagFile + cluster
272 (where DagFile is the primary DAG file of the top-level DAGMan, and
273 cluster is the HTCondor cluster of the top-level DAGMan); the
274 default will override any lower-level batch names.
275 -autorescue 0|1
279 Whether to automatically run the newest rescue DAG for the given DAG
281 file, if one exists (0 = false , 1 = true ).
282 -dorescuefrom number
286 Forces condor_dagman to run the specified rescue DAG number for the
288 given DAG. A value of 0 is the same as not specifying this option.
289 Specifying a non-existent rescue DAG is a fatal error.
290 -allowversionmismatch
294 This optional argument causes condor_dagman to allow a version mis‐
296 match between condor_dagman itself and the .condor.sub file pro‐
297 duced by condor_submit_dag (or, in other words, between condor_sub‐
298 mit_dag and condor_dagman ). WARNING! This option should be used
299 only if absolutely necessary. Allowing version mismatches can cause
300 subtle problems when running DAGs. (Note that, starting with version
301 7.4.0, condor_dagman no longer requires an exact version match
302 between itself and the .condor.sub file. Instead, a "minimum com‐
303 patible version" is defined, and any .condor.sub file of that ver‐
304 sion or newer is accepted.)
305 -no_recurse
309 This optional argument causes condor_submit_dag to not run itself
311 recursively on nested DAGs (this is now the default; this flag has
312 been kept mainly for backwards compatibility).
313 -do_recurse
317 This optional argument causes condor_submit_dag to run itself recur‐
319 sively on nested DAGs. The default is now that it does not run
320 itself recursively; instead the .condor.sub files for nested DAGs
321 are generated "lazily" by condor_dagman itself. DAG nodes specified
322 with the SUBDAG EXTERNAL keyword or with submit file names ending in
323 .condor.sub are considered nested DAGs. The DAGMAN_GENERATE_SUB‐
324 DAG_SUBMITS configuration variable may be relevant.
325 -update_submit
329 This optional argument causes an existing .condor.sub file to not
331 be treated as an error; rather, the .condor.sub file will be over‐
332 written, but the existing values of -maxjobs , -maxidle , -maxpre ,
333 and -maxpost will be preserved.
334 -import_env
338 This optional argument causes condor_submit_dag to import the cur‐
340 rent environment into the environment command of the .condor.sub
341 file it generates.
342 -DumpRescue
346 This optional argument tells condor_dagman to immediately dump a
348 rescue DAG and then exit, as opposed to actually running the DAG.
349 This feature is mainly intended for testing. The Rescue DAG file is
350 produced whether or not there are parse errors reading the original
351 DAG input file. The name of the file differs if there was a parse
352 error.
353 -valgrind
357 This optional argument causes the submit description file generated
359 for the submission of condor_dagman to be modified. The executable
360 becomes valgrind run on condor_dagman , with a specific set of argu‐
361 ments intended for testing condor_dagman . Note that this argument
362 is intended for testing purposes only. Using the -valgrind option
363 without the necessary valgrind software installed will cause the DAG
364 to fail. If the DAG does run, it will run much more slowly than
365 usual.
366 -DontAlwaysRunPost
370 This option causes the submit description file generated for the
372 submission of condor_dagman to be modified. It causes condor_dagman
373 to not run the POST script of a node if the PRE script fails. (This
374 was the default behavior prior to HTCondor version 7.7.2, and is
375 again the default behavior from version 8.5.4 onwards.)
376 -AlwaysRunPost
380 This option causes the submit description file generated for the
382 submission of condor_dagman to be modified. It causes condor_dagman
383 to always run the POST script of a node, even if the PRE script
384 fails. (This was the default behavior for HTCondor version 7.7.2
385 through version 8.5.3.)
386 -priority number
390 Sets the minimum job priority of node jobs submitted and running
392 under the condor_dagman job submitted by this condor_submit_dag com‐
393 mand.
394 -dont_use_default_node_log
398 This option is disabled as of HTCondor version 8.3.1. This causes a
400 compatibility error if the HTCondor version number of the con‐
401 dor_schedd is 7.9.0 or older. Tells condor_dagman to use the file
402 specified by the job ClassAd attribute UserLog to monitor job sta‐
403 tus. If this command line argument is used, then the job event log
404 file cannot be defined with a macro.
405 -schedd-daemon-ad-file FileName
409 Specifies a full path to a daemon ad file dropped by a condor_schedd
411 . Therefore this allows submission to a specific scheduler if sev‐
412 eral are available without repeatedly querying the condor_collector
413 . The value for this argument defaults to the configuration
414 attribute SCHEDD_DAEMON_AD_FILE .
415 -schedd-address-file FileName
419 Specifies a full path to an address file dropped by a condor_schedd
421 . Therefore this allows submission to a specific scheduler if sev‐
422 eral are available without repeatedly querying the condor_collector
423 . The value for this argument defaults to the configuration
424 attribute SCHEDD_ADDRESS_FILE .
425 -suppress_notification
429 Causes jobs submitted by condor_dagman to not send email notifica‐
431 tion for events. The same effect can be achieved by setting configu‐
432 ration variable DAGMAN_SUPPRESS_NOTIFICATION to True . This com‐
433 mand line option is independent of the -notification command line
434 option, which controls notification for the condor_dagman job
435 itself.
436 -dont_suppress_notification
440 Causes jobs submitted by condor_dagman to defer to content within
442 the submit description file when deciding to send email notification
443 for events. The same effect can be achieved by setting configuration
444 variable DAGMAN_SUPPRESS_NOTIFICATION to False . This command line
445 flag is independent of the -notification command line option, which
446 controls notification for the condor_dagman job itself. If both
447 -dont_suppress_notification and -suppress_notification are specified
448 with the same command line, the last argument is used.
449 -DoRecovery
453 Causes condor_dagman to start in recovery mode. (This means that it
455 reads the relevant job user log(s) and "catches up" to the given
456 DAG's previous state before submitting any new jobs.)
457
461 condor_submit_dag will exit with a status value of 0 (zero) upon suc‐
462 cess, and it will exit with the value 1 (one) upon failure.
463
465 To run a single DAG:
466 % condor_submit_dag diamond.dag
468 To run a DAG when it has already been run and the output files exist:
470 % condor_submit_dag -force diamond.dag
472 To run a DAG, limiting the number of idle node jobs in the DAG to a
474 maximum of five:
475 % condor_submit_dag -maxidle 5 diamond.dag
477 To run a DAG, limiting the number of concurrent PRE scripts to 10 and
479 the number of concurrent POST scripts to five:
480 % condor_submit_dag -maxpre 10 -maxpost 5 diamond.dag
482 To run two DAGs, each of which is set up to run in its own directory:
484 % condor_submit_dag -usedagdir dag1/diamond1.dag dag2/diamond2.dag
486
488 Center for High Throughput Computing, University of Wisconsin-Madison
489
491 Copyright (C) 1990-2018 Center for High Throughput Computing, Computer
492 Sciences Department, University of Wisconsin-Madison, Madison, WI. All
493 Rights Reserved. Licensed under the Apache License, Version 2.0.
494 date just-man-pages/condor_submit_dag(1)