1condor_qsub(1) General Commands Manual condor_qsub(1)
2
6 condor_qsubQueue jobs that use PBS/SGE-style submission
7
9 condor_qsub[ -- version]
10 condor_qsub[Specific options] [Directory options] [Environmental
12 options] [File options] [Notification options] [Resource options] [Sta‐
13 tus options] [Submission options] commandfile
14
16 condor_qsubsubmits an HTCondor job. This job is specified in a
17 PBS/Torque style or an SGE style. condor_qsubpermits the submission of
18 dependent jobs without the need to specify the full dependency graph at
19 submission time. Doing things this way is neither as efficient as
20 HTCondor's DAGMan, nor as functional as SGE's qsubor qalter. con‐
21 dor_qsubserves as a minimal translator to be able to use software orig‐
22 inally written to interact with PBS, Torque, and SGE in an HTCondor
23 pool.
24 condor_qsubattempts to behave like qsub. Less than half of the qsub‐
26 functionality is implemented. Option descriptions describe the differ‐
27 ences between the behavior of qsuband condor_qsub. qsuboptions not
28 listed here are not supported. Some concepts present in PBS and SGE do
29 not apply to HTCondor, and so these options are not implemented.
30 For a full listing of qsuboptions, please see
32 POSIX
34 : http://pubs.opengroup.org/onlinepubs/9699919799/utili‐
36 ties/qsub.html
37 SGE
41 : http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html
43 PBS/Torque
47 : http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/com‐
49 mands/qsub.htm
50 condor_qsubaccepts either command line options or the single file, com‐
54 mandfile, that contains all of the commands.
55 condor_qsubdoes the opposite of job submission within the griduniverse
57 batchgrid type, which takes HTCondor jobs submitted with HTCondor syn‐
58 tax and submits them to PBS, SGE, or LSF.
59
61 -a date_time
62 (Submission option) Specify a deferred execution date and time. The
64 PBS/Torque syntax of date_timeis a string in the form
65 [[[[CC]YY]MM]DD]hhmm[.SS]. The portions of this string which are
66 optional are CC, YY, MM, DD, and SS. For SGE, MMand DDare notop‐
67 tional. For PBS, MMand DDare optional. condor_qsubfollows the PBS
68 style.
69 -A account_string
73 (Status option) Uses group accounting where the string
75 account_stringis the accounting group associated with this job.
76 Unlike SGE, there is no default group of "sge".
77 -b y|n
81 (Submission option) Using the SGE definition of its -boption, a
83 value of ycauses condor_qsubto notparse the file for additional con‐
84 dor_qsubcommands. The default value is n. If the command line argu‐
85 ment -f filenameis also specified, it negates a value of y.
86 -c checkpoint_option
90 (Submission option) For standard universe jobs only, controls the
92 how HTCondor produces checkpoints. checkpoint_optionsmay be one of
93 n or N
95 Do not produce checkpoints.
97 s or S
101 Do not produce periodic checkpoints. A job will only produce a
103 checkpoint when the job is evicted.
104 More options may be implemented in the future.
106 —condor-keep-files
110 (Specific option) Directs HTCondor to notremove temporary files gen‐
112 erated by condor_qsub, such as HTCondor submit files and sentinel
113 jobs. These temporary files may be important for debugging.
114 -cwd
118 (Directory option) Specifies the initial directory in which the job
120 will run to be the current directory from which the job was submit‐
121 ted. This sets initialdirfor condor_submit.
122 -d pathor -wd path
126 (Directory option) Specifies the initial directory in which the job
128 will run to be path. This sets initialdirfor condor_submit.
129 -e filename
133 (File option) Specifies the condor_submitcommand error, the file
135 where stderris written. If not specified, set to the default name of
136 <commandfile>.e<ClusterId>, where <commandfile>is the condor_qsubar‐
137 gument, and <ClusterId>is the job attribute ClusterIdassigned for
138 the job.
139 —f qsub_file
143 (Specific option) Parse qsub_fileto search for and set additional
145 condor_submitcommands. Within the file, commands will appear as
146 #PBSor #SGE. condor_qsubwill parse the batch file listed as
147 qsub_file.
148 -h
152 (Status option) Placed submitted job directly into the hold state.
154 —help
158 (Specific option) Print usage information and exit.
160 -hold_jid <jid>
164 (Status option) Submits a job in the hold state. This job is
166 released only when a previously submitted job, identified by its
167 cluster ID as <jid>, exits successfully. Successful completion is
168 defined as not exiting with exit code 100. In implementation, there
169 are three jobs that define this SGE feature. The first job is the
170 previously submitted job. The second job is the newly submitted one
171 that is waiting for the first to finish successfully. The third job
172 is what SGE calls a sentineljob; this is an HTCondor local universe
173 job that watches the history for the first job's exit code. This
174 third job will exit once it has seen the exit code and, for a suc‐
175 cessful termination of the first job, run condor_releaseon the sec‐
176 ond job. If the first job is an array job, the second job will only
177 be released after all individual jobs of the first job have com‐
178 pleted.
179 -i [hostname:]filename
183 (File option) Specifies the condor_submitcommand input, the file
185 from which stdinis read.
186 -j characters
190 (File option) Acceptable characters for this option are e, o, and n.
192 The only sequence that is relevant is eo; it specifies that both
193 standard output and standard error are to be sent to the same file.
194 The file will be the one specified by the -ooption, if both the
195 -oand -eoptions exist. The file will be the one specified by the
196 -eoption, if only the -eoption is provided. If neither the -onor the
197 -eoptions are provided, the file will be the default used for the
198 -ooption.
199 -l resource_spec
203 (Resource option) Specifies requirements for the job, such as the
205 amount of RAM and the number of CPUs. Only PBS-style resource
206 requests are supported. resource_specis a comma separated list of
207 key/value pairs. Each pair is of the form resource_name=value.
208 resource_nameand valuemay be
209 resource_namevalueDescriptionarchstringSets Archmachine attribute.
211 Enclose in double quotes.filesizeDisk space requested.hoststringHost
212 machine on which the job must run.memsizeAmount of memory
213 requested.nodes{<node_count> | <hostname>} [:ppn=<ppn>]
214 [:gpus=<gpu>] [:<property> [:<property>] ...] [+ ...]Number and/or
215 properties of nodes to be used. For examples, please see
216 http://docs.adaptivecomputing.com/torque/4-1-3/Content/top‐
217 ics/2-jobs/requestingRes.htm#qsubopsysstringSets OpSysmachine
218 attribute. Enclose in double quotes.procsintegerNumber of CPUs
219 requested.A size value is an integer specified in bytes, following
220 the PBS/Torque default. Append Kb, Mb, Gb, or Tbto specify the value
221 in powers of two quantities greater than bytes.
222 -m a|e|n
226 (Notification option) Identify when HTCondor sends notification e-
228 mail. If a, send e-mail when the job terminates abnormally. If e,
229 send e-mail when the job terminates. If n, never send e-mail.
230 -M e-mail_address
234 (Notification option) Sets the destination address for HTCondor e-
236 mail.
237 -o filename
241 (File option) Specifies the condor_submitcommand output, the file
243 where stdoutis written. If not specified, set to the default name of
244 <commandfile>.o<ClusterId>, where <commandfile>is the condor_qsubar‐
245 gument, and <ClusterId>is the job attribute ClusterIdassigned for
246 the job.
247 -p integer
251 (Status option) Sets the prioritysubmit command for the job, with 0
253 being the default. Jobs with higher numerical priority will run
254 before jobs with lower numerical priority.
255 —print
259 (Specific option) Send to stdoutthe contents of the HTCondor submit
261 description file that condor_qsubgenerates.
262 -r y|n
266 (Status option) The default value of yimplements the default HTCon‐
268 dor policy of assuming that jobs that do not complete are placed
269 back in the queue to be run again. When n, job submission is
270 restricted to only running the job if the job ClassAd attribute
271 NumJobStartsis currently 0. This identifies the job as not re-
272 runnable, limiting it to start once.
273 -S shell
277 (Submission option) Specifies the path and executable name of a
279 shell. Alters the HTCondor submit description file produced, such
280 that the executable becomes a wrapper script. Within the submit
281 description file will be executable = <shell>and arguments = <com‐
282 mandfile>.
283 -t start [-stop:step]
287 (Submission option) Queues a set of nearly identical jobs. The SGE-
289 style syntax is supported. start, stop, and stepare all integers.
290 startis the starting index of the jobs, stopis the ending index
291 (inclusive) of the jobs, and stepis the step size through the
292 indices. Note that using more than one processor or node in a job
293 will not work with this option.
294 —test
298 (Specific option) With the intention of testing a potential job sub‐
300 mission, parse files and commands to generate error output. Pro‐
301 duces, but then removes the HTCondor submit description file. Never
302 submits the job, even if no errors are encountered.
303 -v variable list
307 (Environmental option) Used to set the submit command environmentfor
309 the job. variable listis as that defined for the submit command.
310 Note that the syntax needed is specialized to deal with quote marks
311 and white space characters.
312 -V
316 (Environmental option) Sets getenv = Truein the submit description
318 file.
319 -W attr_name=attr_value[,attr_name=attr_value...]
323 (File option) PBS/Torque supports a number of attributes. However,
325 condor_qsubonly supports the names stageinand stageoutfor attr_name.
326 The format of attr_valuefor stageinand stageoutis local_file@host‐
327 name:remote_file[,...]and we strip it to remote_file[,...]. HTCon‐
328 dor's file transfer mechanism is then used if needed.
329 —version
333 (Specific option) Print version information for the condor_qsubpro‐
335 gram and exit. Note that condor_qsubhas its own version numbers
336 which are separate from those of HTCondor.
337
341 condor_qsubwill exit with a status value of 0 (zero) upon success, and
342 it will exit with the value 1 (one) upon failure to submit a job.
343
345 Center for High Throughput Computing, University of Wiscon‐
346 sin–Madison
347
349 Copyright © 1990-2019 Center for High Throughput Computing, Computer
350 Sciences Department, University of Wisconsin-Madison, Madison, WI. All
351 Rights Reserved. Licensed under the Apache License, Version 2.0.
352 date condor_qsub(1)