1CONDOR_QSUB(1) HTCondor Manual CONDOR_QSUB(1)
2
6 condor_qsub - HTCondor Manual
7 Queue jobs that use PBS/SGE-style submission
9
12 condor_qsub [--version]
13 condor_qsub [Specific options ] [Directory options ] [Environmental op‐
15 tions ] [File options ] [Notification options ] [Resource options ]
16 [Status options ] [Submission options ] commandfile
17
19 condor_qsub submits an HTCondor job. This job is specified in a
20 PBS/Torque style or an SGE style. condor_qsub permits the submission of
21 dependent jobs without the need to specify the full dependency graph at
22 submission time. Doing things this way is neither as efficient as HT‐
23 Condor's DAGMan, nor as functional as SGE's qsub or qalter. con‐
24 dor_qsub serves as a minimal translator to be able to use software
25 originally written to interact with PBS, Torque, and SGE in an HTCondor
26 pool.
27 condor_qsub attempts to behave like qsub. Less than half of the qsub
29 functionality is implemented. Option descriptions describe the differ‐
30 ences between the behavior of qsub and condor_qsub. qsub options not
31 listed here are not supported. Some concepts present in PBS and SGE do
32 not apply to HTCondor, and so these options are not implemented.
33 For a full listing of qsub options, please see
35 POSIX :
37 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/qsub.html
38 SGE :
40 http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html
41 PBS/Torque
43 :
44 http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/commands/qsub.htm
45 condor_qsub accepts either command line options or the single file,
47 commandfile, that contains all of the commands.
48 condor_qsub does the opposite of job submission within the grid uni‐
50 verse batch grid type, which takes HTCondor jobs submitted with HTCon‐
51 dor syntax and submits them to PBS, SGE, or LSF.
52
54 -a date_time
55 (Submission option) Specify a deferred execution date and
56 time. The PBS/Torque syntax of date_time is a string in the
57 form [[[[CC]YY]MM]DD]hhmm[.SS]. The portions of this string
58 which are optional are CC, YY, MM, DD, and SS. For SGE, MM
59 and DD are not optional. For PBS, MM and DD are optional.
60 condor_qsub follows the PBS style.
61 -A account_string
63 (Status option) Uses group accounting where the string ac‐
64 count_string is the accounting group associated with this
65 job. Unlike SGE, there is no default group of "sge".
66 -b y|n (Submission option) Using the SGE definition of its -b op‐
68 tion, a value of y causes condor_qsub to not parse the file
69 for additional condor_qsub commands. The default value is n.
70 If the command line argument -f filename is also specified,
71 it negates a value of y.
72 -c checkpoint_option
74 (Submission option) For standard universe jobs only, controls
75 the how HTCondor produces checkpoints. checkpoint_options may
76 be one of
77 n or N Do not produce checkpoints.
79 s or S Do not produce periodic checkpoints. A job will
81 only produce a checkpoint when the job is evicted.
82 More options may be implemented in the future.
83 -condor-keep-files
86 (Specific option) Directs HTCondor to not remove temporary
87 files generated by condor_qsub, such as HTCondor submit files
88 and sentinel jobs. These temporary files may be important for
89 debugging.
90 -cwd (Directory option) Specifies the initial directory in which
92 the job will run to be the current directory from which the
93 job was submitted. This sets initialdir for condor_submit.
94 -d path or -wd path
96 (Directory option) Specifies the initial directory in which
97 the job will run to be path. This sets initialdir for con‐
98 dor_submit.
99 -e filename
101 (File option) Specifies the condor_submit command error ,
102 the file where stderr is written. If not specified, set to
103 the default name of `` <commandfile>.e<ClusterId>``, where
104 <commandfile> is the condor_qsub argument, and `` <Clus‐
105 terId>`` is the job attribute ClusterId assigned for the job.
106 -f qsub_file
108 (Specific option) Parse qsub_file to search for and set addi‐
109 tional condor_submit commands. Within the file, commands will
110 appear as #PBS or #SGE. condor_qsub will parse the batch file
111 listed as qsub_file.
112 -h (Status option) Placed submitted job directly into the hold
114 state.
115 -help (Specific option) Print usage information and exit.
117 -hold_jid <jid>
119 (Status option) Submits a job in the hold state. This job is
120 released only when a previously submitted job, identified by
121 its cluster ID as <jid>, exits successfully. Successful com‐
122 pletion is defined as not exiting with exit code 100. In im‐
123 plementation, there are three jobs that define this SGE fea‐
124 ture. The first job is the previously submitted job. The sec‐
125 ond job is the newly submitted one that is waiting for the
126 first to finish successfully. The third job is what SGE calls
127 a sentinel job; this is an HTCondor local universe job that
128 watches the history for the first job's exit code. This third
129 job will exit once it has seen the exit code and, for a suc‐
130 cessful termination of the first job, run condor_release on
131 the second job. If the first job is an array job, the second
132 job will only be released after all individual jobs of the
133 first job have completed.
134 -i [hostname:]filename
136 (File option) Specifies the condor_submit command input ,
137 the file from which stdin is read.
138 -j characters
140 (File option) Acceptable characters for this option are e, o,
141 and n. The only sequence that is relevant is eo; it specifies
142 that both standard output and standard error are to be sent
143 to the same file. The file will be the one specified by the
144 -o option, if both the -o and -e options exist. The file will
145 be the one specified by the -e option, if only the -e option
146 is provided. If neither the -o nor the -e options are pro‐
147 vided, the file will be the default used for the -o option.
148 -l resource_spec
150 (Resource option) Specifies requirements for the job, such as
151 the amount of RAM and the number of CPUs. Only PBS-style re‐
152 source requests are supported. resource_spec is a comma sepa‐
153 rated list of key/value pairs. Each pair is of the form re‐
154 source_name=value. resource_name and value may be
155 +--------------------------+--------------------------+--------------------------+
156 | resource_name | value | Description
157 |
158 +--------------------------+--------------------------+--------------------------+
159 | arch | string | Sets
160 Arch machine | | |
161 | attribute. Enclose in | | |
162 | double quotes. |
163 +--------------------------+--------------------------+--------------------------+
164 | file | size | Disk
165 space requested. |
166 +--------------------------+--------------------------+--------------------------+
167 | host | string | Host
168 machine on which | | |
169 | the job must run. |
170 +--------------------------+--------------------------+--------------------------+
171 | mem | size |
172 Amount of memory | | |
173 | requested. |
174 +--------------------------+--------------------------+--------------------------+
175 | nodes | {<node_count> | <hostn | Number
176 and/or properties | | | ame>}
177 [:ppn=<ppn>] [:gpu | of nodes to be used. For | |
178 | s=<gpu>] [:<property> [: | examples, please see | |
179 | <property>] ...] [+ ...] | http://docs.adaptivecom | |
180 | | puting.com/torque/4-1-3/ | |
181 | | Content/topics/2-jobs/re | |
182 | | questingRes.htm#qsub |
183 +--------------------------+--------------------------+--------------------------+
184 | opsys | string | Sets
185 OpSys machine | | |
186 | attribute. Enclose in | | |
187 | double quotes. |
188 +--------------------------+--------------------------+--------------------------+
189 | procs | integer | Num‐
190 ber of CPUs | | |
191 | requested. |
192 +--------------------------+--------------------------+--------------------------+
193 A size value is an integer specified in bytes, following the
195 PBS/Torque default. Append Kb, Mb, Gb, or Tb to specify the
196 value in powers of two quantities greater than bytes.
197 -m a|e|n
199 (Notification option) Identify when HTCondor sends notifica‐
200 tion e-mail. If a, send e-mail when the job terminates abnor‐
201 mally. If e, send e-mail when the job terminates. If n, never
202 send e-mail.
203 -M e-mail_address
205 (Notification option) Sets the destination address for HTCon‐
206 dor e-mail.
207 -o filename
209 (File option) Specifies the condor_submit command output ,
210 the file where stdout is written. If not specified, set to
211 the default name of `` <commandfile>.o<ClusterId>``, where
212 <commandfile> is the condor_qsub argument, and `` <Clus‐
213 terId>`` is the job attribute ClusterId assigned for the job.
214 -p integer
216 (Status option) Sets the priority submit command for the
217 job, with 0 being the default. Jobs with higher numerical
218 priority will run before jobs with lower numerical priority.
219 -print (Specific option) Send to stdout the contents of the HTCondor
221 submit description file that condor_qsub generates.
222 -r y|n (Status option) The default value of y implements the default
224 HTCondor policy of assuming that jobs that do not complete
225 are placed back in the queue to be run again. When n, job
226 submission is restricted to only running the job if the job
227 ClassAd attribute NumJobStarts is currently 0. This identi‐
228 fies the job as not re-runnable, limiting it to start once.
229 -S shell
231 (Submission option) Specifies the path and executable name of
232 a shell. Alters the HTCondor submit description file pro‐
233 duced, such that the executable becomes a wrapper script.
234 Within the submit description file will be executable =
235 <shell> and arguments = <commandfile>.
236 -t start [-stop:step]
238 (Submission option) Queues a set of nearly identical jobs.
239 The SGE-style syntax is supported. start, stop, and step are
240 all integers. start is the starting index of the jobs, stop
241 is the ending index (inclusive) of the jobs, and step is the
242 step size through the indices. Note that using more than one
243 processor or node in a job will not work with this option.
244 -test (Specific option) With the intention of testing a potential
246 job submission, parse files and commands to generate error
247 output. Produces, but then removes the HTCondor submit de‐
248 scription file. Never submits the job, even if no errors are
249 encountered.
250 -v variable list
252 (Environmental option) Used to set the submit command envi‐
253 ronment for the job. variable list is as that defined for
254 the submit command. Note that the syntax needed is special‐
255 ized to deal with quote marks and white space characters.
256 -V (Environmental option) Sets getenv = True in the submit de‐
258 scription file.
259 -W attr_name=attr_value[,attr_name=attr_value...]
261 (File option) PBS/Torque supports a number of attributes.
262 However, condor_qsub only supports the names stagein and sta‐
263 geout for attr_name. The format of attr_value for stagein and
264 stageout is local_file@hostname:remote_file[,...] and we
265 strip it to remote_file[,...]. HTCondor's file transfer mech‐
266 anism is then used if needed.
267 -version
269 (Specific option) Print version information for the con‐
270 dor_qsub program and exit. Note that condor_qsub has its own
271 version numbers which are separate from those of HTCondor.
272
274 condor_qsub will exit with a status value of 0 (zero) upon success, and
275 it will exit with the value 1 (one) upon failure to submit a job.
276
278 HTCondor Team
279
281 1990-2021, Center for High Throughput Computing, Computer Sciences De‐
282 partment, University of Wisconsin-Madison, Madison, WI, US. Licensed
283 under the Apache License, Version 2.0.
2848.8 Jan 26, 2021 CONDOR_QSUB(1)