1condor_qsub(1)              General Commands Manual             condor_qsub(1)
2
3
4

Name

6       condor_qsubQueue jobs that use PBS/SGE-style submission
7

Synopsis

9       condor_qsub[ -- version]
10
11       condor_qsub[Specific   options]   [Directory   options]  [Environmental
12       options] [File options] [Notification options] [Resource options] [Sta‐
13       tus options] [Submission options] commandfile
14

Description

16       condor_qsubsubmits  an  HTCondor  job.  This  job  is  specified  in  a
17       PBS/Torque style or an SGE style. condor_qsubpermits the submission  of
18       dependent jobs without the need to specify the full dependency graph at
19       submission time. Doing things this  way  is  neither  as  efficient  as
20       HTCondor's  DAGMan,  nor  as  functional  as  SGE's qsubor qalter. con‐
21       dor_qsubserves as a minimal translator to be able to use software orig‐
22       inally  written  to  interact  with PBS, Torque, and SGE in an HTCondor
23       pool.
24
25       condor_qsubattempts to behave like qsub. Less than half  of  the  qsub‐
26       functionality  is implemented. Option descriptions describe the differ‐
27       ences between the behavior  of  qsuband  condor_qsub.  qsuboptions  not
28       listed  here are not supported. Some concepts present in PBS and SGE do
29       not apply to HTCondor, and so these options are not implemented.
30
31       For a full listing of qsuboptions, please see
32
33       POSIX
34
35          :             http://pubs.opengroup.org/onlinepubs/9699919799/utili
36          ties/qsub.html
37
38
39
40       SGE
41
42          : http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html
43
44
45
46       PBS/Torque
47
48          : http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/com
49          mands/qsub.htm
50
51
52
53       condor_qsubaccepts either command line options or the single file, com‐
54       mandfile, that contains all of the commands.
55
56       condor_qsubdoes  the opposite of job submission within the griduniverse
57       batchgrid type, which takes HTCondor jobs submitted with HTCondor  syn‐
58       tax and submits them to PBS, SGE, or LSF.
59

Options

61       -a date_time
62
63          (Submission  option) Specify a deferred execution date and time. The
64          PBS/Torque  syntax   of   date_timeis   a   string   in   the   form
65          [[[[CC]YY]MM]DD]hhmm[.SS].  The  portions  of  this string which are
66          optional are CC, YY, MM, DD, and SS. For  SGE,  MMand  DDare  notop‐
67          tional.  For  PBS,  MMand DDare optional. condor_qsubfollows the PBS
68          style.
69
70
71
72       -A account_string
73
74          (Status   option)   Uses   group   accounting   where   the   string
75          account_stringis  the  accounting  group  associated  with this job.
76          Unlike SGE, there is no default group of "sge".
77
78
79
80       -b y|n
81
82          (Submission option) Using the SGE  definition  of  its  -boption,  a
83          value of ycauses condor_qsubto notparse the file for additional con‐
84          dor_qsubcommands. The default value is n. If the command line  argu‐
85          ment -f filenameis also specified, it negates a value of y.
86
87
88
89       -c checkpoint_option
90
91          (Submission  option)  For  standard universe jobs only, controls the
92          how HTCondor produces checkpoints. checkpoint_optionsmay be one of
93
94          n or N
95
96             Do not produce checkpoints.
97
98
99
100          s or S
101
102             Do not produce periodic checkpoints. A job will  only  produce  a
103             checkpoint when the job is evicted.
104
105             More options may be implemented in the future.
106
107
108
109       --condor-keep-files
110
111          (Specific option) Directs HTCondor to notremove temporary files gen‐
112          erated by condor_qsub, such as HTCondor submit  files  and  sentinel
113          jobs. These temporary files may be important for debugging.
114
115
116
117       -cwd
118
119          (Directory  option) Specifies the initial directory in which the job
120          will run to be the current directory from which the job was  submit‐
121          ted. This sets initialdirfor condor_submit.
122
123
124
125       -d pathor -wd path
126
127          (Directory  option) Specifies the initial directory in which the job
128          will run to be path. This sets initialdirfor condor_submit.
129
130
131
132       -e filename
133
134          (File option) Specifies the  condor_submitcommand  error,  the  file
135          where stderris written. If not specified, set to the default name of
136          <commandfile>.e<ClusterId>, where <commandfile>is the condor_qsubar‐
137          gument,  and  <ClusterId>is  the job attribute ClusterIdassigned for
138          the job.
139
140
141
142       --f qsub_file
143
144          (Specific option) Parse qsub_fileto search for  and  set  additional
145          condor_submitcommands.  Within  the  file,  commands  will appear as
146          #PBSor  #SGE.  condor_qsubwill  parse  the  batch  file  listed   as
147          qsub_file.
148
149
150
151       -h
152
153          (Status option) Placed submitted job directly into the hold state.
154
155
156
157       --help
158
159          (Specific option) Print usage information and exit.
160
161
162
163       -hold_jid <jid>
164
165          (Status  option)  Submits  a  job  in  the  hold  state. This job is
166          released only when a previously submitted  job,  identified  by  its
167          cluster  ID  as  <jid>, exits successfully. Successful completion is
168          defined as not exiting with exit code 100. In implementation,  there
169          are  three  jobs  that define this SGE feature. The first job is the
170          previously submitted job. The second job is the newly submitted  one
171          that  is waiting for the first to finish successfully. The third job
172          is what SGE calls a sentineljob; this is an HTCondor local  universe
173          job  that  watches  the  history for the first job's exit code. This
174          third job will exit once it has seen the exit code and, for  a  suc‐
175          cessful  termination of the first job, run condor_releaseon the sec‐
176          ond job. If the first job is an array job, the second job will  only
177          be  released  after  all  individual jobs of the first job have com‐
178          pleted.
179
180
181
182       -i [hostname:]filename
183
184          (File option) Specifies the  condor_submitcommand  input,  the  file
185          from which stdinis read.
186
187
188
189       -j characters
190
191          (File option) Acceptable characters for this option are e, o, and n.
192          The only sequence that is relevant is eo;  it  specifies  that  both
193          standard  output and standard error are to be sent to the same file.
194          The file will be the one specified by  the  -ooption,  if  both  the
195          -oand  -eoptions  exist.  The  file will be the one specified by the
196          -eoption, if only the -eoption is provided. If neither the -onor the
197          -eoptions  are  provided,  the file will be the default used for the
198          -ooption.
199
200
201
202       -l resource_spec
203
204          (Resource option) Specifies requirements for the job,  such  as  the
205          amount  of  RAM  and  the  number  of  CPUs. Only PBS-style resource
206          requests are supported. resource_specis a comma  separated  list  of
207          key/value  pairs.  Each  pair  is  of  the form resource_name=value.
208          resource_nameand valuemay be
209
210          resource_namevalueDescriptionarchstringSets  Archmachine  attribute.
211          Enclose in double quotes.filesizeDisk space requested.hoststringHost
212          machine  on  which  the  job  must   run.memsizeAmount   of   memory
213          requested.nodes{<node_count>      |     <hostname>}     [:ppn=<ppn>]
214          [:gpus=<gpu>] [:<property> [:<property>] ...] [+  ...]Number  and/or
215          properties   of   nodes   to  be  used.  For  examples,  please  see
216          http://docs.adaptivecomputing.com/torque/4-1-3/Content/top
217          ics/2-jobs/requestingRes.htm#qsubopsysstringSets        OpSysmachine
218          attribute.  Enclose  in  double  quotes.procsintegerNumber  of  CPUs
219          requested.A  size  value is an integer specified in bytes, following
220          the PBS/Torque default. Append Kb, Mb, Gb, or Tbto specify the value
221          in powers of two quantities greater than bytes.
222
223
224
225       -m a|e|n
226
227          (Notification  option)  Identify when HTCondor sends notification e-
228          mail. If a, send e-mail when the job terminates  abnormally.  If  e,
229          send e-mail when the job terminates. If n, never send e-mail.
230
231
232
233       -M e-mail_address
234
235          (Notification  option)  Sets the destination address for HTCondor e-
236          mail.
237
238
239
240       -o filename
241
242          (File option) Specifies the condor_submitcommand  output,  the  file
243          where stdoutis written. If not specified, set to the default name of
244          <commandfile>.o<ClusterId>, where <commandfile>is the condor_qsubar‐
245          gument,  and  <ClusterId>is  the job attribute ClusterIdassigned for
246          the job.
247
248
249
250       -p integer
251
252          (Status option) Sets the prioritysubmit command for the job, with  0
253          being  the  default.  Jobs  with  higher numerical priority will run
254          before jobs with lower numerical priority.
255
256
257
258       --print
259
260          (Specific option) Send to stdoutthe contents of the HTCondor  submit
261          description file that condor_qsubgenerates.
262
263
264
265       -r y|n
266
267          (Status  option) The default value of yimplements the default HTCon‐
268          dor policy of assuming that jobs that do  not  complete  are  placed
269          back  in  the  queue  to  be  run  again.  When n, job submission is
270          restricted to only running the job  if  the  job  ClassAd  attribute
271          NumJobStartsis  currently  0.  This  identifies  the  job as not re-
272          runnable, limiting it to start once.
273
274
275
276       -S shell
277
278          (Submission option) Specifies the path  and  executable  name  of  a
279          shell.  Alters  the  HTCondor submit description file produced, such
280          that the executable becomes a  wrapper  script.  Within  the  submit
281          description  file  will be executable = <shell>and arguments = <com‐
282          mandfile>.
283
284
285
286       -t start [-stop:step]
287
288          (Submission option) Queues a set of nearly identical jobs. The  SGE-
289          style  syntax  is  supported. start, stop, and stepare all integers.
290          startis the starting index of the  jobs,  stopis  the  ending  index
291          (inclusive)  of  the  jobs,  and  stepis  the  step size through the
292          indices. Note that using more than one processor or node  in  a  job
293          will not work with this option.
294
295
296
297       --test
298
299          (Specific option) With the intention of testing a potential job sub‐
300          mission, parse files and commands to  generate  error  output.  Pro‐
301          duces,  but then removes the HTCondor submit description file. Never
302          submits the job, even if no errors are encountered.
303
304
305
306       -v variable list
307
308          (Environmental option) Used to set the submit command environmentfor
309          the  job.  variable  listis  as that defined for the submit command.
310          Note that the syntax needed is specialized to deal with quote  marks
311          and white space characters.
312
313
314
315       -V
316
317          (Environmental  option)  Sets getenv = Truein the submit description
318          file.
319
320
321
322       -W attr_name=attr_value[,attr_name=attr_value...]
323
324          (File option) PBS/Torque supports a number of  attributes.  However,
325          condor_qsubonly supports the names stageinand stageoutfor attr_name.
326          The format of attr_valuefor stageinand  stageoutis  local_file@host‐
327          name:remote_file[,...]and  we  strip it to remote_file[,...]. HTCon‐
328          dor's file transfer mechanism is then used if needed.
329
330
331
332       --version
333
334          (Specific option) Print version information for the  condor_qsubpro‐
335          gram  and  exit.  Note  that  condor_qsubhas its own version numbers
336          which are separate from those of HTCondor.
337
338
339

Exit Status

341       condor_qsubwill exit with a status value of 0 (zero) upon success,  and
342       it will exit with the value 1 (one) upon failure to submit a job.
343

Author

345       Center for High Throughput Computing, University of Wisconsin-Madison
346
348       Copyright  (C) 1990-2019 Center for High Throughput Computing, Computer
349       Sciences Department, University of Wisconsin-Madison, Madison, WI.  All
350       Rights Reserved. Licensed under the Apache License, Version 2.0.
351
352                                     date                       condor_qsub(1)
Impressum