1CONDOR_QSUB(1)                  HTCondor Manual                 CONDOR_QSUB(1)
2
3
4

NAME

6       condor_qsub - HTCondor Manual
7
8       Queue jobs that use PBS/SGE-style submission
9
10

SYNOPSIS

12       condor_qsub [--version]
13
14       condor_qsub [Specific options ] [Directory options ] [Environmental op‐
15       tions ] [File options ] [Notification options  ]  [Resource  options  ]
16       [Status options ] [Submission options ] commandfile
17

DESCRIPTION

19       condor_qsub  submits  an  HTCondor  job.  This  job  is  specified in a
20       PBS/Torque style or an SGE style. condor_qsub permits the submission of
21       dependent jobs without the need to specify the full dependency graph at
22       submission time. Doing things this way is neither as efficient  as  HT‐
23       Condor's  DAGMan,  nor  as  functional  as  SGE's qsub or qalter.  con‐
24       dor_qsub serves as a minimal translator to  be  able  to  use  software
25       originally written to interact with PBS, Torque, and SGE in an HTCondor
26       pool.
27
28       condor_qsub attempts to behave like qsub. Less than half  of  the  qsub
29       functionality  is implemented. Option descriptions describe the differ‐
30       ences between the behavior of qsub and condor_qsub.  qsub  options  not
31       listed  here are not supported. Some concepts present in PBS and SGE do
32       not apply to HTCondor, and so these options are not implemented.
33
34       For a full listing of qsub options, please see
35
36          POSIX  :
37                 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/qsub.html
38
39          SGE    :
40                 http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html
41
42          PBS/Torque
43                 :
44                 http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/commands/qsub.htm
45
46       condor_qsub accepts either command line options  or  the  single  file,
47       commandfile, that contains all of the commands.
48
49       condor_qsub  does  the  opposite of job submission within the grid uni‐
50       verse batch grid type, which takes HTCondor jobs submitted with  HTCon‐
51       dor syntax and submits them to PBS, SGE, or LSF.
52

OPTIONS

54          -a date_time
55                 (Submission  option)  Specify  a  deferred execution date and
56                 time. The PBS/Torque syntax of date_time is a string  in  the
57                 form  [[[[CC]YY]MM]DD]hhmm[.SS].  The portions of this string
58                 which are optional are CC, YY, MM, DD, and SS.  For  SGE,  MM
59                 and  DD  are  not  optional. For PBS, MM and DD are optional.
60                 condor_qsub follows the PBS style.
61
62          -A account_string
63                 (Status option) Uses group accounting where  the  string  ac‐
64                 count_string  is  the  accounting  group associated with this
65                 job.  Unlike SGE, there is no default group of "sge".
66
67          -b y|n (Submission option) Using the SGE definition of  its  -b  op‐
68                 tion,  a  value of y causes condor_qsub to not parse the file
69                 for additional condor_qsub commands. The default value is  n.
70                 If  the  command line argument -f filename is also specified,
71                 it negates a value of y.
72
73          -c checkpoint_option
74                 (Submission option) For standard universe jobs only, controls
75                 the how HTCondor produces checkpoints. checkpoint_options may
76                 be one of
77
78                     n or N Do not produce checkpoints.
79
80                     s or S Do not produce periodic checkpoints.  A  job  will
81                            only produce a checkpoint when the job is evicted.
82                 More options may be implemented in the future.
83
84
85          -condor-keep-files
86                 (Specific  option)  Directs  HTCondor to not remove temporary
87                 files generated by condor_qsub, such as HTCondor submit files
88                 and sentinel jobs. These temporary files may be important for
89                 debugging.
90
91          -cwd   (Directory option) Specifies the initial directory  in  which
92                 the  job  will run to be the current directory from which the
93                 job was submitted. This sets initialdir   for condor_submit.
94
95          -d path or -wd path
96                 (Directory option) Specifies the initial directory  in  which
97                 the  job will run to be path. This sets initialdir   for con‐
98                 dor_submit.
99
100          -e filename
101                 (File option) Specifies the condor_submit  command  error   ,
102                 the  file  where  stderr is written. If not specified, set to
103                 the default name of ``   <commandfile>.e<ClusterId>``,  where
104                 <commandfile>  is  the  condor_qsub  argument, and ``  <Clus‐
105                 terId>`` is the job attribute ClusterId assigned for the job.
106
107          -f qsub_file
108                 (Specific option) Parse qsub_file to search for and set addi‐
109                 tional condor_submit commands. Within the file, commands will
110                 appear as #PBS or #SGE. condor_qsub will parse the batch file
111                 listed as qsub_file.
112
113          -h     (Status  option)  Placed submitted job directly into the hold
114                 state.
115
116          -help  (Specific option) Print usage information and exit.
117
118          -hold_jid <jid>
119                 (Status option) Submits a job in the hold state. This job  is
120                 released  only when a previously submitted job, identified by
121                 its cluster ID as <jid>, exits successfully. Successful  com‐
122                 pletion  is defined as not exiting with exit code 100. In im‐
123                 plementation, there are three jobs that define this SGE  fea‐
124                 ture. The first job is the previously submitted job. The sec‐
125                 ond job is the newly submitted one that is  waiting  for  the
126                 first to finish successfully. The third job is what SGE calls
127                 a sentinel job; this is an HTCondor local universe  job  that
128                 watches the history for the first job's exit code. This third
129                 job will exit once it has seen the exit code and, for a  suc‐
130                 cessful  termination  of the first job, run condor_release on
131                 the second job. If the first job is an array job, the  second
132                 job  will  only  be released after all individual jobs of the
133                 first job have completed.
134
135          -i [hostname:]filename
136                 (File option) Specifies the condor_submit  command  input   ,
137                 the file from which stdin is read.
138
139          -j characters
140                 (File option) Acceptable characters for this option are e, o,
141                 and n. The only sequence that is relevant is eo; it specifies
142                 that  both  standard output and standard error are to be sent
143                 to the same file. The file will be the one specified  by  the
144                 -o option, if both the -o and -e options exist. The file will
145                 be the one specified by the -e option, if only the -e  option
146                 is  provided.  If  neither the -o nor the -e options are pro‐
147                 vided, the file will be the default used for the -o option.
148
149          -l resource_spec
150                 (Resource option) Specifies requirements for the job, such as
151                 the  amount of RAM and the number of CPUs. Only PBS-style re‐
152                 source requests are supported. resource_spec is a comma sepa‐
153                 rated  list  of key/value pairs. Each pair is of the form re‐
154                 source_name=value.   resource_name   and   value    may    be
155                 +--------------------------+--------------------------+--------------------------+
156                 | resource_name        | value                 |  Description
157                 |
158                 +--------------------------+--------------------------+--------------------------+
159                 |  arch                     | string                   | Sets
160                 Arch    machine        |     |                              |
161                 |  attribute.  Enclose  in     | |                          |
162                 |            double            quotes.                      |
163                 +--------------------------+--------------------------+--------------------------+
164                 | file                     | size                     |  Disk
165                 space                     requested.                        |
166                 +--------------------------+--------------------------+--------------------------+
167                 |  host                     | string                   | Host
168                 machine   on   which      |   |                             |
169                 |        the        job        must       run.              |
170                 +--------------------------+--------------------------+--------------------------+
171                 |   mem                        |  size                      |
172                 Amount  of  memory          |  |                            |
173                 |                  requested.                               |
174                 +--------------------------+--------------------------+--------------------------+
175                 |  nodes                    | {<node_count> | <hostn | Number
176                 and/or  properties  |  |                            |   ame>}
177                 [:ppn=<ppn>]   [:gpu   |  of  nodes  to  be  used.  For  |  |
178                 | s=<gpu>] [:<property> [: | examples,  please  see      |  |
179                 |  <property>]  ...]  [+ ...]   | http://docs.adaptivecom | |
180                 |                           |  puting.com/torque/4-1-3/  |  |
181                 |                           |  Content/topics/2-jobs/re  |  |
182                 |                            |   questingRes.htm#qsub       |
183                 +--------------------------+--------------------------+--------------------------+
184                 | opsys                    | string                   |  Sets
185                 OpSys     machine       |     |                             |
186                 | attribute. Enclose  in     |  |                           |
187                 |            double            quotes.                      |
188                 +--------------------------+--------------------------+--------------------------+
189                 |  procs                    | integer                  | Num‐
190                 ber  of   CPUs             |   |                            |
191                 |                  requested.                               |
192                 +--------------------------+--------------------------+--------------------------+
193
194                 A  size value is an integer specified in bytes, following the
195                 PBS/Torque default. Append Kb, Mb, Gb, or Tb to  specify  the
196                 value in powers of two quantities greater than bytes.
197
198          -m a|e|n
199                 (Notification  option) Identify when HTCondor sends notifica‐
200                 tion e-mail. If a, send e-mail when the job terminates abnor‐
201                 mally. If e, send e-mail when the job terminates. If n, never
202                 send e-mail.
203
204          -M e-mail_address
205                 (Notification option) Sets the destination address for HTCon‐
206                 dor e-mail.
207
208          -o filename
209                 (File  option)  Specifies the condor_submit command output  ,
210                 the file where stdout is written. If not  specified,  set  to
211                 the  default  name of ``  <commandfile>.o<ClusterId>``, where
212                 <commandfile> is the condor_qsub  argument,  and  ``   <Clus‐
213                 terId>`` is the job attribute ClusterId assigned for the job.
214
215          -p integer
216                 (Status  option)  Sets  the priority   submit command for the
217                 job, with 0 being the default.  Jobs  with  higher  numerical
218                 priority will run before jobs with lower numerical priority.
219
220          -print (Specific option) Send to stdout the contents of the HTCondor
221                 submit description file that condor_qsub generates.
222
223          -r y|n (Status option) The default value of y implements the default
224                 HTCondor  policy  of  assuming that jobs that do not complete
225                 are placed back in the queue to be run  again.  When  n,  job
226                 submission  is  restricted to only running the job if the job
227                 ClassAd attribute NumJobStarts is currently 0.  This  identi‐
228                 fies the job as not re-runnable, limiting it to start once.
229
230          -S shell
231                 (Submission option) Specifies the path and executable name of
232                 a shell. Alters the HTCondor  submit  description  file  pro‐
233                 duced,  such  that  the  executable becomes a wrapper script.
234                 Within the submit  description  file  will  be  executable  =
235                 <shell> and arguments = <commandfile>.
236
237          -t start [-stop:step]
238                 (Submission  option)  Queues  a set of nearly identical jobs.
239                 The SGE-style syntax is supported. start, stop, and step  are
240                 all  integers.  start is the starting index of the jobs, stop
241                 is the ending index (inclusive) of the jobs, and step is  the
242                 step  size through the indices. Note that using more than one
243                 processor or node in a job will not work with this option.
244
245          -test  (Specific option) With the intention of testing  a  potential
246                 job  submission,  parse  files and commands to generate error
247                 output.  Produces, but then removes the HTCondor  submit  de‐
248                 scription file.  Never submits the job, even if no errors are
249                 encountered.
250
251          -v variable list
252                 (Environmental option) Used to set the submit  command  envi‐
253                 ronment    for  the job. variable list is as that defined for
254                 the submit command.  Note that the syntax needed is  special‐
255                 ized to deal with quote marks and white space characters.
256
257          -V     (Environmental  option)  Sets getenv = True in the submit de‐
258                 scription file.
259
260          -W attr_name=attr_value[,attr_name=attr_value...]
261                 (File option) PBS/Torque supports  a  number  of  attributes.
262                 However, condor_qsub only supports the names stagein and sta‐
263                 geout for attr_name. The format of attr_value for stagein and
264                 stageout   is  local_file@hostname:remote_file[,...]  and  we
265                 strip it to remote_file[,...]. HTCondor's file transfer mech‐
266                 anism is then used if needed.
267
268          -version
269                 (Specific  option)  Print  version  information  for the con‐
270                 dor_qsub program and exit. Note that condor_qsub has its  own
271                 version numbers which are separate from those of HTCondor.
272

EXIT STATUS

274       condor_qsub will exit with a status value of 0 (zero) upon success, and
275       it will exit with the value 1 (one) upon failure to submit a job.
276

AUTHOR

278       HTCondor Team
279
281       1990-2021, Center for High Throughput Computing, Computer Sciences  De‐
282       partment,  University  of  Wisconsin-Madison, Madison, WI, US. Licensed
283       under the Apache License, Version 2.0.
284
285
286
287
2888.8                              Jan 26, 2021                   CONDOR_QSUB(1)
Impressum