1qsub(1B) PBS qsub(1B)
2
6 qsub - submit pbs job
7
9 qsub [-a date_time] [-A account_string] [-b secs] [-c interval] [-C
10 directive_prefix] [-d path] [-D path] [-e path] [-h] [-I] [-j join] [-k
11 keep] [-l resource_list] [-m mail_options] [-M user_list] [-N name] [-o
12 path] [-p priority] [-q destination] [-r c] [-S path_list] [-u
13 user_list] [-v variable_list] [-V] [-W additional_attributes] [-X] [-z]
14 [script]
15
17 To create a job is to submit an executable script to a batch server.
18 The batch server will be the default server unless the -q option is
19 specified. See discussion of PBS_DEFAULT under Environment Variables
20 below. Typically, the script is a shell script which will be executed
21 by a command shell such as sh or csh.
22 Options on the qsub command allow the specification of attributes which
24 affect the behavior of the job.
25 The qsub command will pass certain environment variables in the Vari‐
27 able_List attribute of the job. These variables will be available to
28 the job. The value for the following variables will be taken from the
29 environment of the qsub command: HOME, LANG, LOGNAME, PATH, MAIL,
30 SHELL, and TZ. These values will be assigned to a new name which is
31 the current name prefixed with the string "PBS_O_". For example, the
32 job will have access to an environment variable named PBS_O_HOME which
33 have the value of the variable HOME in the qsub command environment.
34 In addition to the above, the following environment variables will be
36 available to the batch job.
37 PBS_O_HOST
39 the name of the host upon which the qsub command is running.
40 PBS_O_QUEUE
42 the name of the original queue to which the job was submitted.
43 PBS_O_WORKDIR
45 the absolute path of the current working directory of the qsub
46 command.
47 PBS_ENVIRONMENT
49 set to PBS_BATCH to indicate the job is a batch job, or to
50 PBS_INTERACTIVE to indicate the job is a PBS interactive job,
51 see -I option.
52 PBS_JOBID
54 the job identifier assigned to the job by the batch system.
55 PBS_JOBNAME
57 the job name supplied by the user.
58 PBS_NODEFILE
60 the name of the file contain the list of nodes assigned to the
61 job (for parallel and cluster systems).
62 PBS_QUEUE
64 the name of the queue from which the job is executed.
65
67 -a date_time
68 Declares the time after which the job is eligible for execu‐
69 tion.
70 The date_time argument is in the form:
72 [[[[CC]YY]MM]DD]hhmm[.SS]
73 Where CC is the first two digits of the year (the century), YY
75 is the second two digits of the year, MM is the two digits for
76 the month, DD is the day of the month, hh is the hour, mm is
77 the minute, and the optional SS is the seconds.
78 If the month, MM, is not specified, it will default to the cur‐
80 rent month if the specified day DD, is in the future. Other‐
81 wise, the month will be set to next month. Likewise, if the
82 day, DD, is not specified, it will default to today if the time
83 hhmm is in the future. Otherwise, the day will be set to
84 tomorrow. For example, if you submit a job at 11:15am with a
85 time of -a 1110, the job will be eligible to run at 11:10am
86 tomorrow.
87 -A account_string
89 Defines the account string associated with the job. The
90 account_string is an undefined string of characters and is
91 interpreted by the server which executes the job. See section
92 2.7.1 of the PBS ERS.
93 -b seconds
95 Defines the maximum number of seconds qsub will block attempt‐
96 ing to contact pbs_server. If pbs_server is down, or for a
97 variety of communication failures, qsub will continually retry
98 connecting to pbs_server for job submission. This value over‐
99 rides the CLIENTRETRY parameter in torque.cfg. This is a non-
100 portable TORQUE extension. Portability-minded users can use
101 the PBS_CLIENTRETRY environmental variable. A negative value
102 is interpreted as infinity. The default is 0.
103 -c interval
105 Defines the interval at which the job will be checkpointed. If
106 the job executes upon a host which does not support checkpoint,
107 this option will be ignored.
108 The interval argument is specified as:
110 n No checkpointing is to be performed.
112 s Checkpointing is to be performed only when the server exe‐
114 cuting the job is shutdown.
115 c Checkpointing is to be performed at the default minimum time
117 for the server executing the job.
118 c=minutes
120 Checkpointing is to be performed at an interval of minutes,
121 which is the integer number of minutes of CPU time used by
122 the job. This value must be greater than zero.
123 -C directive_prefix
125 Defines the prefix that declares a directive to the qsub com‐
126 mand within the script file. See the paragraph on script
127 directives in the Extended Description section.
128 If the -C option is presented with a directive_prefix argument
130 that is the null string, qsub will not scan the script file for
131 directives.
132 -d path Defines the working directory path to be used for the job. If
134 the -d option is not specified, the default working directory
135 is the home directory. This option sets the environment vari‐
136 able PBS_O_INITDIR.
137 -D path Defines the root directory to be used for the job. This option
139 sets the environment variable PBS_O_ROOTDIR.
140 -e path Defines the path to be used for the standard error stream of
142 the batch job. The path argument is of the form:
143 [hostname:]path_name
144 where hostname is the name of a host to which the file will be
145 returned and path_name is the path name on that host in the
146 syntax recognized by POSIX. The argument will be interpreted
147 as follows:
148 path_name
150 Where path_name is not an absolute path name, then the
151 qsub command will expand the path name relative to the
152 current working directory of the command. The command
153 will supply the name of the host upon which it is exe‐
154 cuting for the hostname component.
155 hostname:path_name
157 Where path_name is not an absolute path name, then the
158 qsub command will not expand the path name relative to
159 the current working directory of the command. On deliv‐
160 ery of the standard error, the path name will be
161 expanded relative to the user's home directory on the
162 hostname system.
163 path_name
165 Where path_name specifies an absolute path name, then
166 the qsub will supply the name of the host on which it is
167 executing for the hostname.
168 hostname:path_name
170 Where path_name specifies an absolute path name, the
171 path will be used as specified.
172 If the -e option is not specified, the default file name for
174 the standard error stream will be used. The default name has
175 the following form:
176 job_name.esequence_number
177 where job_name is the name of the job, see -N option, and
178 sequence_number is the job number assigned when the job is sub‐
179 mitted.
180 -h Specifies that a user hold be applied to the job at submission
182 time.
183 -I Declares that the job is to be run "interactively". The job
185 will be queued and scheduled as any PBS batch job, but when
186 executed, the standard input, output, and error streams of the
187 job are connected through qsub to the terminal session in which
188 qsub is running. Interactive jobs are forced to not rerunable.
189 See the "Extended Description" paragraph for addition informa‐
190 tion of interactive jobs.
191 -j join Declares if the standard error stream of the job will be merged
193 with the standard output stream of the job.
194 An option argument value of oe directs that the two streams
196 will be merged, intermixed, as standard output. An option
197 argument value of eo directs that the two streams will be
198 merged, intermixed, as standard error.
199 If the join argument is n or the option is not specified, the
201 two streams will be two separate files.
202 -k keep Defines which (if either) of standard output or standard error
204 will be retained on the execution host. If set for a stream,
205 this option overrides the path name for that stream. If not
206 set, neither stream is retained on the execution host.
207 The argument is either the single letter "e" or "o", or the
209 letters "e" and "o" combined in either order. Or the argument
210 is the letter "n".
211 e The standard error stream is to retained on the execution
213 host. The stream will be placed in the home directory of
214 the user under whose user id the job executed. The file
215 name will be the default file name given by: job_name.ese‐
216 quence where job_name is the name specified for the job, and
217 sequence is the sequence number component of the job identi‐
218 fier.
219 o The standard output stream is to retained on the execution
221 host. The stream will be placed in the home directory of
222 the user under whose user id the job executed. The file
223 name will be the default file name given by: job_name.ose‐
224 quence where job_name is the name specified for the job, and
225 sequence is the sequence number component of the job identi‐
226 fier.
227 eo Both the standard output and standard error streams will be
229 retained.
230 oe Both the standard output and standard error streams will be
232 retained.
233 n Neither stream is retained.
235 -l resource_list
237 Defines the resources that are required by the job and estab‐
238 lishes a limit to the amount of resource that can be consumed.
239 If not set for a generally available resource, such as CPU
240 time, the limit is infinite. The resource_list argument is of
241 the form:
242 resource_name[=[value]][,resource_name[=[value]],...]
243 -m mail_options
245 Defines the set of conditions under which the execution server
246 will send a mail message about the job. The mail_options argu‐
247 ment is a string which consists of either the single character
248 "n", or one or more of the characters "a", "b", and "e".
249 If the character "n" is specified, no mail will be sent.
251 For the letters "a", "b", and "e":
253 a mail is sent when the job is aborted by the batch system.
255 b mail is sent when the job begins execution.
257 e mail is sent when the job terminates.
259 If the -m option is not specified, mail will be sent if the job
261 is aborted.
262 -M user_list
264 Declares the list of users to whom mail is sent by the execu‐
265 tion server when it sends mail about the job.
266 The user_list argument is of the form:
268 user[@host][,user[@host],...]
269 If unset, the list defaults to the submitting user at the qsub
270 host, i.e. the job owner.
271 -N name Declares a name for the job. The name specified may be up to
273 and including 15 characters in length. It must consist of
274 printable, non white space characters with the first character
275 alphabetic.
276 If the -N option is not specified, the job name will be the
278 base name of the job script file specified on the command line.
279 If no script file name was specified and the script was read
280 from the standard input, then the job name will be set to
281 STDIN.
282 -o path Defines the path to be used for the standard output stream of
284 the batch job. The path argument is of the form:
285 [hostname:]path_name
286 where hostname is the name of a host to which the file will be
287 returned and path_name is the path name on that host in the
288 syntax recognized by POSIX. The argument will be interpreted
289 as follows:
290 path_name
292 Where path_name is not an absolute path name, then the
293 qsub command will expand the path name relative to the
294 current working directory of the command. The command
295 will supply the name of the host upon which it is exe‐
296 cuting for the hostname component.
297 hostname:path_name
299 Where path_name is not an absolute path name, then the
300 qsub command will not expand the path name relative to
301 the current working directory of the command. On deliv‐
302 ery of the standard output, the path name will be
303 expanded relative to the user's home directory on the
304 hostname system.
305 path_name
307 Where path_name specifies an absolute path name, then
308 the qsub will supply the name of the host on which it is
309 executing for the hostname.
310 hostname:path_name
312 Where path_name specifies an absolute path name, the
313 path will be used as specified.
314 If the -o option is not specified, the default file name for
316 the standard output stream will be used. The default name has
317 the following form:
318 job_name.osequence_number
319 where job_name is the name of the job, see -N option, and
320 sequence_number is the job number assigned when the job is sub‐
321 mitted.
322 -p priority
324 Defines the priority of the job. The priority argument must be
325 a integer between -1024 and +1023 inclusive. The default is no
326 priority which is equivalent to a priority of zero.
327 -q destination
329 Defines the destination of the job. The destination names a
330 queue, a server, or a queue at a server.
331 The qsub command will submit the script to the server defined
333 by the destination argument. If the destination is a routing
334 queue, the job may be routed by the server to a new destina‐
335 tion.
336 If the -q option is not specified, the qsub command will submit
338 the script to the default server. See PBS_DEFAULT under the
339 Environment Variables section on this man page and the PBS ERS
340 section 2.7.4, "Default Server".
341 If the -q option is specified, it is in one of the following
343 three forms:
344 queue
345 @server
346 queue@server
347 If the destination argument names a queue and does not name a
349 server, the job will be submitted to the named queue at the
350 default server.
351 If the destination argument names a server and does not name a
353 queue, the job will be submitted to the default queue at the
354 named server.
355 If the destination argument names both a queue and a server,
357 the job will be submitted to the named queue at the named
358 server.
359 -r y|n Declares whether the job is rerunable. See the qrerun command.
361 The option argument is a single character, either y or n.
362 If the argument is "y", the job is rerunable. If the argument
364 is "n", the job is not rerunable. The default value is 'y',
365 rerunable.
366 -S path_list
368 Declares the shell that interprets the job script.
369 The option argument path_list is in the form:
371 path[@host][,path[@host],...]
372 Only one path may be specified for any host named. Only one
373 path may be specified without the corresponding host name. The
374 path selected will be the one with the host name that matched
375 the name of the execution host. If no matching host is found,
376 then the path specified without a host will be selected, if
377 present.
378 If the -S option is not specified, the option argument is the
380 null string, or no entry from the path_list is selected, the
381 execution will use the user's login shell on the execution
382 host.
383 -u user_list
385 Defines the user name under which the job is to run on the exe‐
386 cution system.
387 The user_list argument is of the form:
389 user[@host][,user[@host],...]
390 Only one user name may be given per specified host. Only one
391 of the user specifications may be supplied without the corre‐
392 sponding host specification. That user name will used for exe‐
393 cution on any host not named in the argument list. If unset,
394 the user list defaults to the user who is running qsub.
395 -v variable_list
397 Expands the list of environment variables that are exported to
398 the job.
399 In addition to the variables described in the "Description"
401 section above, variable_list names environment variables from
402 the qsub command environment which are made available to the
403 job when it executes. The variable_list is a comma separated
404 list of strings of the form variable or variable=value. These
405 variables and their values are passed to the job.
406 -V Declares that all environment variables in the qsub command's
408 environment are to be exported to the batch job.
409 -W additional_attributes
411 The -W option allows for the specification of additional job
412 attributes. The general syntax of the -W is in the form:
413 -W attr_name=attr_value[,attr_name=attr_value...]
414 Note if white space occurs anywhere within the option argument
415 string or the equal sign, "=", occurs within an attribute_value
416 string, then the string must be enclosed with either single or
417 double quote marks.
418 PBS currently supports the following attributes within the -W
420 option.
421 depend=dependency_list
423 Defines the dependency between this and other jobs. The depen‐
424 dency_list is in the form:
425 type[:argument[:argument...][,type:argument...].
426 The argument is either a numeric count or a PBS job id accord‐
427 ing to type . If argument is a count, it must be greater than
428 0. If it is a job id and not fully specified in the form
429 seq_number.server.name, it will be expanded according to the
430 default server rules which apply to job IDs on most commands.
431 If argument is null (the preceding colon need not be speci‐
432 fied), the dependency of the corresponding type is cleared
433 (unset).
434 synccount:count
436 This job is the first in a set of jobs to be executed
437 at the same time. Count is the number of additional
438 jobs in the set.
439 syncwith:jobid
441 This job is an additional member of a set of jobs to be
442 executed at the same time. In the above and following
443 dependency types, jobid is the job identifier of the
444 first job in the set.
445 after:jobid[:jobid...]
447 This job may be scheduled for execution at any point
448 after jobs jobid have started execution.
449 afterok:jobid[:jobid...]
451 This job may be scheduled for execution only after jobs
452 jobid have terminated with no errors. See the csh
453 warning under "Extended Description".
454 afternotok:jobid[:jobid...]
456 This job may be scheduled for execution only after jobs
457 jobid have terminated with errors. See the csh warning
458 under "Extended Description".
459 afterany:jobid[:jobid...]
461 This job may be scheduled for execution after jobs
462 jobid have terminated, with or without errors.
463 on:count
465 This job may be scheduled for execution after count
466 dependencies on other jobs have been satisfied. This
467 form is used in conjunction with one of the before
468 forms, see below.
469 before:jobid[:jobid...]
471 When this job has begun execution, then jobs jobid...
472 may begin.
473 beforeok:jobid[:jobid...]
475 If this job terminates execution without errors, then
476 jobs jobid... may begin. See the csh warning under
477 "Extended Description".
478 beforenotok:jobid[:jobid...]
480 If this job terminates execution with errors, then jobs
481 jobid... may begin. See the csh warning under
482 "Extended Description".
483 beforeany:jobid[:jobid...]
485 When this job terminates execution, jobs jobid... may
486 begin.
487 If any of the before forms are used, the jobs refer‐
489 enced by jobid must have been submitted with a depen‐
490 dency type of on.
491 If any of the before forms are used, the jobs refer‐
493 enced by jobid must have the same owner as the job
494 being submitted. Otherwise, the dependency is ignored.
495 Error processing of the existence, state, or condition of
497 the job on which the newly submitted job is a deferred ser‐
498 vice, i.e. the check is performed after the job is queued.
499 If an error is detected, the new job will be deleted by the
500 server. Mail will be sent to the job submitter stating the
501 error.
502 Dependency examples:
504 qsub -W depend=afterok:123.big.iron.com /tmp/script
505 qsub -W depend=before:234.hunk1.com:235.hunk1.com
506 /tmp/script
507 group_list=g_list
509 Defines the group name under which the job is to run on the
510 execution system. The g_list argument is of the form:
511 group[@host][,group[@host],...]
512 Only one group name may be given per specified host. Only one
513 of the group specifications may be supplied without the corre‐
514 sponding host specification. That group name will used for
515 execution on any host not named in the argument list. If not
516 set, the group_list defaults to the primary group of the user
517 under which the job will be run.
518 interactive=true
520 If the interactive attribute is specified, the job is an inter‐
521 active job. The -I option is a alternative method of specify‐
522 ing this attribute.
523 stagein=file_list
525 stageout=file_list
526 Specifies which files are staged (copied) in before job start
527 or staged out after the job completes execution. On completion
528 of the job, all staged-in and staged-out files are removed from
529 the execution system. The file_list is in the form
530 local_file@hostname:remote_file[,...]
531 regardless of the direction of the copy. The name local_file
532 is the name of the file on the system where the job executed.
533 It may be an absolute path or relative to the home directory of
534 the user. The name remote_file is the destination name on the
535 host specified by hostname. The name may be absolute or rela‐
536 tive to the user's home directory on the destination host. The
537 use of wildcards in the file name is not recommended. The file
538 names map to a remote copy program (rcp) call on the execution
539 system in the follow manner:
540 For stagein: rcp hostname:remote_file local_file
541 For stageout: rcp local_file hostname:remote_file
542 Data staging examples:
543 -W stagein=/tmp/input.txt@headnode:/home/user/input.txt
544 -W stageout=/tmp/output.txt@headnode:/home/user/output.txt
545 If TORQUE has been compiled with wordexp support, then vari‐
546 ables can be used in the specified paths. Currently only
547 $PBS_JOBID, $HOME, and $TMPDIR are supported for stagein.
548 -X Enables X11 forwarding. The DISPLAY environment variable must
550 be set.
551 -z Directs that the qsub command is not to write the job identi‐
553 fier assigned to the job to the command's standard output.
554
556 The qsub command accepts a script operand that is the path to the
557 script of the job. If the path is relative, it will be expanded rela‐
558 tive to the working directory of the qsub command.
559 If the script operand is not provided or the operand is the single
561 character "-", the qsub command reads the script from standard input.
562 When the script is being read from Standard Input, qsub will copy the
563 file to a temporary file. This temporary file is passed to the library
564 interface routine pbs_submit. The temporary file is removed by qsub
565 after pbs_submit returns or upon the receipt of a signal which would
566 cause qsub to terminate.
567
569 The qsub command reads the script for the job from standard input if
570 the script operand is missing or is the single character "-".
571
573 The script file is read by the qsub command. Qsub acts upon any direc‐
574 tives found in the script.
575 When the job is created, a copy of the script file is made and that
577 copy cannot be modified.
578
580 Unless the -z option is set, the job identifier assigned to the job
581 will be written to standard output if the job is successfully created.
582
584 The qsub command will write a diagnostic message to standard error for
585 each error occurrence.
586
588 The values of some or all of the variables in the qsub command's envi‐
589 ronment are exported with the job, see the -v and -V options.
590 The environment variable PBS_DEFAULT defines the name of the default
592 server. Typically, it corresponds to the system name of the host on
593 which the server is running. If PBS_DEFAULT is not set, the default is
594 defined by an administrator established file.
595 The environment variable PBS_DPREFIX determines the prefix string which
597 identifies directives in the script.
598 The environment variable PBS_CLIENTRETRY defines the maximum number of
600 seconds qsub will block. See the -b option above. Despite the name,
601 currently qsub is the only client that supports this option.
602
604 Script Processing:
605 A job script may consist of PBS directives, comments and executable
607 statements. A PBS directive provides a way of specifying job
608 attributes in addition to the command line options. For example:
609 :
610 #PBS -N Job_name
611 #PBS -l walltime=10:30,mem=320kb
612 #PBS -m be
613 #
614 step1 arg1 arg2
615 step2 arg3 arg4
616 The qsub command scans the lines of the script file for directives. An
619 initial line in the script that begins with the characters "#!" or the
620 character ":" will be ignored and scanning will start with the next
621 line. Scanning will continue until the first executable line, that is
622 a line that is not blank, not a directive line, nor a line whose first
623 non white space character is "#". If directives occur on subsequent
624 lines, they will be ignored.
625 A line in the script file will be processed as a directive to qsub if
627 and only if the string of characters starting with the first non white
628 space character on the line and of the same length as the directive
629 prefix matches the directive prefix.
630 The remainder of the directive line consists of the options to qsub in
632 the same syntax as they appear on the command line. The option charac‐
633 ter is to be preceded with the "-" character.
634 If an option is present in both a directive and on the command line,
636 that option and its argument, if any, will be ignored in the directive.
637 The command line takes precedence.
638 If an option is present in a directive and not on the command line,
640 that option and its argument, if any, will be processed as if it had
641 occurred on the command line.
642 The directive prefix string will be determined in order of preference
644 from:
645 The value of the -C option argument if the option is specified on
647 the command line.
648 The value of the environment variable PBS_DPREFIX if it is defined.
650 The four character string #PBS.
652 If the -C option is found in a directive in the script file, it will be
654 ignored.
655 User Authorization:
657 When the user submits a job from a system other than the one on which
659 the PBS Server is running, the name under which the job is to be exe‐
660 cuted is selected according to the rules listed under the -u option.
661 The user submitting the job must be authorized to run the job under the
662 execution user name. This authorization is provided if
663 (1) The host on which qsub is run is trusted by the execution
665 host (see /etc/hosts.equiv),
666 (2) The execution user has an .rhosts file naming the submit‐
668 ting user on the submitting host.
669 C-Shell .logout File:
671 The following warning applies for users of the c-shell, csh. If the
673 job is executed under the csh and a .logout file exists in the home
674 directory in which the job executes, the exit status of the job is that
675 of the .logout script, not the job script. This may impact any inter-
676 job dependencies. To preserve the job exit status, either remove the
677 .logout file or place the following line as the first line in the
678 .logout file
679 set EXITVAL = $status
680 and the following line as the last executable line in .logout
681 exit $EXITVAL
682 Interactive Jobs:
684 If the -I option is specified on the command line or in a script direc‐
686 tive, or if the "interactive" job attribute declared true via the -W
687 option, -W interactive=true, either on the command line or in a script
688 directive, the job is an interactive job. The script will be processed
689 for directives, but will not be included with the job. When the job
690 begins execution, all input to the job is from the terminal session in
691 which qsub is running.
692 When an interactive job is submitted, the qsub command will not termi‐
694 nate when the job is submitted. Qsub will remain running until the job
695 terminates, is aborted, or the user interrupts qsub with an SIGINT (the
696 control-C key). If qsub is interrupted prior to job start, it will
697 query if the user wishes to exit. If the user response "yes", qsub
698 exits and the job is aborted.
699 One the interactive job has started execution, input to and output from
701 the job pass through qsub. Keyboard generated interrupts are passed to
702 the job. Lines entered that begin with the tilde ('~') character and
703 contain special sequences are escaped by qsub. The recognized escape
704 sequences are:
705 ~. Qsub terminates execution. The batch job is also termi‐
707 nated.
708 ~susp Suspend the qsub program if running under the C shell.
710 "susp" is the suspend character, usually CNTL-Z.
711 ~asusp Suspend the input half of qsub (terminal to job), but
713 allow output to continue to be displayed. Only works
714 under the C shell. "asusp" is the auxiliary suspend
715 character, usually CNTL-Y.
716
718 Upon successful processing, the qsub exit status will be a value of
719 zero.
720 If the qsub command fails, the command exits with a value greater than
722 zero.
723
725 qalter(1B), qdel(1B), qhold(1B), qmove(1B), qmsg(1B), qrerun(1B),
726 qrls(1B), qselect(1B), qsig(1B), qstat(1B), pbs_connect(3B),
727 pbs_job_attributes(7B), pbs_queue_attributes(7B),
728 pbs_resources_irix5(7B), pbs_resources_sp2(7B),
729 pbs_resources_sunos4(7B), pbs_resources_unicos8(7B),
730 pbs_server_attributes(7B), and pbs_server(8B)
731Local qsub(1B)