1qsub(1B) PBS qsub(1B)
2
6 qsub - submit pbs job
7
9 qsub [-a date_time] [-A account_string] [-b secs] [-c checkpoint_op‐
10 tions] [-C directive_prefix] [-d path] [-D path] [-e path] [-f] [-F]
11 [-h] [-i idle_slot_limit] [-I] [-j join] [-k keep] [-K kill_delay] [-l
12 resource_list] [-L NUMA_resource_list] [-m mail_options] [-M user_list]
13 [-n node_exclusive] [-N name] [-o path] [-p priority] [-P proxy_user‐
14 name[:group]] [-q destination] [-r c] [-S path_list] [-t array_re‐
15 quest] [-T prologue/epilogue script_name] [-u user_list] [-v vari‐
16 able_list] [-V] [-w path] [-W additional_attributes] [-x] [-X] [-z]
17 [script]
18
20 To create a job is to submit an executable script to a batch server.
21 The batch server will be the default server unless the -q option is
22 specified. See discussion of PBS_DEFAULT under Environment Variables
23 below. Typically, the script is a shell script which will be executed
24 by a command shell such as sh or csh.
25 Options on the qsub command allow the specification of attributes which
27 affect the behavior of the job.
28 The qsub command will pass certain environment variables in the Vari‐
30 able_List attribute of the job. These variables will be available to
31 the job. The value for the following variables will be taken from the
32 environment of the qsub command: HOME, LANG, LOGNAME, PATH, MAIL,
33 SHELL, and TZ. These values will be assigned to a new name which is
34 the current name prefixed with the string "PBS_O_". For example, the
35 job will have access to an environment variable named PBS_O_HOME which
36 have the value of the variable HOME in the qsub command environment.
37 In addition to the above, the following environment variables will be
39 available to the batch job.
40 PBS_O_HOST
42 the name of the host upon which the qsub command is running.
43 PBS_SERVER
45 the hostname of the pbs_server which qsub submits the job to.
46 PBS_O_QUEUE
48 the name of the original queue to which the job was submitted.
49 PBS_O_WORKDIR
51 the absolute path of the current working directory of the qsub
52 command.
53 PBS_ARRAYID
55 each member of a job array is assigned a unique identifier (see
56 -t)
57 PBS_ENVIRONMENT
59 set to PBS_BATCH to indicate the job is a batch job, or to
60 PBS_INTERACTIVE to indicate the job is a PBS interactive job,
61 see -I option.
62 PBS_JOBID
64 the job identifier assigned to the job by the batch system.
65 PBS_JOBNAME
67 the job name supplied by the user.
68 PBS_NODEFILE
70 the name of the file contain the list of nodes assigned to the
71 job (for parallel and cluster systems).
72 PBS_QUEUE
74 the name of the queue from which the job is executed.
75
77 -a date_time
78 Declares the time after which the job is eligible for execu‐
79 tion.
80 The date_time argument is in the form:
82 [[[[CC]YY]MM]DD]hhmm[.SS]
83 Where CC is the first two digits of the year (the century), YY
85 is the second two digits of the year, MM is the two digits for
86 the month, DD is the day of the month, hh is the hour, mm is
87 the minute, and the optional SS is the seconds.
88 If the month, MM, is not specified, it will default to the cur‐
90 rent month if the specified day DD, is in the future. Other‐
91 wise, the month will be set to next month. Likewise, if the
92 day, DD, is not specified, it will default to today if the time
93 hhmm is in the future. Otherwise, the day will be set to to‐
94 morrow. For example, if you submit a job at 11:15am with a
95 time of -a 1110, the job will be eligible to run at 11:10am to‐
96 morrow.
97 -A account_string
99 Defines the account string associated with the job. The ac‐
100 count_string is an undefined string of characters and is inter‐
101 preted by the server which executes the job. See section 2.7.1
102 of the PBS ERS.
103 -b seconds
105 Defines the maximum number of seconds qsub will block attempt‐
106 ing to contact pbs_server. If pbs_server is down, or for a va‐
107 riety of communication failures, qsub will continually retry
108 connecting to pbs_server for job submission. This value over‐
109 rides the CLIENTRETRY parameter in torque.cfg. This is a non-
110 portable TORQUE extension. Portability-minded users can use
111 the PBS_CLIENTRETRY environmental variable. A negative value
112 is interpreted as infinity. The default is 0.
113 -c checkpoint_options
115 Defines the options that will apply to the job. If the job ex‐
116 ecutes upon a host which does not support checkpoint, these op‐
117 tions will be ignored.
118 Valid checkpoint options are:
120 none
122 No checkpointing is to be performed.
123 enabled
125 Specify that checkpointing is allowed but must be explicitly
126 invoked by either the qhold or qchkpt commands.
127 shutdown
129 Specify that checkpointing is to be done on a job at pbs_mom
130 shutdown.
131 periodic
133 Specify that periodic checkpointing is enabled. The default
134 interval is 10 minutes and can be changed by the $check‐
135 point_interval option in the mom config file or by specify‐
136 ing an interval when the job is submitted
137 interval=minutes
139 Checkpointing is to be performed at an interval of minutes,
140 which is the integer number of minutes of wall time used by
141 the job. This value must be greater than zero.
142 depth=number
144 Specify a number (depth) of checkpoint images to be kept in
145 the checkpoint directory.
146 dir=path
148 Specify a checkpoint directory (default is
149 /var/spool/torque/checkpoint).
150 -C directive_prefix
152 Defines the prefix that declares a directive to the qsub com‐
153 mand within the script file. See the paragraph on script di‐
154 rectives in the Extended Description section.
155 If the -C option is presented with a directive_prefix argument
157 that is the null string, qsub will not scan the script file for
158 directives.
159 -d path Defines the working directory path to be used for the job. If
161 the -d option is not specified, the default working directory
162 is the home directory. This option sets the environment vari‐
163 able PBS_O_INITDIR.
164 -D path Defines the root directory to be used for the job. This option
166 sets the environment variable PBS_O_ROOTDIR.
167 -e path Defines the path to be used for the standard error stream of
169 the batch job. The path argument is of the form:
170 [hostname:][path_name]
171 where hostname is the name of a host to which the file will be
172 returned and path_name is the path name on that host in the
173 syntax recognized by POSIX. The argument will be interpreted
174 as follows:
175 path_name
177 Where path_name is not an absolute path name, then the
178 qsub command will expand the path name relative to the
179 current working directory of the command. The command
180 will supply the name of the host upon which it is exe‐
181 cuting for the hostname component.
182 hostname:path_name
184 Where path_name is not an absolute path name, then the
185 qsub command will not expand the path name relative to
186 the current working directory of the command. On deliv‐
187 ery of the standard error, the path name will be ex‐
188 panded relative to the user's home directory on the
189 hostname system.
190 path_name
192 Where path_name specifies an absolute path name, then
193 the qsub will supply the name of the host on which it is
194 executing for the hostname
195 hostname:path_name
197 Where path_name specifies an absolute path name, the
198 path will be used as specified. hostname.
199 hostname:
201 Where hostname specifies the name of the host that the
202 file should be returned to. The path will be the default
203 file name.
204 If the -e option is not specified or the path_name is not spec‐
206 ified or is specified and is a directory, the default file name
207 for the standard error stream will be used. The default name
208 has the following form:
209 job_name.esequence_number
210 where job_name is the name of the job, see -N option, and se‐
211 quence_number is the job number assigned when the job is sub‐
212 mitted.
213 -f Specifies that the job is fault tolerant. The fault_tolerant
215 attribute will be set to true, which indicates that the job can
216 survive the loss of a mom other than the "mother superior" mom
217 (the first node in the exec hosts )
218 -F Specifies the arguments that will be passed to the job script
220 when the script is launched.
221 The accepted syntax is:
222 qsub -F "myarg1 myarg2 myarg3=myarg3value" myscript2.sh
223 -h Specifies that a user hold be applied to the job at submission
225 time.
226 -i idle_slot_lmit
228 Sets an idle slot limit for the job array being submitted. If
229 this parameter is set for a non-array job, it will be rejected.
230 Additionally, if the user requests an idle slot limit that ex‐
231 ceeds the the server parameter's default, the job will be re‐
232 jected. See also the idle_slot_limit server parameter.
233 $ qsub -t 0-99 -i 10 script.sh
234 The submitted array will only instantiate 10 idle jobs; in‐
235 stead of all 100 jobs at submission time.
236 -I Declares that the job is to be run "interactively". The job
238 will be queued and scheduled as any PBS batch job, but when ex‐
239 ecuted, the standard input, output, and error streams of the
240 job are connected through qsub to the terminal session in which
241 qsub is running. Interactive jobs are forced to not rerunable.
242 See the "Extended Description" paragraph for addition informa‐
243 tion of interactive jobs.
244 -j join Declares if the standard error stream of the job will be merged
246 with the standard output stream of the job.
247 An option argument value of oe directs that the two streams
249 will be merged, intermixed, as standard output. An option ar‐
250 gument value of eo directs that the two streams will be merged,
251 intermixed, as standard error.
252 If the join argument is n or the option is not specified, the
254 two streams will be two separate files.
255 -k keep Defines which (if either) of standard output or standard error
257 will be retained on the execution host. If set for a stream,
258 this option overrides the path name for that stream. If not
259 set, neither stream is retained on the execution host.
260 The argument is either the single letter "e" or "o", or the
262 letters "e" and "o" combined in either order. Or the argument
263 is the letter "n".
264 e The standard error stream is to retained on the execution
266 host. The stream will be placed in the home directory of
267 the user under whose user id the job executed. The file
268 name will be the default file name given by: job_name.ese‐
269 quence where job_name is the name specified for the job, and
270 sequence is the sequence number component of the job identi‐
271 fier.
272 o The standard output stream is to retained on the execution
274 host. The stream will be placed in the home directory of
275 the user under whose user id the job executed. The file
276 name will be the default file name given by: job_name.ose‐
277 quence where job_name is the name specified for the job, and
278 sequence is the sequence number component of the job identi‐
279 fier.
280 eo Both the standard output and standard error streams will be
282 retained.
283 oe Both the standard output and standard error streams will be
285 retained.
286 n Neither stream is retained.
288 -K kill_delay
290 Specifies a job specific kill delay in seconds. When specified,
291 this job will be sent a SIGTERM, followed by a SIGKILL the
292 specified number of seconds later.
293 -l resource_list
295 Defines the resources that are required by the job and estab‐
296 lishes a limit to the amount of resource that can be consumed.
297 If not set for a generally available resource, such as CPU
298 time, the limit is infinite. The resource_list argument is of
299 the form:
300 resource_name[=[value]][,resource_name[=[value]],...]
301 -L NUMA_resource_list
303 Defines the NUMA-aware resource requests for NUMA hardware.
304 This option will work with non-NUMA hardware. Syntax for
305 NUMA_resource_list is:
306 tasks=#[:lprocs=#|all]
307 [:{usecores|usethreads|allowthreads}]
308 [:place={socket|numanode|core|thread}[=#]{node}][:mem‐
309 ory=#][:swap=#][:maxtpn=#][:gpus=#[:<mode>]][:mics=#]
310 [:gres=<gres>][:feature=<feature>]
311 [[:{cpt|cgroup_per_task}]|[:{cph|cgroup_per_host}]]
312 -m mail_options
314 Defines the set of conditions under which the execution server
315 will send a mail message about the job. The mail_options argu‐
316 ment is a string which consists of either the single character
317 "n" or "p", or one or more of the characters "a", "b", "e", and
318 "f".
319 If the character "n" is specified, no normal mail is sent. Mail
321 for job cancels and other events outside of normal job process‐
322 ing are still sent. If the character "p" is specified, mail
323 will never be sent for the job.
324 For the characters "a", "b", "e", and "f":
326 a Mail is sent when the job is aborted by the batch system.
328 b Mail is sent when the job begins execution.
330 e Mail is sent when the job terminates.
332 f Mail is sent when the job terminates with a non-zero exit
334 code.
335 If the -m option is not specified, mail will be sent if the job
337 is aborted.
338 -M user_list
341 Declares the list of users to whom mail is sent by the execu‐
342 tion server when it sends mail about the job.
343 The user_list argument is of the form:
345 user[@host][,user[@host],...]
346 If unset, the list defaults to the submitting user at the qsub
347 host, i.e. the job owner.
348 -n node_exclusive
350 Allows a user to specify an exclusive-node access/allocation
352 request for the job. This will set node_exclusive = True in the
353 output of qstat -f <job ID>. By default, this only applies for
354 cpusets, and only for compatible schedulers.
355 For systems that use Moab and have cgroups enabled, the recom‐
357 mended manner for assigning all cores is to use NUMA syntax:
358 "-L tasks=<count>:lprocs=all:place=node".
359 With cgroups, the ("-l") syntax (lowercase L) will, by default,
361 restrict to the number of cores requested, or to the re‐
362 sources_default.procs value (i.e., 1 core, typically). In order
363 to override this behavior and have Moab assign all the cores on
364 a node while using "-l...singlejob" and/or "-n" (in other
365 words, without "-L ...lprocs=all..."), you must also set RM‐
366 CFG[<torque>] FLAGS=MigrateAllJobAttributes in moab.cfg.
367 -N name Declares a name for the job. The name specified may be up to
371 and including 15 characters in length. It must consist of
372 printable, non white space characters with the first character
373 alphabetic.
374 If the -N option is not specified, the job name will be the
376 base name of the job script file specified on the command line.
377 If no script file name was specified and the script was read
378 from the standard input, then the job name will be set to
379 STDIN.
380 -o path Defines the path to be used for the standard output stream of
382 the batch job. The path argument is of the form:
383 [hostname:][path_name]
384 where hostname is the name of a host to which the file will be
385 returned and path_name is the path name on that host in the
386 syntax recognized by POSIX. The argument will be interpreted
387 as follows:
388 path_name
390 Where path_name is not an absolute path name, then the
391 qsub command will expand the path name relative to the
392 current working directory of the command. The command
393 will supply the name of the host upon which it is exe‐
394 cuting for the hostname component.
395 hostname:path_name
397 Where path_name is not an absolute path name, then the
398 qsub command will not expand the path name relative to
399 the current working directory of the command. On deliv‐
400 ery of the standard output, the path name will be ex‐
401 panded relative to the user's home directory on the
402 hostname system.
403 path_name
405 Where path_name specifies an absolute path name, then
406 the qsub will supply the name of the host on which it is
407 executing for the hostname
408 hostname:path_name
410 Where path_name specifies an absolute path name, the
411 path will be used as specified. hostname.
412 hostname:
414 Where hostname specifies the name of the host that the
415 file should be returned to. The path will be the default
416 file name.
417 If the -o option is not specified or the path_name is not spec‐
419 ified or is specified and is a directory, the default file name
420 for the standard output stream will be used. The default name
421 has the following form:
422 job_name.osequence_number
423 where job_name is the name of the job, see -N option, and se‐
424 quence_number is the job number assigned when the job is sub‐
425 mitted.
426 -p priority
428 Defines the priority of the job. The priority argument must be
429 a integer between -1024 and +1023 inclusive. The default is no
430 priority which is equivalent to a priority of zero.
431 -P proxy_user[:group]
433 Proxy user for whom the job should be submitted. This option
434 is only available for the super user.
435 -q destination
437 Defines the destination of the job. The destination names a
438 queue, a server, or a queue at a server.
439 The qsub command will submit the script to the server defined
441 by the destination argument. If the destination is a routing
442 queue, the job may be routed by the server to a new destina‐
443 tion.
444 If the -q option is not specified, the qsub command will submit
446 the script to the default server. See PBS_DEFAULT under the
447 Environment Variables section on this man page and the PBS ERS
448 section 2.7.4, "Default Server".
449 If the -q option is specified, it is in one of the following
451 three forms:
452 queue
453 @server
454 queue@server
455 If the destination argument names a queue and does not name a
457 server, the job will be submitted to the named queue at the de‐
458 fault server.
459 If the destination argument names a server and does not name a
461 queue, the job will be submitted to the default queue at the
462 named server.
463 If the destination argument names both a queue and a server,
465 the job will be submitted to the named queue at the named
466 server.
467 -r y|n Declares whether the job is rerunable. See the qrerun command.
469 The option argument is a single character, either y or n.
470 If the argument is "y", the job is rerunable. If the argument
472 is "n", the job is not rerunable. The default value is 'y',
473 rerunable.
474 -S path_list
476 Declares the shell that interprets the job script.
477 The option argument path_list is in the form:
479 path[@host][,path[@host],...]
480 Only one path may be specified for any host named. Only one
481 path may be specified without the corresponding host name. The
482 path selected will be the one with the host name that matched
483 the name of the execution host. If no matching host is found,
484 then the path specified without a host will be selected, if
485 present.
486 If the -S option is not specified, the option argument is the
488 null string, or no entry from the path_list is selected, the
489 execution will use the user's login shell on the execution
490 host.
491 -t array_request
493 Specifies the task ids of a job array. Single task arrays are
494 allowed.
495 The array_request argument is an integer id or a range of inte‐
497 gers. Multiple ids or id ranges can be combined in a comma de‐
498 limted list. Examples : -t 1-100 or -t 1,10,50-100
499 An optional slot limit can be specified to limit the amount of
501 jobs that can run concurrently in the job array. The default
502 value is unlimited. The slot limit must be the last thing spec‐
503 ified in the array_request and is delimited from the array by a
504 percent sign (%).
505 qsub script.sh -t 0-299%5
507 This sets the slot limit to 5. Only 5 jobs from this array can
509 run at the same time.
510 Note: You can use qalter to modify slot limits on an array. The
512 server parameter max_slot_limit can be used to set a global
513 slot limit policy.
514 -T script_name
516 Allows for per job prologue and epilogue scripts. The full
517 script name will be prologue.[name] or epilogue.[name]. For the
518 job submission, only request the name of the prologue or epi‐
519 logue script.
520 Example: qsub -T prescript
522 Specifies to use the script prologue.prescript
523 -u user_list
525 Defines the user name under which the job is to run on the exe‐
526 cution system.
527 The user_list argument is of the form:
529 user[@host][,user[@host],...]
530 Only one user name may be given per specified host. Only one
531 of the user specifications may be supplied without the corre‐
532 sponding host specification. That user name will used for exe‐
533 cution on any host not named in the argument list. If unset,
534 the user list defaults to the user who is running qsub.
535 -v variable_list
537 Expands the list of environment variables that are exported to
538 the job.
539 In addition to the variables described in the "Description"
541 section above, variable_list names environment variables from
542 the qsub command environment which are made available to the
543 job when it executes. The variable_list is a comma separated
544 list of strings of the form variable or variable=value. These
545 variables and their values are passed to the job.
546 -V Declares that all environment variables in the qsub command's
548 environment are to be exported to the batch job.
549 -w path Defines the working directory path to be used for the job. If
551 the -w option is not specified, the default working directory
552 is the current directory. This option sets the environment
553 variable PBS_O_WORKDIR.
554 -W additional_attributes
556 The -W option allows for the specification of additional job
557 attributes. The general syntax of the -W is in the form:
558 -W attr_name=attr_value[,attr_name=attr_value...]
559 Note if white space occurs anywhere within the option argument
560 string or the equal sign, "=", occurs within an attribute_value
561 string, then the string must be enclosed with either single or
562 double quote marks.
563 PBS currently supports the following attributes within the -W
565 option.
566 depend=dependency_list
568 Defines the dependency between this and other jobs. The depen‐
569 dency_list is in the form:
570 type[:argument[:argument...][,type:argument...].
571 The argument is either a numeric count or a PBS job id accord‐
572 ing to type . If argument is a count, it must be greater than
573 0. If it is a job id and not fully specified in the form
574 seq_number.server.name, it will be expanded according to the
575 default server rules which apply to job IDs on most commands.
576 If argument is null (the preceding colon need not be speci‐
577 fied), the dependency of the corresponding type is cleared (un‐
578 set).
579 synccount:count
581 This job is the first in a set of jobs to be executed
582 at the same time. Count is the number of additional
583 jobs in the set.
584 syncwith:jobid
586 This job is an additional member of a set of jobs to be
587 executed at the same time. In the above and following
588 dependency types, jobid is the job identifier of the
589 first job in the set.
590 after:jobid[:jobid...]
592 This job may be scheduled for execution at any point
593 after jobs jobid have started execution.
594 afterok:jobid[:jobid...]
596 This job may be scheduled for execution only after jobs
597 jobid have terminated with no errors. See the csh
598 warning under "Extended Description".
599 afternotok:jobid[:jobid...]
601 This job may be scheduled for execution only after jobs
602 jobid have terminated with errors. See the csh warning
603 under "Extended Description".
604 afterany:jobid[:jobid...]
606 This job may be scheduled for execution after jobs jo‐
607 bid have terminated, with or without errors.
608 on:count
610 This job may be scheduled for execution after count de‐
611 pendencies on other jobs have been satisfied. This
612 form is used in conjunction with one of the before
613 forms, see below.
614 before:jobid[:jobid...]
616 When this job has begun execution, then jobs jobid...
617 may begin.
618 beforeok:jobid[:jobid...]
620 If this job terminates execution without errors, then
621 jobs jobid... may begin. See the csh warning under
622 "Extended Description".
623 beforenotok:jobid[:jobid...]
625 If this job terminates execution with errors, then jobs
626 jobid... may begin. See the csh warning under "Ex‐
627 tended Description".
628 beforeany:jobid[:jobid...]
630 When this job terminates execution, jobs jobid... may
631 begin.
632 If any of the before forms are used, the jobs refer‐
634 enced by jobid must have been submitted with a depen‐
635 dency type of on.
636 Array Dependencies
638 It is now possible to have a job depend on an array.
639 These dependencies are in the form depend=arraydep:ar‐
640 rayid[num]. If [num] is not present, then the dependen‐
641 cies applies to the entire array. If [num] is present,
642 then num means the number of jobs that must meet the
643 condition for the dependency to be satisfied.
644 afterstartarray:arrayid[count]
646 This job may be scheduled for execution only after jobs
647 in arrayid have started execution.
648 afterokarray:arrayid[count]
650 This job may be scheduled for execution only after jobs
651 in arrayid have terminated with no errors.
652 afternotok:arrayid[count]
654 This job may be scheduled for execution only after jobs
655 in arrayid have terminated with errors.
656 afteranyarray:arrayid[count]
658 This job may be scheduled for execution after jobs in
659 array id have terminated, with or without errors.
660 beforestartarray:arrayid[count]
662 This job may be scheduled for execution only before
663 jobs in arrayid have started execution.
664 beforeokarray:arrayid[count]
666 This job may be scheduled for execution only before
667 jobs in arrayid have terminated with no errors.
668 beforenotok:arrayid[count]
670 This job may be scheduled for execution only before
671 jobs in arrayid have terminated with errors.
672 beforeanyarray:arrayid[count]
674 This job may be scheduled for execution before jobs in
675 array id have terminated, with or without errors.
676 If any of the before forms are used, the jobs refer‐
678 enced by jobid must have the same owner as the job be‐
679 ing submitted. Otherwise, the dependency is ignored.
680 Error processing of the existence, state, or condition of
682 the job on which the newly submitted job is a deferred ser‐
683 vice, i.e. the check is performed after the job is queued.
684 If an error is detected, the new job will be deleted by the
685 server. Mail will be sent to the job submitter stating the
686 error.
687 Dependency examples:
689 qsub -W depend=afterok:123.big.iron.com /tmp/script
690 qsub -W depend=before:234.hunk1.com:235.hunk1.com
691 /tmp/script
692 qsub -W depend=afterokarray:21.tom.com[] /tmp/script
693 qsub -W depend=beforenotokarray:22.tom.com[][5] /tmp/script
694 group_list=g_list
696 Defines the group name under which the job is to run on the ex‐
697 ecution system. The g_list argument is of the form:
698 group[@host][,group[@host],...]
699 Only one group name may be given per specified host. Only one
700 of the group specifications may be supplied without the corre‐
701 sponding host specification. That group name will used for ex‐
702 ecution on any host not named in the argument list. If not
703 set, the group_list defaults to the primary group of the user
704 under which the job will be run.
705 interactive=true
707 If the interactive attribute is specified, the job is an inter‐
708 active job. The -I option is a alternative method of specify‐
709 ing this attribute.
710 stagein=file_list
712 stageout=file_list
713 Specifies which files are staged (copied) in before job start
714 or staged out after the job completes execution. On completion
715 of the job, all staged-in and staged-out files are removed from
716 the execution system. The file_list is in the form
717 local_file@hostname:remote_file[,...]
718 regardless of the direction of the copy. The name local_file
719 is the name of the file on the system where the job executed.
720 It may be an absolute path or relative to the home directory of
721 the user. The name remote_file is the destination name on the
722 host specified by hostname. The name may be absolute or rela‐
723 tive to the user's home directory on the destination host. The
724 use of wildcards in the file name is not recommended. The file
725 names map to a remote copy program (rcp) call on the execution
726 system in the follow manner:
727 For stagein: rcp hostname:remote_file local_file
728 For stageout: rcp local_file hostname:remote_file
729 Data staging examples:
730 -W stagein=/tmp/input.txt@headnode:/home/user/input.txt
731 -W stageout=/tmp/output.txt@headnode:/home/user/output.txt
732 If TORQUE has been compiled with wordexp support, then vari‐
733 ables can be used in the specified paths. Currently only
734 $PBS_JOBID, $HOME, and $TMPDIR are supported for stagein.
735 umask=XXX
737 Sets umask used to create stdout and stderr spool files in
738 pbs_mom spool directory. Values starting with 0 are treated as
739 octal values, otherwise the value is treated as a decimal umask
740 value.
741 -x When running an interactive job, the -x flag makes it so that
743 the script won't be parsed for PBS directives, but instead will
744 be a command that is launched once the interactive job has
745 started. The job will terminate at the completion of this com‐
746 mand.
747 -X Enables X11 forwarding. The DISPLAY environment variable must
749 be set.
750 -z Directs that the qsub command is not to write the job identi‐
752 fier assigned to the job to the command's standard output.
753
755 The qsub command accepts a script operand that is the path to the
756 script of the job. If the path is relative, it will be expanded rela‐
757 tive to the working directory of the qsub command.
758 If the script operand is not provided or the operand is the single
760 character "-", the qsub command reads the script from standard input.
761 When the script is being read from Standard Input, qsub will copy the
762 file to a temporary file. This temporary file is passed to the library
763 interface routine pbs_submit. The temporary file is removed by qsub
764 after pbs_submit returns or upon the receipt of a signal which would
765 cause qsub to terminate.
766
768 The qsub command reads the script for the job from standard input if
769 the script operand is missing or is the single character "-".
770
772 The script file is read by the qsub command. Qsub acts upon any direc‐
773 tives found in the script.
774 When the job is created, a copy of the script file is made and that
776 copy cannot be modified.
777
779 Unless the -z option is set, the job identifier assigned to the job
780 will be written to standard output if the job is successfully created.
781
783 The qsub command will write a diagnostic message to standard error for
784 each error occurrence.
785
787 The values of some or all of the variables in the qsub command's envi‐
788 ronment are exported with the job, see the -v and -V options.
789 The environment variable PBS_DEFAULT defines the name of the default
791 server. Typically, it corresponds to the system name of the host on
792 which the server is running. If PBS_DEFAULT is not set, the default is
793 defined by an administrator established file.
794 The environment variable PBS_DPREFIX determines the prefix string which
796 identifies directives in the script.
797 The environment variable PBS_CLIENTRETRY defines the maximum number of
799 seconds qsub will block. See the -b option above. Despite the name,
800 currently qsub is the only client that supports this option.
801
803 The torque.cfg file, located in PBS_SERVER_HOME (/var/spool/torque by
804 default) controls the behavior of the qsub command. This file contains
805 a list of parameters and values separated by whitespace
806 QSUBSLEEP takes an integer operand which specifies time to sleep when
808 running qsub command. Used to prevent users from overwhelming the
809 scheduler.
810 SUBMITFILTER specifies the path to the submit filter used to pre-
812 process job submission. The default path is $(libexecdir)/qsub_filter,
813 which falls back to /usr/local/sbin/torque_submitfilter for backwards
814 compatibility. This torque.cfg parameter overrides this default.
815 SERVERHOST specifies the value for the PBS_SERVER environment variable
817 QSUBHOST specifies the hostname for the jobs QSUB_O_HOST variable
819 QSUBSENDUID specifies a uid to use for the jobs PBS_O_UID variable
821 XAUTHPATH specifies the path to xauth
823 CLIENTRETRY specifies the integer seconds between retry attempts to
825 communicate with pbs_server
826 VALIDATEGROUP set this parameter to force qsub to verify the submit‐
828 ter's group id
829 DEFAULTCKPT specifies the default value for the jobs checkpoint attri‐
831 bute. The user overrides this with the -c qsub option.
832 VALIDATEPATH set this parameter to force qsub to validate local exis‐
834 tence of a "-d" working directory
835 RERUNNABLEBYDEFAULT this parameter specifies if a job is rerunnable by
837 default. The default is true, setting this to false causes the re‐
838 runnable attribute value to be false unless the users specifies other‐
839 wise with the -r option
840 FAULT_TOLERANT_BY_DEFAULT this parameter specifies if a job is fault
842 tolerant by default. The default value for the fault_tolerant job at‐
843 tribute is false, setting this parameter to true causes the default
844 value of the attribute to be true. The user can specify their prefer‐
845 ence with the -f qsub option.
846 For example:
848 QSUBSLEEP 2
849 RERUNNABLEBYDEFAULT false
850
853 Script Processing:
854 A job script may consist of PBS directives, comments and executable
856 statements. A PBS directive provides a way of specifying job at‐
857 tributes in addition to the command line options. For example:
858 :
859 #PBS -N Job_name
860 #PBS -l walltime=10:30,mem=320kb
861 #PBS -m be
862 #
863 step1 arg1 arg2
864 step2 arg3 arg4
865 The qsub command scans the lines of the script file for directives. An
868 initial line in the script that begins with the characters "#!" or the
869 character ":" will be ignored and scanning will start with the next
870 line. Scanning will continue until the first executable line, that is
871 a line that is not blank, not a directive line, nor a line whose first
872 non white space character is "#". If directives occur on subsequent
873 lines, they will be ignored.
874 A line in the script file will be processed as a directive to qsub if
876 and only if the string of characters starting with the first non white
877 space character on the line and of the same length as the directive
878 prefix matches the directive prefix.
879 The remainder of the directive line consists of the options to qsub in
881 the same syntax as they appear on the command line. The option charac‐
882 ter is to be preceded with the "-" character.
883 If an option is present in both a directive and on the command line,
885 that option and its argument, if any, will be ignored in the directive.
886 The command line takes precedence.
887 If an option is present in a directive and not on the command line,
889 that option and its argument, if any, will be processed as if it had
890 occurred on the command line.
891 The directive prefix string will be determined in order of preference
893 from:
894 The value of the -C option argument if the option is specified on
896 the command line.
897 The value of the environment variable PBS_DPREFIX if it is defined.
899 The four character string #PBS.
901 If the -C option is found in a directive in the script file, it will be
903 ignored.
904 User Authorization:
906 When the user submits a job from a system other than the one on which
908 the PBS Server is running, the name under which the job is to be exe‐
909 cuted is selected according to the rules listed under the -u option.
910 The user submitting the job must be authorized to run the job under the
911 execution user name. This authorization is provided if
912 (1) The host on which qsub is run is trusted by the execution
914 host (see /etc/hosts.equiv),
915 (2) The execution user has an .rhosts file naming the submit‐
917 ting user on the submitting host.
918 C-Shell .logout File:
920 The following warning applies for users of the c-shell, csh. If the
922 job is executed under the csh and a .logout file exists in the home di‐
923 rectory in which the job executes, the exit status of the job is that
924 of the .logout script, not the job script. This may impact any inter-
925 job dependencies. To preserve the job exit status, either remove the
926 .logout file or place the following line as the first line in the .lo‐
927 gout file
928 set EXITVAL = $status
929 and the following line as the last executable line in .logout
930 exit $EXITVAL
931 Interactive Jobs:
933 If the -I option is specified on the command line or in a script direc‐
935 tive, or if the "interactive" job attribute declared true via the -W
936 option, -W interactive=true, either on the command line or in a script
937 directive, the job is an interactive job. The script will be processed
938 for directives, but will not be included with the job. When the job
939 begins execution, all input to the job is from the terminal session in
940 which qsub is running.
941 When an interactive job is submitted, the qsub command will not termi‐
943 nate when the job is submitted. Qsub will remain running until the job
944 terminates, is aborted, or the user interrupts qsub with an SIGINT (the
945 control-C key). If qsub is interrupted prior to job start, it will
946 query if the user wishes to exit. If the user response "yes", qsub ex‐
947 its and the job is aborted.
948 Once the interactive job has started execution, input to and output
950 from the job pass through qsub. Keyboard generated interrupts are
951 passed to the job. Lines entered that begin with the tilde ('~') char‐
952 acter and contain special sequences are escaped by qsub. The recog‐
953 nized escape sequences are:
954 ~. Qsub terminates execution. The batch job is also termi‐
956 nated.
957 ~susp Suspend the qsub program if running under the C shell.
959 "susp" is the suspend character, usually CNTL-Z.
960 ~asusp Suspend the input half of qsub (terminal to job), but al‐
962 low output to continue to be displayed. Only works under
963 the C shell. "asusp" is the auxiliary suspend character,
964 usually CNTL-Y.
965
967 Upon successful processing, the qsub exit status will be a value of
968 zero.
969 If the qsub command fails, the command exits with a value greater than
971 zero.
972
974 qalter(1B), qdel(1B), qhold(1B), qmove(1B), qmsg(1B), qrerun(1B),
975 qrls(1B), qselect(1B), qsig(1B), qstat(1B), pbs_connect(3B),
976 pbs_job_attributes(7B), pbs_queue_attributes(7B), pbs_re‐
977 sources_irix5(7B), pbs_resources_sp2(7B), pbs_resources_sunos4(7B),
978 pbs_resources_unicos8(7B), pbs_server_attributes(7B), and
979 pbs_server(8B)
980Local qsub(1B)