1SUBMIT(1) Grid Engine User Commands SUBMIT(1)
2
6 qsub - submit a batch job to Grid Engine.
7 qsh - submit an interactive X-windows session to Grid Engine.
9 qlogin - submit an interactive login session to Grid Engine.
11 qrsh - submit an interactive rsh session to Grid Engine.
13 qalter - modify a pending or running batch job of Grid Engine.
15 qresub - submit a copy of an existing Grid Engine job.
17
19 qsub [ options ] [ command | -- [ command_args ]]
20 qsh [ options ] [ -- xterm_args ]
22 qlogin [ options ]
24 qrsh [ options ] [ command [ command_args ]]
26 qalter [ options ] wc_job_range_list [ -- [ command_args ]]
28 qalter [ options ] -u user_list | -uall [ -- [ command_args ]]
30 qresub [ options ] job_id_list
32
34 Qsub submits batch jobs to the Grid Engine queuing system. Grid Engine
35 supports single- and multiple-node jobs. Command can be a path to a
36 binary or a script (see -b below) which contains the commands to be run
37 by the job using a shell (for example, sh(1) or csh(1)). Arguments to
38 the command are given as command_args to qsub . If command is handled
39 as a script then it is possible to embed flags in the script. If the
40 first two characters of a script line either match '#$' or are equal to
41 the prefix string defined with the -C option described below, the line
42 is parsed for embedded command flags.
43 Qsh submits an interactive X-windows session to Grid Engine. An
45 xterm(1) is brought up from the executing machine with the display
46 directed either to the X-server indicated by the DISPLAY environment
47 variable or as specified with the -display qsh option. Interactive jobs
48 are not spooled if no resource is available to execute them. They are
49 either dispatched to a suitable machine for execution immediately or
50 the user submitting the job is notified by qsh that appropriate
51 resources to execute the job are not available. xterm_args are passed
52 to the xterm(1) executable. Note, however, that the -e and -ls xterm
53 options do not work with qsh .
54 Qlogin is similar to qsh in that it submits an interactive job to the
56 queuing system. It does not open an xterm(1) window on the X display,
57 but uses the current terminal for user I/O. Usually, qlogin establishes
58 a telnet(1) connection with the remote host, using standard client- and
59 server-side commands. These commands can be configured with the qlo‐
60 gin_daemon (server-side, Grid Engine telnetd if not set, otherwise
61 something like /usr/sbin/in.telnetd) and qlogin_command (client-side,
62 Grid Engine telnet if not set, otherwise something like /usr/bin/tel‐
63 net) parameters in the global and local configuration settings of
64 ge_conf(5). The client side command is automatically parameterized
65 with the remote host name and port number to which to connect, result‐
66 ing in an invocation like
67 /usr/bin/telnet my_exec_host 2442
69 for example. Qlogin is invoked exactly like qsh and its jobs can only
71 run on INTERACTIVE queues. Qlogin jobs can only be used if the
72 ge_execd(8) is running under the root account.
73 Qrsh is similar to qlogin in that it submits an interactive job to the
75 queuing system. It uses the current terminal for user I/O. Usually,
76 qrsh establishes a rsh(1) connection with the remote host. If no com‐
77 mand is given to qrsh, an rlogin(1) session is established. The
78 server-side commands used can be configured with the rsh_daemon and
79 rlogin_daemon parameters in the global and local configuration settings
80 of ge_conf(5). An Grid Engine rshd or rlogind is used if the parame‐
81 ters are not set. If the parameters are set, they should be set to
82 something like /usr/sbin/in.rshd or /usr/sbin/in.rlogind. On the
83 client-side, the rsh_command and rlogin_command parameters can be set
84 in the global and local configuration settings of ge_conf(5). If they
85 are not set, special Grid Engine rsh(1) and rlogin(1) binaries deliv‐
86 ered with Grid Engine are used. Use the cluster configuration parame‐
87 ters to integrate mechanisms like ssh or the rsh(1) and rlogin(1)
88 facilities supplied with the operating system.
89 Qrsh jobs can only run in INTERACTIVE queues unless the option -now no
91 is used (see below). They can also only be run, if the ge_execd(8) is
92 running under the root account.
93 Qrsh provides an additional useful feature for integrating with inter‐
95 active tools providing a specific command shell. If the environment
96 variable QRSH_WRAPPER is set when qrsh is invoked, the command inter‐
97 preter pointed to by QRSH_WRAPPER will be executed to run qrsh commands
98 instead of the users login shell or any shell specified in the qrsh
99 command-line. The options -cwd, -v, -V, and -display only apply to
100 batch jobs.
101 Qalter can be used to change the attributes of pending jobs. For array
103 jobs with a mix of running and pending tasks (see the -t option below),
104 modification with qalter only affects the pending tasks. Qalter can
105 change most of the characteristics of a job (see the corresponding
106 statements in the OPTIONS section below), including those which were
107 defined as embedded flags in the script file (see above). Some submit
108 options, such as the job script, cannot be changed with I. qalter.
109 Qresub allows the user to create jobs as copies of existing pending or
111 running jobs. The copied jobs will have exactly the same attributes as
112 the ones from which they were copied, except with a new job ID and with
113 a cleared hold state. The only modification to the copied jobs sup‐
114 ported by qresub is assignment of a new hold state with the -h option.
115 This option can be used to first copy a job and then change its
116 attributes via qalter.
117 Only a manager can use qresub on jobs submitted by another user. Regu‐
119 lar users can only use qresub on their own jobs.
120 For qsub, qsh, qrsh, and qlogin the administrator and the user may
122 define default request files (see ge_request(5)) which can contain any
123 of the options described below. If an option in a default request file
124 is understood by qsub and qlogin but not by qsh the option is silently
125 ignored if qsh is invoked. Thus you can maintain shared default request
126 files for both qsub and qsh.
127 A cluster wide default request file may be placed under
129 $GE_ROOT/$GE_CELL/common/ge_request. User private default request
130 files are processed under the locations $HOME/.ge_request and
131 $cwd/.ge_request. The working directory local default request file has
132 the highest precedence, then the home directory located file and then
133 the cluster global file. The option arguments, the embedded script
134 flags and the options in the default request files are processed in the
135 following order:
136 left to right in the script line,
138 left to right in the default request files,
139 from top to bottom of the script file (qsub only),
140 from top to bottom of default request files,
141 from left to right of the command line.
142 In other words, the command line can be used to override the embedded
144 flags and the default request settings. The embedded flags, however,
145 will override the default settings.
146 Note, that the -clear option can be used to discard any previous set‐
148 tings at any time in a default request file, in the embedded script
149 flags, or in a command-line option. It is, however, not available with
150 qalter.
151 The options described below can be requested either hard or soft. By
153 default, all requests are considered hard until the -soft option (see
154 below) is encountered. The hard/soft status remains in effect until its
155 counterpart is encountered again. If all the hard requests for a job
156 cannot be met, the job will not be scheduled. Jobs which cannot be run
157 at the present time remain spooled.
158
160 -@ optionfile
161 Forces qsub, qrsh, qsh, or qlogin to use the options contained
162 in optionfile. The indicated file may contain all valid options.
163 Comment lines must start with a "#" sign.
164 -a date_time
166 Available for qsub and qalter only.
167 Defines or redefines the time and date at which a job is eligi‐
169 ble for execution. Date_time conforms to [[CC]]YY]MMDDhhmm[.SS],
170 for the details, please see Date_time in: sge_types(1).
171 If this option is used with qsub or if a corresponding value is
173 specified in qmon then a parameter named a and the value in the
174 format CCYYMMDDhhmm.SS will be passed to the defined JSV
175 instances (see -jsv option below or find more information con‐
176 cerning JSV in jsv(1))
177 -ac variable[=value],...
179 Available for qsub, qsh, qrsh, qlogin and qalter only.
180 Adds the given name/value pair(s) to the job's context. Value
182 may be omitted. Grid Engine appends the given argument to the
183 list of context variables for the job. Multiple -ac, -dc, and
184 -sc options may be given. The order is important here.
185 The outcome of the evaluation of all -ac, -dc, and -sc options
187 or corresponding values in qmon is passed to defined JSV
188 instances as parameter with the name ac. (see -jsv option below
189 or find more information concerning JSV in jsv(1)) QALTER allows
190 changing this option even while the job executes.
191 -ar ar_id
193 Available for qsub, qalter, qrsh, qsh, or qlogin only.
194 Assigns the submitted job to be a part of an existing Advance
196 Reservation. The complete list of existing Advance Reservations
197 can be obtained using the qrstat(1) command.
198 Note that the -ar option adds implicitly the -w e option if not
200 otherwise requested.
201 Qalter allows changing this option even while the job executes.
203 The modified parameter will only be in effect after a restart or
204 migration of the job however.
205 If this option or a corresponding value in qmon is specified
207 then this value will be passed to defined JSV instances as
208 parameter with the name ar. (see -jsv option below or find more
209 information concerning JSV in jsv(1))
210 -A account_string
212 Available for qsub, qsh, qrsh, qlogin and qalter only.
213 Identifies the account to which the resource consumption of the
215 job should be charged. The account_string should conform to the
216 name definition in M sge_types 1 . In the absence of this
217 parameter Grid Engine will place the default account string "ge"
218 in the accounting record of the job.
219 Qalter allows changing this option even while the job executes.
221 If this option or a corresponding value in qmon is specified
223 then this value will be passed to defined JSV instances as
224 parameter with the name A. (see -jsv option below or find more
225 information concerning JSV in jsv(1))
226 -binding [ binding_instance ] binding_strategy
228 A job can request a specific processor core binding (processor
229 affinity) with this parameter. This request is neither a hard
230 nor a soft request, it is a hint for the execution host to do
231 this if possible. Please note that the requested binding strat‐
232 egy is not used for resource selection within Grid Engine.
233 As a result an execution host might be selected where Grid
234 Engine does not even know the hardware topology and therefore is
235 not able to apply the requested binding.
236 To enforce Grid Engine to select hardware on which the binding
238 can be applied please use the -l switch in combination with the
239 complex attribute m_topology.
240 binding_instance is an optional parameter. It might either be
242 env, pe or set depending on which instance should accomplish the
243 job to core binding. If the value for binding_instance is not
244 specified then set will be used.
245 env means that the environment variable SGE_BINDING will be
247 exported to the job environment of the job. This variable con‐
248 tains the selected operating system internal processor numbers.
249 They might be more than selected cores in presence of SMT or CMT
250 because each core could be represented by multiple processor
251 identifiers. The processor numbers are space separated.
252 pe means that the information about the selected cores appears
254 in the fourth column of the pe_hostfile. Here the logical core
255 and socket numbers are printed (they start at 0 and have no
256 holes) in colon separated pairs (i.e. 0,0:1,0 which means core 0
257 on socket 0 and core 0 on socket 1). For more information about
258 the $pe_hostfile check ge_pe(5)
259 set (default if nothing else is specified). The binding strategy
261 is applied by Grid Engine. How this is achieved depends on the
262 underlying hardware architecture of the execution host where the
263 submitted job will be started.
264 On Solaris 10 hosts a processor set will be created where the
266 job can exclusively run in. Because of operating system limita‐
267 tions at least one core must remain unbound. This resource could
268 of course used by an unbound job.
269 On Linux hosts a processor affinity mask will be set to restrict
271 the job to run exclusively on the selected cores. The operating
272 system allows other unbound processes to use these cores.
273 Please note that on Linux the binding requires a Linux kernel
274 version of 2.6.16 or greater. It might be even possible to use a
275 kernel with lower version number but in that case additional
276 kernel patches have to be applied. The loadcheck tool in the
277 utilbin directory can be used to check if the hosts capabili‐
278 ties. You can also use the -sep in combination with -cb of
279 qconf(5) command to identify if Grid Engine is able to recognize
280 the hardware topology.
281 Possible values for binding_strategy are as follows:
283 linear:<amount>[:<socket>,<core>]
285 striding:<amount>:<n>[:<socket>,<core>]
286 explicit:[<socket>,<core>;...]<socket>,<core>
287 For the binding strategy linear and striding there is an
289 optional socket and core pair attached. These denotes the manda‐
290 tory starting point for the first core to bind on.
291 linear means that Grid Engine tries to bind the job on amount
293 successive cores. If socket and core is omitted then Grid Engine
294 first allocates successive cores on the first empty socket
295 found. Empty means that there are no jobs bound to the socket
296 by Grid Engine. If this is not possible or is not sufficient
297 Grid Engine tries to find (further) cores on the socket with the
298 most unbound cores and so on. If the amount of allocated cores
299 is lower than requested cores, no binding is done for the job.
300 If socket and core is specified then Grid Engine tries to find
301 amount of empty cores beginning with this starting point. If
302 this is not possible then binding is not done.
303 striding means that Grid Engine tries to find cores with a cer‐
305 tain offset. It will select amount of empty cores with a offset
306 of n -1 cores in between. Start point for the search algorithm
307 is socket 0 core 0. As soon as amount cores are found they will
308 be used to do the job binding. If there are not enough empty
309 cores or if correct offset cannot be achieved then there will be
310 no binding done.
311 explicit binds the specified sockets and cores that are men‐
313 tioned in the provided socket/core list. Each socket/core pair
314 has to be specified only once. If a socket/core pair is already
315 in use by a different job the whole binding request will be
316 ignored.
317 Qalter allows changing this option even while the job executes.
319 The modified parameter will only be in effect after a restart or
320 migration of the job, however.
321 If this option or a corresponding value in qmon is specified
323 then these values will be passed to defined JSV instances as
324 parameters with the names binding_strategy, binding_type, bind‐
325 ing_amount, binding_step, binding_socket, binding_core, bind‐
326 ing_exp_n, binding_exp_socket<id>, binding_exp_core<id>.
327 Please note that the length of the socket/core value list of the
329 explicit binding is reported as binding_exp_n. <id> will be
330 replaced by the position of the socket/core pair within the
331 explicit list (0 <= id < binding_exp_n). The first socket/core
332 pair of the explicit binding will be reported with the parameter
333 names binding_exp_socket0 and binding_exp_core0.
334 Values that do not apply for the specified binding will not be
336 reported to JSV. E.g. binding_step will only be reported for
337 the striding binding and all binding_exp_* values will passed to
338 JSV if explicit binding was specified. (see -jsv option below
339 or find more information concerning JSV in jsv(1))
340 -b y[es]|n[o]
342 Available for qsub, qrsh only. Qalter does not allow changing
343 this option. This option cannot be embedded in the script file
344 itself.
345 Gives the user the possibility to indicate explicitly whether
347 command should be treated as binary or script. If the value of
348 -b is 'y', then command may be a binary or script. The command
349 might not be accessible from the submission host. Nothing
350 except the path of the command will be transferred from the sub‐
351 mission host to the execution host. Path aliasing will be
352 applied to the path of command before command will be executed.
353 If the value of -b is 'n' then command needs to be a script and
355 it will be handled as script. The script file has to be accessi‐
356 ble by the submission host. It will be transferred to the execu‐
357 tion host. qsub/qrsh will search directive prefixes within
358 script.
359 qsub will implicitly use -b n whereas qrsh will apply the -b y
361 option if nothing else is specified.
362 The value specified with this option or the corresponding value
364 specified in qmon will only be passed to defined JSV instances
365 if the value is yes. The name of the parameter will be b. The
366 value will be y also when then long form yes was specified dur‐
367 ing submission. (see -jsv option below or find more information
368 concerning JSV in jsv(1))
369 Please note that submission of command as script (-b n) can have
371 a significant performance impact, especially for short running
372 jobs and big job scripts. Script submission adds a number of
373 operations to the submission process: The job script needs to be
374 - parsed at client side (for special comments)
375 - transferred from submit client to qmaster
376 - spooled in qmaster
377 - transferred to execd at job execution
378 - spooled in execd
379 - removed from spooling both in execd and qmaster once the job is done
380 If job scripts are available on the execution nodes, e.g. via
381 NFS, binary submission can be the better choice.
382 -c occasion_specifier
384 Available for qsub and qalter only.
385 Defines or redefines whether the job should be checkpointed, and
387 if so, under what circumstances. The specification of the check‐
388 pointing occasions with this option overwrites the definitions
389 of the when parameter in the checkpointing environment (see
390 checkpoint(5)) referenced by the qsub -ckpt switch. Possible
391 values for occasion_specifier are
392 n no checkpoint is performed.
394 s checkpoint when batch server is shut down.
395 m checkpoint at minimum CPU interval.
396 x checkpoint when job gets suspended.
397 <interval> checkpoint in the specified time interval.
398 The minimum CPU interval is defined in the queue configuration
400 (see queue_conf(5) for details). <interval> has to be specified
401 in the format hh:mm:ss. The maximum of <interval> and the
402 queue's minimum CPU interval is used if <interval> is specified.
403 This is done to ensure that a machine is not overloaded by
404 checkpoints being generated too frequently.
405 The value specified with this option or the corresponding value
407 specified in qmon will be passed to defined JSV instances. The
408 <interval> will be available as parameter with the name c_inter‐
409 val. The character sequence specified will be available as
410 parameter with the name c_occasion. Please note that if you
411 change c_occasion via JSV then the last setting of c_interval
412 will be overwritten and vice versa. (see -jsv option below or
413 find more information concerning JSV in jsv(1))
414 -ckpt ckpt_name
417 Available for qsub and qalter only.
418 Selects the checkpointing environment (see checkpoint(5)) to be
420 used for checkpointing the job. Also declares the job to be a
421 checkpointing job.
422 If this option or a corresponding value in qmon is specified
424 then this value will be passed to defined JSV instances as
425 parameter with the name ckpt. (see -jsv option below or find
426 more information concerning JSV in jsv(1))
427 -clear Available for qsub, qsh, qrsh, and qlogin only.
429 Causes all elements of the job to be reset to the initial
431 default status prior to applying any modifications (if any)
432 appearing in this specific command.
433 -cwd Available for qsub, qsh, qrsh and qalter only.
435 Execute the job from the current working directory. This switch
437 will activate Grid Engine's path aliasing facility, if the cor‐
438 responding configuration files are present (see ge_aliases(5)).
439 In the case of qalter, the previous definition of the current
441 working directory will be overwritten if qalter is executed from
442 a different directory than the preceding qsub or qalter.
443 Qalter allows changing this option even while the job executes.
445 The modified parameter will only be in effect after a restart or
446 migration of the job, however.
447 If this option or a corresponding value in qmon is specified
449 then this value will be passed to defined JSV instances as
450 parameter with the name cwd. The value of this parameter will be
451 the absolute path to the current working directory. JSV scripts
452 can remove the path from jobs during the verification process by
453 setting the value of this parameter to an empty string. As a
454 result the job behaves as if -cwd was not specified during job
455 submission. (see -jsv option below or find more information
456 concerning JSV in jsv(1))
457 -C prefix_string
460 Available for qsub and qrsh with script submission (-b n).
461 Prefix_string defines the prefix that declares a directive in
463 the job's command. The prefix is not a job attribute, but
464 affects the behavior of qsub and qrsh. If prefix is a null
465 string, the command will not be scanned for embedded directives.
466 The directive prefix consists of two ASCII characters which,
467 when appearing in the first two bytes of a script line, indicate
468 that what follows is an Grid Engine command. The default is
469 "#$".
470 The user should be aware that changing the first delimiting
471 character can produce unforeseen side effects. If the script
472 file contains anything other than a "#" character in the first
473 byte position of the line, the shell processor for the job will
474 reject the line and may exit the job prematurely.
475 If the -C option is present in the script file, it is ignored.
476 -dc variable,...
478 Available for qsub, qsh, qrsh, qlogin and qalter only.
479 Removes the given variable(s) from the job's context. Multiple
481 -ac, -dc, and -sc options may be given. The order is important.
482 Qalter allows changing this option even while the job executes.
484 The outcome of the evaluation of all -ac, -dc, and -sc options
486 or corresponding values in qmon is passed to defined JSV
487 instances as parameter with the name ac. (see -jsv option below
488 or find more information concerning JSV in jsv(1))
489 -display display_specifier
491 Available for qsh and qrsh.
492 Directs xterm(1) to use display_specifier in order to contact
494 the X server. The display_specifier has to contain the hostname
495 part of the display name (e.g. myhost:1). Local display names
496 (e.g. :0) cannot be used in grid environments. Values set with
497 the -display option overwrite settings from the submission envi‐
498 ronment and from -v command line options.
499 If this option or a corresponding value in qmon is specified
501 then this value will be passed to defined JSV instances as
502 parameter with the name display. This value will also be avail‐
503 able in the job environment which might optionally be passed to
504 JSV scripts. The variable name will be DISPLAY. (see -jsv
505 option below or find more information concerning JSV in jsv(1))
506 -dl date_time
508 Available for qsub, qsh, qrsh, qlogin and qalter only.
509 Specifies the deadline initiation time in [[CC]YY]MMDDhhmm[.SS]
511 format (see -a option above). The deadline initiation time is
512 the time at which a deadline job has to reach top priority to be
513 able to complete within a given deadline. Before the deadline
514 initiation time the priority of a deadline job will be raised
515 steadily until it reaches the maximum as configured by the Grid
516 Engine administrator.
517 This option is applicable only for users allowed to submit dead‐
519 line jobs.
520 If this option or a corresponding value in qmon is specified
522 then this value will be passed to defined JSV instances as
523 parameter with the name dl. The format for the date_time value
524 is CCYYMMDDhhmm.SS (see -jsv option below or find more informa‐
525 tion concerning JSV in jsv(1))
526 -e [[hostname]:]path,...
529 Available for qsub, qsh, qrsh, qlogin and qalter only.
530 Defines or redefines the path used for the standard error stream
532 of the job. For qsh, qrsh and qlogin only the standard error
533 stream of prolog and epilog is redirected. If the path consti‐
534 tutes an absolute path name, the error-path attribute of the job
535 is set to path, including the hostname. If the path name is rel‐
536 ative, Grid Engine expands path either with the current working
537 directory path (if the -cwd switch (see above) is also speci‐
538 fied) or with the home directory path. If hostname is present,
539 the standard error stream will be placed in the corresponding
540 location only if the job runs on the specified host. If the path
541 contains a ":" without a hostname, a leading ":" has to be spec‐
542 ified.
543 By default the file name for interactive jobs is /dev/null. For
545 batch jobs the default file name has the form job_name.ejob_id
546 and job_name.ejob_id.task_id for array job tasks (see -t option
547 below).
548 If path is a directory, the standard error stream of the job
550 will be put in this directory under the default file name. If
551 the pathname contains certain pseudo environment variables,
552 their value will be expanded at runtime of the job and will be
553 used to constitute the standard error stream path name. The fol‐
554 lowing pseudo environment variables are supported currently:
555 $HOME home directory on execution machine
557 $USER user ID of job owner
558 $JOB_ID current job ID
559 $JOB_NAME current job name (see -N option)
560 $HOSTNAME name of the execution host
561 $TASK_ID array job task index number
562 Alternatively to $HOME the tilde sign "~" can be used as common
564 in csh(1) or ksh(1). Note, that the "~" sign also works in com‐
565 bination with user names, so that "~<user>" expands to the home
566 directory of <user>. Using another user ID than that of the job
567 owner requires corresponding permissions, of course.
568 Qalter allows changing this option even while the job executes.
570 The modified parameter will only be in effect after a restart or
571 migration of the job, however.
572 If this option or a corresponding value in qmon is specified
574 then this value will be passed to defined JSV instances as
575 parameter with the name e. (see -jsv option below or find more
576 information concerning JSV in jsv(1))
577 -hard Available for qsub, qsh, qrsh, qlogin and qalter only.
579 Signifies that all -q and -l resource requirements following in
581 the command line will be hard requirements and must be satisfied
582 in full before a job can be scheduled.
583 As Grid Engine scans the command line and script file for Grid
584 Engine options and parameters it builds a list of resources
585 required by a job. All such resource requests are considered as
586 absolutely essential for the job to commence. If the -soft
587 option (see below) is encountered during the scan then all fol‐
588 lowing resources are designated as "soft requirements" for exe‐
589 cution, or "nice-to-have, but not essential". If the -hard flag
590 is encountered at a later stage of the scan, all resource
591 requests following it once again become "essential". The -hard
592 and -soft options in effect act as "toggles" during the scan.
593 If this option or a corresponding value in qmon is specified
595 then the corresponding -q and -l resource requirements will be
596 passed to defined JSV instances as parameter with the names
597 q_hard and l_hard. Find for information in the sections describ‐
598 ing -q and -l. (see -jsv option below or find more information
599 concerning JSV in jsv(1))
600 -h | -h {u|s|o|n|U|O|S}...
602 Available for qsub (only -h), qrsh, qalter and qresub (hold
603 state is removed when not set explicitly).
604 List of holds to place on a job, a task or some tasks of a job.
606 `u' denotes a user hold.
608 `s' denotes a system hold.
609 `o' denotes a operator hold.
610 `n' denotes no hold (requires manager privileges).
611 As long as any hold other than `n' is assigned to the job the
613 job is not eligible for execution. Holds can be released via
614 qalter and qrls(1). In case of qalter this is supported by the
615 following additional option specifiers for the -h switch:
616 `U' removes a user hold.
618 `S' removes a system hold.
619 `O' removes a operator hold.
620 Grid Engine managers can assign and remove all hold types, Grid
622 Engine operators can assign and remove user and operator holds,
623 and users can only assign or remove user holds.
624 In the case of qsub only user holds can be placed on a job and
626 thus only the first form of the option with the -h switch alone
627 is allowed. As opposed to this, qalter requires the second form
628 described above.
629 An alternate means to assign hold is provided by the qhold(1)
631 facility.
632 If the job is a array job (see the -t option below), all tasks
634 specified via -t are affected by the -h operation simultane‐
635 ously.
636 Qalter allows changing this option even while the job executes.
638 The modified parameter will only be in effect after a restart or
639 migration of the job, however.
640 If this option is specified with qsub or during the submission
642 of a job in qmon then the parameter h with the value u will be
643 passed to the defined JSV instances indicating that the job will
644 be in user hold after the submission finishes. (see -jsv option
645 below or find more information concerning JSV in jsv(1))
646 -help Prints a listing of all options.
648 -hold_jid wc_job_list
650 Available for qsub, qrsh, and qalter only. See sge_types(1).
651 for wc_job_list definition.
652 Defines or redefines the job dependency list of the submitted
654 job. A reference by job name or pattern is only accepted if the
655 referenced job is owned by the same user as the referring job.
656 The submitted job is not eligible for execution unless all jobs
657 referenced in the comma-separated job id and/or job name list
658 have completed. If any of the referenced jobs exits with exit
659 code 100, the submitted job will remain ineligible for execu‐
660 tion.
661 With the help of job names or regular pattern one can specify a
663 job dependency on multiple jobs satisfying the regular pattern
664 or on all jobs with the requested name. The name dependencies
665 are resolved at submit time and can only be changed via qalter.
666 New jobs or name changes of other jobs will not be taken into
667 account.
668 Qalter allows changing this option even while the job executes.
670 The modified parameter will only be in effect after a restart or
671 migration of the job, however.
672 If this option or a corresponding value in qmon is specified
674 then this value will be passed to defined JSV instances as
675 parameter with the name hold_jid. (see -jsv option below or
676 find more information concerning JSV in jsv(1))
677 -hold_jid_ad wc_job_list
679 Available for qsub, qrsh, and qalter only. See sge_types(1).
680 for wc_job_list definition.
681 Defines or redefines the job array dependency list of the sub‐
683 mitted job. A reference by job name or pattern is only accepted
684 if the referenced job is owned by the same user as the referring
685 job. Each sub-task of the submitted job is not eligible for exe‐
686 cution unless the corresponding sub-tasks of all jobs referenced
687 in the comma-separated job id and/or job name list have com‐
688 pleted. If any array task of the referenced jobs exits with
689 exit code 100, the dependent tasks of the submitted job will
690 remain ineligible for execution.
691 With the help of job names or regular pattern one can specify a
693 job dependency on multiple jobs satisfying the regular pattern
694 or on all jobs with the requested name. The name dependencies
695 are resolved at submit time and can only be changed via qalter.
696 New jobs or name changes of other jobs will not be taken into
697 account.
698 If either the submitted job or any job in wc_job_list are not
700 array jobs with the same range of sub-tasks (see -t option
701 below), the request list will be rejected and the job create or
702 modify operation will error.
703 qalter allows changing this option even while the job executes.
705 The modified parameter will only be in effect after a restart or
706 migration of the job, however.
707 If this option or a corresponding value in qmon is specified
709 then this value will be passed to defined JSV instances as
710 parameter with the name hold_jid_ad. (see -jsv option below or
711 find more information concerning JSV in jsv(1))
712 -i [[hostname]:]file,...
714 Available for qsub, and qalter only.
715 Defines or redefines the file used for the standard input stream
717 of the job. If the file constitutes an absolute filename, the
718 input-path attribute of the job is set to path, including the
719 hostname. If the path name is relative, Grid Engine expands path
720 either with the current working directory path (if the -cwd
721 switch (see above) is also specified) or with the home directory
722 path. If hostname is present, the standard input stream will be
723 placed in the corresponding location only if the job runs on the
724 specified host. If the path contains a ":" without a hostname, a
725 leading ":" has to be specified.
726 By default /dev/null is the input stream for the job.
728 It is possible to use certain pseudo variables, whose values
730 will be expanded at runtime of the job and will be used to
731 express the standard input stream as described in the -e option
732 for the standard error stream.
733 Qalter allows changing this option even while the job executes.
735 The modified parameter will only be in effect after a restart or
736 migration of the job, however.
737 If this option or a corresponding value in qmon is specified
739 then this value will be passed to defined JSV instances as
740 parameter with the name i. (see -jsv option below or find more
741 information concerning JSV in jsv(1))
742 -inherit
744 Available only for qrsh and qmake(1).
745 qrsh allows the user to start a task in an already scheduled
747 parallel job. The option -inherit tells qrsh to read a job id
748 from the environment variable JOB_ID and start the specified
749 command as a task in this job. Please note that in this case,
750 the hostname of the host where the command will be executed must
751 precede the command to execute; the syntax changes to
752 qrsh -inherit [ other options ] hostname command [ command_args
754 ]
755 Note also, that in combination with -inherit, most other command
757 line options will be ignored. Only the options -verbose, -v and
758 -V will be interpreted. As a replacement to option -cwd please
759 use -v PWD.
760 Usually a task should have the same environment (including the
762 current working directory) as the corresponding job, so specify‐
763 ing the option -V should be suitable for most applications.
764 Note: If in your system the qmaster tcp port is not configured
766 as a service, but rather via the environment variable GE_QMAS‐
767 TER_PORT, make sure that this variable is set in the environment
768 when calling qrsh or qmake with the -inherit option. If you call
769 qrsh or qmake with the -inherit option from within a job script,
770 export GE_QMASTER_PORT with the option "-v GE_QMASTER_PORT"
771 either as a command argument or an embedded directive.
772 This parameter is not available in the JSV context. (see -jsv
774 option below or find more information concerning JSV in jsv(1))
775 -j y[es]|n[o]
778 Available for qsub, qsh, qrsh, qlogin and qalter only.
779 Specifies whether or not the standard error stream of the job is
781 merged into the standard output stream.
782 If both the -j y and the -e options are present, Grid Engine
783 sets but ignores the error-path attribute.
784 Qalter allows changing this option even while the job executes.
786 The modified parameter will only be in effect after a restart or
787 migration of the job, however.
788 The value specified with this option or the corresponding value
790 specified in qmon will only be passed to defined JSV instances
791 if the value is yes. The name of the parameter will be j. The
792 value will be y also when then long form yes was specified dur‐
793 ing submission. (see -jsv option below or find more information
794 concerning JSV in jsv(1))
795 -js job_share
797 Available for qsub, qsh, qrsh, qlogin and qalter only.
798 Defines or redefines the job share of the job relative to other
800 jobs. Job share is an unsigned integer value. The default job
801 share value for jobs is 0.
802 The job share influences the Share Tree Policy and the Func‐
804 tional Policy. It has no effect on the Urgency and Override
805 Policies (see share_tree(5), sched_conf(5) and the Grid Engine
806 Installation and Administration Guide for further information on
807 the resource management policies supported by Grid Engine).
808 In case of the Share Tree Policy, users can distribute the tick‐
810 ets to which they are currently entitled among their jobs using
811 different shares assigned via -js. If all jobs have the same job
812 share value, the tickets are distributed evenly. Otherwise, jobs
813 receive tickets relative to the different job shares. Job shares
814 are treated like an additional level in the share tree in the
815 latter case.
816 In connection with the Functional Policy, the job share can be
818 used to weight jobs within the functional job category. Tickets
819 are distributed relative to any uneven job share distribution
820 treated as a virtual share distribution level underneath the
821 functional job category.
822 If both the Share Tree and the Functional Policy are active, the
824 job shares will have an effect in both policies, and the tickets
825 independently derived in each of them are added to the total
826 number of tickets for each job.
827 If this option or a corresponding value in qmon is specified
829 then this value will be passed to defined JSV instances as
830 parameter with the name js. (see -jsv option below or find more
831 information concerning JSV in jsv(1))
832 -jsv jsv_url
834 Available for qsub, qsh, qrsh and qlogin only.
835 Defines a client JSV instance which will be executed to verify
837 the job specification before the job is sent to qmaster.
838 In contrast to other options this switch will not be overwritten
840 if it is also used in sge_request files. Instead all specified
841 JSV instances will be executed to verify the job to be submit‐
842 ted.
843 The JSV instance which is directly passed with the commandline
845 of a client is executed as first to verify the job specifica‐
846 tion. After that the JSV instance which might have been defined
847 in various sge_request files will be triggered to check the job.
848 Find more details in man page jsv(1) and sge_request(5).
849 The syntax of the jsv_url is specified in sge_types(1).()
851 -l resource=value,...
853 Available for qsub, qsh, qrsh, qlogin and qalter only.
854 Launch the job in a Grid Engine queue meeting the given resource
856 request list. In case of qalter the previous definition is
857 replaced by the specified one.
858 complex(5) describes how a list of available resources and their
860 associated valid value specifiers can be obtained.
861 There may be multiple -l switches in a single command. You may
863 request multiple -l options to be soft or hard both in the same
864 command line. In case of a serial job multiple -l switches
865 refine the definition for the sought queue.
866 Qalter allows changing the value of this option even while the
868 job is running, but only if the initial list of resources does
869 not contain a resource that is marked as consumable. However the
870 modification will only be effective after a restart or migration
871 of the job.
872 If this option or a corresponding value in qmon is specified the
874 these hard and soft resource requirements will be passed to
875 defined JSV instances as parameter with the names l_hard and
876 l_soft. If regular expressions will be used for resource
877 requests, then these expressions will be passed as they are.
878 Also shortcut names will not be expanded. (see -jsv option
879 above or find more information concerning JSV in jsv(1))
880 -m b|e|a|s|n,...
882 Available for qsub, qsh, qrsh, qlogin and qalter only.
883 Defines or redefines under which circumstances mail is to be
885 sent to the job owner or to the users defined with the -M option
886 described below. The option arguments have the following mean‐
887 ing:
888 `b' Mail is sent at the beginning of the job.
890 `e' Mail is sent at the end of the job.
891 `a' Mail is sent when the job is aborted or
892 rescheduled.
893 `s' Mail is sent when the job is suspended.
894 `n' No mail is sent.
895 Currently no mail is sent when a job is suspended.
897 Qalter allows changing the b, e, and a option arguments even
899 while the job executes. The modification of the b option argu‐
900 ment will only be in effect after a restart or migration of the
901 job, however.
902 If this option or a corresponding value in qmon is specified
904 then this value will be passed to defined JSV instances as
905 parameter with the name m. (see -jsv option above or find more
906 information concerning JSV in
907 -M user[@host],...
909 Available for qsub, qsh, qrsh, qlogin and qalter only.
910 Defines or redefines the list of users to which the server that
912 executes the job has to send mail, if the server sends mail
913 about the job. Default is the job owner at the originating
914 host.
915 Qalter allows changing this option even while the job executes.
917 If this option or a corresponding value in qmon is specified
919 then this value will be passed to defined JSV instances as
920 parameter with the name M. (see -jsv option above or find more
921 information concerning JSV in jsv(1))
922 -masterq wc_queue_list
924 Available for qsub, qrsh, qsh, qlogin and qalter. Only meaning‐
925 ful for parallel jobs, i.e. together with the -pe option.
926 Defines or redefines a list of cluster queues, queue domains and
928 queue instances which may be used to become the so called master
929 queue of this parallel job. A more detailed description of
930 wc_queue_list can be found in sge_types(1). The master queue is
931 defined as the queue where the parallel job is started. The
932 other queues to which the parallel job spawns tasks are called
933 slave queues. A parallel job only has one master queue.
934 This parameter has all the properties of a resource request and
936 will be merged with requirements derived from the -l option
937 described above.
938 Qalter allows changing this option even while the job executes.
940 The modified parameter will only be in effect after a restart or
941 migration of the job, however.
942 If this option or a corresponding value in qmon is specified the
944 this hard resource requirement will be passed to defined JSV
945 instances as parameter with the name masterq. (see -jsv option
946 above or find more information concerning JSV in jsv(1))
947 -notify
949 Available for qsub, qrsh (with command) and qalter only.
950 This flag, when set causes Grid Engine to send "warning" signals
952 to a running job prior to sending the signals themselves. If a
953 SIGSTOP is pending, the job will receive a SIGUSR1 several sec‐
954 onds before the SIGSTOP. If a SIGKILL is pending, the job will
955 receive a SIGUSR2 several seconds before the SIGKILL. This
956 option provides the running job, before receiving the SIGSTOP or
957 SIGKILL, a configured time interval to do e.g. cleanup opera‐
958 tions. The amount of time delay is controlled by the notify
959 parameter in each queue configuration (see queue_conf(5)).
960 Note that the Linux operating system "misused" the user signals
962 SIGUSR1 and SIGUSR2 in some early Posix thread implementations.
963 You might not want to use the -notify option if you are running
964 multi-threaded applications in your jobs under Linux, particu‐
965 larly on 2.0 or earlier kernels.
966 Qalter allows changing this option even while the job executes.
968 Only if this option is used the parameter named notify with the
970 value y will be passed to defined JSV instances. (see -jsv
971 option above or find more information concerning JSV in jsv(1))
972 -now y[es]|n[o]
974 Available for qsub, qsh, qlogin and qrsh.
975 -now y tries to start the job immediately or not at all. The
977 command returns 0 on success, or 1 on failure (also if the job
978 could not be scheduled immediately). For array jobs submitted
979 with the -now option, if all tasks cannot be immediately sched‐
980 uled, no tasks are scheduled. -now y is default for qsh, qlogin
981 and qrsh
982 With the -now n option, the job will be put into the pending
984 queue if it cannot be executed immediately. -now n is default
985 for qsub.
986 The value specified with this option or the corresponding value
988 specified in qmon will only be passed to defined JSV instances
989 if the value is yes. The name of the parameter will be now. The
990 value will be y also when then long form yes was specified dur‐
991 ing submission. (see -jsv option above or find more information
992 concerning JSV in jsv(1))
993 -N name
995 Available for qsub, qsh, qrsh, qlogin and qalter only.
996 The name of the job. The name should follow the "name" defini‐
998 tion in sge_types(1). Invalid job names will be denied at sub‐
999 mit time.
1000 If the -N option is not present, Grid Engine assigns the name of
1002 the job script to the job after any directory pathname has been
1003 removed from the script-name. If the script is read from stan‐
1004 dard input, the job name defaults to STDIN.
1005 In the case of qsh or qlogin with the -N option is absent, the
1007 string `INTERACT' is assigned to the job.
1008 In the case of qrsh if the -N option is absent, the resulting
1010 job name is determined from the qrsh command line by using the
1011 argument string up to the first occurrence of a semicolon or
1012 whitespace and removing the directory pathname.
1013 Qalter allows changing this option even while the job executes.
1015 The value specified with this option or the corresponding value
1017 specified in qmon will be passed to defined JSV instances as
1018 parameter with the name N. (see -jsv option above or find more
1019 information concerning JSV in jsv(1))
1020 -noshell
1022 Available only for qrsh with a command line.
1023 Do not start the command line given to qrsh in a user's login
1025 shell, i.e. execute it without the wrapping shell.
1026 This option can be used to speed up execution as some overhead,
1028 like the shell startup and sourcing the shell resource files, is
1029 avoided.
1030 This option can only be used if no shell-specific command line
1032 parsing is required. If the command line contains shell syntax
1033 like environment variable substitution or (back) quoting, a
1034 shell must be started. In this case, either do not use the
1035 -noshell option or include the shell call in the command line.
1036 Example:
1038 qrsh echo '$HOSTNAME'
1039 Alternative call with the -noshell option
1040 qrsh -noshell /bin/tcsh -f -c 'echo $HOSTNAME'
1041 -nostdin
1043 Available only for qrsh.
1044 Suppress the input stream STDIN - qrsh will pass the option -n
1046 to the rsh(1) command. This is especially useful, if multiple
1047 tasks are executed in parallel using qrsh, e.g. in a make(1)
1048 process - it would be undefined, which process would get the
1049 input.
1050 -o [[hostname]:]path,...
1052 Available for qsub, qsh, qrsh, qlogin and qalter only.
1053 The path used for the standard output stream of the job. The
1055 path is handled as described in the -e option for the standard
1056 error stream.
1057 By default the file name for standard output has the form
1059 job_name.ojob_id and job_name.ojob_id.task_id for array job
1060 tasks (see -t option below).
1061 Qalter allows changing this option even while the job executes.
1063 The modified parameter will only be in effect after a restart or
1064 migration of the job, however.
1065 If this option or a corresponding value in qmon is specified
1067 then this value will be passed to defined JSV instances as
1068 parameter with the name o. (see -jsv option above or find more
1069 information concerning JSV in jsv(1))
1070 -ot override_tickets
1072 Available for qalter only.
1073 Changes the number of override tickets for the specified job.
1075 Requires manager/operator privileges.
1076 -P project_name
1078 Available for qsub, qsh, qrsh, qlogin and qalter only.
1079 Specifies the project to which this job is assigned. The admin‐
1081 istrator needs to give permission to individual users to submit
1082 jobs to a specific project. (see -aprj option to qconf(1)).
1083 If this option or a corresponding value in qmon is specified
1085 then this value will be passed to defined JSV instances as
1086 parameter with the name ot. (see -jsv option above or find more
1087 information concerning JSV in jsv(1))
1088 -p priority
1090 Available for qsub, qsh, qrsh, qlogin and qalter only.
1091 Defines or redefines the priority of the job relative to other
1093 jobs. Priority is an integer in the range -1023 to 1024. The
1094 default priority value for jobs is 0.
1095 Users may only decrease the priority of their jobs. Grid Engine
1097 managers and administrators may also increase the priority asso‐
1098 ciated with jobs. If a pending job has higher priority, it is
1099 earlier eligible for being dispatched by the Grid Engine sched‐
1100 uler.
1101 If this option or a corresponding value in qmon is specified and
1103 the priority is not 0 then this value will be passed to defined
1104 JSV instances as parameter with the name p. (see -jsv option
1105 above or find more information concerning JSV in jsv(1))
1106 -pe parallel_environment n[-[m]]|[-]m,...
1108 Available for qsub, qsh, qrsh, qlogin and qalter only.
1109 Parallel programming environment (PE) to instantiate. For more
1111 detail about PEs, please see the sge_types(1).
1112 Qalter allows changing this option even while the job executes.
1114 The modified parameter will only be in effect after a restart or
1115 migration of the job, however.
1116 If this option or a corresponding value in qmon is specified
1118 then the parameters pe_name, pe_min and pe_max will be passed to
1119 configured JSV instances where pe_name will be the name of the
1120 parallel environment and the values pe_min and pe_max represent
1121 the values n and m which have been provided with the -pe option.
1122 A missing specification of m will be expanded as value 9999999
1123 in JSV scripts and it represents the value infinity. (see -jsv
1124 option above or find more information concerning JSV in jsv(1))
1125 -pty y[es]|n[o]
1127 Available for qrsh and qlogin only.
1128 -pty yes enforces the job to be started in a pseudo terminal
1130 (pty). If no pty is available, the job start fails. -pty no
1131 enforces the job to be started without a pty. By default, qrsh
1132 without a command and qlogin start the job in a pty, qrsh with a
1133 command starts the job without a pty.
1134 This parameter is not available in the JSV context. (see -jsv
1136 option above or find more information concerning JSV in jsv(1))
1137 -q wc_queue_list
1139 Available for qsub, qrsh, qsh, qlogin and qalter.
1140 Defines or redefines a list of cluster queues, queue domains or
1142 queue instances which may be used to execute this job. Please
1143 find a description of wc_queue_list in sge_types(1). This
1144 parameter has all the properties of a resource request and will
1145 be merged with requirements derived from the -l option described
1146 above.
1147 Qalter allows changing this option even while the job executes.
1149 The modified parameter will only be in effect after a restart or
1150 migration of the job, however.
1151 If this option or a corresponding value in qmon is specified the
1153 these hard and soft resource requirements will be passed to
1154 defined JSV instances as parameters with the names q_hard and
1155 q_soft. If regular expressions will be used for resource
1156 requests, then these expressions will be passed as they are.
1157 Also shortcut names will not be expanded. (see -jsv option
1158 above or find more information concerning JSV in jsv(1))
1159 -R y[es]|n[o]
1161 Available for qsub, qrsh, qsh, qlogin and qalter.
1162 Indicates whether a reservation for this job should be done.
1164 Reservation is never done for immediate jobs, i.e. jobs submit‐
1165 ted using the -now yes option. Please note that regardless of
1166 the reservation request, job reservation might be disabled using
1167 max_reservation in sched_conf(5) and might be limited only to a
1168 certain number of high priority jobs.
1169 By default jobs are submitted with the -R n option.
1171 The value specified with this option or the corresponding value
1173 specified in qmon will only be passed to defined JSV instances
1174 if the value is yes. The name of the parameter will be R. The
1175 value will be y also when then long form yes was specified dur‐
1176 ing submission. (see -jsv option above or find more information
1177 concerning JSV in jsv(1))
1178 -r y[es]|n[o]
1180 Available for qsub and qalter only.
1181 Identifies the ability of a job to be rerun or not. If the
1183 value of -r is 'yes', the job will be rerun if the job was
1184 aborted without leaving a consistent exit state. (This is typi‐
1185 cally the case if the node on which the job is running crashes).
1186 If -r is 'no', the job will not be rerun under any circum‐
1187 stances.
1188 Interactive jobs submitted with qsh, qrsh or qlogin are not
1189 rerunnable.
1190 Qalter allows changing this option even while the job executes.
1192 The value specified with this option or the corresponding value
1194 specified in qmon will only be passed to defined JSV instances
1195 if the value is yes. The name of the parameter will be r. The
1196 value will be y also when then long form yes was specified dur‐
1197 ing submission. (see -jsv option above or find more information
1198 concerning JSV in jsv(1))
1199 -sc variable[=value],...
1201 Available for qsub, qsh, qrsh, qlogin and qalter only.
1202 Sets the given name/value pairs as the job's context. Value may
1204 be omitted. Grid Engine replaces the job's previously defined
1205 context with the one given as the argument. Multiple -ac, -dc,
1206 and -sc options may be given. The order is important.
1207 Contexts provide a way to dynamically attach and remove meta-
1209 information to and from a job. The context variables are not
1210 passed to the job's execution context in its environment.
1211 Qalter allows changing this option even while the job executes.
1213 The outcome of the evaluation of all -ac, -dc, and -sc options
1215 or corresponding values in qmon is passed to defined JSV
1216 instances as parameter with the name ac. (see -jsv option above
1217 or find more information concerning JSV in jsv(1))
1218 -shell y[es]|n[o]
1220 Available only for qsub.
1221 -shell n causes qsub to execute the command line directly, as if
1223 by exec(2). No command shell will be executed for the job.
1224 This option only applies when -b y is also used. Without -b y,
1225 -shell n has no effect.
1226 This option can be used to speed up execution as some overhead,
1228 like the shell startup and sourcing the shell resource files is
1229 avoided.
1230 This option can only be used if no shell-specific command line
1232 parsing is required. If the command line contains shell syntax,
1233 like environment variable substitution or (back) quoting, a
1234 shell must be started. In this case either do not use the
1235 -shell n option or execute the shell as the command line and
1236 pass the path to the executable as a parameter.
1237 If a job executed with the -shell n option fails due to a user
1239 error, such as an invalid path to the executable, the job will
1240 enter the error state.
1241 -shell y cancels the effect of a previous -shell n. Otherwise,
1243 it has no effect.
1244 See -b and -noshell for more information.
1246 The value specified with this option or the corresponding value
1248 specified in qmon will only be passed to defined JSV instances
1249 if the value is yes. The name of the parameter will be shell.
1250 The value will be y also when then long form yes was specified
1251 during submission. (see -jsv option above or find more informa‐
1252 tion concerning JSV in jsv(1))
1253 -soft Available for qsub, qsh, qrsh, qlogin and qalter only.
1255 Signifies that all resource requirements following in the com‐
1257 mand line will be soft requirements and are to be filled on an
1258 "as available" basis.
1259 As Grid Engine scans the command line and script file for Grid
1260 Engine options and parameters, it builds a list of resources
1261 required by the job. All such resource requests are considered
1262 as absolutely essential for the job to commence. If the -soft
1263 option is encountered during the scan then all following
1264 resources are designated as "soft requirements" for execution,
1265 or "nice-to-have, but not essential". If the -hard flag (see
1266 above) is encountered at a later stage of the scan, all resource
1267 requests following it once again become "essential". The -hard
1268 and -soft options in effect act as "toggles" during the scan.
1269 If this option or a corresponding value in qmon is specified
1271 then the corresponding -q and -l resource requirements will be
1272 passed to defined JSV instances as parameter with the names
1273 q_soft and l_soft. Find for information in the sections describ‐
1274 ing -q and -l. (see -jsv option above or find more information
1275 concerning JSV in jsv(1))
1276 -sync y[es]|n[o]
1278 Available for qsub.
1279 -sync y causes qsub to wait for the job to complete before exit‐
1281 ing. If the job completes successfully, qsub's exit code will
1282 be that of the completed job. If the job fails to complete suc‐
1283 cessfully, qsub will print out a error message indicating why
1284 the job failed and will have an exit code of 1. If qsub is
1285 interrupted, e.g. with CTRL-C, before the job completes, the job
1286 will be canceled.
1287 With the -sync n option, qsub will exit with an exit code of 0
1288 as soon as the job is submitted successfully. -sync n is
1289 default for qsub.
1290 If -sync y is used in conjunction with -now y, qsub will behave
1291 as though only -now y were given until the job has been success‐
1292 fully scheduled, after which time qsub will behave as though
1293 only -sync y were given.
1294 If -sync y is used in conjunction with -t n[-m[:i]], qsub will
1295 wait for all the job's tasks to complete before exiting. If all
1296 the job's tasks complete successfully, qsub's exit code will be
1297 that of the first completed job tasks with a non-zero exit code,
1298 or 0 if all job tasks exited with an exit code of 0. If any of
1299 the job's tasks fail to complete successfully, qsub will print
1300 out an error message indicating why the job task(s) failed and
1301 will have an exit code of 1. If qsub is interrupted, e.g. with
1302 CTRL-C, before the job completes, all of the job's tasks will be
1303 canceled.
1304 Information that this switch was specified during submission is
1306 not available in the JSV context. (see -jsv option above or
1307 find more information concerning JSV in jsv(1))
1308 -S [[hostname]:]pathname,...
1310 Available for qsub, qsh and qalter.
1311 Specifies the interpreting shell for the job. Only one pathname
1313 component without a host specifier is valid and only one path
1314 name for a given host is allowed. Shell paths with host assign‐
1315 ments define the interpreting shell for the job if the host is
1316 the execution host. The shell path without host specification is
1317 used if the execution host matches none of the hosts in the
1318 list.
1319 Furthermore, the pathname can be constructed with pseudo envi‐
1321 ronment variables as described for the -e option above.
1322 In the case of qsh the specified shell path is used to execute
1324 the corresponding command interpreter in the xterm(1) (via its
1325 -e option) started on behalf of the interactive job. Qalter
1326 allows changing this option even while the job executes. The
1327 modified parameter will only be in effect after a restart or
1328 migration of the job, however.
1329 If this option or a corresponding value in qmon is specified
1331 then this value will be passed to defined JSV instances as
1332 parameter with the name S. (see -jsv option above or find more
1333 information concerning JSV in jsv(1))
1334 -t n[-m[:s]]
1336 Available for qsub and qalter only.
1337 Submits a so called Array Job, i.e. an array of identical tasks
1339 being differentiated only by an index number and being treated
1340 by Grid Engine almost like a series of jobs. The option argument
1341 to -t specifies the number of array job tasks and the index num‐
1342 ber which will be associated with the tasks. The index numbers
1343 will be exported to the job tasks via the environment variable
1344 GE_TASK_ID. The option arguments n, m and s will be available
1345 through the environment variables GE_TASK_FIRST, GE_TASK_LAST
1346 and GE_TASK_STEPSIZE.
1347 Following restrictions apply to the values n and m:
1349 1 <= n <= MIN(2^31-1, max_aj_tasks)
1351 1 <= m <= MIN(2^31-1, max_aj_tasks)
1352 n <= m
1353 max_aj_tasks is defined in the cluster configuration (see
1355 sge_conf(5))
1356 The task id range specified in the option argument may be a sin‐
1358 gle number, a simple range of the form n-m or a range with a
1359 step size. Hence, the task id range specified by 2-10:2 would
1360 result in the task id indexes 2, 4, 6, 8, and 10, for a total of
1361 5 identical tasks, each with the environment variable GE_TASK_ID
1362 containing one of the 5 index numbers.
1363 All array job tasks inherit the same resource requests and
1365 attribute definitions as specified in the qsub or qalter command
1366 line, except for the -t option. The tasks are scheduled indepen‐
1367 dently and, provided enough resources exist, concurrently, very
1368 much like separate jobs. However, an array job or a sub-array
1369 there of can be accessed as a single unit by commands like
1370 qmod(1) or qdel(1). See the corresponding manual pages for fur‐
1371 ther detail.
1372 Array jobs are commonly used to execute the same type of opera‐
1374 tion on varying input data sets correlated with the task index
1375 number. The number of tasks in a array job is unlimited.
1376 STDOUT and STDERR of array job tasks will be written into dif‐
1378 ferent files with the default location
1379 <jobname>.['e'|'o']<job_id>'.'<task_id>
1381 In order to change this default, the -e and -o options (see
1383 above) can be used together with the pseudo environment vari‐
1384 ables $HOME, $USER, $JOB_ID, $JOB_NAME, $HOSTNAME, and
1385 $GE_TASK_ID.
1386 Note, that you can use the output redirection to divert the out‐
1388 put of all tasks into the same file, but the result of this is
1389 undefined.
1390 If this option or a corresponding value in qmon is specified
1392 then this value will be passed to defined JSV instances as
1393 parameters with the name t_min, t_max and t_step (see -jsv
1394 option above or find more information concerning JSV in jsv(1))
1395 -tc max_running_tasks
1399 -allow users to limit concurrent array job task execution.
1400 Parameter max_running_tasks specifies maximum number of simulta‐
1401 neously running tasks. For example we have running SGE with 10
1402 free slots. We call qsub -t 1-100 -tc 2 jobscript. Then only 2
1403 tasks will be scheduled to run even when 8 slots are free.
1404 -terse Available for qsub only.
1406 -terse causes the qsub to display only the job-id of the job
1408 being submitted rather than the regular "Your job ..." string.
1409 In case of an error the error is reported on stderr as usual.
1410 This can be helpful for scripts which need to parse qsub output
1411 to get the job-id.
1412 Information that this switch was specified during submission is
1414 not available in the JSV context. (see -jsv option above or
1415 find more information concerning JSV in jsv(1))
1416 -u username,...
1418 Available for qalter only. Changes are only made on those jobs
1419 which were submitted by users specified in the list of user‐
1420 names. For managers it is possible to use the qalter -u '*'
1421 command to modify all jobs of all users.
1422 If you use the -u switch it is not permitted to specify an addi‐
1424 tional wc_job_range_list.
1425 -v variable[=value],...
1427 Available for qsub, qrsh (with command argument) and qalter.
1428 Defines or redefines the environment variables to be exported to
1430 the execution context of the job. If the -v option is present
1431 Grid Engine will add the environment variables defined as argu‐
1432 ments to the switch and, optionally, values of specified vari‐
1433 ables, to the execution context of the job.
1434 Qalter allows changing this option even while the job executes.
1436 The modified parameter will only be in effect after a restart or
1437 migration of the job, however.
1438 All environment variables specified with -v, -V or the DISPLAY
1440 variable provided with -display will be exported to the defined
1441 JSV instances only optionally when this is requested explicitly
1442 during the job submission verification. (see -jsv option above
1443 or find more information concerning JSV in jsv(1))
1444 -verbose
1446 Available only for qrsh and qmake(1).
1447 Unlike qsh and qlogin, qrsh does not output any informational
1449 messages while establishing the session, compliant with the
1450 standard rsh(1) and rlogin(1) system calls. If the option -ver‐
1451 bose is set, qrsh behaves like the qsh and qlogin commands,
1452 printing information about the process of establishing the
1453 rsh(1) or rlogin(1) session.
1454 -verify
1456 Available for qsub, qsh, qrsh, qlogin and qalter.
1457 Instead of submitting a job, prints detailed information about
1459 the would-be job as though qstat(1) -j were used, including the
1460 effects of command-line parameters and the external environment.
1461 -V Available for qsub, qsh, qrsh with command and qalter.
1463 Specifies that all environment variables active within the qsub
1465 utility be exported to the context of the job.
1466 All environment variables specified with -v, -V or the DISPLAY
1468 variable provided with -display will be exported to the defined
1469 JSV instances only optionally when this is requested explicitly
1470 during the job submission verification. (see -jsv option above
1471 or find more information concerning JSV in jsv(1))
1472 -w e|w|n|p|v
1474 Available for qsub, qsh, qrsh, qlogin and qalter.
1475 Specifies a validation level applied to the job to be submitted
1477 (qsub, qlogin, and qsh) or the specified queued job (qalter).
1478 The information displayed indicates whether the job can possibly
1479 be scheduled assuming an empty system with no other jobs.
1480 Resource requests exceeding the configured maximal thresholds or
1481 requesting unavailable resource attributes are possible causes
1482 for jobs to fail this validation.
1483 The specifiers e, w, n and v define the following validation
1485 modes:
1486 `e' error - jobs with invalid requests will be
1488 rejected.
1489 `w' warning - only a warning will be displayed
1490 for invalid requests.
1491 `n' none - switches off validation; the default for
1492 qsub, qalter, qrsh, qsh
1493 and qlogin.
1494 `p' poke - does not submit the job but prints a
1495 validation report based on a cluster as is with
1496 all resource utilizations in place.
1497 `v' verify - does not submit the job but prints a
1498 validation report based on an empty cluster.
1499 Note, that the necessary checks are performance consuming and
1501 hence the checking is switched off by default. It should also
1502 be noted that load values are not taken into account with the
1503 verification since they are assumed to be too volatile. To cause
1504 -w e verification to be passed at submission time, it is possi‐
1505 ble to specify non-volatile values (non-consumables) or maximum
1506 values (consumables) in complex_values.
1507 -wd working_dir
1509 Available for qsub, qsh, qrsh and qalter only.
1510 Execute the job from the directory specified in working_dir.
1512 This switch will activate Grid Engine's path aliasing facility,
1513 if the corresponding configuration files are present (see
1514 ge_aliases(5)).
1515 Qalter allows changing this option even while the job executes.
1517 The modified parameter will only be in effect after a restart or
1518 migration of the job, however. The parameter value will be
1519 available in defined JSV instances as parameter with the name
1520 cwd (see -cwd switch above or find more information concerning
1521 JSV in jsv(1))
1522 command
1524 Available for qsub and qrsh only.
1525 The job's scriptfile or binary. If not present or if the oper‐
1527 and is the single-character string '-', qsub reads the script
1528 from standard input.
1529 The command will be available in defined JSV instances as param‐
1531 eter with the name CMDNAME (see -jsv option above or find more
1532 information concerning JSV in jsv(1))
1533 command_args
1535 Available for qsub, qrsh and qalter only.
1536 Arguments to the job. Not valid if the script is entered from
1538 standard input.
1539 Qalter allows changing this option even while the job executes.
1541 The modified parameter will only be in effect after a restart or
1542 migration of the job, however.
1543 The number of command arguments is provided to configured JSV
1545 instances as parameter with the name CMDARGS. Also the argument
1546 values can by accessed. Argument names have the format
1547 CMDARG<number> where <number> is a integer between 0 and CMDARGS
1548 - 1. (see -jsv option above or find more information concerning
1549 JSV in jsv(1))
1550 xterm_args
1552 Available for qsh only.
1553 Arguments to the xterm(1) executable, as defined in the configu‐
1555 ration. For details, refer to ge_conf(5)).
1556 Information concerning xterm_args will be available in JSV con‐
1558 text as parameters with the name CMDARGS and CMDARG<number>.
1559 Find more information above in section command_args. (see -jsv
1560 option above or find more information concerning JSV in jsv(1))
1561
1563 GE_ROOT Specifies the location of the Grid Engine standard con‐
1564 figuration files.
1565 GE_CELL If set, specifies the default Grid Engine cell. To
1567 address a Grid Engine cell qsub, qsh, qlogin or qalter
1568 use (in the order of precedence):
1569 The name of the cell specified in the environment
1571 variable GE_CELL, if it is set.
1572 The name of the default cell, i.e. default.
1574 GE_DEBUG_LEVEL If set, specifies that debug information should be writ‐
1577 ten to stderr. In addition the level of detail in which
1578 debug information is generated is defined.
1579 GE_QMASTER_PORT
1581 If set, specifies the tcp port on which ge_qmaster(8) is
1582 expected to listen for communication requests. Most
1583 installations will use a services map entry for the ser‐
1584 vice "sge_qmaster" instead to define that port.
1585 DISPLAY For qsh jobs the DISPLAY has to be specified at job sub‐
1587 mission. If the DISPLAY is not set by using the -dis‐
1588 play or the -v switch, the contents of the DISPLAY envi‐
1589 ronment variable are used as default.
1590 In addition to those environment variables specified to be exported to
1592 the job via the -v or the -V option (see above) qsub, qsh, and qlogin
1593 add the following variables with the indicated values to the variable
1594 list:
1595 GE_O_HOME the home directory of the submitting client.
1598 GE_O_HOST the name of the host on which the submitting client is
1600 running.
1601 GE_O_LOGNAME the LOGNAME of the submitting client.
1603 GE_O_MAIL the MAIL of the submitting client. This is the mail
1605 directory of the submitting client.
1606 GE_O_PATH the executable search path of the submitting client.
1608 GE_O_SHELL the SHELL of the submitting client.
1610 GE_O_TZ the time zone of the submitting client.
1612 GE_O_WORKDIR the absolute path of the current working directory of
1614 the submitting client.
1615 Furthermore, Grid Engine sets additional variables into the job's envi‐
1617 ronment, as listed below.
1618 ARC
1620 SGE_ARCH The Grid Engine architecture name of the node on which
1622 the job is running. The name is compiled-in into the
1623 ge_execd(8) binary.
1624 GE_CKPT_ENV Specifies the checkpointing environment (as selected
1626 with the -ckpt option) under which a checkpointing job
1627 executes. Only set for checkpointing jobs.
1628 GE_CKPT_DIR Only set for checkpointing jobs. Contains path ckpt_dir
1630 (see checkpoint(5) ) of the checkpoint interface.
1631 GE_STDERR_PATH the pathname of the file to which the standard error
1633 stream of the job is diverted. Commonly used for enhanc‐
1634 ing the output with error messages from prolog, epilog,
1635 parallel environment start/stop or checkpointing
1636 scripts.
1637 GE_STDOUT_PATH the pathname of the file to which the standard output
1639 stream of the job is diverted. Commonly used for enhanc‐
1640 ing the output with messages from prolog, epilog, paral‐
1641 lel environment start/stop or checkpointing scripts.
1642 GE_STDIN_PATH the pathname of the file from which the standard input
1644 stream of the job is taken. This variable might be used
1645 in combination with GE_O_HOST in prolog/epilog scripts
1646 to transfer the input file from the submit to the execu‐
1647 tion host.
1648 GE_JOB_SPOOL_DIR
1650 The directory used by ge_shepherd(8) to store job
1651 related data during job execution. This directory is
1652 owned by root or by a Grid Engine administrative account
1653 and commonly is not open for read or write access to
1654 regular users.
1655 GE_TASK_ID The index number of the current array job task (see -t
1657 option above). This is an unique number in each array
1658 job and can be used to reference different input data
1659 records, for example. This environment variable is set
1660 to "undefined" for non-array jobs. It is possible to
1661 change the predefined value of this variable with -v or
1662 -V (see options above).
1663 GE_TASK_FIRST The index number of the first array job task (see -t
1665 option above). It is possible to change the predefined
1666 value of this variable with -v or -V (see options
1667 above).
1668 GE_TASK_LAST The index number of the last array job task (see -t
1670 option above). It is possible to change the predefined
1671 value of this variable with -v or -V (see options
1672 above).
1673 GE_TASK_STEPSIZE
1675 The step size of the array job specification (see -t
1676 option above). It is possible to change the predefined
1677 value of this variable with -v or -V (see options
1678 above).
1679 ENVIRONMENT The ENVIRONMENT variable is set to BATCH to identify
1681 that the job is being executed under Grid Engine con‐
1682 trol.
1683 HOME The user's home directory path from the passwd(5) file.
1685 HOSTNAME The hostname of the node on which the job is running.
1687 JOB_ID A unique identifier assigned by the ge_qmaster(8) when
1689 the job was submitted. The job ID is a decimal integer
1690 in the range 1 to 99999.
1691 JOB_NAME The job name. For batch jobs or jobs submitted by qrsh
1693 with a command, the job name is built as basename of the
1694 qsub script filename resp. the qrsh command. For inter‐
1695 active jobs it is set to `INTERACTIVE' for qsh jobs,
1696 `QLOGIN' for qlogin jobs and `QRLOGIN' for qrsh jobs
1697 without a command.
1698 This default may be overwritten by the -N. option.
1700 JOB_SCRIPT The path to the job script which is executed. The value
1702 can not be overwritten by the -v or -V option.
1703 LOGNAME The user's login name from the passwd(5) file.
1705 NHOSTS The number of hosts in use by a parallel job.
1707 NQUEUES The number of queues allocated for the job (always 1 for
1709 serial jobs).
1710 NSLOTS The number of queue slots in use by a parallel job.
1712 PATH A default shell search path of:
1714 /usr/local/bin:/usr/ucb:/bin:/usr/bin
1715 SGE_BINARY_PATH
1717 The path where the Grid Engine binaries are installed.
1718 The value is the concatenation of the cluster configura‐
1719 tion value binary_path and the architecture name
1720 $SGE_ARCH environment variable.
1721 PE The parallel environment under which the job executes
1723 (for parallel jobs only).
1724 PE_HOSTFILE The path of a file containing the definition of the vir‐
1726 tual parallel machine assigned to a parallel job by Grid
1727 Engine. See the description of the $pe_hostfile parame‐
1728 ter in ge_pe(5) for details on the format of this file.
1729 The environment variable is only available for parallel
1730 jobs.
1731 QUEUE The name of the cluster queue in which the job is run‐
1733 ning.
1734 REQUEST Available for batch jobs only.
1736 The request name of a job as specified with the -N
1738 switch (see above) or taken as the name of the job
1739 script file.
1740 RESTARTED This variable is set to 1 if a job was restarted either
1742 after a system crash or after a migration in case of a
1743 checkpointing job. The variable has the value 0 other‐
1744 wise.
1745 SHELL The user's login shell from the passwd(5) file. Note:
1747 This is not necessarily the shell in use for the job.
1748 TMPDIR The absolute path to the job's temporary working direc‐
1750 tory.
1751 TMP The same as TMPDIR; provided for compatibility with NQS.
1753 TZ The time zone variable imported from ge_execd(8) if set.
1755 USER The user's login name from the passwd(5) file.
1757 SGE_JSV_TIMEOUT
1759 If the response time of the client JSV is greater than
1760 this timeout value, then the JSV will attempt to be re-
1761 started. The default value is 10 seconds, and this value
1762 must be greater than 0. If the timeout has been reached,
1763 the JSV will only try to re-start once, if the timeout
1764 is reached again an error will occur.
1765
1767 There is no controlling terminal for batch jobs under Grid Engine, and
1768 any tests or actions on a controlling terminal will fail. If these
1769 operations are in your .login or .cshrc file, they may cause your job
1770 to abort.
1771 Insert the following test before any commands that are not pertinent to
1773 batch jobs in your .login:
1774 if ( $?JOB_NAME) then
1776 echo "Grid Engine spooled job"
1777 exit 0
1778 endif
1779 Don't forget to set your shell's search path in your shell start-up
1781 before this code.
1782
1784 The following exit values are returned:
1785 0 Operation was executed successfully.
1787 25 It was not possible to register a new job according to the config‐
1789 ured max_u_jobs or max_jobs limit. Additional information may be
1790 found in sge_conf(5)
1791 >0 Error occurred.
1793
1795 The following is the simplest form of a Grid Engine script file.
1796 =====================================================
1798 #!/bin/csh
1801 a.out
1802 =====================================================
1805 The next example is a more complex Grid Engine script.
1807 =====================================================
1809 #!/bin/csh
1811 # Which account to be charged cpu time
1813 #$ -A santa_claus
1814 # date-time to run, format [[CC]yy]MMDDhhmm[.SS]
1816 #$ -a 12241200
1817 # to run I want 6 or more parallel processes
1819 # under the PE pvm. the processes require
1820 # 128M of memory
1821 #$ -pe pvm 6- -l mem=128
1822 # If I run on dec_x put stderr in /tmp/foo, if I
1824 # run on sun_y, put stderr in /usr/me/foo
1825 #$ -e dec_x:/tmp/foo,sun_y:/usr/me/foo
1826 # Send mail to these users
1828 #$ -M santa@nothpole,claus@northpole
1829 # Mail at beginning/end/on suspension
1831 #$ -m bes
1832 # Export these environmental variables
1834 #$ -v PVM_ROOT,FOOBAR=BAR
1835 # The job is located in the current
1837 # working directory.
1838 #$ -cwd
1839 a.out
1841 ==========================================================
1843
1846 $REQUEST.oJID[.TASKID] STDOUT of job #JID
1847 $REQUEST.eJID[.TASKID] STDERR of job
1848 $REQUEST.poJID[.TASKID] STDOUT of par. env. of job
1849 $REQUEST.peJID[.TASKID] STDERR of par. env. of job
1850 $cwd/.ge_aliases cwd path aliases
1852 $cwd/.ge_request cwd default request
1853 $HOME/.ge_aliases user path aliases
1854 $HOME/.ge_request user default request
1855 <ge_root>/<cell>/common/ge_aliases
1856 cluster path aliases
1857 <ge_root>/<cell>/common/ge_request
1858 cluster default request
1859 <ge_root>/<cell>/common/act_qmaster
1860 Grid Engine master host file
1861
1863 ge_intro(1), qconf(1), qdel(1), qhold(1), qmod(1), qrls(1), qstat(1),
1864 accounting(5), ge_aliases(5), ge_conf(5), ge_request(5), ge_pe(5), com‐
1865 plex(5).
1866
1868 If configured correspondingly, qrsh and qlogin contain portions of the
1869 rsh, rshd, telnet and telnetd code copyrighted by The Regents of the
1870 University of California. Therefore, the following note applies with
1871 respect to qrsh and qlogin: This product includes software developed by
1872 the University of California, Berkeley and its contributors.
1873 See ge_intro(1) as well as the information provided in
1875 <ge_root>/3rd_party/qrsh and <ge_root>/3rd_party/qlogin for a statement
1876 of further rights and permissions.
1877GE 6.2u5 $Date: 2009/12/01 12:24:06 $ SUBMIT(1)