1salloc(1) Slurm Commands salloc(1)
2
6 salloc - Obtain a Slurm job allocation (a set of nodes), execute a com‐
7 mand, and then release the allocation when the command is finished.
8
11 salloc [OPTIONS(0)...] [ : [OPTIONS(N)...]] [command(0) [args(0)...]]
12 Option(s) define multiple jobs in a co-scheduled heterogeneous job.
14 For more details about heterogeneous jobs see the document
15 https://slurm.schedmd.com/heterogeneous_jobs.html
16
19 salloc is used to allocate a Slurm job allocation, which is a set of
20 resources (nodes), possibly with some set of constraints (e.g. number
21 of processors per node). When salloc successfully obtains the re‐
22 quested allocation, it then runs the command specified by the user.
23 Finally, when the user specified command is complete, salloc relin‐
24 quishes the job allocation.
25 The command may be any program the user wishes. Some typical commands
27 are xterm, a shell script containing srun commands, and srun (see the
28 EXAMPLES section). If no command is specified, then salloc runs the
29 user's default shell.
30 The following document describes the influence of various options on
32 the allocation of cpus to jobs and tasks.
33 https://slurm.schedmd.com/cpu_management.html
34 NOTE: The salloc logic includes support to save and restore the termi‐
36 nal line settings and is designed to be executed in the foreground. If
37 you need to execute salloc in the background, set its standard input to
38 some file, for example: "salloc -n16 a.out </dev/null &"
39
42 If salloc is unable to execute the user command, it will return 1 and
43 print errors to stderr. Else if success or if killed by signals HUP,
44 INT, KILL, or QUIT: it will return 0.
45
48 If provided, the command is resolved in the following order:
49 1. If command starts with ".", then path is constructed as: current
51 working directory / command
52 2. If command starts with a "/", then path is considered absolute.
53 3. If command can be resolved through PATH. See path_resolution(7).
54 4. If command is in current working directory.
55 Current working directory is the calling process working directory un‐
57 less the --chdir argument is passed, which will override the current
58 working directory.
59
62 -A, --account=<account>
63 Charge resources used by this job to specified account. The ac‐
64 count is an arbitrary string. The account name may be changed
65 after job submission using the scontrol command.
66 --acctg-freq=<datatype>=<interval>[,<datatype>=<interval>...]
68 Define the job accounting and profiling sampling intervals in
69 seconds. This can be used to override the JobAcctGatherFre‐
70 quency parameter in the slurm.conf file. <datatype>=<interval>
71 specifies the task sampling interval for the jobacct_gather
72 plugin or a sampling interval for a profiling type by the
73 acct_gather_profile plugin. Multiple comma-separated
74 <datatype>=<interval> pairs may be specified. Supported datatype
75 values are:
76 task Sampling interval for the jobacct_gather plugins and
78 for task profiling by the acct_gather_profile
79 plugin.
80 NOTE: This frequency is used to monitor memory us‐
81 age. If memory limits are enforced the highest fre‐
82 quency a user can request is what is configured in
83 the slurm.conf file. It can not be disabled.
84 energy Sampling interval for energy profiling using the
86 acct_gather_energy plugin.
87 network Sampling interval for infiniband profiling using the
89 acct_gather_interconnect plugin.
90 filesystem Sampling interval for filesystem profiling using the
92 acct_gather_filesystem plugin.
93 The default value for the task sampling interval is 30 seconds.
95 The default value for all other intervals is 0. An interval of
96 0 disables sampling of the specified type. If the task sampling
97 interval is 0, accounting information is collected only at job
98 termination (reducing Slurm interference with the job).
99 Smaller (non-zero) values have a greater impact upon job perfor‐
100 mance, but a value of 30 seconds is not likely to be noticeable
101 for applications having less than 10,000 tasks.
102 --bb=<spec>
105 Burst buffer specification. The form of the specification is
106 system dependent. Note the burst buffer may not be accessible
107 from a login node, but require that salloc spawn a shell on one
108 of its allocated compute nodes. When the --bb option is used,
109 Slurm parses this option and creates a temporary burst buffer
110 script file that is used internally by the burst buffer plugins.
111 See Slurm's burst buffer guide for more information and exam‐
112 ples:
113 https://slurm.schedmd.com/burst_buffer.html
114 --bbf=<file_name>
116 Path of file containing burst buffer specification. The form of
117 the specification is system dependent. Also see --bb. Note the
118 burst buffer may not be accessible from a login node, but re‐
119 quire that salloc spawn a shell on one of its allocated compute
120 nodes. See Slurm's burst buffer guide for more information and
121 examples:
122 https://slurm.schedmd.com/burst_buffer.html
123 --begin=<time>
125 Defer eligibility of this job allocation until the specified
126 time.
127 Time may be of the form HH:MM:SS to run a job at a specific time
129 of day (seconds are optional). (If that time is already past,
130 the next day is assumed.) You may also specify midnight, noon,
131 fika (3 PM) or teatime (4 PM) and you can have a time-of-day
132 suffixed with AM or PM for running in the morning or the
133 evening. You can also say what day the job will be run, by
134 specifying a date of the form MMDDYY or MM/DD/YY YYYY-MM-DD.
135 Combine date and time using the following format
136 YYYY-MM-DD[THH:MM[:SS]]. You can also give times like now +
137 count time-units, where the time-units can be seconds (default),
138 minutes, hours, days, or weeks and you can tell Slurm to run the
139 job today with the keyword today and to run the job tomorrow
140 with the keyword tomorrow. The value may be changed after job
141 submission using the scontrol command. For example:
142 --begin=16:00
143 --begin=now+1hour
144 --begin=now+60 (seconds by default)
145 --begin=2010-01-20T12:34:00
146 Notes on date/time specifications:
149 - Although the 'seconds' field of the HH:MM:SS time specifica‐
150 tion is allowed by the code, note that the poll time of the
151 Slurm scheduler is not precise enough to guarantee dispatch of
152 the job on the exact second. The job will be eligible to start
153 on the next poll following the specified time. The exact poll
154 interval depends on the Slurm scheduler (e.g., 60 seconds with
155 the default sched/builtin).
156 - If no time (HH:MM:SS) is specified, the default is
157 (00:00:00).
158 - If a date is specified without a year (e.g., MM/DD) then the
159 current year is assumed, unless the combination of MM/DD and
160 HH:MM:SS has already passed for that year, in which case the
161 next year is used.
162 --bell Force salloc to ring the terminal bell when the job allocation
165 is granted (and only if stdout is a tty). By default, salloc
166 only rings the bell if the allocation is pending for more than
167 ten seconds (and only if stdout is a tty). Also see the option
168 --no-bell.
169 -D, --chdir=<path>
171 Change directory to path before beginning execution. The path
172 can be specified as full path or relative path to the directory
173 where the command is executed.
174 --cluster-constraint=<list>
176 Specifies features that a federated cluster must have to have a
177 sibling job submitted to it. Slurm will attempt to submit a sib‐
178 ling job to a cluster if it has at least one of the specified
179 features.
180 -M, --clusters=<string>
182 Clusters to issue commands to. Multiple cluster names may be
183 comma separated. The job will be submitted to the one cluster
184 providing the earliest expected job initiation time. The default
185 value is the current cluster. A value of 'all' will query to run
186 on all clusters. Note that the SlurmDBD must be up for this op‐
187 tion to work properly.
188 --comment=<string>
190 An arbitrary comment.
191 -C, --constraint=<list>
193 Nodes can have features assigned to them by the Slurm adminis‐
194 trator. Users can specify which of these features are required
195 by their job using the constraint option. If you are looking for
196 'soft' constraints please see see --prefer for more information.
197 Only nodes having features matching the job constraints will be
198 used to satisfy the request. Multiple constraints may be speci‐
199 fied with AND, OR, matching OR, resource counts, etc. (some op‐
200 erators are not supported on all system types).
201 NOTE: If features that are part of the node_features/helpers
203 plugin are requested, then only the Single Name and AND options
204 are supported.
205 Supported --constraint options include:
207 Single Name
209 Only nodes which have the specified feature will be used.
210 For example, --constraint="intel"
211 Node Count
213 A request can specify the number of nodes needed with
214 some feature by appending an asterisk and count after the
215 feature name. For example, --nodes=16 --con‐
216 straint="graphics*4 ..." indicates that the job requires
217 16 nodes and that at least four of those nodes must have
218 the feature "graphics."
219 AND If only nodes with all of specified features will be
221 used. The ampersand is used for an AND operator. For
222 example, --constraint="intel&gpu"
223 OR If only nodes with at least one of specified features
225 will be used. The vertical bar is used for an OR opera‐
226 tor. For example, --constraint="intel|amd"
227 Matching OR
229 If only one of a set of possible options should be used
230 for all allocated nodes, then use the OR operator and en‐
231 close the options within square brackets. For example,
232 --constraint="[rack1|rack2|rack3|rack4]" might be used to
233 specify that all nodes must be allocated on a single rack
234 of the cluster, but any of those four racks can be used.
235 Multiple Counts
237 Specific counts of multiple resources may be specified by
238 using the AND operator and enclosing the options within
239 square brackets. For example, --con‐
240 straint="[rack1*2&rack2*4]" might be used to specify that
241 two nodes must be allocated from nodes with the feature
242 of "rack1" and four nodes must be allocated from nodes
243 with the feature "rack2".
244 NOTE: This construct does not support multiple Intel KNL
246 NUMA or MCDRAM modes. For example, while --con‐
247 straint="[(knl&quad)*2&(knl&hemi)*4]" is not supported,
248 --constraint="[haswell*2&(knl&hemi)*4]" is supported.
249 Specification of multiple KNL modes requires the use of a
250 heterogeneous job.
251 NOTE: Multiple Counts can cause jobs to be allocated with
253 a non-optimal network layout.
254 Brackets
256 Brackets can be used to indicate that you are looking for
257 a set of nodes with the different requirements contained
258 within the brackets. For example, --con‐
259 straint="[(rack1|rack2)*1&(rack3)*2]" will get you one
260 node with either the "rack1" or "rack2" features and two
261 nodes with the "rack3" feature. The same request without
262 the brackets will try to find a single node that meets
263 those requirements.
264 NOTE: Brackets are only reserved for Multiple Counts and
266 Matching OR syntax. AND operators require a count for
267 each feature inside square brackets (i.e.
268 "[quad*2&hemi*1]"). Slurm will only allow a single set of
269 bracketed constraints per job.
270 Parenthesis
272 Parenthesis can be used to group like node features to‐
273 gether. For example, --con‐
274 straint="[(knl&snc4&flat)*4&haswell*1]" might be used to
275 specify that four nodes with the features "knl", "snc4"
276 and "flat" plus one node with the feature "haswell" are
277 required. All options within parenthesis should be
278 grouped with AND (e.g. "&") operands.
279 --container=<path_to_container>
281 Absolute path to OCI container bundle.
282 --contiguous
284 If set, then the allocated nodes must form a contiguous set.
285 NOTE: If SelectPlugin=cons_res this option won't be honored with
287 the topology/tree or topology/3d_torus plugins, both of which
288 can modify the node ordering.
289 -S, --core-spec=<num>
291 Count of specialized cores per node reserved by the job for sys‐
292 tem operations and not used by the application. The application
293 will not use these cores, but will be charged for their alloca‐
294 tion. Default value is dependent upon the node's configured
295 CoreSpecCount value. If a value of zero is designated and the
296 Slurm configuration option AllowSpecResourcesUsage is enabled,
297 the job will be allowed to override CoreSpecCount and use the
298 specialized resources on nodes it is allocated. This option can
299 not be used with the --thread-spec option.
300 NOTE: Explicitly setting a job's specialized core value implic‐
302 itly sets its --exclusive option, reserving entire nodes for the
303 job.
304 --cores-per-socket=<cores>
306 Restrict node selection to nodes with at least the specified
307 number of cores per socket. See additional information under -B
308 option above when task/affinity plugin is enabled.
309 NOTE: This option may implicitly set the number of tasks (if -n
310 was not specified) as one task per requested thread.
311 --cpu-freq=<p1>[-p2[:p3]]
313 Request that job steps initiated by srun commands inside this
315 allocation be run at some requested frequency if possible, on
316 the CPUs selected for the step on the compute node(s).
317 p1 can be [#### | low | medium | high | highm1] which will set
319 the frequency scaling_speed to the corresponding value, and set
320 the frequency scaling_governor to UserSpace. See below for defi‐
321 nition of the values.
322 p1 can be [Conservative | OnDemand | Performance | PowerSave]
324 which will set the scaling_governor to the corresponding value.
325 The governor has to be in the list set by the slurm.conf option
326 CpuFreqGovernors.
327 When p2 is present, p1 will be the minimum scaling frequency and
329 p2 will be the maximum scaling frequency.
330 p2 can be [#### | medium | high | highm1] p2 must be greater
332 than p1.
333 p3 can be [Conservative | OnDemand | Performance | PowerSave |
335 SchedUtil | UserSpace] which will set the governor to the corre‐
336 sponding value.
337 If p3 is UserSpace, the frequency scaling_speed will be set by a
339 power or energy aware scheduling strategy to a value between p1
340 and p2 that lets the job run within the site's power goal. The
341 job may be delayed if p1 is higher than a frequency that allows
342 the job to run within the goal.
343 If the current frequency is < min, it will be set to min. Like‐
345 wise, if the current frequency is > max, it will be set to max.
346 Acceptable values at present include:
348 #### frequency in kilohertz
350 Low the lowest available frequency
352 High the highest available frequency
354 HighM1 (high minus one) will select the next highest
356 available frequency
357 Medium attempts to set a frequency in the middle of the
359 available range
360 Conservative attempts to use the Conservative CPU governor
362 OnDemand attempts to use the OnDemand CPU governor (the de‐
364 fault value)
365 Performance attempts to use the Performance CPU governor
367 PowerSave attempts to use the PowerSave CPU governor
369 UserSpace attempts to use the UserSpace CPU governor
371 The following informational environment variable is set
373 in the job
374 step when --cpu-freq option is requested.
375 SLURM_CPU_FREQ_REQ
376 This environment variable can also be used to supply the value
378 for the CPU frequency request if it is set when the 'srun' com‐
379 mand is issued. The --cpu-freq on the command line will over‐
380 ride the environment variable value. The form on the environ‐
381 ment variable is the same as the command line. See the ENVIRON‐
382 MENT VARIABLES section for a description of the
383 SLURM_CPU_FREQ_REQ variable.
384 NOTE: This parameter is treated as a request, not a requirement.
386 If the job step's node does not support setting the CPU fre‐
387 quency, or the requested value is outside the bounds of the le‐
388 gal frequencies, an error is logged, but the job step is allowed
389 to continue.
390 NOTE: Setting the frequency for just the CPUs of the job step
392 implies that the tasks are confined to those CPUs. If task con‐
393 finement (i.e. the task/affinity TaskPlugin is enabled, or the
394 task/cgroup TaskPlugin is enabled with "ConstrainCores=yes" set
395 in cgroup.conf) is not configured, this parameter is ignored.
396 NOTE: When the step completes, the frequency and governor of
398 each selected CPU is reset to the previous values.
399 NOTE: When submitting jobs with the --cpu-freq option with lin‐
401 uxproc as the ProctrackType can cause jobs to run too quickly
402 before Accounting is able to poll for job information. As a re‐
403 sult not all of accounting information will be present.
404 --cpus-per-gpu=<ncpus>
406 Advise Slurm that ensuing job steps will require ncpus proces‐
407 sors per allocated GPU. Not compatible with the --cpus-per-task
408 option.
409 -c, --cpus-per-task=<ncpus>
411 Advise Slurm that ensuing job steps will require ncpus proces‐
412 sors per task. By default Slurm will allocate one processor per
413 task.
414 For instance, consider an application that has 4 tasks, each re‐
416 quiring 3 processors. If our cluster is comprised of quad-pro‐
417 cessors nodes and we simply ask for 12 processors, the con‐
418 troller might give us only 3 nodes. However, by using the
419 --cpus-per-task=3 options, the controller knows that each task
420 requires 3 processors on the same node, and the controller will
421 grant an allocation of 4 nodes, one for each of the 4 tasks.
422 NOTE: Beginning with 22.05, srun will not inherit the
424 --cpus-per-task value requested by salloc or sbatch. It must be
425 requested again with the call to srun or set with the
426 SRUN_CPUS_PER_TASK environment variable if desired for the
427 task(s).
428 --deadline=<OPT>
430 remove the job if no ending is possible before this deadline
431 (start > (deadline - time[-min])). Default is no deadline.
432 Valid time formats are:
433 HH:MM[:SS] [AM|PM]
434 MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
435 MM/DD[/YY]-HH:MM[:SS]
436 YYYY-MM-DD[THH:MM[:SS]]]
437 now[+count[seconds(default)|minutes|hours|days|weeks]]
438 --delay-boot=<minutes>
440 Do not reboot nodes in order to satisfied this job's feature
441 specification if the job has been eligible to run for less than
442 this time period. If the job has waited for less than the spec‐
443 ified period, it will use only nodes which already have the
444 specified features. The argument is in units of minutes. A de‐
445 fault value may be set by a system administrator using the de‐
446 lay_boot option of the SchedulerParameters configuration parame‐
447 ter in the slurm.conf file, otherwise the default value is zero
448 (no delay).
449 -d, --dependency=<dependency_list>
451 Defer the start of this job until the specified dependencies
452 have been satisfied completed. <dependency_list> is of the form
453 <type:job_id[:job_id][,type:job_id[:job_id]]> or
454 <type:job_id[:job_id][?type:job_id[:job_id]]>. All dependencies
455 must be satisfied if the "," separator is used. Any dependency
456 may be satisfied if the "?" separator is used. Only one separa‐
457 tor may be used. Many jobs can share the same dependency and
458 these jobs may even belong to different users. The value may
459 be changed after job submission using the scontrol command. De‐
460 pendencies on remote jobs are allowed in a federation. Once a
461 job dependency fails due to the termination state of a preceding
462 job, the dependent job will never be run, even if the preceding
463 job is requeued and has a different termination state in a sub‐
464 sequent execution.
465 after:job_id[[+time][:jobid[+time]...]]
467 After the specified jobs start or are cancelled and
468 'time' in minutes from job start or cancellation happens,
469 this job can begin execution. If no 'time' is given then
470 there is no delay after start or cancellation.
471 afterany:job_id[:jobid...]
473 This job can begin execution after the specified jobs
474 have terminated.
475 afterburstbuffer:job_id[:jobid...]
477 This job can begin execution after the specified jobs
478 have terminated and any associated burst buffer stage out
479 operations have completed.
480 aftercorr:job_id[:jobid...]
482 A task of this job array can begin execution after the
483 corresponding task ID in the specified job has completed
484 successfully (ran to completion with an exit code of
485 zero).
486 afternotok:job_id[:jobid...]
488 This job can begin execution after the specified jobs
489 have terminated in some failed state (non-zero exit code,
490 node failure, timed out, etc).
491 afterok:job_id[:jobid...]
493 This job can begin execution after the specified jobs
494 have successfully executed (ran to completion with an
495 exit code of zero).
496 singleton
498 This job can begin execution after any previously
499 launched jobs sharing the same job name and user have
500 terminated. In other words, only one job by that name
501 and owned by that user can be running or suspended at any
502 point in time. In a federation, a singleton dependency
503 must be fulfilled on all clusters unless DependencyParam‐
504 eters=disable_remote_singleton is used in slurm.conf.
505 -m, --distribution={*|block|cyclic|arbi‐
507 trary|plane=<size>}[:{*|block|cyclic|fcyclic}[:{*|block|cyclic|fcyclic}]][,{Pack|NoPack}]
508 Specify alternate distribution methods for remote processes.
510 For job allocation, this sets environment variables that will be
511 used by subsequent srun requests and also affects which cores
512 will be selected for job allocation.
513 This option controls the distribution of tasks to the nodes on
515 which resources have been allocated, and the distribution of
516 those resources to tasks for binding (task affinity). The first
517 distribution method (before the first ":") controls the distri‐
518 bution of tasks to nodes. The second distribution method (after
519 the first ":") controls the distribution of allocated CPUs
520 across sockets for binding to tasks. The third distribution
521 method (after the second ":") controls the distribution of allo‐
522 cated CPUs across cores for binding to tasks. The second and
523 third distributions apply only if task affinity is enabled. The
524 third distribution is supported only if the task/cgroup plugin
525 is configured. The default value for each distribution type is
526 specified by *.
527 Note that with select/cons_res and select/cons_tres, the number
529 of CPUs allocated to each socket and node may be different. Re‐
530 fer to https://slurm.schedmd.com/mc_support.html for more infor‐
531 mation on resource allocation, distribution of tasks to nodes,
532 and binding of tasks to CPUs.
533 First distribution method (distribution of tasks across nodes):
534 * Use the default method for distributing tasks to nodes
537 (block).
538 block The block distribution method will distribute tasks to a
540 node such that consecutive tasks share a node. For exam‐
541 ple, consider an allocation of three nodes each with two
542 cpus. A four-task block distribution request will dis‐
543 tribute those tasks to the nodes with tasks one and two
544 on the first node, task three on the second node, and
545 task four on the third node. Block distribution is the
546 default behavior if the number of tasks exceeds the num‐
547 ber of allocated nodes.
548 cyclic The cyclic distribution method will distribute tasks to a
550 node such that consecutive tasks are distributed over
551 consecutive nodes (in a round-robin fashion). For exam‐
552 ple, consider an allocation of three nodes each with two
553 cpus. A four-task cyclic distribution request will dis‐
554 tribute those tasks to the nodes with tasks one and four
555 on the first node, task two on the second node, and task
556 three on the third node. Note that when SelectType is
557 select/cons_res, the same number of CPUs may not be allo‐
558 cated on each node. Task distribution will be round-robin
559 among all the nodes with CPUs yet to be assigned to
560 tasks. Cyclic distribution is the default behavior if
561 the number of tasks is no larger than the number of allo‐
562 cated nodes.
563 plane The tasks are distributed in blocks of size <size>. The
565 size must be given or SLURM_DIST_PLANESIZE must be set.
566 The number of tasks distributed to each node is the same
567 as for cyclic distribution, but the taskids assigned to
568 each node depend on the plane size. Additional distribu‐
569 tion specifications cannot be combined with this option.
570 For more details (including examples and diagrams),
571 please see https://slurm.schedmd.com/mc_support.html and
572 https://slurm.schedmd.com/dist_plane.html
573 arbitrary
575 The arbitrary method of distribution will allocate pro‐
576 cesses in-order as listed in file designated by the envi‐
577 ronment variable SLURM_HOSTFILE. If this variable is
578 listed it will over ride any other method specified. If
579 not set the method will default to block. Inside the
580 hostfile must contain at minimum the number of hosts re‐
581 quested and be one per line or comma separated. If spec‐
582 ifying a task count (-n, --ntasks=<number>), your tasks
583 will be laid out on the nodes in the order of the file.
584 NOTE: The arbitrary distribution option on a job alloca‐
585 tion only controls the nodes to be allocated to the job
586 and not the allocation of CPUs on those nodes. This op‐
587 tion is meant primarily to control a job step's task lay‐
588 out in an existing job allocation for the srun command.
589 NOTE: If the number of tasks is given and a list of re‐
590 quested nodes is also given, the number of nodes used
591 from that list will be reduced to match that of the num‐
592 ber of tasks if the number of nodes in the list is
593 greater than the number of tasks.
594 Second distribution method (distribution of CPUs across sockets
596 for binding):
597 * Use the default method for distributing CPUs across sock‐
600 ets (cyclic).
601 block The block distribution method will distribute allocated
603 CPUs consecutively from the same socket for binding to
604 tasks, before using the next consecutive socket.
605 cyclic The cyclic distribution method will distribute allocated
607 CPUs for binding to a given task consecutively from the
608 same socket, and from the next consecutive socket for the
609 next task, in a round-robin fashion across sockets.
610 Tasks requiring more than one CPU will have all of those
611 CPUs allocated on a single socket if possible.
612 fcyclic
614 The fcyclic distribution method will distribute allocated
615 CPUs for binding to tasks from consecutive sockets in a
616 round-robin fashion across the sockets. Tasks requiring
617 more than one CPU will have each CPUs allocated in a
618 cyclic fashion across sockets.
619 Third distribution method (distribution of CPUs across cores for
621 binding):
622 * Use the default method for distributing CPUs across cores
625 (inherited from second distribution method).
626 block The block distribution method will distribute allocated
628 CPUs consecutively from the same core for binding to
629 tasks, before using the next consecutive core.
630 cyclic The cyclic distribution method will distribute allocated
632 CPUs for binding to a given task consecutively from the
633 same core, and from the next consecutive core for the
634 next task, in a round-robin fashion across cores.
635 fcyclic
637 The fcyclic distribution method will distribute allocated
638 CPUs for binding to tasks from consecutive cores in a
639 round-robin fashion across the cores.
640 Optional control for task distribution over nodes:
642 Pack Rather than evenly distributing a job step's tasks evenly
645 across its allocated nodes, pack them as tightly as pos‐
646 sible on the nodes. This only applies when the "block"
647 task distribution method is used.
648 NoPack Rather than packing a job step's tasks as tightly as pos‐
650 sible on the nodes, distribute them evenly. This user
651 option will supersede the SelectTypeParameters
652 CR_Pack_Nodes configuration parameter.
653 -x, --exclude=<node_name_list>
655 Explicitly exclude certain nodes from the resources granted to
656 the job.
657 --exclusive[={user|mcs}]
659 The job allocation can not share nodes with other running jobs
660 (or just other users with the "=user" option or with the "=mcs"
661 option). If user/mcs are not specified (i.e. the job allocation
662 can not share nodes with other running jobs), the job is allo‐
663 cated all CPUs and GRES on all nodes in the allocation, but is
664 only allocated as much memory as it requested. This is by design
665 to support gang scheduling, because suspended jobs still reside
666 in memory. To request all the memory on a node, use --mem=0.
667 The default shared/exclusive behavior depends on system configu‐
668 ration and the partition's OverSubscribe option takes precedence
669 over the job's option. NOTE: Since shared GRES (MPS) cannot be
670 allocated at the same time as a sharing GRES (GPU) this option
671 only allocates all sharing GRES and no underlying shared GRES.
672 -B, --extra-node-info=<sockets>[:cores[:threads]]
674 Restrict node selection to nodes with at least the specified
675 number of sockets, cores per socket and/or threads per core.
676 NOTE: These options do not specify the resource allocation size.
677 Each value specified is considered a minimum. An asterisk (*)
678 can be used as a placeholder indicating that all available re‐
679 sources of that type are to be utilized. Values can also be
680 specified as min-max. The individual levels can also be speci‐
681 fied in separate options if desired:
682 --sockets-per-node=<sockets>
683 --cores-per-socket=<cores>
684 --threads-per-core=<threads>
685 If task/affinity plugin is enabled, then specifying an alloca‐
686 tion in this manner also results in subsequently launched tasks
687 being bound to threads if the -B option specifies a thread
688 count, otherwise an option of cores if a core count is speci‐
689 fied, otherwise an option of sockets. If SelectType is config‐
690 ured to select/cons_res, it must have a parameter of CR_Core,
691 CR_Core_Memory, CR_Socket, or CR_Socket_Memory for this option
692 to be honored. If not specified, the scontrol show job will
693 display 'ReqS:C:T=*:*:*'. This option applies to job alloca‐
694 tions.
695 NOTE: This option is mutually exclusive with --hint,
696 --threads-per-core and --ntasks-per-core.
697 NOTE: This option may implicitly set the number of tasks (if -n
698 was not specified) as one task per requested thread.
699 --get-user-env[=timeout][mode]
701 This option will load login environment variables for the user
702 specified in the --uid option. The environment variables are
703 retrieved by running something along the lines of "su - <user‐
704 name> -c /usr/bin/env" and parsing the output. Be aware that
705 any environment variables already set in salloc's environment
706 will take precedence over any environment variables in the
707 user's login environment. The optional timeout value is in sec‐
708 onds. Default value is 3 seconds. The optional mode value con‐
709 trols the "su" options. With a mode value of "S", "su" is exe‐
710 cuted without the "-" option. With a mode value of "L", "su" is
711 executed with the "-" option, replicating the login environment.
712 If mode is not specified, the mode established at Slurm build
713 time is used. Examples of use include "--get-user-env",
714 "--get-user-env=10" "--get-user-env=10L", and
715 "--get-user-env=S". NOTE: This option only works if the caller
716 has an effective uid of "root".
717 --gid=<group>
719 Submit the job with the specified group's group access permis‐
720 sions. group may be the group name or the numerical group ID.
721 In the default Slurm configuration, this option is only valid
722 when used by the user root.
723 --gpu-bind=[verbose,]<type>
725 Bind tasks to specific GPUs. By default every spawned task can
726 access every GPU allocated to the step. If "verbose," is speci‐
727 fied before <type>, then print out GPU binding debug information
728 to the stderr of the tasks. GPU binding is ignored if there is
729 only one task.
730 Supported type options:
732 closest Bind each task to the GPU(s) which are closest. In a
734 NUMA environment, each task may be bound to more than
735 one GPU (i.e. all GPUs in that NUMA environment).
736 map_gpu:<list>
738 Bind by setting GPU masks on tasks (or ranks) as spec‐
739 ified where <list> is
740 <gpu_id_for_task_0>,<gpu_id_for_task_1>,... GPU IDs
741 are interpreted as decimal values. If the number of
742 tasks (or ranks) exceeds the number of elements in
743 this list, elements in the list will be reused as
744 needed starting from the beginning of the list. To
745 simplify support for large task counts, the lists may
746 follow a map with an asterisk and repetition count.
747 For example "map_gpu:0*4,1*4". If the task/cgroup
748 plugin is used and ConstrainDevices is set in
749 cgroup.conf, then the GPU IDs are zero-based indexes
750 relative to the GPUs allocated to the job (e.g. the
751 first GPU is 0, even if the global ID is 3). Other‐
752 wise, the GPU IDs are global IDs, and all GPUs on each
753 node in the job should be allocated for predictable
754 binding results.
755 mask_gpu:<list>
757 Bind by setting GPU masks on tasks (or ranks) as spec‐
758 ified where <list> is
759 <gpu_mask_for_task_0>,<gpu_mask_for_task_1>,... The
760 mapping is specified for a node and identical mapping
761 is applied to the tasks on every node (i.e. the lowest
762 task ID on each node is mapped to the first mask spec‐
763 ified in the list, etc.). GPU masks are always inter‐
764 preted as hexadecimal values but can be preceded with
765 an optional '0x'. To simplify support for large task
766 counts, the lists may follow a map with an asterisk
767 and repetition count. For example
768 "mask_gpu:0x0f*4,0xf0*4". If the task/cgroup plugin
769 is used and ConstrainDevices is set in cgroup.conf,
770 then the GPU IDs are zero-based indexes relative to
771 the GPUs allocated to the job (e.g. the first GPU is
772 0, even if the global ID is 3). Otherwise, the GPU IDs
773 are global IDs, and all GPUs on each node in the job
774 should be allocated for predictable binding results.
775 none Do not bind tasks to GPUs (turns off binding if
777 --gpus-per-task is requested).
778 per_task:<gpus_per_task>
780 Each task will be bound to the number of gpus speci‐
781 fied in <gpus_per_task>. Gpus are assigned in order to
782 tasks. The first task will be assigned the first x
783 number of gpus on the node etc.
784 single:<tasks_per_gpu>
786 Like --gpu-bind=closest, except that each task can
787 only be bound to a single GPU, even when it can be
788 bound to multiple GPUs that are equally close. The
789 GPU to bind to is determined by <tasks_per_gpu>, where
790 the first <tasks_per_gpu> tasks are bound to the first
791 GPU available, the second <tasks_per_gpu> tasks are
792 bound to the second GPU available, etc. This is basi‐
793 cally a block distribution of tasks onto available
794 GPUs, where the available GPUs are determined by the
795 socket affinity of the task and the socket affinity of
796 the GPUs as specified in gres.conf's Cores parameter.
797 --gpu-freq=[<type]=value>[,<type=value>][,verbose]
799 Request that GPUs allocated to the job are configured with spe‐
800 cific frequency values. This option can be used to indepen‐
801 dently configure the GPU and its memory frequencies. After the
802 job is completed, the frequencies of all affected GPUs will be
803 reset to the highest possible values. In some cases, system
804 power caps may override the requested values. The field type
805 can be "memory". If type is not specified, the GPU frequency is
806 implied. The value field can either be "low", "medium", "high",
807 "highm1" or a numeric value in megahertz (MHz). If the speci‐
808 fied numeric value is not possible, a value as close as possible
809 will be used. See below for definition of the values. The ver‐
810 bose option causes current GPU frequency information to be
811 logged. Examples of use include "--gpu-freq=medium,memory=high"
812 and "--gpu-freq=450".
813 Supported value definitions:
815 low the lowest available frequency.
817 medium attempts to set a frequency in the middle of the
819 available range.
820 high the highest available frequency.
822 highm1 (high minus one) will select the next highest avail‐
824 able frequency.
825 -G, --gpus=[type:]<number>
827 Specify the total number of GPUs required for the job. An op‐
828 tional GPU type specification can be supplied. For example
829 "--gpus=volta:3". Multiple options can be requested in a comma
830 separated list, for example: "--gpus=volta:3,kepler:1". See
831 also the --gpus-per-node, --gpus-per-socket and --gpus-per-task
832 options.
833 NOTE: The allocation has to contain at least one GPU per node.
834 --gpus-per-node=[type:]<number>
836 Specify the number of GPUs required for the job on each node in‐
837 cluded in the job's resource allocation. An optional GPU type
838 specification can be supplied. For example
839 "--gpus-per-node=volta:3". Multiple options can be requested in
840 a comma separated list, for example:
841 "--gpus-per-node=volta:3,kepler:1". See also the --gpus,
842 --gpus-per-socket and --gpus-per-task options.
843 --gpus-per-socket=[type:]<number>
845 Specify the number of GPUs required for the job on each socket
846 included in the job's resource allocation. An optional GPU type
847 specification can be supplied. For example
848 "--gpus-per-socket=volta:3". Multiple options can be requested
849 in a comma separated list, for example:
850 "--gpus-per-socket=volta:3,kepler:1". Requires job to specify a
851 sockets per node count ( --sockets-per-node). See also the
852 --gpus, --gpus-per-node and --gpus-per-task options.
853 --gpus-per-task=[type:]<number>
855 Specify the number of GPUs required for the job on each task to
856 be spawned in the job's resource allocation. An optional GPU
857 type specification can be supplied. For example
858 "--gpus-per-task=volta:1". Multiple options can be requested in
859 a comma separated list, for example:
860 "--gpus-per-task=volta:3,kepler:1". See also the --gpus,
861 --gpus-per-socket and --gpus-per-node options. This option re‐
862 quires an explicit task count, e.g. -n, --ntasks or "--gpus=X
863 --gpus-per-task=Y" rather than an ambiguous range of nodes with
864 -N, --nodes. This option will implicitly set
865 --gpu-bind=per_task:<gpus_per_task>, but that can be overridden
866 with an explicit --gpu-bind specification.
867 --gres=<list>
869 Specifies a comma-delimited list of generic consumable re‐
870 sources. The format of each entry on the list is
871 "name[[:type]:count]". The name is that of the consumable re‐
872 source. The count is the number of those resources with a de‐
873 fault value of 1. The count can have a suffix of "k" or "K"
874 (multiple of 1024), "m" or "M" (multiple of 1024 x 1024), "g" or
875 "G" (multiple of 1024 x 1024 x 1024), "t" or "T" (multiple of
876 1024 x 1024 x 1024 x 1024), "p" or "P" (multiple of 1024 x 1024
877 x 1024 x 1024 x 1024). The specified resources will be allo‐
878 cated to the job on each node. The available generic consumable
879 resources is configurable by the system administrator. A list
880 of available generic consumable resources will be printed and
881 the command will exit if the option argument is "help". Exam‐
882 ples of use include "--gres=gpu:2", "--gres=gpu:kepler:2", and
883 "--gres=help".
884 --gres-flags=<type>
886 Specify generic resource task binding options.
887 disable-binding
889 Disable filtering of CPUs with respect to generic re‐
890 source locality. This option is currently required to
891 use more CPUs than are bound to a GRES (i.e. if a GPU is
892 bound to the CPUs on one socket, but resources on more
893 than one socket are required to run the job). This op‐
894 tion may permit a job to be allocated resources sooner
895 than otherwise possible, but may result in lower job per‐
896 formance.
897 NOTE: This option is specific to SelectType=cons_res.
898 enforce-binding
900 The only CPUs available to the job will be those bound to
901 the selected GRES (i.e. the CPUs identified in the
902 gres.conf file will be strictly enforced). This option
903 may result in delayed initiation of a job. For example a
904 job requiring two GPUs and one CPU will be delayed until
905 both GPUs on a single socket are available rather than
906 using GPUs bound to separate sockets, however, the appli‐
907 cation performance may be improved due to improved commu‐
908 nication speed. Requires the node to be configured with
909 more than one socket and resource filtering will be per‐
910 formed on a per-socket basis.
911 NOTE: This option is specific to SelectType=cons_tres.
912 -h, --help
914 Display help information and exit.
915 --hint=<type>
917 Bind tasks according to application hints.
918 NOTE: This option cannot be used in conjunction with
919 --ntasks-per-core, --threads-per-core or -B. If --hint is speci‐
920 fied as a command line argument, it will take precedence over
921 the environment.
922 compute_bound
924 Select settings for compute bound applications: use all
925 cores in each socket, one thread per core.
926 memory_bound
928 Select settings for memory bound applications: use only
929 one core in each socket, one thread per core.
930 [no]multithread
932 [don't] use extra threads with in-core multi-threading
933 which can benefit communication intensive applications.
934 Only supported with the task/affinity plugin.
935 help show this help message
937 -H, --hold
939 Specify the job is to be submitted in a held state (priority of
940 zero). A held job can now be released using scontrol to reset
941 its priority (e.g. "scontrol release <job_id>").
942 -I, --immediate[=<seconds>]
944 exit if resources are not available within the time period spec‐
945 ified. If no argument is given (seconds defaults to 1), re‐
946 sources must be available immediately for the request to suc‐
947 ceed. If defer is configured in SchedulerParameters and sec‐
948 onds=1 the allocation request will fail immediately; defer con‐
949 flicts and takes precedence over this option. By default, --im‐
950 mediate is off, and the command will block until resources be‐
951 come available. Since this option's argument is optional, for
952 proper parsing the single letter option must be followed immedi‐
953 ately with the value and not include a space between them. For
954 example "-I60" and not "-I 60".
955 -J, --job-name=<jobname>
957 Specify a name for the job allocation. The specified name will
958 appear along with the job id number when querying running jobs
959 on the system. The default job name is the name of the "com‐
960 mand" specified on the command line.
961 -K, --kill-command[=signal]
963 salloc always runs a user-specified command once the allocation
964 is granted. salloc will wait indefinitely for that command to
965 exit. If you specify the --kill-command option salloc will send
966 a signal to your command any time that the Slurm controller
967 tells salloc that its job allocation has been revoked. The job
968 allocation can be revoked for a couple of reasons: someone used
969 scancel to revoke the allocation, or the allocation reached its
970 time limit. If you do not specify a signal name or number and
971 Slurm is configured to signal the spawned command at job termi‐
972 nation, the default signal is SIGHUP for interactive and SIGTERM
973 for non-interactive sessions. Since this option's argument is
974 optional, for proper parsing the single letter option must be
975 followed immediately with the value and not include a space be‐
976 tween them. For example "-K1" and not "-K 1".
977 -L, --licenses=<license>[@db][:count][,license[@db][:count]...]
979 Specification of licenses (or other resources available on all
980 nodes of the cluster) which must be allocated to this job. Li‐
981 cense names can be followed by a colon and count (the default
982 count is one). Multiple license names should be comma separated
983 (e.g. "--licenses=foo:4,bar").
984 NOTE: When submitting heterogeneous jobs, license requests only
986 work correctly when made on the first component job. For exam‐
987 ple "salloc -L ansys:2 :".
988 --mail-type=<type>
990 Notify user by email when certain event types occur. Valid type
991 values are NONE, BEGIN, END, FAIL, REQUEUE, ALL (equivalent to
992 BEGIN, END, FAIL, INVALID_DEPEND, REQUEUE, and STAGE_OUT), IN‐
993 VALID_DEPEND (dependency never satisfied), STAGE_OUT (burst buf‐
994 fer stage out and teardown completed), TIME_LIMIT, TIME_LIMIT_90
995 (reached 90 percent of time limit), TIME_LIMIT_80 (reached 80
996 percent of time limit), and TIME_LIMIT_50 (reached 50 percent of
997 time limit). Multiple type values may be specified in a comma
998 separated list. The user to be notified is indicated with
999 --mail-user.
1000 --mail-user=<user>
1002 User to receive email notification of state changes as defined
1003 by --mail-type. The default value is the submitting user.
1004 --mcs-label=<mcs>
1006 Used only when the mcs/group plugin is enabled. This parameter
1007 is a group among the groups of the user. Default value is cal‐
1008 culated by the Plugin mcs if it's enabled.
1009 --mem=<size>[units]
1011 Specify the real memory required per node. Default units are
1012 megabytes. Different units can be specified using the suffix
1013 [K|M|G|T]. Default value is DefMemPerNode and the maximum value
1014 is MaxMemPerNode. If configured, both of parameters can be seen
1015 using the scontrol show config command. This parameter would
1016 generally be used if whole nodes are allocated to jobs (Select‐
1017 Type=select/linear). Also see --mem-per-cpu and --mem-per-gpu.
1018 The --mem, --mem-per-cpu and --mem-per-gpu options are mutually
1019 exclusive. If --mem, --mem-per-cpu or --mem-per-gpu are speci‐
1020 fied as command line arguments, then they will take precedence
1021 over the environment.
1022 NOTE: A memory size specification of zero is treated as a spe‐
1024 cial case and grants the job access to all of the memory on each
1025 node.
1026 NOTE: Enforcement of memory limits currently relies upon the
1028 task/cgroup plugin or enabling of accounting, which samples mem‐
1029 ory use on a periodic basis (data need not be stored, just col‐
1030 lected). In both cases memory use is based upon the job's Resi‐
1031 dent Set Size (RSS). A task may exceed the memory limit until
1032 the next periodic accounting sample.
1033 --mem-bind=[{quiet|verbose},]<type>
1035 Bind tasks to memory. Used only when the task/affinity plugin is
1036 enabled and the NUMA memory functions are available. Note that
1037 the resolution of CPU and memory binding may differ on some ar‐
1038 chitectures. For example, CPU binding may be performed at the
1039 level of the cores within a processor while memory binding will
1040 be performed at the level of nodes, where the definition of
1041 "nodes" may differ from system to system. By default no memory
1042 binding is performed; any task using any CPU can use any memory.
1043 This option is typically used to ensure that each task is bound
1044 to the memory closest to its assigned CPU. The use of any type
1045 other than "none" or "local" is not recommended.
1046 NOTE: To have Slurm always report on the selected memory binding
1048 for all commands executed in a shell, you can enable verbose
1049 mode by setting the SLURM_MEM_BIND environment variable value to
1050 "verbose".
1051 The following informational environment variables are set when
1053 --mem-bind is in use:
1054 SLURM_MEM_BIND_LIST
1056 SLURM_MEM_BIND_PREFER
1057 SLURM_MEM_BIND_SORT
1058 SLURM_MEM_BIND_TYPE
1059 SLURM_MEM_BIND_VERBOSE
1060 See the ENVIRONMENT VARIABLES section for a more detailed de‐
1062 scription of the individual SLURM_MEM_BIND* variables.
1063 Supported options include:
1065 help show this help message
1067 local Use memory local to the processor in use
1069 map_mem:<list>
1071 Bind by setting memory masks on tasks (or ranks) as spec‐
1072 ified where <list> is
1073 <numa_id_for_task_0>,<numa_id_for_task_1>,... The map‐
1074 ping is specified for a node and identical mapping is ap‐
1075 plied to the tasks on every node (i.e. the lowest task ID
1076 on each node is mapped to the first ID specified in the
1077 list, etc.). NUMA IDs are interpreted as decimal values
1078 unless they are preceded with '0x' in which case they in‐
1079 terpreted as hexadecimal values. If the number of tasks
1080 (or ranks) exceeds the number of elements in this list,
1081 elements in the list will be reused as needed starting
1082 from the beginning of the list. To simplify support for
1083 large task counts, the lists may follow a map with an as‐
1084 terisk and repetition count. For example
1085 "map_mem:0x0f*4,0xf0*4". For predictable binding re‐
1086 sults, all CPUs for each node in the job should be allo‐
1087 cated to the job.
1088 mask_mem:<list>
1090 Bind by setting memory masks on tasks (or ranks) as spec‐
1091 ified where <list> is
1092 <numa_mask_for_task_0>,<numa_mask_for_task_1>,... The
1093 mapping is specified for a node and identical mapping is
1094 applied to the tasks on every node (i.e. the lowest task
1095 ID on each node is mapped to the first mask specified in
1096 the list, etc.). NUMA masks are always interpreted as
1097 hexadecimal values. Note that masks must be preceded
1098 with a '0x' if they don't begin with [0-9] so they are
1099 seen as numerical values. If the number of tasks (or
1100 ranks) exceeds the number of elements in this list, ele‐
1101 ments in the list will be reused as needed starting from
1102 the beginning of the list. To simplify support for large
1103 task counts, the lists may follow a mask with an asterisk
1104 and repetition count. For example "mask_mem:0*4,1*4".
1105 For predictable binding results, all CPUs for each node
1106 in the job should be allocated to the job.
1107 no[ne] don't bind tasks to memory (default)
1109 p[refer]
1111 Prefer use of first specified NUMA node, but permit
1112 use of other available NUMA nodes.
1113 q[uiet]
1115 quietly bind before task runs (default)
1116 rank bind by task rank (not recommended)
1118 sort sort free cache pages (run zonesort on Intel KNL nodes)
1120 v[erbose]
1122 verbosely report binding before task runs
1123 --mem-per-cpu=<size>[units]
1125 Minimum memory required per usable allocated CPU. Default units
1126 are megabytes. Different units can be specified using the suf‐
1127 fix [K|M|G|T]. The default value is DefMemPerCPU and the maxi‐
1128 mum value is MaxMemPerCPU (see exception below). If configured,
1129 both parameters can be seen using the scontrol show config com‐
1130 mand. Note that if the job's --mem-per-cpu value exceeds the
1131 configured MaxMemPerCPU, then the user's limit will be treated
1132 as a memory limit per task; --mem-per-cpu will be reduced to a
1133 value no larger than MaxMemPerCPU; --cpus-per-task will be set
1134 and the value of --cpus-per-task multiplied by the new
1135 --mem-per-cpu value will equal the original --mem-per-cpu value
1136 specified by the user. This parameter would generally be used
1137 if individual processors are allocated to jobs (SelectType=se‐
1138 lect/cons_res). If resources are allocated by core, socket, or
1139 whole nodes, then the number of CPUs allocated to a job may be
1140 higher than the task count and the value of --mem-per-cpu should
1141 be adjusted accordingly. Also see --mem and --mem-per-gpu. The
1142 --mem, --mem-per-cpu and --mem-per-gpu options are mutually ex‐
1143 clusive.
1144 NOTE: If the final amount of memory requested by a job can't be
1146 satisfied by any of the nodes configured in the partition, the
1147 job will be rejected. This could happen if --mem-per-cpu is
1148 used with the --exclusive option for a job allocation and
1149 --mem-per-cpu times the number of CPUs on a node is greater than
1150 the total memory of that node.
1151 NOTE: This applies to usable allocated CPUs in a job allocation.
1153 This is important when more than one thread per core is config‐
1154 ured. If a job requests --threads-per-core with fewer threads
1155 on a core than exist on the core (or --hint=nomultithread which
1156 implies --threads-per-core=1), the job will be unable to use
1157 those extra threads on the core and those threads will not be
1158 included in the memory per CPU calculation. But if the job has
1159 access to all threads on the core, those threads will be in‐
1160 cluded in the memory per CPU calculation even if the job did not
1161 explicitly request those threads.
1162 In the following examples, each core has two threads.
1164 In this first example, two tasks can run on separate hyper‐
1166 threads in the same core because --threads-per-core is not used.
1167 The third task uses both threads of the second core. The allo‐
1168 cated memory per cpu includes all threads:
1169 $ salloc -n3 --mem-per-cpu=100
1171 salloc: Granted job allocation 17199
1172 $ sacct -j $SLURM_JOB_ID -X -o jobid%7,reqtres%35,alloctres%35
1173 JobID ReqTRES AllocTRES
1174 ------- ----------------------------------- -----------------------------------
1175 17199 billing=3,cpu=3,mem=300M,node=1 billing=4,cpu=4,mem=400M,node=1
1176 In this second example, because of --threads-per-core=1, each
1178 task is allocated an entire core but is only able to use one
1179 thread per core. Allocated CPUs includes all threads on each
1180 core. However, allocated memory per cpu includes only the usable
1181 thread in each core.
1182 $ salloc -n3 --mem-per-cpu=100 --threads-per-core=1
1184 salloc: Granted job allocation 17200
1185 $ sacct -j $SLURM_JOB_ID -X -o jobid%7,reqtres%35,alloctres%35
1186 JobID ReqTRES AllocTRES
1187 ------- ----------------------------------- -----------------------------------
1188 17200 billing=3,cpu=3,mem=300M,node=1 billing=6,cpu=6,mem=300M,node=1
1189 --mem-per-gpu=<size>[units]
1191 Minimum memory required per allocated GPU. Default units are
1192 megabytes. Different units can be specified using the suffix
1193 [K|M|G|T]. Default value is DefMemPerGPU and is available on
1194 both a global and per partition basis. If configured, the pa‐
1195 rameters can be seen using the scontrol show config and scontrol
1196 show partition commands. Also see --mem. The --mem,
1197 --mem-per-cpu and --mem-per-gpu options are mutually exclusive.
1198 --mincpus=<n>
1200 Specify a minimum number of logical cpus/processors per node.
1201 --network=<type>
1203 Specify information pertaining to the switch or network. The
1204 interpretation of type is system dependent. This option is sup‐
1205 ported when running Slurm on a Cray natively. It is used to re‐
1206 quest using Network Performance Counters. Only one value per
1207 request is valid. All options are case in-sensitive. In this
1208 configuration supported values include:
1209 system
1211 Use the system-wide network performance counters. Only
1212 nodes requested will be marked in use for the job alloca‐
1213 tion. If the job does not fill up the entire system the
1214 rest of the nodes are not able to be used by other jobs
1215 using NPC, if idle their state will appear as PerfCnts.
1216 These nodes are still available for other jobs not using
1217 NPC.
1218 blade Use the blade network performance counters. Only nodes re‐
1220 quested will be marked in use for the job allocation. If
1221 the job does not fill up the entire blade(s) allocated to
1222 the job those blade(s) are not able to be used by other
1223 jobs using NPC, if idle their state will appear as PerfC‐
1224 nts. These nodes are still available for other jobs not
1225 using NPC.
1226 In all cases the job allocation request must specify the --ex‐
1228 clusive option. Otherwise the request will be denied.
1229 Also with any of these options steps are not allowed to share
1231 blades, so resources would remain idle inside an allocation if
1232 the step running on a blade does not take up all the nodes on
1233 the blade.
1234 --nice[=adjustment]
1236 Run the job with an adjusted scheduling priority within Slurm.
1237 With no adjustment value the scheduling priority is decreased by
1238 100. A negative nice value increases the priority, otherwise de‐
1239 creases it. The adjustment range is +/- 2147483645. Only privi‐
1240 leged users can specify a negative adjustment.
1241 --no-bell
1243 Silence salloc's use of the terminal bell. Also see the option
1244 --bell.
1245 -k, --no-kill[=off]
1247 Do not automatically terminate a job if one of the nodes it has
1248 been allocated fails. The user will assume the responsibilities
1249 for fault-tolerance should a node fail. When there is a node
1250 failure, any active job steps (usually MPI jobs) on that node
1251 will almost certainly suffer a fatal error, but with --no-kill,
1252 the job allocation will not be revoked so the user may launch
1253 new job steps on the remaining nodes in their allocation.
1254 Specify an optional argument of "off" disable the effect of the
1256 SALLOC_NO_KILL environment variable.
1257 By default Slurm terminates the entire job allocation if any
1259 node fails in its range of allocated nodes.
1260 --no-shell
1262 immediately exit after allocating resources, without running a
1263 command. However, the Slurm job will still be created and will
1264 remain active and will own the allocated resources as long as it
1265 is active. You will have a Slurm job id with no associated pro‐
1266 cesses or tasks. You can submit srun commands against this re‐
1267 source allocation, if you specify the --jobid= option with the
1268 job id of this Slurm job. Or, this can be used to temporarily
1269 reserve a set of resources so that other jobs cannot use them
1270 for some period of time. (Note that the Slurm job is subject to
1271 the normal constraints on jobs, including time limits, so that
1272 eventually the job will terminate and the resources will be
1273 freed, or you can terminate the job manually using the scancel
1274 command.)
1275 -F, --nodefile=<node_file>
1277 Much like --nodelist, but the list is contained in a file of
1278 name node file. The node names of the list may also span multi‐
1279 ple lines in the file. Duplicate node names in the file will
1280 be ignored. The order of the node names in the list is not im‐
1281 portant; the node names will be sorted by Slurm.
1282 -w, --nodelist=<node_name_list>
1284 Request a specific list of hosts. The job will contain all of
1285 these hosts and possibly additional hosts as needed to satisfy
1286 resource requirements. The list may be specified as a
1287 comma-separated list of hosts, a range of hosts (host[1-5,7,...]
1288 for example), or a filename. The host list will be assumed to
1289 be a filename if it contains a "/" character. If you specify a
1290 minimum node or processor count larger than can be satisfied by
1291 the supplied host list, additional resources will be allocated
1292 on other nodes as needed. Duplicate node names in the list will
1293 be ignored. The order of the node names in the list is not im‐
1294 portant; the node names will be sorted by Slurm.
1295 -N, --nodes=<minnodes>[-maxnodes]
1297 Request that a minimum of minnodes nodes be allocated to this
1298 job. A maximum node count may also be specified with maxnodes.
1299 If only one number is specified, this is used as both the mini‐
1300 mum and maximum node count. The partition's node limits super‐
1301 sede those of the job. If a job's node limits are outside of
1302 the range permitted for its associated partition, the job will
1303 be left in a PENDING state. This permits possible execution at
1304 a later time, when the partition limit is changed. If a job
1305 node limit exceeds the number of nodes configured in the parti‐
1306 tion, the job will be rejected. Note that the environment vari‐
1307 able SLURM_JOB_NUM_NODES will be set to the count of nodes actu‐
1308 ally allocated to the job. See the ENVIRONMENT VARIABLES sec‐
1309 tion for more information. If -N is not specified, the default
1310 behavior is to allocate enough nodes to satisfy the requested
1311 resources as expressed by per-job specification options, e.g.
1312 -n, -c and --gpus. The job will be allocated as many nodes as
1313 possible within the range specified and without delaying the
1314 initiation of the job. The node count specification may include
1315 a numeric value followed by a suffix of "k" (multiplies numeric
1316 value by 1,024) or "m" (multiplies numeric value by 1,048,576).
1317 -n, --ntasks=<number>
1319 salloc does not launch tasks, it requests an allocation of re‐
1320 sources and executed some command. This option advises the Slurm
1321 controller that job steps run within this allocation will launch
1322 a maximum of number tasks and sufficient resources are allocated
1323 to accomplish this. The default is one task per node, but note
1324 that the --cpus-per-task option will change this default.
1325 --ntasks-per-core=<ntasks>
1327 Request the maximum ntasks be invoked on each core. Meant to be
1328 used with the --ntasks option. Related to --ntasks-per-node ex‐
1329 cept at the core level instead of the node level. NOTE: This
1330 option is not supported when using SelectType=select/linear.
1331 --ntasks-per-gpu=<ntasks>
1333 Request that there are ntasks tasks invoked for every GPU. This
1334 option can work in two ways: 1) either specify --ntasks in addi‐
1335 tion, in which case a type-less GPU specification will be auto‐
1336 matically determined to satisfy --ntasks-per-gpu, or 2) specify
1337 the GPUs wanted (e.g. via --gpus or --gres) without specifying
1338 --ntasks, and the total task count will be automatically deter‐
1339 mined. The number of CPUs needed will be automatically in‐
1340 creased if necessary to allow for any calculated task count.
1341 This option will implicitly set --gpu-bind=single:<ntasks>, but
1342 that can be overridden with an explicit --gpu-bind specifica‐
1343 tion. This option is not compatible with a node range (i.e.
1344 -N<minnodes-maxnodes>). This option is not compatible with
1345 --gpus-per-task, --gpus-per-socket, or --ntasks-per-node. This
1346 option is not supported unless SelectType=cons_tres is config‐
1347 ured (either directly or indirectly on Cray systems).
1348 --ntasks-per-node=<ntasks>
1350 Request that ntasks be invoked on each node. If used with the
1351 --ntasks option, the --ntasks option will take precedence and
1352 the --ntasks-per-node will be treated as a maximum count of
1353 tasks per node. Meant to be used with the --nodes option. This
1354 is related to --cpus-per-task=ncpus, but does not require knowl‐
1355 edge of the actual number of cpus on each node. In some cases,
1356 it is more convenient to be able to request that no more than a
1357 specific number of tasks be invoked on each node. Examples of
1358 this include submitting a hybrid MPI/OpenMP app where only one
1359 MPI "task/rank" should be assigned to each node while allowing
1360 the OpenMP portion to utilize all of the parallelism present in
1361 the node, or submitting a single setup/cleanup/monitoring job to
1362 each node of a pre-existing allocation as one step in a larger
1363 job script.
1364 --ntasks-per-socket=<ntasks>
1366 Request the maximum ntasks be invoked on each socket. Meant to
1367 be used with the --ntasks option. Related to --ntasks-per-node
1368 except at the socket level instead of the node level. NOTE:
1369 This option is not supported when using SelectType=select/lin‐
1370 ear.
1371 -O, --overcommit
1373 Overcommit resources.
1374 When applied to a job allocation (not including jobs requesting
1376 exclusive access to the nodes) the resources are allocated as if
1377 only one task per node is requested. This means that the re‐
1378 quested number of cpus per task (-c, --cpus-per-task) are allo‐
1379 cated per node rather than being multiplied by the number of
1380 tasks. Options used to specify the number of tasks per node,
1381 socket, core, etc. are ignored.
1382 When applied to job step allocations (the srun command when exe‐
1384 cuted within an existing job allocation), this option can be
1385 used to launch more than one task per CPU. Normally, srun will
1386 not allocate more than one process per CPU. By specifying
1387 --overcommit you are explicitly allowing more than one process
1388 per CPU. However no more than MAX_TASKS_PER_NODE tasks are per‐
1389 mitted to execute per node. NOTE: MAX_TASKS_PER_NODE is defined
1390 in the file slurm.h and is not a variable, it is set at Slurm
1391 build time.
1392 -s, --oversubscribe
1394 The job allocation can over-subscribe resources with other run‐
1395 ning jobs. The resources to be over-subscribed can be nodes,
1396 sockets, cores, and/or hyperthreads depending upon configura‐
1397 tion. The default over-subscribe behavior depends on system
1398 configuration and the partition's OverSubscribe option takes
1399 precedence over the job's option. This option may result in the
1400 allocation being granted sooner than if the --oversubscribe op‐
1401 tion was not set and allow higher system utilization, but appli‐
1402 cation performance will likely suffer due to competition for re‐
1403 sources. Also see the --exclusive option.
1404 -p, --partition=<partition_names>
1406 Request a specific partition for the resource allocation. If
1407 not specified, the default behavior is to allow the slurm con‐
1408 troller to select the default partition as designated by the
1409 system administrator. If the job can use more than one parti‐
1410 tion, specify their names in a comma separate list and the one
1411 offering earliest initiation will be used with no regard given
1412 to the partition name ordering (although higher priority parti‐
1413 tions will be considered first). When the job is initiated, the
1414 name of the partition used will be placed first in the job
1415 record partition string.
1416 --power=<flags>
1418 Comma separated list of power management plugin options. Cur‐
1419 rently available flags include: level (all nodes allocated to
1420 the job should have identical power caps, may be disabled by the
1421 Slurm configuration option PowerParameters=job_no_level).
1422 --prefer=<list>
1424 Nodes can have features assigned to them by the Slurm adminis‐
1425 trator. Users can specify which of these features are desired
1426 but not required by their job using the prefer option. This op‐
1427 tion operates independently from --constraint and will override
1428 whatever is set there if possible. When scheduling the features
1429 in --prefer are tried first if a node set isn't available with
1430 those features then --constraint is attempted. See --constraint
1431 for more information, this option behaves the same way.
1432 --priority=<value>
1435 Request a specific job priority. May be subject to configura‐
1436 tion specific constraints. value should either be a numeric
1437 value or "TOP" (for highest possible value). Only Slurm opera‐
1438 tors and administrators can set the priority of a job.
1439 --profile={all|none|<type>[,<type>...]}
1441 Enables detailed data collection by the acct_gather_profile
1442 plugin. Detailed data are typically time-series that are stored
1443 in an HDF5 file for the job or an InfluxDB database depending on
1444 the configured plugin.
1445 All All data types are collected. (Cannot be combined with
1447 other values.)
1448 None No data types are collected. This is the default.
1450 (Cannot be combined with other values.)
1451 Valid type values are:
1453 Energy Energy data is collected.
1455 Task Task (I/O, Memory, ...) data is collected.
1457 Lustre Lustre data is collected.
1459 Network
1461 Network (InfiniBand) data is collected.
1462 -q, --qos=<qos>
1464 Request a quality of service for the job. QOS values can be de‐
1465 fined for each user/cluster/account association in the Slurm
1466 database. Users will be limited to their association's defined
1467 set of qos's when the Slurm configuration parameter, Account‐
1468 ingStorageEnforce, includes "qos" in its definition.
1469 -Q, --quiet
1471 Suppress informational messages from salloc. Errors will still
1472 be displayed.
1473 --reboot
1475 Force the allocated nodes to reboot before starting the job.
1476 This is only supported with some system configurations and will
1477 otherwise be silently ignored. Only root, SlurmUser or admins
1478 can reboot nodes.
1479 --reservation=<reservation_names>
1481 Allocate resources for the job from the named reservation. If
1482 the job can use more than one reservation, specify their names
1483 in a comma separate list and the one offering earliest initia‐
1484 tion. Each reservation will be considered in the order it was
1485 requested. All reservations will be listed in scontrol/squeue
1486 through the life of the job. In accounting the first reserva‐
1487 tion will be seen and after the job starts the reservation used
1488 will replace it.
1489 --signal=[R:]<sig_num>[@sig_time]
1491 When a job is within sig_time seconds of its end time, send it
1492 the signal sig_num. Due to the resolution of event handling by
1493 Slurm, the signal may be sent up to 60 seconds earlier than
1494 specified. sig_num may either be a signal number or name (e.g.
1495 "10" or "USR1"). sig_time must have an integer value between 0
1496 and 65535. By default, no signal is sent before the job's end
1497 time. If a sig_num is specified without any sig_time, the de‐
1498 fault time will be 60 seconds. Use the "R:" option to allow
1499 this job to overlap with a reservation with MaxStartDelay set.
1500 To have the signal sent at preemption time see the pre‐
1501 empt_send_user_signal SlurmctldParameter.
1502 --sockets-per-node=<sockets>
1504 Restrict node selection to nodes with at least the specified
1505 number of sockets. See additional information under -B option
1506 above when task/affinity plugin is enabled.
1507 NOTE: This option may implicitly set the number of tasks (if -n
1508 was not specified) as one task per requested thread.
1509 --spread-job
1511 Spread the job allocation over as many nodes as possible and at‐
1512 tempt to evenly distribute tasks across the allocated nodes.
1513 This option disables the topology/tree plugin.
1514 --switches=<count>[@max-time]
1516 When a tree topology is used, this defines the maximum count of
1517 leaf switches desired for the job allocation and optionally the
1518 maximum time to wait for that number of switches. If Slurm finds
1519 an allocation containing more switches than the count specified,
1520 the job remains pending until it either finds an allocation with
1521 desired switch count or the time limit expires. It there is no
1522 switch count limit, there is no delay in starting the job. Ac‐
1523 ceptable time formats include "minutes", "minutes:seconds",
1524 "hours:minutes:seconds", "days-hours", "days-hours:minutes" and
1525 "days-hours:minutes:seconds". The job's maximum time delay may
1526 be limited by the system administrator using the SchedulerParam‐
1527 eters configuration parameter with the max_switch_wait parameter
1528 option. On a dragonfly network the only switch count supported
1529 is 1 since communication performance will be highest when a job
1530 is allocate resources on one leaf switch or more than 2 leaf
1531 switches. The default max-time is the max_switch_wait Sched‐
1532 ulerParameters.
1533 --thread-spec=<num>
1535 Count of specialized threads per node reserved by the job for
1536 system operations and not used by the application. The applica‐
1537 tion will not use these threads, but will be charged for their
1538 allocation. This option can not be used with the --core-spec
1539 option.
1540 NOTE: Explicitly setting a job's specialized thread value im‐
1542 plicitly sets its --exclusive option, reserving entire nodes for
1543 the job.
1544 --threads-per-core=<threads>
1546 Restrict node selection to nodes with at least the specified
1547 number of threads per core. In task layout, use the specified
1548 maximum number of threads per core. NOTE: "Threads" refers to
1549 the number of processing units on each core rather than the num‐
1550 ber of application tasks to be launched per core. See addi‐
1551 tional information under -B option above when task/affinity
1552 plugin is enabled.
1553 NOTE: This option may implicitly set the number of tasks (if -n
1554 was not specified) as one task per requested thread.
1555 -t, --time=<time>
1557 Set a limit on the total run time of the job allocation. If the
1558 requested time limit exceeds the partition's time limit, the job
1559 will be left in a PENDING state (possibly indefinitely). The
1560 default time limit is the partition's default time limit. When
1561 the time limit is reached, each task in each job step is sent
1562 SIGTERM followed by SIGKILL. The interval between signals is
1563 specified by the Slurm configuration parameter KillWait. The
1564 OverTimeLimit configuration parameter may permit the job to run
1565 longer than scheduled. Time resolution is one minute and second
1566 values are rounded up to the next minute.
1567 A time limit of zero requests that no time limit be imposed.
1569 Acceptable time formats include "minutes", "minutes:seconds",
1570 "hours:minutes:seconds", "days-hours", "days-hours:minutes" and
1571 "days-hours:minutes:seconds".
1572 --time-min=<time>
1574 Set a minimum time limit on the job allocation. If specified,
1575 the job may have its --time limit lowered to a value no lower
1576 than --time-min if doing so permits the job to begin execution
1577 earlier than otherwise possible. The job's time limit will not
1578 be changed after the job is allocated resources. This is per‐
1579 formed by a backfill scheduling algorithm to allocate resources
1580 otherwise reserved for higher priority jobs. Acceptable time
1581 formats include "minutes", "minutes:seconds", "hours:min‐
1582 utes:seconds", "days-hours", "days-hours:minutes" and
1583 "days-hours:minutes:seconds".
1584 --tmp=<size>[units]
1586 Specify a minimum amount of temporary disk space per node. De‐
1587 fault units are megabytes. Different units can be specified us‐
1588 ing the suffix [K|M|G|T].
1589 --uid=<user>
1591 Attempt to submit and/or run a job as user instead of the invok‐
1592 ing user id. The invoking user's credentials will be used to
1593 check access permissions for the target partition. This option
1594 is only valid for user root. This option may be used by user
1595 root may use this option to run jobs as a normal user in a
1596 RootOnly partition for example. If run as root, salloc will drop
1597 its permissions to the uid specified after node allocation is
1598 successful. user may be the user name or numerical user ID.
1599 --usage
1601 Display brief help message and exit.
1602 --use-min-nodes
1604 If a range of node counts is given, prefer the smaller count.
1605 -v, --verbose
1607 Increase the verbosity of salloc's informational messages. Mul‐
1608 tiple -v's will further increase salloc's verbosity. By default
1609 only errors will be displayed.
1610 -V, --version
1612 Display version information and exit.
1613 --wait-all-nodes=<value>
1615 Controls when the execution of the command begins with respect
1616 to when nodes are ready for use (i.e. booted). By default, the
1617 salloc command will return as soon as the allocation is made.
1618 This default can be altered using the salloc_wait_nodes option
1619 to the SchedulerParameters parameter in the slurm.conf file.
1620 0 Begin execution as soon as allocation can be made. Do not
1622 wait for all nodes to be ready for use (i.e. booted).
1623 1 Do not begin execution until all nodes are ready for use.
1625 --wckey=<wckey>
1627 Specify wckey to be used with job. If TrackWCKey=no (default)
1628 in the slurm.conf this value is ignored.
1629 --x11[={all|first|last}]
1631 Sets up X11 forwarding on "all", "first" or "last" node(s) of
1632 the allocation. This option is only enabled if Slurm was com‐
1633 piled with X11 support and PrologFlags=x11 is defined in the
1634 slurm.conf. Default is "all".
1635
1637 Executing salloc sends a remote procedure call to slurmctld. If enough
1638 calls from salloc or other Slurm client commands that send remote pro‐
1639 cedure calls to the slurmctld daemon come in at once, it can result in
1640 a degradation of performance of the slurmctld daemon, possibly result‐
1641 ing in a denial of service.
1642 Do not run salloc or other Slurm client commands that send remote pro‐
1644 cedure calls to slurmctld from loops in shell scripts or other pro‐
1645 grams. Ensure that programs limit calls to salloc to the minimum neces‐
1646 sary for the information you are trying to gather.
1647
1650 Upon startup, salloc will read and handle the options set in the fol‐
1651 lowing environment variables. The majority of these variables are set
1652 the same way the options are set, as defined above. For flag options
1653 that are defined to expect no argument, the option can be enabled by
1654 setting the environment variable without a value (empty or NULL
1655 string), the string 'yes', or a non-zero number. Any other value for
1656 the environment variable will result in the option not being set.
1657 There are a couple exceptions to these rules that are noted below.
1658 NOTE: Command line options always override environment variables set‐
1659 tings.
1660 SALLOC_ACCOUNT Same as -A, --account
1663 SALLOC_ACCTG_FREQ Same as --acctg-freq
1665 SALLOC_BELL Same as --bell
1667 SALLOC_BURST_BUFFER Same as --bb
1669 SALLOC_CLUSTERS or SLURM_CLUSTERS
1671 Same as --clusters
1672 SALLOC_CONSTRAINT Same as -C, --constraint
1674 SALLOC_CONTAINER Same as --container.
1676 SALLOC_CORE_SPEC Same as --core-spec
1678 SALLOC_CPUS_PER_GPU Same as --cpus-per-gpu
1680 SALLOC_DEBUG Same as -v, --verbose. Must be set to 0 or 1 to
1682 disable or enable the option.
1683 SALLOC_DELAY_BOOT Same as --delay-boot
1685 SALLOC_EXCLUSIVE Same as --exclusive
1687 SALLOC_GPU_BIND Same as --gpu-bind
1689 SALLOC_GPU_FREQ Same as --gpu-freq
1691 SALLOC_GPUS Same as -G, --gpus
1693 SALLOC_GPUS_PER_NODE Same as --gpus-per-node
1695 SALLOC_GPUS_PER_TASK Same as --gpus-per-task
1697 SALLOC_GRES Same as --gres
1699 SALLOC_GRES_FLAGS Same as --gres-flags
1701 SALLOC_HINT or SLURM_HINT
1703 Same as --hint
1704 SALLOC_IMMEDIATE Same as -I, --immediate
1706 SALLOC_KILL_CMD Same as -K, --kill-command
1708 SALLOC_MEM_BIND Same as --mem-bind
1710 SALLOC_MEM_PER_CPU Same as --mem-per-cpu
1712 SALLOC_MEM_PER_GPU Same as --mem-per-gpu
1714 SALLOC_MEM_PER_NODE Same as --mem
1716 SALLOC_NETWORK Same as --network
1718 SALLOC_NO_BELL Same as --no-bell
1720 SALLOC_NO_KILL Same as -k, --no-kill
1722 SALLOC_OVERCOMMIT Same as -O, --overcommit
1724 SALLOC_PARTITION Same as -p, --partition
1726 SALLOC_POWER Same as --power
1728 SALLOC_PROFILE Same as --profile
1730 SALLOC_QOS Same as --qos
1732 SALLOC_REQ_SWITCH When a tree topology is used, this defines the
1734 maximum count of switches desired for the job al‐
1735 location and optionally the maximum time to wait
1736 for that number of switches. See --switches.
1737 SALLOC_RESERVATION Same as --reservation
1739 SALLOC_SIGNAL Same as --signal
1741 SALLOC_SPREAD_JOB Same as --spread-job
1743 SALLOC_THREAD_SPEC Same as --thread-spec
1745 SALLOC_THREADS_PER_CORE
1747 Same as --threads-per-core
1748 SALLOC_TIMELIMIT Same as -t, --time
1750 SALLOC_USE_MIN_NODES Same as --use-min-nodes
1752 SALLOC_WAIT_ALL_NODES Same as --wait-all-nodes. Must be set to 0 or 1
1754 to disable or enable the option.
1755 SALLOC_WAIT4SWITCH Max time waiting for requested switches. See
1757 --switches
1758 SALLOC_WCKEY Same as --wckey
1760 SLURM_CONF The location of the Slurm configuration file.
1762 SLURM_DEBUG_FLAGS Specify debug flags for salloc to use. See De‐
1764 bugFlags in the slurm.conf(5) man page for a full
1765 list of flags. The environment variable takes
1766 precedence over the setting in the slurm.conf.
1767 SLURM_EXIT_ERROR Specifies the exit code generated when a Slurm
1769 error occurs (e.g. invalid options). This can be
1770 used by a script to distinguish application exit
1771 codes from various Slurm error conditions. Also
1772 see SLURM_EXIT_IMMEDIATE.
1773 SLURM_EXIT_IMMEDIATE Specifies the exit code generated when the --im‐
1775 mediate option is used and resources are not cur‐
1776 rently available. This can be used by a script
1777 to distinguish application exit codes from vari‐
1778 ous Slurm error conditions. Also see
1779 SLURM_EXIT_ERROR.
1780
1782 salloc will set the following environment variables in the environment
1783 of the executed program:
1784 SLURM_*_HET_GROUP_#
1786 For a heterogeneous job allocation, the environment variables
1787 are set separately for each component.
1788 SLURM_CLUSTER_NAME
1790 Name of the cluster on which the job is executing.
1791 SLURM_CONTAINER
1793 OCI Bundle for job. Only set if --container is specified.
1794 SLURM_CPUS_PER_GPU
1796 Number of CPUs requested per allocated GPU. Only set if the
1797 --cpus-per-gpu option is specified.
1798 SLURM_CPUS_PER_TASK
1800 Number of CPUs requested per task. Only set if the
1801 --cpus-per-task option is specified.
1802 SLURM_DIST_PLANESIZE
1804 Plane distribution size. Only set for plane distributions. See
1805 -m, --distribution.
1806 SLURM_DISTRIBUTION
1808 Only set if the -m, --distribution option is specified.
1809 SLURM_GPU_BIND
1811 Requested binding of tasks to GPU. Only set if the --gpu-bind
1812 option is specified.
1813 SLURM_GPU_FREQ
1815 Requested GPU frequency. Only set if the --gpu-freq option is
1816 specified.
1817 SLURM_GPUS
1819 Number of GPUs requested. Only set if the -G, --gpus option is
1820 specified.
1821 SLURM_GPUS_PER_NODE
1823 Requested GPU count per allocated node. Only set if the
1824 --gpus-per-node option is specified.
1825 SLURM_GPUS_PER_SOCKET
1827 Requested GPU count per allocated socket. Only set if the
1828 --gpus-per-socket option is specified.
1829 SLURM_GPUS_PER_TASK
1831 Requested GPU count per allocated task. Only set if the
1832 --gpus-per-task option is specified.
1833 SLURM_HET_SIZE
1835 Set to count of components in heterogeneous job.
1836 SLURM_JOB_ACCOUNT
1838 Account name associated of the job allocation.
1839 SLURM_JOB_ID
1841 The ID of the job allocation.
1842 SLURM_JOB_CPUS_PER_NODE
1844 Count of CPUs available to the job on the nodes in the alloca‐
1845 tion, using the format CPU_count[(xnumber_of_nodes)][,CPU_count
1846 [(xnumber_of_nodes)] ...]. For example:
1847 SLURM_JOB_CPUS_PER_NODE='72(x2),36' indicates that on the first
1848 and second nodes (as listed by SLURM_JOB_NODELIST) the alloca‐
1849 tion has 72 CPUs, while the third node has 36 CPUs. NOTE: The
1850 select/linear plugin allocates entire nodes to jobs, so the
1851 value indicates the total count of CPUs on allocated nodes. The
1852 select/cons_res and select/cons_tres plugins allocate individual
1853 CPUs to jobs, so this number indicates the number of CPUs allo‐
1854 cated to the job.
1855 SLURM_JOB_GPUS
1857 The global GPU IDs of the GPUs allocated to this job. The GPU
1858 IDs are not relative to any device cgroup, even if devices are
1859 constrained with task/cgroup. Only set in batch and interactive
1860 jobs.
1861 SLURM_JOB_NODELIST
1863 List of nodes allocated to the job.
1864 SLURM_JOB_NUM_NODES
1866 Total number of nodes in the job allocation.
1867 SLURM_JOB_PARTITION
1869 Name of the partition in which the job is running.
1870 SLURM_JOB_QOS
1872 Quality Of Service (QOS) of the job allocation.
1873 SLURM_JOB_RESERVATION
1875 Advanced reservation containing the job allocation, if any.
1876 SLURM_JOBID
1878 The ID of the job allocation. See SLURM_JOB_ID. Included for
1879 backwards compatibility.
1880 SLURM_MEM_BIND
1882 Set to value of the --mem-bind option.
1883 SLURM_MEM_BIND_LIST
1885 Set to bit mask used for memory binding.
1886 SLURM_MEM_BIND_PREFER
1888 Set to "prefer" if the --mem-bind option includes the prefer op‐
1889 tion.
1890 SLURM_MEM_BIND_SORT
1892 Sort free cache pages (run zonesort on Intel KNL nodes)
1893 SLURM_MEM_BIND_TYPE
1895 Set to the memory binding type specified with the --mem-bind op‐
1896 tion. Possible values are "none", "rank", "map_map", "mask_mem"
1897 and "local".
1898 SLURM_MEM_BIND_VERBOSE
1900 Set to "verbose" if the --mem-bind option includes the verbose
1901 option. Set to "quiet" otherwise.
1902 SLURM_MEM_PER_CPU
1904 Same as --mem-per-cpu
1905 SLURM_MEM_PER_GPU
1907 Requested memory per allocated GPU. Only set if the
1908 --mem-per-gpu option is specified.
1909 SLURM_MEM_PER_NODE
1911 Same as --mem
1912 SLURM_NNODES
1914 Total number of nodes in the job allocation. See
1915 SLURM_JOB_NUM_NODES. Included for backwards compatibility.
1916 SLURM_NODELIST
1918 List of nodes allocated to the job. See SLURM_JOB_NODELIST. In‐
1919 cluded for backwards compabitility.
1920 SLURM_NODE_ALIASES
1922 Sets of node name, communication address and hostname for nodes
1923 allocated to the job from the cloud. Each element in the set if
1924 colon separated and each set is comma separated. For example:
1925 SLURM_NODE_ALIASES=ec0:1.2.3.4:foo,ec1:1.2.3.5:bar
1926 SLURM_NTASKS
1928 Same as -n, --ntasks
1929 SLURM_NTASKS_PER_CORE
1931 Set to value of the --ntasks-per-core option, if specified.
1932 SLURM_NTASKS_PER_GPU
1934 Set to value of the --ntasks-per-gpu option, if specified.
1935 SLURM_NTASKS_PER_NODE
1937 Set to value of the --ntasks-per-node option, if specified.
1938 SLURM_NTASKS_PER_SOCKET
1940 Set to value of the --ntasks-per-socket option, if specified.
1941 SLURM_OVERCOMMIT
1943 Set to 1 if --overcommit was specified.
1944 SLURM_PROFILE
1946 Same as --profile
1947 SLURM_SHARDS_ON_NODE
1949 Number of GPU Shards available to the step on this node.
1950 SLURM_SUBMIT_DIR
1952 The directory from which salloc was invoked or, if applicable,
1953 the directory specified by the -D, --chdir option.
1954 SLURM_SUBMIT_HOST
1956 The hostname of the computer from which salloc was invoked.
1957 SLURM_TASKS_PER_NODE
1959 Number of tasks to be initiated on each node. Values are comma
1960 separated and in the same order as SLURM_JOB_NODELIST. If two
1961 or more consecutive nodes are to have the same task count, that
1962 count is followed by "(x#)" where "#" is the repetition count.
1963 For example, "SLURM_TASKS_PER_NODE=2(x3),1" indicates that the
1964 first three nodes will each execute two tasks and the fourth
1965 node will execute one task.
1966 SLURM_THREADS_PER_CORE
1968 This is only set if --threads-per-core or SAL‐
1969 LOC_THREADS_PER_CORE were specified. The value will be set to
1970 the value specified by --threads-per-core or SAL‐
1971 LOC_THREADS_PER_CORE. This is used by subsequent srun calls
1972 within the job allocation.
1973
1975 While salloc is waiting for a PENDING job allocation, most signals will
1976 cause salloc to revoke the allocation request and exit.
1977 However if the allocation has been granted and salloc has already
1979 started the specified command, then salloc will ignore most signals.
1980 salloc will not exit or release the allocation until the command exits.
1981 One notable exception is SIGHUP. A SIGHUP signal will cause salloc to
1982 release the allocation and exit without waiting for the command to fin‐
1983 ish. Another exception is SIGTERM, which will be forwarded to the
1984 spawned process.
1985
1988 To get an allocation, and open a new xterm in which srun commands may
1989 be typed interactively:
1990 $ salloc -N16 xterm
1992 salloc: Granted job allocation 65537
1993 # (at this point the xterm appears, and salloc waits for xterm to exit)
1994 salloc: Relinquishing job allocation 65537
1995 To grab an allocation of nodes and launch a parallel application on one
1998 command line:
1999 $ salloc -N5 srun -n10 myprogram
2001 To create a heterogeneous job with 3 components, each allocating a
2004 unique set of nodes:
2005 $ salloc -w node[2-3] : -w node4 : -w node[5-7] bash
2007 salloc: job 32294 queued and waiting for resources
2008 salloc: job 32294 has been allocated resources
2009 salloc: Granted job allocation 32294
2010
2013 Copyright (C) 2006-2007 The Regents of the University of California.
2014 Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
2015 Copyright (C) 2008-2010 Lawrence Livermore National Security.
2016 Copyright (C) 2010-2022 SchedMD LLC.
2017 This file is part of Slurm, a resource management program. For de‐
2019 tails, see <https://slurm.schedmd.com/>.
2020 Slurm is free software; you can redistribute it and/or modify it under
2022 the terms of the GNU General Public License as published by the Free
2023 Software Foundation; either version 2 of the License, or (at your op‐
2024 tion) any later version.
2025 Slurm is distributed in the hope that it will be useful, but WITHOUT
2027 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
2028 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
2029 for more details.
2030
2033 sinfo(1), sattach(1), sbatch(1), squeue(1), scancel(1), scontrol(1),
2034 slurm.conf(5), sched_setaffinity (2), numa (3)
2035September 2022 Slurm Commands salloc(1)