1sacct(1) Slurm Commands sacct(1)
2
6 sacct - displays accounting data for all jobs and job steps in the
7 Slurm job accounting log or Slurm database
8
11 sacct [OPTIONS...]
12
15 Accounting information for jobs invoked with Slurm are either logged in
16 the job accounting log file or saved to the Slurm database.
17 The sacct command displays job accounting data stored in the job
19 accounting log file or Slurm database in a variety of forms for your
20 analysis. The sacct command displays information on jobs, job steps,
21 status, and exitcodes by default. You can tailor the output with the
22 use of the --format= option to specify the fields to be shown.
23 For the root user, the sacct command displays job accounting data for
25 all users, although there are options to filter the output to report
26 only the jobs from a specified user or group.
27 For the non-root user, the sacct command limits the display of job
29 accounting data to jobs that were launched with their own user identi‐
30 fier (UID) by default. Data for other users can be displayed with the
31 --allusers, --user, or --uid options.
32 Note: If designated, the slurmdbd.conf option PrivateData may further
34 restrict the accounting data visible to users which are not
35 SlurmUser, root, or a user with AdminLevel=Admin. See the slur‐
36 mdbd.conf man page for additional details on restricting access
37 to accounting data.
38 Note: If the AccountingStorageType is set to "accounting_stor‐
40 age/filetxt", space characters embedded within account names,
41 job names, and step names will be replaced by underscores. If
42 account names with embedded spaces are needed, it is recommended
43 that a database type of accounting storage be configured.
44 Note: The content's of Slurm's database are maintained in lower case.
46 This may result in some sacct output differing from that of
47 other Slurm commands.
48 Note: Much of the data reported by sacct has been generated by the
50 wait3() and getrusage() system calls. Some systems gather and
51 report incomplete information for these calls; sacct reports
52 values of 0 for this missing data. See your systems getrusage
53 [4m(3) man page for information about which data are actually
54 available on your system.
55 Elapsed time fields are presented as [days-]hours:minutes:sec‐
57 onds[.microseconds]. Only 'CPU' fields will ever have microsec‐
58 onds.
59 The default input file is the file named in the AccountingStor‐
61 ageLoc parameter in slurm.conf.
62
65 -a, --allusers
66 Displays all users jobs when run by user root or if Private‐
67 Data is not configured to jobs. Otherwise display the cur‐
68 rent user's jobs
69 -A account_list , --accounts=account_list
71 Displays jobs when a comma separated list of accounts are
72 given as the argument.
73 -b, --brief
75 Displays a brief listing, which includes the following data:
76 jobid
78 status
80 exitcode
82 -c, --completion
84 Use job completion data instead of job accounting. The Job‐
85 CompType parameter in the slurm.conf file must be defined to
86 a non-none option. Does not support federated cluster infor‐
87 mation (local data only).
88 --delimiter=characters
90 ASCII characters used to separate the fields when specifying
91 the -p or -P options. The default delimiter is a '|'. This
92 options is ignored if -p or -P options are not specified.
93 -D, --duplicates
96 If Slurm job ids are reset, some job numbers will probably
97 appear more than once in the accounting log file but refer to
98 different jobs. Such jobs can be distinguished by the "sub‐
99 mit" time stamp in the data records.
100 When data for specific jobs are requested with the --jobs
103 option, sacct returns the most recent job with that number.
104 This behavior can be overridden by specifying --duplicates,
105 in which case all records that match the selection criteria
106 will be returned.
107 NOTE: Revoked federated sibling jobs are hidden unless the
110 --duplicates option is specified.
111 -e, --helpformat
114 Print a list of fields that can be specified with the --for‐
116 mat option.
117 Fields available:
119 Account AdminComment AllocCPUS AllocGRES
121 AllocNodes AllocTRES AssocID AveCPU
122 AveCPUFreq AveDiskRead AveDiskWrite AvePages
123 AveRSS AveVMSize BlockID Cluster
124 Comment ConsumedEnergy ConsumedEnergyRaw CPUTime
125 CPUTimeRAW DerivedExitCode Elapsed ElapsedRaw
126 Eligible End ExitCode GID
127 Group JobID JobIDRaw JobName
128 Layout MaxDiskRead MaxDiskReadNode MaxDiskReadTask
129 MaxDiskWrite MaxDiskWriteNode MaxDiskWriteTask MaxPages
130 MaxPagesNode MaxPagesTask MaxRSS MaxRSSNode
131 MaxRSSTask MaxVMSize MaxVMSizeNode MaxVMSizeTask
132 McsLabel MinCPU MinCPUNode MinCPUTask
133 NCPUS NNodes NodeList NTasks
134 Priority Partition QOS QOSRAW
135 ReqCPUFreq ReqCPUFreqMin ReqCPUFreqMax ReqCPUFreqGov
136 ReqCPUS ReqGRES ReqMem ReqNodes
137 ReqTRES Reservation ReservationId Reserved
138 ResvCPU ResvCPURAW Start State
139 Submit Suspended SystemCPU SystemComment
140 Timelimit TimelimitRaw TotalCPU TRESUsageInAve
141 TRESUsageInMax TRESUsageInMaxNode TRESUsageInMaxTask TRESUsageInMin
142 TRESUsageInMinNode TRESUsageInMinTask TRESUsageInTot TRESUsageOutAve
143 TRESUsageOutMax TRESUsageOutMaxNode TRESUsageOutMaxTask TRESUsageOutMin
144 TRESUsageOutMinNode TRESUsageOutMinTask TRESUsageOutTot UID
145 User UserCPU WCKey WCKeyID
146 WorkDir
147 NOTE: When using with Ave[RSS|VM]Size or their values in
149 TRESUsageIn[Ave|Tot]. They represent the average/total of the highest
150 watermarks over all ranks in the step. When using sstat they represent the
151 average/total at the moment the command was ran.
152 NOTE: TRESUsage*Min* values represent the lowest highwater mark in the step.
154 The section titled "Job Accounting Fields" describes these
157 fields.
158 -E end_time, --endtime=end_time
161 Select eligible jobs in any state before the specified time.
163 If states are given with the -s option return jobs in this
164 state before this period. See the DEFAULT TIME WINDOW for
165 more details.
166 Valid time formats are...
168 HH:MM[:SS] [AM|PM]
170 MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
171 MM/DD[/YY]-HH:MM[:SS]
172 YYYY-MM-DD[THH:MM[:SS]]
173 --federation
175 Show jobs from the federation if a member of one.
176 -f file, --file=file
179 Causes the sacct command to read job accounting data from the
180 named file instead of the current Slurm job accounting log
181 file. Only applicable when running the filetxt plugin.
182 -g gid_list, --gid=gid_list --group=group_list
185 Displays the statistics only for the jobs started with the
186 GID or the GROUP specified by the gid_list or thegroup_list
187 operand, which is a comma-separated list. Space characters
188 are not allowed. Default is no restrictions..
189 -h, --help
192 Displays a general help message.
193 -i, --nnodes=N
196 Return jobs which ran on this many nodes (N = min[-max])
197 -j job(.step) , --jobs=job(.step)
200 Displays information about the specified job(.step) or list
201 of job(.step)s.
202 The job(.step) parameter is a comma-separated list of jobs.
204 Space characters are not permitted in this list. NOTE: A
205 step id of 'batch' will display the information about the
206 batch step. The batch step information is only available
207 after the batch job is complete unlike regular steps which
208 are available when they start.
209 The default is to display information on all jobs. See the
211 DEFAULT TIME WINDOW for details about how this option changes
212 the default -S and -E options.
213 -k, --timelimit-min
216 Only send data about jobs with this timelimit. If used with
217 timelimit_max this will be the minimum timelimit of the
218 range. Default is no restriction.
219 -K, --timelimit-max
222 Ignored by itself, but if timelimit_min is set this will be
223 the maximum timelimit of the range. Default is no restric‐
224 tion.
225 --local Show only jobs local to this cluster. Ignore other clusters
228 in this federation (if any). Overrides --federation.
229 -l, --long
232 Equivalent to specifying:
233 --format=jobid,jobname,partition,maxvmsize,maxvmsizen‐
235 ode,maxvmsizetask, avevm‐
236 size,maxrss,maxrssnode,maxrsstask,averss,maxpages,max‐
237 pagesnode, maxpagestask,avepages,mincpu,mincpun‐
238 ode,mincputask,avecpu,ntasks, alloccpus,elapsed,state,exit‐
239 code,maxdiskread,maxdiskreadnode,maxdiskreadtask,
240 avediskread,maxdiskwrite,maxdiskwritenode,maxdiskwrite‐
241 task,avediskwrite, allocgres,reqgres,avecpufreq,reqcpufre‐
242 qmin,reqcpufreqmax,reqcpufreqgov, tresusageinmax,tresusagein‐
243 maxn,tresusageinmaxt,tresusageoutmax,tresusageoutmaxn, tre‐
244 susageoutmaxt,tresusageoutave,tresusageinave
245 -L, --allclusters
248 Display jobs ran on all clusters. By default, only jobs ran
249 on the cluster from where sacct is called are displayed.
250 -M cluster_list, --clusters=cluster_list
253 Displays the statistics only for the jobs started on the
254 clusters specified by the cluster_list operand, which is a
255 comma-separated list of clusters. Space characters are not
256 allowed in the cluster_list. A value of 'all' will query to
257 run on all clusters. The default is current cluster you are
258 executing the sacct command on or all clusters in the federa‐
259 tion when executed on a federated cluster. This option
260 implicitly sets the --local option.
261 -n, --noheader
264 No heading will be added to the output. The default action is
265 to display a header.
266 --noconvert
268 Don't convert units from their original type (e.g. 2048M
269 won't be converted to 2G).
270 -N node_list, --nodelist=node_list
272 Display jobs that ran on any of these node(s). node_list can
273 be a ranged string.
274 --name=jobname_list
276 Display jobs that have any of these name(s).
277 -o, --format
279 Comma separated list of fields. (use "--helpformat" for a
280 list of available fields).
281 NOTE: When using the format option for listing various fields
283 you can put a %NUMBER afterwards to specify how many charac‐
284 ters should be printed.
285 e.g. format=name%30 will print 30 characters of field name
287 right justified. A %-30 will print 30 characters left justi‐
288 fied.
289 When set, the SACCT_FORMAT environment variable will override
291 the default format. For example:
292 SACCT_FORMAT="jobid,user,account,cluster"
294 -p, --parsable
297 output will be '|' delimited with a '|' at the end
298 -P, --parsable2
301 output will be '|' delimited without a '|' at the end
302 -q, --qos Only send data about jobs using these qos. Default is all.
305 -r, --partition
308 Comma separated list of partitions to select jobs and job
310 steps from. The default is all partitions.
311 -s state_list , --state=state_list
314 Selects jobs based on their state during the time period
315 given. Unless otherwise specified, the start and end time
316 will be the current time when the --state option is specified
317 and only currently running jobs can be displayed. A start
318 and/or end time must be specified to view information about
319 jobs not currently running. See the JOB STATE CODES section
320 below for a list of state designators. Multiple state names
321 may be specified using comma separators. Either the short or
322 long form of the state name may be used (e.g. CA or CAN‐
323 CELLED) and the name is case insensitive (e.g. ca and CA both
324 work).
325 NOTE: Note for a job to be selected in the PENDING state it
327 must have "EligibleTime" in the requested time interval or
328 different from "Unknown". The "EligibleTime" is displayed by
329 the "scontrol show job" command. For example jobs submitted
330 with the "--hold" option will have "EligibleTime=Unknown" as
331 they are pending indefinitely.
332 NOTE: When specifying states and no start time is given the
334 default start time is 'now'. This is only when -j is not
335 used. If -j is used the start time will default to 'Epoch'.
336 In both cases if no end time is given it will default to
337 'now'. See the DEFAULT TIME WINDOW for more details.
338 -S, --starttime
341 Select eligible jobs in any state after the specified time.
342 Default is 00:00:00 of the current day, unless the '-s' or
343 '-j' options are used. If the '-s' option is used, then the
344 default is 'now'. If states are given with the '-s' option
345 then only jobs in this state at this time will be returned.
346 If the '-j' option is used, then the default time is Unix
347 Epoch 0. See the DEFAULT TIME WINDOW for more details.
348 Valid time formats are...
350 HH:MM[:SS] [AM|PM]
352 MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
353 MM/DD[/YY]-HH:MM[:SS]
354 YYYY-MM-DD[THH:MM[:SS]]
355 -T, --truncate
358 Truncate time. So if a job started before --starttime the
359 start time would be truncated to --starttime. The same for
360 end time and --endtime.
361 -u uid_list, --uid=uid_list, --user=user_list
364 Use this comma separated list of uids or user names to select
365 jobs to display. By default, the running user's uid is used.
366 --units=[KMGTP]
369 Display values in specified unit type. Takes precedence over
370 --noconvert option.
371 --usage Display a command usage summary.
374 -v, --verbose
377 Primarily for debugging purposes, report the state of various
378 variables during processing.
379 -V, --version
382 Print version.
383 --whole-hetjob=[yes|no]
386 When querying heterogeneous jobs, Slurm by default retrieves
387 the information about all the components of the job if the
388 pack_job_id (leader) is selected. If a non-leader hetjob com‐
389 ponent is selected only that component is retrieved by
390 default. This behavior can be changed by using this option.
391 If set to 'yes' or no value is set, then information about
392 all the components will be retrieved no matter which compo‐
393 nent is selected in the job filter. Otherwise, if set to 'no'
394 then only the selected hetjob components will be retrieved,
395 even when selecting the leader.
396 -W wckey_list, --wckeys=wckey_list
399 Displays the statistics only for the jobs started on the wck‐
400 eys specified by the wckey_list operand, which is a
401 comma-separated list of wckey names. Space characters are
402 not allowed in the wckey_list. Default is all wckeys.
403 -x associd_list, --associations=assoc_list
406 Displays the statistics only for the jobs running under the
407 association ids specified by the assoc_list operand, which is
408 a comma-separated list of association ids. Space characters
409 are not allowed in the assoc_list. Default is all associa‐
410 tions.
411 -X, --allocations
414 Only show statistics relevant to the job allocation itself,
415 not taking steps into consideration.
416 Job Accounting Fields
419 The following describes each job accounting field:
420 ALL Print all fields listed below.
422 AllocCPUs Count of allocated CPUs. Equivalent to NCPUS.
425 AllocGRES Names and counts of generic resources allocated.
428 AllocNodes
431 Number of nodes allocated to the job/step. 0 if the
432 job is pending.
433 AllocTres Trackable resources. These are the resources allocated
436 to the job/step after the job started running. For
437 pending jobs this should be blank. For more details
438 see AccountingStorageTRES in slurm.conf.
439 Account Account the job ran under.
442 AssocID Reference to the association of user, account and
445 cluster.
446 AveCPU Average (system + user) CPU time of all tasks in job.
449 AveCPUFreq
452 Average weighted CPU frequency of all tasks in job, in
453 kHz.
454 AveDiskRead
457 Average number of bytes read by all tasks in job.
458 AveDiskWrite
461 Average number of bytes written by all tasks in job.
462 AvePages Average number of page faults of all tasks in job.
465 AveRSS Average resident set size of all tasks in job.
468 AveVMSize Average Virtual Memory size of all tasks in job.
471 Cluster Cluster name.
474 Comment The job's comment string when the AccountingStoreJob‐
477 Comment parameter in the slurm.conf file is set (or
478 defaults) to YES. The Comment string can be modified
479 by invoking sacctmgr modify job or the specialized
480 sjobexitmod command.
481 ConsumedEnergy
484 Total energy consumed by all tasks in job, in joules.
485 Note: Only in case of exclusive job allocation this
486 value reflects the jobs' real energy consumption.
487 CPUTime Formatted (Elapsed time * CPU) count used by a job or
490 step.
491 CPUTimeRAW
494 Unlike above non formatted (Elapsed time * CPU) count
495 for a job or step. Units are cpu-seconds.
496 DerivedExitCode
499 The highest exit code returned by the job's job steps
500 (srun invocations). Following the colon is the signal
501 that caused the process to terminate if it was termi‐
502 nated by a signal. The DerivedExitCode can be modi‐
503 fied by invoking sacctmgr modify job or the special‐
504 ized sjobexitmod command.
505 Elapsed The jobs elapsed time.
508 The format of this fields output is as follows:
510 [DD-[HH:]]MM:SS
511 as defined by the following:
513 DD days
515 hh hours
517 mm minutes
519 ss seconds
521 Eligible When the job became eligible to run.
524 End Termination time of the job. Format output is,
527 YYYY-MM-DDTHH:MM:SS, unless changed through the
528 SLURM_TIME_FORMAT environment variable.
529 ExitCode The exit code returned by the job script or salloc,
532 typically as set by the exit() function. Following
533 the colon is the signal that caused the process to
534 terminate if it was terminated by a signal.
535 GID The group identifier of the user who ran the job.
538 Group The group name of the user who ran the job.
541 JobID The number of the job or job step. It is in the form:
544 job.jobstep.
545 JobIDRaw In case of job array print the JobId instead of the
548 ArrayJobId. For non job arrays the output is the
549 JobId in the format job.jobstep.
550 JobName The name of the job or job step. The slurm_account‐
553 ing.log file is a space delimited file. Because of
554 this if a space is used in the jobname an underscore
555 is substituted for the space before the record is
556 written to the accounting file. So when the jobname is
557 displayed by sacct the jobname that had a space in it
558 will now have an underscore in place of the space.
559 Layout What the layout of a step was when it was running.
562 This can be used to give you an idea of which node ran
563 which rank in your job.
564 MaxDiskRead
567 Maximum number of bytes read by all tasks in job.
568 MaxDiskReadNode
571 The node on which the maxdiskread occurred.
572 MaxDiskReadTask
575 The task ID where the maxdiskread occurred.
576 MaxDiskWrite
579 Maximum number of bytes written by all tasks in job.
580 MaxDiskWriteNode
583 The node on which the maxdiskwrite occurred.
584 MaxDiskWriteTask
587 The task ID where the maxdiskwrite occurred.
588 MaxPages Maximum number of page faults of all tasks in job.
591 MaxPagesNode
594 The node on which the maxpages occurred.
595 MaxPagesTask
598 The task ID where the maxpages occurred.
599 MaxRSS Maximum resident set size of all tasks in job.
602 MaxRSSNode
605 The node on which the maxrss occurred.
606 MaxRSSTask
609 The task ID where the maxrss occurred.
610 MaxVMSize Maximum Virtual Memory size of all tasks in job.
613 MaxVMSizeNode
616 The node on which the maxvmsize occurred.
617 MaxVMSizeTask
620 The task ID where the maxvmsize occurred.
621 MinCPU Minimum (system + user) CPU time of all tasks in job.
624 MinCPUNode
627 The node on which the mincpu occurred.
628 MinCPUTask
631 The task ID where the mincpu occurred.
632 NCPUS Count of allocated CPUs. Equivalent to AllocCPUS
635 Total number of CPUs allocated to the job.
637 NodeList List of nodes in job/step.
640 NNodes Number of nodes in a job or step. If the job is run‐
643 ning, or ran, this count will be the number allocated,
644 else the number will be the number requested.
645 NTasks Total number of tasks in a job or step.
648 Priority Slurm priority.
651 Partition Identifies the partition on which the job ran.
654 QOS Name of Quality of Service.
657 QOSRAW Id of Quality of Service.
660 ReqCPUFreq
663 Requested CPU frequency for the step, in kHz. Note:
664 This value applies only to a job step. No value is
665 reported for the job.
666 ReqCPUS Required CPUs.
669 ReqGRES Names and counts of generic resources requested.
672 ReqMem Minimum required memory for the job, in MB. A 'c' at
675 the end of number represents Memory Per CPU, a 'n'
676 represents Memory Per Node. Note: This value is only
677 from the job allocation, not the step.
678 ReqNodes Requested minimum Node count for the job/step.
681 ReqTres Trackable resources. These are the minimum resource
684 counts requested by the job/step at submission time.
685 For more details see AccountingStorageTRES in
686 slurm.conf.
687 Reservation
690 Reservation Name.
691 ReservationId
694 Reservation Id.
695 Reserved How much wall clock time was used as reserved time for
698 this job. This is derived from how long a job was
699 waiting from eligible time to when it actually
700 started.
701 ResvCPU Formatted time for how long (cpu secs) a job was
704 reserved for.
705 ResvCPURAW
708 Reserved CPUs in second format, not formatted.
709 Start Initiation time of the job in the same format as End.
712 State Displays the job status, or state. See the JOB STATE
715 CODES section below for a list of possible states.
716 If more information is available on the job state than
718 will fit into the current field width (for example,
719 the uid that CANCELLED a job) the state will be fol‐
720 lowed by a "+". You can increase the size of the dis‐
721 played state using the "%NUMBER" format modifier
722 described earlier.
723 NOTE: The RUNNING state will return suspended jobs as
725 well. In order to print suspended jobs you must
726 request SUSPENDED at a different call from RUNNING.
727 NOTE: The RUNNING state will return any jobs completed
729 (cancelled or otherwise) in the time period requested
730 as the job was also RUNNING durning that time. If you
731 are only looking for job that finished please choose
732 the appropriate state(s) without the RUNNING state.
733 Submit The time and date stamp (in localtime) the job was
736 submitted. The format of the output is identical to
737 that of the End field.
738 NOTE: If a job is requeued, the submit time is reset.
740 To obtain the original submit time it is necessary to
741 use the -D or --duplicate option to display all dupli‐
742 cate entries for a job.
743 Suspended How long the job was suspended for.
746 SystemCPU The amount of system CPU time used by the job or job
749 step. The format of the output is identical to that
750 of the Elapsed field.
751 NOTE: SystemCPU provides a measure of the task's par‐
753 ent process and does not include CPU time of child
754 processes.
755 Timelimit What the timelimit was/is for the job.
758 TimelimitRaw
761 What the timelimit was/is for the job, not formatted.
762 TotalCPU The sum of the SystemCPU and UserCPU time used by the
765 job or job step. The total CPU time of the job may
766 exceed the job's elapsed time for jobs that include
767 multiple job steps. The format of the output is iden‐
768 tical to that of the Elapsed field.
769 NOTE: TotalCPU provides a measure of the task's parent
771 process and does not include CPU time of child pro‐
772 cesses.
773 TresUsageInAve
776 Tres average usage in by all tasks in job. NOTE: If
777 corresponding TresUsageInMaxTask is -1 the metric is
778 node centric instead of task.
779 TresUsageInMax
782 Tres maximum usage in by all tasks in job. NOTE: If
783 corresponding TresUsageInMaxTask is -1 the metric is
784 node centric instead of task.
785 TresUsageInMaxNode
788 Node for which each maximum TRES usage out occurred.
789 TresUsageInMaxTask
792 Task for which each maximum TRES usage out occurred.
793 TresUsageOutAve
796 Tres average usage out by all tasks in job. NOTE: If
797 corresponding TresUsageOutMaxTask is -1 the metric is
798 node centric instead of task.
799 TresUsageOutMax
802 Tres maximum usage out by all tasks in job. NOTE: If
803 corresponding TresUsageOutMaxTask is -1 the metric is
804 node centric instead of task.
805 TresUsageOutMaxNode
808 Node for which each maximum TRES usage out occurred.
809 TresUsageOutMaxTask
812 Task for which each maximum TRES usage out occurred.
813 UID The user identifier of the user who ran the job.
816 User The user name of the user who ran the job.
819 UserCPU The amount of user CPU time used by the job or job
822 step. The format of the output is identical to that
823 of the Elapsed field.
824 NOTE: UserCPU provides a measure of the task's parent
826 process and does not include CPU time of child pro‐
827 cesses.
828 WCKey Workload Characterization Key. Arbitrary string
831 for grouping orthogonal accounts together.
832 WCKeyID Reference to the wckey.
835
839 BF BOOT_FAIL Job terminated due to launch failure, typically due
840 to a hardware failure (e.g. unable to boot the node
841 or block and the job can not be requeued).
842 CA CANCELLED Job was explicitly cancelled by the user or system
844 administrator. The job may or may not have been
845 initiated.
846 CD COMPLETED Job has terminated all processes on all nodes with
848 an exit code of zero.
849 DL DEADLINE Job terminated on deadline.
851 F FAILED Job terminated with non-zero exit code or other
853 failure condition.
854 NF NODE_FAIL Job terminated due to failure of one or more allo‐
856 cated nodes.
857 OOM OUT_OF_MEMORY Job experienced out of memory error.
859 PD PENDING Job is awaiting resource allocation.
861 PR PREEMPTED Job terminated due to preemption.
863 R RUNNING Job currently has an allocation.
865 RQ REQUEUED Job was requeued.
867 RS RESIZING Job is about to change size.
869 RV REVOKED Sibling was removed from cluster due to other clus‐
871 ter starting the job.
872 S SUSPENDED Job has an allocation, but execution has been sus‐
874 pended and CPUs have been released for other jobs.
875 TO TIMEOUT Job terminated upon reaching its time limit.
877
880 The options --starttime and --endtime define the time window in between
881 sacct is going to search. For historical and practical reasons their
882 default values (i.e. the default time window) depends on ohter options:
883 --jobs and --state.
884 Depending on if --jobs and/or --state are specified, the default values
886 of --starttime and --endtime options are:
887 WITHOUT EITHER --jobs NOR --state specified:
890 --starttime
892 Defaults to Midnight.
893 --endtime
895 Defaults to Now.
896 WITH --jobs AND WITHOUT --state specified:
899 --starttime
901 Dfaults to Epoch 0.
902 --endtime
904 Defaults to Now.
905 WITHOUT --jobs AND WITH --state specified:
908 ---starttime
910 Defaults to Now.
911 ---endtime
913 Defaults to --starttime and to Now if --starttime is not
914 specified.
915 WITH BOTH --jobs AND --state specified:
918 --starttime
920 Defaults to Epoch 0.
921 --endtime
923 Defaults to --starttime or to Now if --starttime is not
924 specified.
925 NOTE: With -v/--verbose a message about the actual time window in use
928 is shown.
929
932 Some sacct options may be set via environment variables. These environ‐
933 ment variables, along with their corresponding options, are listed
934 below. (Note: Commandline options will always override these settings.)
935 SACCT_FEDERATION Same as --federation
937 SACCT_LOCAL Same as --local
939 SLURM_CONF The location of the Slurm configuration file.
941 SLURM_TIME_FORMAT Specify the format used to report time stamps. A
943 value of standard, the default value, generates
944 output in the form
945 "year-month-dateThour:minute:second". A value of
946 relative returns only "hour:minute:second" if the
947 current day. For other dates in the current year
948 it prints the "hour:minute" preceded by "Tomorr"
949 (tomorrow), "Ystday" (yesterday), the name of the
950 day for the coming week (e.g. "Mon", "Tue", etc.),
951 otherwise the date (e.g. "25 Apr"). For other
952 years it returns a date month and year without a
953 time (e.g. "6 Jun 2012"). All of the time stamps
954 use a 24 hour format.
955 A valid strftime() format can also be specified.
957 For example, a value of "%a %T" will report the day
958 of the week and a time stamp (e.g. "Mon 12:34:56").
959
962 This example illustrates the default invocation of the sacct command:
963 # sacct
965 Jobid Jobname Partition Account AllocCPUS State ExitCode
966 ---------- ---------- ---------- ---------- ---------- ---------- --------
967 2 script01 srun acct1 1 RUNNING 0
968 3 script02 srun acct1 1 RUNNING 0
969 4 endscript srun acct1 1 RUNNING 0
970 4.0 srun acct1 1 COMPLETED 0
971 This example shows the same job accounting information with the brief
974 option.
975 # sacct --brief
977 Jobid State ExitCode
978 ---------- ---------- --------
979 2 RUNNING 0
980 3 RUNNING 0
981 4 RUNNING 0
982 4.0 COMPLETED 0
983 # sacct --allocations
985 Jobid Jobname Partition Account AllocCPUS State ExitCode
986 ---------- ---------- ---------- ---------- ------- ---------- --------
987 3 sja_init andy acct1 1 COMPLETED 0
988 4 sjaload andy acct1 2 COMPLETED 0
989 5 sja_scr1 andy acct1 1 COMPLETED 0
990 6 sja_scr2 andy acct1 18 COMPLETED 2
991 7 sja_scr3 andy acct1 18 COMPLETED 0
992 8 sja_scr5 andy acct1 2 COMPLETED 0
993 9 sja_scr7 andy acct1 90 COMPLETED 1
994 10 endscript andy acct1 186 COMPLETED 0
995 This example demonstrates the ability to customize the output of the
998 sacct command. The fields are displayed in the order designated on the
999 command line.
1000 # sacct --format=jobid,elapsed,ncpus,ntasks,state
1002 Jobid Elapsed Ncpus Ntasks State
1003 ---------- ---------- ---------- -------- ----------
1004 3 00:01:30 2 1 COMPLETED
1005 3.0 00:01:30 2 1 COMPLETED
1006 4 00:00:00 2 2 COMPLETED
1007 4.0 00:00:01 2 2 COMPLETED
1008 5 00:01:23 2 1 COMPLETED
1009 5.0 00:01:31 2 1 COMPLETED
1010 This example demonstrates the use of the -T (--truncate) option when
1013 used with -S (--starttime) and -E (--endtime). When the -T option is
1014 used, the start time of the job will be the specified -S value if the
1015 job was started before the specified time, otherwise the time will be
1016 the job's start time. The end time will be the specified -E option if
1017 the job ends after the specified time, otherwise it will be the jobs
1018 end time.
1019 NOTE: If no -s (--state) option is given sacct will display jobs that
1021 ran during the specified time, otherwise it returns jobs that were in
1022 the state requested during that period of time.
1023 Without -T (normal operation) sacct output would be like this.
1025 # sacct -S2014-07-03-11:40 -E2014-07-03-12:00 -X -ojobid,start,end,state
1027 JobID Start End State
1028 --------- --------------------- -------------------- ------------
1029 2 2014-07-03T11:33:16 2014-07-03T11:59:01 COMPLETED
1030 3 2014-07-03T11:35:21 Unknown RUNNING
1031 4 2014-07-03T11:35:21 2014-07-03T11:45:21 COMPLETED
1032 5 2014-07-03T11:41:01 Unknown RUNNING
1033 By adding the -T option the job's start and end times are truncated to
1035 reflect only the time requested. If a job started after the start time
1036 requested or finished before the end time requested those times are not
1037 altered. The -T option is useful when determining exact run times dur‐
1038 ing any given period.
1039 # sacct -T -S2014-07-03-11:40 -E2014-07-03-12:00 -X -ojobid,jobname,user,start,end,state
1041 JobID Start End State
1042 --------- --------------------- -------------------- ------------
1043 2 2014-07-03T11:40:00 2014-07-03T11:59:01 COMPLETED
1044 3 2014-07-03T11:40:00 2014-07-03T12:00:00 RUNNING
1045 4 2014-07-03T11:40:00 2014-07-03T11:45:21 COMPLETED
1046 5 2014-07-03T11:41:01 2014-07-03T12:00:00 RUNNING
1047
1051 Copyright (C) 2005-2007 Copyright Hewlett-Packard Development Company
1052 L.P.
1053 Copyright (C) 2008-2010 Lawrence Livermore National Security. Produced
1054 at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
1055 Copyright (C) 2010-2014 SchedMD LLC.
1056 This file is part of Slurm, a resource management program. For
1058 details, see <https://slurm.schedmd.com/>.
1059 Slurm is free software; you can redistribute it and/or modify it under
1061 the terms of the GNU General Public License as published by the Free
1062 Software Foundation; either version 2 of the License, or (at your
1063 option) any later version.
1064 Slurm is distributed in the hope that it will be useful, but WITHOUT
1066 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
1067 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
1068 for more details.
1069
1071 /etc/slurm.conf
1072 Entries to this file enable job accounting and designate the
1073 job accounting log file that collects system job accounting.
1074 /var/log/slurm_accounting.log
1076 The default job accounting log file. By default, this file
1077 is set to read and write permission for root only.
1078
1080 sstat(1), ps (1), srun(1), squeue(1), getrusage (2), time (2)
1081June 2018 Slurm Commands sacct(1)