1sh5util(1)                      Slurm Commands                      sh5util(1)
2
3
4

NAME

6       sh5util  -  Tool  for  merging  HDF5 files from the acct_gather_profile
7       plugin that gathers detailed data for jobs running under Slurm
8
9

SYNOPSIS

11       sh5util
12
13

DESCRIPTION

15       sh5util merges HDF5 files produced on each node for each step of a  job
16       into  one  HDF5  file for the job. The resulting file can be viewed and
17       manipulated by common HDF5 tools such as HDF5View, h5dump,  h5edit,  or
18       h5ls.
19
20       sh5util  also has two extract modes. The first, writes a limited set of
21       data for specific nodes, steps, and data  series  in  "comma  separated
22       value"  form  to a file which can be imported into other analysis tools
23       such as spreadsheets.
24
25       The second, (Item-Extract) extracts one data time from one time  series
26       for all the samples on all the nodes from a jobs HDF5 profile.
27
28       - Finds sample with maximum value of the item.
29
30       -  Write CSV file with min, ave, max, and item totals for each node for
31       each sample
32
33
34

OPTIONS

36       -E, --extract
37
38              Extract data series from a merged job file.
39
40
41              Extract mode options
42
43
44              -i, --input=path
45                        merged file to extract from (default ./job_$jobid.h5)
46
47
48              -N, --node=nodename
49                        Node name to extract (default is all)
50
51
52              -l, --level=[Node:Totals | Node:TimeSeries]
53                        Level to which series is attached.  (default  Node:To‐
54                        tals)
55
56
57              -s, --series=[Energy | Filesystem | Network | Task | Task_#]
58                        Task is all tasks, Task_# (# is a task id) (default is
59                        everything)
60
61
62       -h, --help
63              Print this description of use.
64
65
66       -I, --item-extract
67
68              Extract one data item from all samples of one data  series  from
69              all nodes in a merged job file.
70
71
72              Item-Extract mode options
73
74
75              -s, --series=[Energy | Filesystem | Network | Task]
76
77
78              -d, --data
79                        Name of data item in series (See note below).
80
81
82
83       -j, --jobs=<job[.step]>
84              Format  is  <job[.step]>.  Merge this job/step (or a comma-sepa‐
85              rated list of job steps). This option is required.  Not specify‐
86              ing a step will result in all steps found to be processed.
87
88
89       -L, --list
90
91              Print the items of a series contained in a job file.
92
93              List mode options
94
95
96              -i, --input=path
97                        Merged file to extract from (default ./job_$jobid.h5)
98
99
100              -s, --series=[Energy | Filesystem | Network | Task]
101
102
103       -o, --output=<path>
104              Path to a file into which to write.
105              Default for merge is ./job_$jobid.h5
106              Default for extract is ./extract_$jobid.csv
107
108
109       -p, --profiledir=<dir>
110              Directory location where node-step files exist default is set in
111              acct_gather.conf.
112
113
114       -S, --savefiles
115              Instead of removing node-step files after merging them into  the
116              job file, keep them around.
117
118
119       --usage
120              Display brief usage message.
121
122
123       --user=<user>
124              User  who  profiled job.  (Handy for root user, defaults to user
125              running this command.)
126
127

Data Items per Series

129       Energy
130              Power
131              CPU_Frequency
132
133
134       Filesystem
135              Reads
136              Megabytes_Read
137              Writes
138              Megabytes_Write
139
140
141       Network
142              Packets_In
143              Megabytes_In
144              Packets_Out
145              Megabytes_Out
146
147
148       Task
149              CPU_Frequency
150              CPU_Time
151              CPU_Utilization
152              RSS
153              VM_Size
154              Pages
155              Read_Megabytes
156              Write_Megabytes
157
158

PERFORMANCE

160       Executing sh5util sends a remote procedure call to slurmctld. If enough
161       calls from sh5util or other Slurm client commands that send remote pro‐
162       cedure calls to the slurmctld daemon come in at once, it can result  in
163       a  degradation of performance of the slurmctld daemon, possibly result‐
164       ing in a denial of service.
165
166       Do not run sh5util or other Slurm client commands that send remote pro‐
167       cedure  calls  to  slurmctld  from loops in shell scripts or other pro‐
168       grams. Ensure that programs limit calls to sh5util to the minimum  nec‐
169       essary for the information you are trying to gather.
170
171

EXAMPLES

173       Merge node-step files (as part of a sbatch script):
174
175              $ sbatch -n1 -d$SLURM_JOB_ID --wrap="sh5util --savefiles -j $SLURM_JOB_ID"
176
177
178       Extract all task data from a node:
179
180              $ sh5util -j 42 -N snowflake01 --level=Node:TimeSeries --series=Tasks
181
182
183       Extract all energy data:
184
185              $ sh5util -j 42 --series=Energy --data=power
186
187

COPYING

189       Copyright (C) 2013 Bull.
190       Copyright  (C)  2013-2021 SchedMD LLC.  Slurm is free software; you can
191       redistribute it and/or modify it under the terms  of  the  GNU  General
192       Public  License  as  published  by the Free Software Foundation; either
193       version 2 of the License, or (at your option) any later version.
194
195       Slurm is distributed in the hope that it will be  useful,  but  WITHOUT
196       ANY  WARRANTY;  without even the implied warranty of MERCHANTABILITY or
197       FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General  Public  License
198       for more details.
199
200

SEE ALSO

202February 2021                   Slurm Commands                      sh5util(1)
Impressum