sh5util(1)

1sh5util(1)                      Slurm Commands                      sh5util(1)
2
3
4

NAME

6       sh5util  -  Tool  for  merging  HDF5 files from the acct_gather_profile
7       plugin that gathers detailed data for jobs running under Slurm
8
9

SYNOPSIS

11       sh5util
12
13

DESCRIPTION

15       sh5util merges HDF5 files produced on each node for each step of a  job
16       into  one  HDF5  file for the job. The resulting file can be viewed and
17       manipulated by common HDF5 tools such as HDF5View, h5dump,  h5edit,  or
18       h5ls.
19
20       sh5util  also has two extract modes. The first, writes a limited set of
21       data for specific nodes, steps, and data  series  in  "comma  separated
22       value"  form  to a file which can be imported into other analysis tools
23       such as spreadsheets.
24
25       The second, (Item-Extract) extracts one data time from one time  series
26       for all the samples on all the nodes from a jobs HDF5 profile.
27
28       - Finds sample with maximum value of the item.
29
30       -  Write CSV file with min, ave, max, and item totals for each node for
31       each
32              sample
33
34
35

OPTIONS

37       -E, --extract
38
39              Extract data series from a merged job file.
40
41
42              Extract mode options
43
44
45              -i, --input=path
46                        merged file to extract from (default ./job_$jobid.h5)
47
48
49              -N, --node=nodename
50                        Node name to extract (default is all)
51
52
53              -l, --level=[Node:Totals|Node:TimeSeries]
54                        Level  to   which   series   is   attached.   (default
55                        Node:Totals)
56
57
58              -s, --series=[Energy | Lustre | Network | Tasks | Task_#]
59                        Tasks  is  all tasks, Task_# (# is a task id) (default
60                        is everything)
61
62
63
64       -I, --item-extract
65
66              Extract one data item from all samples of one data  series  from
67              all nodes in a merged job file.
68
69
70              Item-Extract mode options
71
72
73              -s, --series=[Energy | Lustre | Network | Task]
74
75
76              -d, --data
77                        Name of data item in series (See note below).
78
79
80
81       -j, --jobs=<job(.step)>
82              Format  is  <job(.step)>.  Merge this job/step (or a comma-sepa‐
83              rated list of job steps). This option is required.  Not specify‐
84              ing a step will result in all steps found to be processed.
85
86
87       -h, --help
88              Print this description of use.
89
90
91       -o, --output=path
92              Path to a file into which to write.
93              Default for merge is ./job_$jobid.h5
94              Default for extract is ./extract_$jobid.csv
95
96
97       -p, --profiledir=dir
98              Directory location where node-step files exist default is set in
99              acct_gather.conf.
100
101
102       -S, --savefiles
103              Instead of removing node-step files after merging them into  the
104              job file, keep them around.
105
106
107       --user=user
108              User  who  profiled job.  (Handy for root user, defaults to user
109              running this command.)
110
111
112       --usage
113              Display brief usage message.
114
115

Data Items per Series

117       Energy
118              Power
119              CPU_Frequency
120
121
122       Lustre
123              Reads
124              Megabytes_Read
125              Writes
126              Megabytes_Write
127
128
129       Network
130              Packets_In
131              Megabytes_In
132              Packets_Out
133              Megabytes_Out
134
135
136       Task
137              CPU_Frequency
138              CPU_Time
139              CPU_Utilization
140              RSS
141              VM_Size
142              Pages
143              Read_Megabytes
144              Write_Megabytes
145
146

Examples

148       Merge node-step files (as part of a sbatch script)
149
150       sbatch   -n1    -d$SLURM_JOB_ID    --wrap="sh5util    --savefiles    -j
151       $SLURM_JOB_ID"
152
153
154       Extract all task data from a node
155
156       sh5util -j 42 -N snowflake01 --level=Node:TimeSeries --series=Tasks
157
158
159       Extract all energy data
160              sh5util -j 42 --series=Energy --data=power
161
162

COPYING

164       Copyright (C) 2013 Bull.
165       Copyright (C) 2013 SchedMD LLC.  Slurm is free software; you can redis‐
166       tribute it and/or modify it under the terms of the GNU  General  Public
167       License  as published by the Free Software Foundation; either version 2
168       of the License, or (at your option) any later version.
169
170       Slurm is distributed in the hope that it will be  useful,  but  WITHOUT
171       ANY  WARRANTY;  without even the implied warranty of MERCHANTABILITY or
172       FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General  Public  License
173       for more details.
174
175