1PMLOGGER(1) General Commands Manual PMLOGGER(1)
2
6 pmlogger - create archive log for performance metrics
7
9 $PCP_BINADM_DIR/pmlogger [-c configfile] [-h host] [-l logfile] [-L]
10 [-n pmnsfile] [-P] [-r] [-s endsize] [-t interval] [-T endtime] [-u]
11 [-v volsize] [-V version] [-x fd] archive
12
14 pmlogger creates the archive logs of performance metric values that may
15 be ``played back'' by other Performance Co-Pilot (see PCPIntro(1))
16 tools. These logs form the basis of the VCR paradigm and retrospective
17 performance analysis services common to the PCP toolkit.
18 The mandatory argument archive is the base name for the physical files
20 that constitute an archive log.
21 The -V option specifies whether a version 1 or version 2 archive is
23 generated. A version 2 archive also stores the associated Performance
24 Metrics Name Space (PMNS). By default a version 2 archive is generated.
25 Unless directed to another host by the -h option, pmlogger will contact
27 the Performance Metrics Collector Daemon (PMCD) on the local host and
28 use that as the source of the metric values to be logged.
29 To support the required flexibility and control over what is logged and
31 when, pmlogger maintains an independent two level logging state for
32 each instance of each performance metric. At the first (mandatory)
33 level, logging is allowed to be on (with an associated interval between
34 samples), or off or maybe. In the latter case, the second (advisory)
35 level logging is allowed to be on (with an associated interval between
36 samples), or off.
37 The mandatory level allows universal specification that some metrics
39 must be logged, or must not be logged. The default state for all
40 instances of all metrics when pmlogger starts is mandatory maybe and
41 advisory off.
42 Use pmlc(1) to interrogate and change the logging state once pmlogger
44 is running.
45 If a metric's state is mandatory (on or off) and a request is made to
47 change it to mandatory maybe, the new state is mandatory maybe and
48 advisory off. If a metric's state is already advisory (on or off) and
49 a request is made to change it to mandatory maybe, the current state is
50 retained.
51 It is not possible for pmlogger to log specific instances of a metric
53 and all instances of the same metric concurrently. If specific
54 instances are being logged and a request to log all instances is made,
55 then all instances of the metric will be logged according to the new
56 request, superseding any prior logging request for the metric. A
57 request to log all instances of a metric will supersede any previous
58 request to log all instances. A request to log specific instances of a
59 metric when all instances are already being logged is refused. To do
60 this one must turn off logging for all instances of the metric first.
61 In each case, the validity of the request is checked first; for example
62 a request to change a metric's logging state to advisory on when it is
63 currently mandatory off is never permitted (it is necessary to change
64 the state to mandatory maybe first).
65 Optionally, each system running pmcd(1) may also be configured to run a
67 ``primary'' pmlogger instance. Like pmcd(1), this pmlogger instance is
68 launched by $PCP_RC_DIR/pcp, and is affected by the files /etc/con‐
69 fig/pmlogger (use chkconfig(1M) to activate or disable the primary
70 pmlogger instance), /etc/config/pmlogger.options (command line options
71 passed to the primary pmlogger) and $PCP_VAR_DIR/config/pmlogger/con‐
72 fig.default (the default initial configuration file for the primary
73 pmlogger).
74 The primary pmlogger instance is identified by the -P option. There
76 may be at most one ``primary'' pmlogger instance on each system with an
77 active pmcd(1). The primary pmlogger instance (if any) must be running
78 on the same host as the pmcd(1) to which it connects, so the -h and -P
79 options are mutually exclusive.
80 When launched as a non-primary instance, pmlogger will exit immediately
82 if the configuration file causes no metric logging to be scheduled.
83 The -L option overrides this behavior, and causes a non-primary pmlog‐
84 ger instance to ``linger'', presumably pending some future dynamic re-
85 configuration and state change via pmlc(1). pmlogger will also linger
86 without the -L option being used if all the metrics to be logged are
87 logged as once only metrics. When the once only metrics have been
88 logged, a warning message will be generated stating that the event
89 queue is empty and no more events will be scheduled.
90 By default all diagnostics and errors from pmlogger are written to the
92 file pmlogger.log in the directory where pmlogger is launched. The -l
93 option may be used to override the default behavior. If the log file
94 cannot be created or is not writable, output is written to standard
95 error instead.
96 If specified, the -s option instructs pmlogger to terminate after a
98 certain size in records, bytes or time units has been accumulated. If
99 endsize is an integer then endsize records will be written to the log.
100 If endsize is suffixed by b or bytes then endsize bytes of the archive
101 data will be written out (note, however, that archive log record bound‐
102 aries will not be broken and so this limit may be slightly surpassed).
103 Other viable file size units include: K, Kb, Kilobyte for kilobytes and
104 M, Mb, Megabyte for megabytes. These units may be optionally suffixed
105 by an s and may be of mixed case. Alternatively endsize may be suf‐
106 fixed using a time unit as described in PCPIntro(1) for the interval
107 argument (to the standard PCP tool option -t).
108 Some examples of different formats:
109 -s 100
110 -s 100bytes
111 -s 100K
112 -s 100Mb
113 -s 10mins
114 -s 10hours
115 The default is for pmlogger to run forever.
116 The -r option causes the size of the physical record(s) for each group
118 of metrics and the expected contribution of the group to the size of
119 the PCP archive for one full day of collection to be reported in the
120 log file. This information is reported the first time each group is
121 successfully written to the archive.
122 The log file is potentially a multi-volume data set, and the -v option
124 causes pmlogger to start a new volume after a certain size in records,
125 bytes, or time units has been accumulated for the current volume. The
126 format of this size specification is identical to that of the -s option
127 (see above). The default is for pmlogger to create a single volume
128 log. Additional volume switches can also be forced asynchronously by
129 either using pmlc(1) or sending pmlogger a SIGHUP signal (see below).
130 Note, if a scheduled volume switch is in operation due to the -v
131 option, then its counters will be reset after an asynchronous switch.
132 Normally pmlogger operates on the distributed Performance Metrics Name
134 Space (PMNS), however if the -n option is specified an alternative
135 local PMNS is loaded from the file pmnsfile.
136 Under normal circumstances, pmlogger will run forever (except for a -s
138 option or a termination signal). The -T option may be used to limit
139 the execution time using the format of time as prescribed by PCPIn‐
140 tro(1).
141 Some examples of different formats:
142 -T 10mins
143 -T '@ 11:30'
144 From this it can be seen that -T 10mins and -s 10mins perform identical
145 actions.
146 When pmlogger receives a SIGHUP signal, the current volume of the log
148 is closed, and a new volume is opened. This mechanism (or the alterna‐
149 tive mechanism via pmlc(1)) may be used to manage the growth of the log
150 files - once a log volume is closed, that file may be archived without
151 ill-effect on the continued operation of pmlogger. See also the -v
152 option above.
153 The buffers for the current log may be flushed to disk using the flush
155 command of pmlc(1), or by sending pmlogger a SIGUSR1 signal or by using
156 the -u option (the latter forces every log write to be unbuffered).
157 This is useful when the log needs to be read while pmlogger is still
158 running.
159 When launched with the -x option, pmlogger will accept asynchronous
161 control requests on the file descriptor fd. This option is only
162 expected to be used internally by PCP applications that support ``live
163 record mode''.
164
166 The configuration file may be specified with the -c option. If it is
167 not, configuration specifications are read from standard input.
168 If configfile does not exist, then a search is made in the directory
170 $PCP_VAR_DIR/config/pmlogger for a file of the same name, and if found
171 that file is used, e.g. if config.mumble does not exist in the current
172 directory and the file $PCP_VAR_DIR/config/pmlogger/config.mumble does
173 exist, then -c config.mumble and -c $PCP_VAR_DIR/config/pmlogger/con‐
174 fig.mumble are equivalent.
175 The syntax for the configuration file is as follows.
177 1. Words are separated by white space (space, tab or newline).
179 2. The symbol ``#'' (hash) introduces a comment, and all text up to
181 the next newline is ignored.
182 3. Keywords (shown in bold below) must appear literally (i.e. in
184 lower case).
185 4. Each specification begins with the optional keyword log, fol‐
187 lowed by one of the states mandatory on, mandatory off, manda‐
188 tory maybe, advisory on or advisory off.
189 5. For the on states, a logging interval must follow using the syn‐
191 tax ``once'', or ``default'', or ``every N timeunits'', or sim‐
192 ply ``N timeunits'' - N is an unsigned integer, and timeunits is
193 one of the keywords msec, millisecond, sec, second, min, minute,
194 hour or the plural form of one of the above.
195 Internal limitations require the interval to be smaller than
196 (approximately) 74 hours. An interval value of zero is a syn‐
197 onym for once. An interval of default means to use the default
198 logging interval of 60 seconds; this default value may be
199 changed to interval with the -t command line option.
200 The interval argument follows the syntax described in PCPIn‐
202 tro(1), and in the simplest form may be an unsigned integer (the
203 implied units in this case are seconds).
204 6. Following the state and possible interval specifications comes a
206 ``{'', followed by a list of one or more metric specifications
207 and a closing ``}''. The list is white space (or comma) sepa‐
208 rated. If there is only one metric specification in the list,
209 the braces are optional.
210 7. A metric specification consists of a metric name optionally fol‐
212 lowed by a set of instance names. The metric name follows the
213 standard PCP naming conventions, see pmns(4), and if the metric
214 name is a non-leaf node in the PMNS (see pmns(4)), then pmlogger
215 will recursively descend the PMNS and apply the logging specifi‐
216 cation to all descendent metric names that are leaf nodes in the
217 PMNS. The set of instance names is a ``['', followed by a list
218 of one or more space (or comma) separated names, numbers or
219 strings, and a closing ``]''. Elements in the list that are
220 numbers are assumed to be internal instance identifiers, other
221 elements are assumed to be external instance identifiers - see
222 pmGetInDom(3) for more information.
223 If no instances are given, then the logging specification is
225 applied to all instances of the associated metric.
226 8. There may be an arbitrary number of logging specifications.
228 9. Following all of the logging specifications, there may be an
230 optional access control section, introduced by the literal token
231 [access]. Thereafter come access control rules of the form
232 ``allow hostlist : operation ;'' and ``disallow hostlist : oper‐
233 ation ;''.
234 The base operations are advisory, mandatory and enquire. In all
236 other aspects, these access control statements follow the syn‐
237 tactic and semantic rules defined for the access control mecha‐
238 nisms used by PMCD and are fully documented in pmcd(1).
239
241 For each PCP utility, there is a sample pmlogger configuration file
242 that could be used to create an archive log suitable for replaying with
243 that tool (i.e. includes all of the performance metrics used by the
244 tool). For a tool named foo this configuration file is located in
245 $PCP_VAR_DIR/config/pmlogger/config.foo.
246 The following is a simple default configuration file for a primary
248 pmlogger instance, and demonstrates most of the capabilities of the
249 configuration specification language.
250 log mandatory on once { hinv.ncpu hinv.ndisk }
252 log mandatory on every 10 minutes {
253 disk.all.write
254 disk.all.read
255 network.interface.in.packets [ "et0" ]
256 network.interface.out.packets [ "et0" ]
257 nfs.server.reqs [ "lookup" "getattr" "read" "write" ]
258 }
259 log advisory on every 30 minutes {
261 environ.temp
262 pmcd.pdu_in.total
263 pmcd.pdu_out.total
264 }
265 [access]
267 disallow * : all except enquire;
268 allow localhost : mandatory, advisory;
269
271 It is often useful for pmlogger processes (other than the primary
272 instance) to be started and stopped when the local host is booted or
273 shutdown. The script $PCP_RC_DIR/pcplocal and the necessary soft-links
274 are provided, and can be modified by root to run PCP tools automati‐
275 cally. If you want to find out more before starting, read the manual
276 pages for rc2(1), rc0(1), shutdown(1) and the file /etc/init.d/README.
277 For example, changing $PCP_RC_DIR/pcplocal so that it contains:
279 ´start´)
281 # Add startup actions here
282 ($PCP_BINADM_DIR/pmlogger_check &)
283 ;;
284 ´stop´)
286 # Add shutdown actions here
287 ($PCP_BINADM_DIR/pmsignal -a -s TERM pmlogger &)
288 ;;
289 will start pmlogger instances at boot time and terminate them in an
291 orderly fashion at system shutdown.
292 This script runs as root, so any pmlogger instances it launches are
294 also run as root. To run some pmlogger instances as a particular user,
295 create your own archive logger control file (see pmlogger_check(1)) and
296 use the su(1) command. e.g.
297 ´start´)
299 # Add startup actions here
300 (su tanya -c "$PCP_BINADM_DIR/pmlogger_check -c /usr/people/tanya/ctl" &)
301 ;;
302 at boot time will start the pmlogger instances described in /usr/peo‐
304 ple/tanya/ctl, running as user tanya.
305
307 archive.meta
308 metadata (metric descriptions, instance domains, etc.) for
309 the archive log
310 archive.0 initial volume of metrics values (subsequent volumes have
311 suffixes 1, 2, ...)
312 archive.index
313 temporal index to support rapid random access to the other
314 files in the archive log
315 $PCP_TMP_DIR/pmlogger
316 pmlogger maintains the files in this directory as the map
317 between the process id of the pmlogger instance and the IPC
318 port that may be used to control each pmlogger instance (as
319 used by pmlc(1))
320 /etc/config/pmlogger
321 chkconfig(1M) control flag, to control launching of pmlogger
322 from $PCP_RC_DIR/pcp
323 /etc/config/pmlogger.options
324 command line options to pmlogger when launched from
325 $PCP_RC_DIR/pcp
326 $PCP_VAR_DIR/config/pmlogger/config.default
327 default configuration file for the primary logger instance
328 launched from $PCP_RC_DIR/pcp
329 $PCP_VAR_DIR/config/pmlogger/config.*
330 assorted configuration files suitable for creating logs that
331 may be subsequently replayed with the PCP visualization and
332 monitoring tools
333 $PCP_LOG_DIR/pmlogger/hostname
334 Default directory for PCP archive files for performance met‐
335 ric values collected from the host hostname.
336 ./pmlogger.log
337 (or $PCP_LOG_DIR/pmlogger/hostname/pmlogger.log when started
338 automatically by either $PCP_RC_DIR/pcp or one of the pmlog‐
339 ger(1) monitoring scripts such as pmlogger_check(1))
340 all messages and diagnostics are directed here
341 $PCP_RC_DIR/pcplocal
342 contains ``hooks'' to enable automatic restart at system boot
343 time
344
346 Normally pmlogger creates a socket to receive control messages from
347 pmlc(1) on the first available TCP/IP port numbered 4330 or higher.
348 The environment variable PMLOGGER_PORT may be used to specify an alter‐
349 native starting port number.
350
352 Environment variables with the prefix PCP_ are used to parameterize the
353 file and directory names used by PCP. On each installation, the file
354 /etc/pcp.conf contains the local values for these variables. The
355 $PCP_CONF variable may be used to specify an alternative configuration
356 file, as described in pcp.conf(4).
357
359 PCPIntro(1), pmcd(1), pmdumplog(1), pmlc(1), pmlogger_check(1),
360 pcp.conf(4), pcp.env(4) and pmns(4).
361
363 The archive logs are sufficiently precious that pmlogger will not trun‐
364 cate an existing physical file. A message of the form
365 __pmLogNewFile: "foo.index" already exists, not over-written
366 __pmLogCreate: File exists
367 indicates this situation has arisen. You must explicitly remove the
368 files and launch pmlogger again.
369 There may be at most one primary pmlogger instance per monitored host;
371 attempting to bend this rule produces the error:
372 pmlogger: there is already a primary pmlogger running
373 Various other messages relating to the creation and/or deletion of
375 files in $PCP_TMP_DIR/pmlogger suggest a permission problem on this
376 directory, or some feral files have appeared therein.
377Performance Co-Pilot SGI PMLOGGER(1)