1PMLOGGER(1)                 General Commands Manual                PMLOGGER(1)
2
3
4

NAME

6       pmlogger - create archive log for performance metrics
7

SYNOPSIS

9       pmlogger  [-CLoPruy]  [-c configfile] [-h host] [-H hostname] [-K spec]
10       [-l logfile] [-m note] [-n pmnsfile] [-p pid] [-s endsize]  [-t  inter‐
11       val]  [-T  endtime] [-U username] [-v volsize] [-V version] [-x fd] ar‐
12       chive
13

DESCRIPTION

15       pmlogger creates the archive logs of performance metric values that may
16       be  ``played  back''  by  other  Performance Co-Pilot (see PCPIntro(1))
17       tools.  These logs form the basis of the VCR paradigm and retrospective
18       performance analysis services common to the PCP toolkit.
19
20       The  mandatory argument archive is the base name for the physical files
21       that constitute an archive log.
22
23       The -V option specifies the version for the archive that is  generated.
24       By  default  a  version 2 archive is generated, and the only value cur‐
25       rently supported for version is 2.
26
27       Unless directed to another host by the -h option or when directly using
28       PMDAs  via the -o option, pmlogger will contact the Performance Metrics
29       Collector Daemon (PMCD) on the local host and use that as the source of
30       the metric values to be logged.
31
32       To support the required flexibility and control over what is logged and
33       when, pmlogger maintains an independent two  level  logging  state  for
34       each  instance  of  each  performance metric.  At the first (mandatory)
35       level, logging is allowed to be on (with an associated interval between
36       samples),  or  off or maybe.  In the latter case, the second (advisory)
37       level logging is allowed to be on (with an associated interval  between
38       samples), or off.
39
40       The  mandatory  level  allows universal specification that some metrics
41       must be logged, or must not be  logged.   The  default  state  for  all
42       instances  of  all  metrics when pmlogger starts is mandatory maybe and
43       advisory off.
44
45       Use pmlc(1) to interrogate and change the logging state  once  pmlogger
46       is running.
47
48       If  a  metric's state is mandatory (on or off) and a request is made to
49       change it to mandatory maybe, the new  state  is  mandatory  maybe  and
50       advisory  off.  If a metric's state is already advisory (on or off) and
51       a request is made to change it to mandatory maybe, the current state is
52       retained.
53
54       It  is  not possible for pmlogger to log specific instances of a metric
55       and all  instances  of  the  same  metric  concurrently.   If  specific
56       instances  are being logged and a request to log all instances is made,
57       then all instances of the metric will be logged according  to  the  new
58       request,  superseding  any  prior  logging  request  for the metric.  A
59       request to log all instances of a metric will  supersede  any  previous
60       request to log all instances.  A request to log specific instances of a
61       metric when all instances are already being logged is refused.   To  do
62       this  one  must turn off logging for all instances of the metric first.
63       In each case, the validity of the request is checked first; for example
64       a  request to change a metric's logging state to advisory on when it is
65       currently mandatory off is never permitted (it is necessary  to  change
66       the state to mandatory maybe first).
67
68       Optionally, each system running pmcd(1) may also be configured to run a
69       ``primary'' pmlogger instance.  This pmlogger instance is  launched  by
70       $PCP_RC_DIR/pmlogger,     and     is     affected    by    the    files
71       $PCP_SYSCONF_DIR/pmlogger/control,  $PCP_SYSCONF_DIR/pmlogger/control.d
72       (use  chkconfig(8),  systemctl(1) or similar platform-specific commands
73       to activate or disable the  primary  pmlogger  instance),  $PCP_SYSCON‐
74       FIG_DIR/pmlogger  (environment variable settings for the primary pmlog‐
75       ger) $PCP_SYSCONF_DIR/pmlogger/pmlogger.options (command  line  options
76       passed  to  the primary pmlogger) and $PCP_VAR_DIR/config/pmlogger/con‐
77       fig.default (the default initial configuration  file  for  the  primary
78       pmlogger).
79
80       The  primary  pmlogger  instance is identified by the -P option.  There
81       may be at most one ``primary'' pmlogger instance on each  system.   The
82       primary  pmlogger instance (if any) must be running on the same host as
83       the pmcd(1) to which it connects (if any), so the -h and -P options are
84       mutually exclusive.
85
86       Logging  of  some  metrics  is  possible even in the absence of a local
87       pmcd(1), using the "local context" mode of operation.   This  is  acti‐
88       vated using the -o option, and causes pmlogger to make use of local DSO
89       PMDAs instead of communicating with pmcd(1).  When  operating  using  a
90       local  context, the -K option may be used to control the DSO PMDAs that
91       should be made accessible.  The spec argument conforms  to  the  syntax
92       described in pmSpecLocalPMDA(3).  More than one -K option may be used.
93
94       When launched as a non-primary instance, pmlogger will exit immediately
95       if the configuration file causes no metric  logging  to  be  scheduled.
96       The  -L option overrides this behavior, and causes a non-primary pmlog‐
97       ger instance to ``linger'', presumably pending some future dynamic  re-
98       configuration  and state change via pmlc(1).  pmlogger will also linger
99       without the -L option being used if all the metrics to  be  logged  are
100       logged  as  once  only  metrics.  When  the once only metrics have been
101       logged, a warning message will be  generated  stating  that  the  event
102       queue is empty and no more events will be scheduled.
103
104       By  default all diagnostics and errors from pmlogger are written to the
105       file pmlogger.log in the directory where pmlogger is launched.  The  -l
106       option  may  be used to override the default behavior.  If the log file
107       cannot be created or is not writable, output  is  written  to  standard
108       error instead.
109
110       If  specified,  the  -s  option instructs pmlogger to terminate after a
111       certain size in records, bytes or time units has been accumulated.   If
112       endsize  is an integer then endsize records will be written to the log.
113       If endsize is an integer suffixed by b or bytes then endsize  bytes  of
114       the  archive  data will be written out (note, however, that archive log
115       record boundaries will not be broken and so this limit may be  slightly
116       surpassed).   Other  viable file size units include: K, Kb, KiB, Kbyte,
117       Kilobyte for kilobytes and M, Mb, MiB, Mbyte,  Megabyte  for  megabytes
118       and  G,  Gb,  GiB,  Gbyte,  Gigabyte for gigabytes.  These units may be
119       optionally suffixed by an s and may be of  mixed  case.   Alternatively
120       endsize  may  be an integer or a floating point number suffixed using a
121       time unit as described in PCPIntro(1) for the interval argument (to the
122       standard PCP -t command line option).
123       Some examples of different formats:
124          -s 100
125          -s 100bytes
126          -s 100K
127          -s 100Mb
128          -s 10Gbyte
129          -s 10mins
130          -s 1.5hours
131       The default is for pmlogger to run forever.
132
133       The  -r option causes the size of the physical record(s) for each group
134       of metrics and the expected contribution of the group to  the  size  of
135       the  PCP  archive  for one full day of collection to be reported in the
136       log file.  This information is reported the first time  each  group  is
137       successfully written to the archive.
138
139       The  -U  option specifies the user account under which to run pmlogger.
140       The default is the current user account for interactive use.  When  run
141       as a daemon, the unprivileged "pcp" account is used in current versions
142       of PCP, but in older versions the superuser account ("root")  was  used
143       by default.
144
145       The  log file is potentially a multi-volume data set, and the -v option
146       causes pmlogger to start a new volume after a certain size in  records,
147       bytes,  or time units has been accumulated for the current volume.  The
148       format of this size specification is identical to that of the -s option
149       (see  above).   The  default  is for pmlogger to create a single volume
150       log.  Additional volume switches can also be forced  asynchronously  by
151       either  using  pmlc(1) or sending pmlogger a SIGHUP signal (see below).
152       Note, if a scheduled volume switch  is  in  operation  due  to  the  -v
153       option, then its counters will be reset after an asynchronous switch.
154
155       Independent  of  any -v option, each volume of an archive is limited to
156       no more than 2^31 bytes, so pmlogger will automatically  create  a  new
157       volume for the archive before this limit is reached.
158
159       Normally  pmlogger operates on the distributed Performance Metrics Name
160       Space (PMNS), however if the -n  option  is  specified  an  alternative
161       local PMNS is loaded from the file pmnsfile.
162
163       Under  normal circumstances, pmlogger will run forever (except for a -s
164       option or a termination signal).  The -T option may be  used  to  limit
165       the  execution  time  using  the format of time as prescribed by PCPIn‐
166       tro(1).  The time is interpreted within  the  time  zone  of  the  PMCD
167       server,  unless the -y option is given, within which case the time zone
168       at this logger host is used.
169       Some examples of different formats:
170          -T 10mins
171          -T '@ 11:30'
172       From this it can be seen that -T 10mins and -s 10mins perform identical
173       actions.
174
175       Alternatively,  pmlogger  runtime  may  be  limited  to the lifetime of
176       another process by using the -p or --PID option to nominate the PID  of
177       the  process of interest.  In this case the pmlogger will exit when the
178       other process no longer exists.
179
180       When pmlogger receives a SIGHUP signal, the current volume of  the  log
181       is closed, and a new volume is opened.  This mechanism (or the alterna‐
182       tive mechanism via pmlc(1)) may be used to manage the growth of the log
183       files  - once a log volume is closed, that file may be archived without
184       ill-effect on the continued operation of pmlogger.   See  also  the  -v
185       option above.
186
187       Historically  the  buffers  for  the current log may be flushed to disk
188       using the flush command of pmlc(1), or by sending  pmlogger  a  SIGUSR1
189       signal  or by using the -u option.  The current version of pmlogger and
190       the  libpcp  routines  that  underpin  pmlogger   unconditionally   use
191       unbuffered  writes and a single fwrite(3) for each logical record writ‐
192       ten, and so ``flushing'' does not force any additional data to be writ‐
193       ten  to  the  file system.  The -u option, the SIGUSR1 handling and the
194       pmlc(1) flush command are retained for backwards compatibility.
195
196       When launched with the -x option,  pmlogger  will  accept  asynchronous
197       control  requests  on  the  file  descriptor  fd.   This option is only
198       expected to be used internally by PCP applications that support  ``live
199       record mode''.
200
201       The -m option allows the string note to be appended to the map file for
202       this instance of pmlogger in the $PCP_TMP_DIR/pmlogger directory.  This
203       is  currently used internally to document the file descriptor (fd) when
204       the -x option is used, or to indicate that this pmlogger  instance  was
205       started under the control of pmlogger_check(1).
206
207       The  -H option allows the hostname written into the archive label to be
208       overridden.  This mirrors the -H option of pmcd(1) , but allows  it  to
209       be  specified  on  the pmlogger process. Without this option, the value
210       returned from the logged pmcd(1) is used.
211
212       The -C option will cause the configuration file to be parsed and pmlog‐
213       ger  will  then  exit without creating an output archive, so when -C is
214       specified, the archive command line  argument  is  not  required.   Any
215       errors in the configuration file are reported.
216

CONFIGURATION FILE SYNTAX

218       The  configuration  file may be specified with the -c option.  If it is
219       not, configuration specifications are read from standard input.
220
221       If configfile does not exist, then a search is made  in  the  directory
222       $PCP_VAR_DIR/config/pmlogger  for a file of the same name, and if found
223       that file is used, e.g. if config.mumble does not exist in the  current
224       directory  and the file $PCP_VAR_DIR/config/pmlogger/config.mumble does
225       exist, then -c config.mumble and  -c  $PCP_VAR_DIR/config/pmlogger/con‐
226       fig.mumble are equivalent.
227
228       The syntax for the configuration file is as follows.
229
230       1.   Words are separated by white space (space, tab or newline).
231
232       2.   The  symbol  ``#'' (hash) introduces a comment, and all text up to
233            the next newline is ignored.
234
235       3.   Keywords (shown in bold below)  must  appear  literally  (i.e.  in
236            lower case).
237
238       4.   Each  specification begins with the optional keyword log, followed
239            by one of the states mandatory on, mandatory off, mandatory maybe,
240            advisory on or advisory off.
241
242       5.   For the on states, a logging interval must follow using the syntax
243            ``once'', or ``default'', or ``every N timeunits'', or simply  ``N
244            timeunits''  -  N  is an unsigned integer, and timeunits is one of
245            the keywords msec, millisecond, sec, second, min, minute, hour  or
246            the plural form of one of the above.
247            Internal  limitations  require  the  interval  to  be smaller than
248            (approximately) 74 hours.  An interval value of zero is a  synonym
249            for once.  An interval of default means to use the default logging
250            interval of 60 seconds; this  default  value  may  be  changed  to
251            interval with the -t command line option.
252
253            The interval argument follows the syntax described in PCPIntro(1),
254            and in the simplest form may be an unsigned integer  (the  implied
255            units in this case are seconds).
256
257       6.   Following  the  state and possible interval specifications comes a
258            ``{'', followed by a list of one or more metric specifications and
259            a  closing  ``}''.   The list is white space (or comma) separated.
260            If there is only one metric specification in the list, the  braces
261            are optional.
262
263       7.   A  metric  specification consists of a metric name optionally fol‐
264            lowed by a set of instance names.  The  metric  name  follows  the
265            standard  PCP  naming  conventions, see pmns(5), and if the metric
266            name is a non-leaf node in the PMNS (see pmns(5)),  then  pmlogger
267            will recursively descend the PMNS and apply the logging specifica‐
268            tion to all descendent metric names that are  leaf  nodes  in  the
269            PMNS.  The set of instance names is a ``['', followed by a list of
270            one or more space (or comma) separated names, numbers or  strings,
271            and  a  closing  ``]''.  Elements in the list that are numbers are
272            assumed to be internal instance identifiers,  other  elements  are
273            assumed  to  be  external instance identifiers - see pmGetInDom(3)
274            for more information.
275
276            If no instances are  given,  then  the  logging  specification  is
277            applied to all instances of the associated metric.
278
279       8.   There may be an arbitrary number of logging specifications.
280
281       9.   As  of  PCP  version  4.0 and later, any metric name specification
282            that does not resolve to a leaf node in the PMNS is  added  to  an
283            internal list of possible dynamic subtree roots.  PMDAs can dynam‐
284            ically create new metrics below a dynamic root node in their PMNS,
285            and  send a notification to clients that the PMNS has changed, see
286            pmdaExtSetFlags(3) and in particular the  METRIC  CHANGES  section
287            for  API  details.   This  mechanism  is  currently  supported  by
288            pmdaprometheus(1) and pmdammv(1).  When a fetch issued by pmlogger
289            returns  with  the  PMDA_EXT_NAMES_CHANGE  flag set, pmlogger will
290            traverse the internal list of possible dynamic subtree  nodes  and
291            dynamically  discover  any new metrics.  In effect, as of PCP ver‐
292            sion 4.0 and later, pmlogger can be configured to dynamically  log
293            new  metrics  that  appear  in the future, after the configuration
294            file is initially parsed.
295
296       10.  Following all of the  logging  specifications,  there  may  be  an
297            optional  access  control section, introduced by the literal token
298            [access].  Thereafter come access control rules that allow or dis‐
299            allow operations from particular hosts or groups of hosts.
300
301            The  operations  may  be  used to interrogate or control a running
302            pmlogger using pmlc(1) and fall into the following classes:
303
304            enquire        interrogate the status of pmlogger and the  metrics
305                           it is logging
306            advisory       Change advisory logging.
307            mandatory      Change mandatory logging.
308            all            All of the above.
309
310            Access  control  rules  are  of the form ``allow hostlist : opera‐
311            tionlist ;'' and ``disallow hostlist : operationlist ;''.
312
313            The hostlist follows the syntax and semantics for the access  con‐
314            trol  mechanisms used by PMCD and are fully documented in pmcd(1).
315            An operationslist is a comma  separated  list  of  the  operations
316            advisory, mandatory, enquire and all.
317
318            A  missing [access] section allows all access and is equivalent to
319            allow * : all;.
320
321       The configuration (either from standard input or  configfile)  is  ini‐
322       tially   scanned   by   pmcpp(1)   with   the   options   -rs   and  -I
323       $PCP_VAR_DIR/config/pmlogger.  This extends the configuration file syn‐
324       tax  with  include  file  processing  (%include),  a common location to
325       search for include files ($PCP_VAR_DIR/config/pmlogger), macro  defini‐
326       tions  (%define),  macro  expansion (%name and %{name}) and conditional
327       inclusion of lines (%ifdef name ... %else ... %endif and  %ifndef  name
328       ... %else ... %endif).
329

EXAMPLES

331       For  each  PCP  utility,  there is a sample pmlogger configuration file
332       that could be used to create an archive log suitable for replaying with
333       that  tool  (i.e.  includes  all of the performance metrics used by the
334       tool).  For a tool named foo this  configuration  file  is  located  in
335       $PCP_VAR_DIR/config/pmlogger/config.foo.
336
337       The  following  is  a  simple  default configuration file for a primary
338       pmlogger instance, and demonstrates most of  the  capabilities  of  the
339       configuration specification language.
340
341            log mandatory on once { hinv.ncpu hinv.ndisk }
342            log mandatory on every 10 minutes {
343                disk.all.write
344                disk.all.read
345                network.interface.in.packets [ "et0" ]
346                network.interface.out.packets [ "et0" ]
347                nfs.server.reqs [ "lookup" "getattr" "read" "write" ]
348            }
349
350            log advisory on every 30 minutes {
351                environ.temp
352                pmcd.pdu_in.total
353                pmcd.pdu_out.total
354            }
355
356            %include "macros.default"
357
358            %ifdef %disk_detail
359            log mandatory on %disk_detail_freq {
360                disk.dev
361            }
362            %endif
363
364            [access]
365            disallow * : all except enquire;
366            allow localhost : mandatory, advisory;
367

FILES

369       archive.meta
370                 metadata  (metric  descriptions,  instance domains, etc.) for
371                 the archive log
372       archive.0 initial volume of metrics  values  (subsequent  volumes  have
373                 suffixes 1, 2, ...)
374       archive.index
375                 temporal  index  to  support rapid random access to the other
376                 files in the archive log
377       $PCP_TMP_DIR/pmlogger
378                 pmlogger maintains the files in this  directory  as  the  map
379                 between  the  process id of the pmlogger instance and the IPC
380                 port that may be used to control each pmlogger  instance  (as
381                 used by pmlc(1))
382       $PCP_VAR_DIR/config/pmlogger/config.default
383                 default  configuration  file  for the primary logger instance
384                 launched from $PCP_RC_DIR/pmlogger
385       $PCP_VAR_DIR/config/pmlogger/config.*
386                 assorted configuration files suitable for creating logs  that
387                 may  be  subsequently replayed with the PCP visualization and
388                 monitoring tools
389       $PCP_LOG_DIR/pmlogger/hostname
390                 Default directory for PCP archive files for performance  met‐
391                 ric values collected from the host hostname.
392       $PCP_SYSCONFIG_DIR/pmlogger
393                 additional  environment  variables  that will be set when the
394                 primary pmlogger instance executes.   Only  settings  of  the
395                 form "PMLOGGER_VARIABLE=value" will be honoured.
396       ./pmlogger.log
397                 (or  $PCP_LOG_DIR/pmlogger/hostname/pmlogger.log when started
398                 automatically by either $PCP_RC_DIR/pmlogger or  one  of  the
399                 pmlogger(1) monitoring scripts such as pmlogger_check(1))
400                 all messages and diagnostics are directed here
401

ENVIRONMENT

403       Normally  pmlogger  creates  a  socket to receive control messages from
404       pmlc(1) on the first available TCP/IP port  numbered  4330  or  higher.
405       The environment variable PMLOGGER_PORT may be used to specify an alter‐
406       native starting port number.
407
408       If set to the value 1, the  PMLOGGER_LOCAL  environment  variable  will
409       cause  pmlogger  to run in a localhost-only mode of operation, where it
410       binds only to the loopback interface.
411
412       The PMLOGGER_MAXPENDING variable can be set  to  indicate  the  maximum
413       length to which the queue of pending pmlc connections may grow.
414
415       The  default  sampling  interval  used by pmlogger can be set using the
416       PMLOGGER_INTERVAL variable (if not set, 60 seconds will be used).  Both
417       the command line and directives in the configuration file will override
418       this value.  It is an integer in units of seconds.
419

PCP ENVIRONMENT

421       Environment variables with the prefix PCP_ are used to parameterize the
422       file  and  directory names used by PCP.  On each installation, the file
423       /etc/pcp.conf contains the  local  values  for  these  variables.   The
424       $PCP_CONF  variable may be used to specify an alternative configuration
425       file, as described in pcp.conf(5).
426

SEE ALSO

428       PCPIntro(1), pmcd(1), pmdumplog(1), pmlc(1), pmlogger_check(1), system‐
429       ctl(1),   pmSpecLocalPMDA(3),   pcp.conf(5),  pcp.env(5),  pmlogger(5),
430       pmns(5) and chkconfig(8).
431

DIAGNOSTICS

433       The archive logs are sufficiently precious that pmlogger will not trun‐
434       cate an existing physical file.  A message of the form
435        ...: "foo.index" already exists, not over-written
436        ...: File exists
437       indicates  this  situation  has arisen.  You must explicitly remove the
438       files and launch pmlogger again.
439
440       There may be at most one primary pmlogger instance per monitored  host;
441       attempting to bend this rule produces the error:
442        pmlogger: there is already a primary pmlogger running
443
444       Various  other  messages  relating  to  the creation and/or deletion of
445       files in $PCP_TMP_DIR/pmlogger suggest a  permission  problem  on  this
446       directory, or some feral files have appeared therein.
447
448
449
450Performance Co-Pilot                  PCP                          PMLOGGER(1)
Impressum