1PMLOGGER_CHECK(1)           General Commands Manual          PMLOGGER_CHECK(1)
2
3
4

NAME

6       pmlogger_check, pmlogger_daily - administration of Performance Co-Pilot
7       archive log files
8

SYNOPSIS

10       $PCP_BINADM_DIR/pmlogger_check [-CNsTV] [-c control] [-l logfile]
11       $PCP_BINADM_DIR/pmlogger_daily [-KMNoprRV] [-c  control]  [-k  discard]
12       [-l  logfile] [-m addresses] [-s size] [-t want] [-x compress] [-X pro‐
13       gram] [-Y regex]
14

DESCRIPTION

16       These shell scripts and associated control files may be used to  create
17       a  customized  regime  of administration and management for Performance
18       Co-Pilot (see PCPintro(1)) archive log files.
19
20       pmlogger_daily is intended to be run once per day,  preferably  in  the
21       early  morning,  as soon after midnight as practicable.  Its task is to
22       aggregate, rotate and perform general housekeeping one or more sets  of
23       PCP archives.
24
25       After  some  period, old PCP archives are discarded.  This period is 14
26       days by default, but may be changed using the -k option.  Some  special
27       values are recognized for the period (discard), namely 0 to keep no ar‐
28       chives beyond the current one, and forever or never to prevent any  ar‐
29       chives being discarded.  Note that the semantics of discard are that it
30       is measured from the time of last modification of each archive, and not
31       from  the  current  day.   This has subtle implications for compression
32       (see below) - the compression process results in the  creation  of  new
33       archive  files  which  have  new modification times.  In this case, the
34       discard period (re)starts from the time of compression.
35
36       Archive data files can optionally be compressed after  some  period  to
37       conserve  disk space.  This is particularly useful for large numbers of
38       pmlogger processes under the control of pmlogger_check.   If  transpar‐
39       ent_decompress  is  enabled  when libpcp was built (can be checked with
40       pmconfig -L), then the default behaviour is compression  ``as  soon  as
41       possible''  otherwise  the  default  behaviour is to not compress files
42       (which  matches  the  historical  default  behaviour  in  earlier   PCP
43       releases).
44
45       The -x option controls compression and compress specifies the number of
46       days after which to compress archive data files and metadata files.  If
47       compress is 0 then compression will be applied as soon as possible.  If
48       compress is never or forever then no compression  will  be  done.   The
49       environment  variable  PCP_COMPRESSAFTER  may be used as an alternative
50       mechanism to define compress.  If both PCP_COMPRESSAFTER and -x specify
51       different  values  for  compress then the environment variable value is
52       used and a warning is issued.
53
54       The -X option specifies the program to use for compression - by default
55       this is xz(1).  The environment variable PCP_COMPRESS may be used as an
56       alternative mechanism to define program.  If both PCP_COMPRESS  and  -X
57       specify  different  compression  programs then the environment variable
58       value is used and a warning is issued.
59
60       Use of the -Y option allows a regular expression to be specified  caus‐
61       ing  files  in the set of files matched for compression to be omitted -
62       this allows only the data file to be compressed, and also prevents  the
63       program  from  attempting  to  compress it more than once.  The default
64       regex is ".(index|Z|gz|bz2|zip|xz|lzma|lzo|lz4)$" - such files are fil‐
65       tered  using  the  -v  option  to  egrep(1).   The environment variable
66       PCP_COMPRESSREGEX may be used as an  alternative  mechanism  to  define
67       regex.   If  both PCP_COMPRESSREGEX and -Y specify different values for
68       regex then the environment variable value is  used  and  a  warning  is
69       issued.
70
71       To accommodate the evolution of PMDAs and changes in production logging
72       environments, pmlogger_daily  is  integrated  with  pmlogrewrite(1)  to
73       allow  optional and automatic rewriting of archives before merging.  If
74       there are global rewriting rules to be applied across all archives men‐
75       tioned   in   the   control   file(s),   then   create   the  directory
76       $PCP_SYSCONF_DIR/pmlogrewrite and place any  pmlogrewrite(1)  rewriting
77       rules in this directory.  For rewriting rules that are specific to only
78       one family of archives, use the directory name from the control file(s)
79       -  i.e. the fourth field - and create a file, or a directory, or a sym‐
80       bolic link named pmlogrewrite  within  this  directory  and  place  the
81       required  rewriting rule(s) in the pmlogrewrite file or in files within
82       the pmlogrewrite subdirectory.  pmlogger_daily  will  choose  rewriting
83       rules  from  the  archive directory if they exist, else rewriting rules
84       from $PCP_SYSCONF_DIR/pmlogrewrite if that directory  exists,  else  no
85       rewriting is attempted.
86
87       The  -r  command  line option acts as an over-ride and prevents all ar‐
88       chive rewriting with pmlogrewrite(1) independent of the presence of any
89       rewriting rule files or directories.
90
91       Sometimes  PMDA  changes require all archives to be rewritten, not just
92       the ones involved in any current merging.  This is required for example
93       after a PCP upgrade where a new version of an existing PMDA has revised
94       metadata.  The -R command line forces this universal-style  of  rewrit‐
95       ing.
96
97       The  -R option to pmlogger_daily is mutually exclusive with both the -r
98       and -M options.
99
100       As an alternate  mechanism,  if  the  file  $PCP_LOG_DIR/pmlogger/.Nee‐
101       dRewrite  exists  when  pmlogger_daily  starts then this is treated the
102       same as specifying -R  on  the  command  line  and  $PCP_LOG_DIR/pmlog‐
103       ger/.NeedRewrite will be removed once all the rewriting has been done.
104
105       By  default  all possible archives will be merged.  The -o option rein‐
106       states the old behaviour in which only  yesterday's  archives  will  be
107       considered as merge candidates.
108
109       In  the  special  case  where  only  a single input archive needs to be
110       merged, pmlogmv(1) is used to  rename  the  archive,  otherwise  pmlog‐
111       ger_merge(1) is used to merge all of the archives for a single host and
112       a single day into a new PCP archive and  the  individual  archives  are
113       removed.
114
115       The  -M option may be used to disable archive merging (or renaming) and
116       rewriting (-M implies -r).  This is most useful in cases where the  ar‐
117       chives  are  being  incrementally  copied  to a remote repository, e.g.
118       using rsync(1).  Merging, renaming and rewriting all risk  an  increase
119       in  the  synchronization  load,  especially  immediately  after  pmlog‐
120       ger_daily has run, so -M may be useful in these cases.
121
122       To assist with debugging or diagnosing  intermittent  failures  the  -t
123       option  may  be used.  This will turn on very verbose tracing (-VV) and
124       capture  the  trace  output  in  a   file   named   $PCP_LOG_DIR/pmlog‐
125       ger/daily.datestamp.trace,  where  datestamp is the time pmlogger_daily
126       was run in the format YYYYMMDD.HH.MM.  In addition, the  want  argument
127       will ensure that trace files created with -t will be kept for want days
128       and then discarded.
129
130       In addition, if the  PCP  ``notices''  file  ($PCP_LOG_DIR/NOTICES)  is
131       larger  than  20480  bytes,  pmlogger_daily will rename the file with a
132       ``.old'' suffix, and start a new ``notices'' file.  The rotate  thresh‐
133       old may be changed from 20480 to size bytes using the -s option.
134
135       Use  of  the  -m option causes pmlogger_daily to construct a summary of
136       the ``notices'' file entries which were generated in the last 24 hours,
137       and  e-mail that summary to the set of space-separated addresses.  This
138       daily summary is stored in the file  $PCP_LOG_DIR/NOTICES.daily,  which
139       will be empty when no new ``notices'' entries were made in the previous
140       24 hour period.
141
142       If the -K option is specified for pmlogger_daily then only the compres‐
143       sion  tasks  are  attempted, so no pmlogger(1) rotation, no culling, no
144       rewriting, etc.  When -K is used and a compress value of 0 is in effect
145       (from -x on the command line or PCP_COMPRESSAFTER in the environment or
146       via the control file) this is intended for environments where  compres‐
147       sion  of archives is desired before the scheduled daily processing hap‐
148       pens.  To achieve this, once pmlogger_check has completed regular  pro‐
149       cessing,  it  calls  pmlogger_daily  with just the -K option.  Provided
150       PCP_COMPRESSAFTER is set to 0 along with any other required compression
151       options  to match the scheduled invocation of pmlogger_daily, then this
152       will compress all volumes except the ones being  currently  written  by
153       pmlogger(1).
154
155       If the -p option is specified for pmlogger_daily then the status of the
156       daily processing is polled  and  if  the  daily  pmlogger(1)  rotation,
157       culling, rewriting, compressing, etc.  has not been done in the last 24
158       hours then it is done now.  The intent is to have pmlogger_daily called
159       regularly  with  the -p option (at 30 mins past the hour, every hour in
160       the default cron(8) set up) to ensure daily processing happens as  soon
161       as  possible if it was missed at the regularly scheduled time (which is
162       00:10 by default), e.g. if the system was down  or  suspended  at  that
163       time.
164
165       With  the  -p option, pmlogger_daily simply exits if the previous day's
166       processing has already been done.
167
168       The -K and -p options to pmlogger_daily are mutually exclusive.
169
170       The script $PCP_BINADM_DIR/pmlogger_daily could be copied and  modified
171       to  implement  a site-specific procedure for end-of-week and/or end-of-
172       month management for a set of PCP archives.
173
174       pmlogger_check may be run at any time, and is intended  to  check  that
175       the desired set of pmlogger(1) processes are running, and if not to re-
176       launch any failed loggers.  Use of the -s option provides  the  reverse
177       functionality,  allowing  the  set  of pmlogger processes to be cleanly
178       shutdown.  Use of the -C option queries  the  system  service  runlevel
179       information  for  pmlogger, and uses that to determine whether to start
180       processes.
181
182       The -T option provides a terser form of output for pmlogger_check  that
183       is most suitable for a pmlogger ``farm'' where many instances of pmlog‐
184       ger are expected to be running.
185
186       Using -N option invokes the scripts in a ``show  me''  or  ``dry  run''
187       mode  where  the  tasks  that  would  be performed are reported, but no
188       changes are made.  This is typically used for debugging in  combination
189       with one (verbose) or two (very verbose) -V options.
190
191       Both  pmlogger_daily  and  pmlogger_check  are controlled by PCP logger
192       control file(s) that specifies the pmlogger instances  to  be  managed.
193       The default control file is $PCP_PMLOGGERCONTROL_PATH, but an alternate
194       may be specified using the -c option.  If the directory  $PCP_PMLOGGER‐
195       CONTROL_PATH.d  (or control.d from the -c option) exists, then the con‐
196       tents of any additional control files therein will be appended  to  the
197       main control file (which must exist).
198
199       Warning:  The $PCP_PMLOGGERCONTROL_PATH and $PCP_PMLOGGERCONTROL_PATH.d
200       files must not be writable by any user other than root.
201
202       The control file(s) should be customized  according  to  the  following
203       rules  that  define  for  the current version (1.1) of the control file
204       format.
205
206       1.  Lines beginning with a ``#'' are comments.
207       2.  Lines beginning with a ``$'' are assumed to be assignments to envi‐
208           ronment variables in the style of sh(1), and all text following the
209           ``$'' will be eval'ed by the script reading the control  file,  and
210           the  corresponding variable exported into the environment.  This is
211           particularly useful to set and export variables into  the  environ‐
212           ment of the administrative scripts, e.g.
213               $ PMCD_CONNECT_TIMEOUT=20
214       3.  There  must  be  a  version line in the initial control file of the
215           form:
216               $ version=1.1
217       4.  There should be one line in the control file(s) for  each  pmlogger
218           instance of the form:
219
220               host y|n y|n directory args
221
222       5.  Fields  within  a line of the control file(s) are usually separated
223           by one or more spaces or tabs (although refer to the description of
224           the directory field for some important exceptions).
225       6.  The  first  field is the name of the host that is the source of the
226           performance metrics for this pmlogger instance.
227       7.  The second field indicates if this is a primary  pmlogger  instance
228           (y)  or  not  (n).   Since the primary logger must run on the local
229           host, and there may be at most one primary logger for a  particular
230           host,  this  field  can  be y for at most one pmlogger instance, in
231           which case the host name must be the name of the local host.
232       8.  The third field indicates if this pmlogger  instance  needs  to  be
233           started  under  the  control  of  pmsocks(1)  to  connect to a pmcd
234           through a firewall (y or n).
235       9.  The fourth field is a directory name.  All  files  associated  with
236           this  pmlogger instance will be created in this directory, and this
237           will be the current directory for the  execution  of  any  programs
238           required in the maintenance of those archives.  A useful convention
239           is that primary logger archives for the local  host  with  hostname
240           myhost are maintained in the directory $PCP_LOG_DIR/pmlogger/myhost
241           (this  is  where  the   default   pmlogger   start-up   script   in
242           $PCP_RC_DIR/pcp  will  create the archives), while archives for the
243           remote host mumble are maintained in $PCP_LOG_DIR/pmlogger/mumble.
244       10. The directory field may contain embedded shell syntax that will  be
245           evaluated  by  sh(1) to produce the real directory name to be used.
246           The allowed constructs are:
247           · Any text (including white space) enclosed with $( and ).
248           · Any text (including white space) enclosed  with  `  and  `  (back
249             quotes).
250           · Any  text  (including  white space) enclosed with " and " (double
251             quotes).
252           · Any word containing a $  (assumed  to  introduce  an  environment
253             variable name).
254       11. All  other  fields  are  interpreted  as  arguments to be passed to
255           pmlogger(1) and/or pmnewlog(1).  Most typically this would  be  the
256           -c option.
257
258       The  following  sample  control  lines  specify a primary logger on the
259       local host (bozo), and non-primary loggers to collect and  log  perfor‐
260       mance metrics from the hosts wobbly and boing.
261
262       $version=1.1
263       bozo   y  n  $PCP_LOG_DIR/pmlogger/bozo   -c config.default
264       wobbly n  n  "/store/wobbly/$(date +%Y)"  -c ./wobbly.config
265       boing  n  n  $PCP_LOG_DIR/pmlogger/boing  -c ./pmlogger.config
266
267       Typical crontab(5) entries for periodic execution of pmlogger_daily and
268       pmlogger_check are given in  $PCP_SYSCONF_DIR/pmlogger/crontab  (unless
269       installed by default in /etc/cron.d already) and shown below.
270
271       # daily processing of archive logs
272       14      0       *       *       *       $PCP_BINADM_DIR/pmlogger_daily
273       # every 30 minutes, check pmlogger instances are running
274       25,55   *       *       *       *       $PCP_BINADM_DIR/pmlogger_check
275
276       In  order  to  ensure  that mail is not unintentionally sent when these
277       scripts are run from cron(8) diagnostics are always sent to a log file.
278       By  default,  this  file is $PCP_LOG_DIR/pmlogger/pmlogger_daily.log or
279       $PCP_LOG_DIR/pmlogger/pmlogger_check.log but this can be changed  using
280       the -l option.  If this log file already exists when the script starts,
281       it will be renamed with a .prev suffix (overwriting any log file  saved
282       earlier)  before diagnostics are generated to the log file.  The -l and
283       -t options cannot be used together.
284
285       The output from the cron execution of the scripts may be extended using
286       the -V option to the scripts which will enable verbose tracing of their
287       activity.  By default the scripts generate no output unless some  error
288       or warning condition is encountered.
289
290

FILES

292       $PCP_PMLOGGERCONTROL_PATH
293                 the PCP logger control file
294                 Warning:  this  file  must  not be writable by any user other
295                 than root.
296
297       $PCP_PMLOGGERCONTROL_PATH.d
298                 optional directory containing additional PCP  logger  control
299                 files, typically one per host
300                 Warning:  the  files  herein must not be writable by any user
301                 other than root.
302
303       $PCP_SYSCONF_DIR/pmlogger/crontab
304                 sample crontab for automated script  execution  by  $PCP_USER
305                 (or  root).  Exists only if the platform does not support the
306                 /etc/cron.d mechanism.
307
308       $PCP_VAR_DIR/config/pmlogger/config.default
309                 default pmlogger configuration file location  for  the  local
310                 primary  logger,  typically generated automatically by pmlog‐
311                 conf(1).
312
313       $PCP_LOG_DIR/pmlogger/hostname
314                 default location for archives of performance information col‐
315                 lected from the host hostname
316
317       $PCP_LOG_DIR/pmlogger/hostname/lock
318                 transient  lock  file  to  guarantee  mutual exclusion during
319                 pmlogger administration for the host hostname -  if  present,
320                 can  be  safely  removed if neither pmlogger_daily nor pmlog‐
321                 ger_check are running
322
323       $PCP_LOG_DIR/pmlogger/hostname/Latest
324                 PCP archive folio created by mkaf(1) for  the  most  recently
325                 launched archive containing performance metrics from the host
326                 hostname
327
328       $PCP_LOG_DIR/NOTICES
329                 PCP ``notices'' file used by pmie(1) and friends
330
331       $PCP_LOG_DIR/pmlogger/pmlogger_check.log
332                 if the previous execution of pmlogger_check produced any out‐
333                 put  it is saved here.  The normal case is no output in which
334                 case the file does not exist.
335
336       $PCP_LOG_DIR/pmlogger/pmlogger_daily.log
337                 if the previous execution of pmlogger_daily produced any out‐
338                 put  it is saved here.  The normal case is no output in which
339                 case the file does not exist.
340
341       $PCP_LOG_DIR/pmlogger/hostname/SaveLogs
342                 if this directory exists, then the log file from the -l argu‐
343                 ment  of  a  newly  launched pmlogger(1) for hostname will be
344                 linked into this directory with the  name  archive.log  where
345                 archive is the basename of the associated pmlogger(1) PCP ar‐
346                 chive files.  This allows the log file to be inspected  at  a
347                 later  time,  even if several pmlogger(1) instances for host‐
348                 name have been launched in the interim.   Because  the  cron-
349                 driven  PCP  archive  management scripts run under the uid of
350                 the  user  ``pcp'',   $PCP_LOG_DIR/pmlogger/hostname/SaveLogs
351                 typically needs to be owned by the user ``pcp''.
352
353       $PCP_LOG_DIR/pmlogger/.NeedRewrite
354                 if  this  file  exists, then this is treated as equivalent to
355                 using -R on the command line and the  file  will  be  removed
356                 once all rewriting has been done.
357

PCP ENVIRONMENT

359       Environment variables with the prefix PCP_ are used to parameterize the
360       file and directory names used by PCP.  On each installation,  the  file
361       /etc/pcp.conf  contains  the  local  values  for  these variables.  The
362       $PCP_CONF variable may be used to specify an alternative  configuration
363       file, as described in pcp.conf(5).
364
365       The default behaviour, when pmlogger(1) configuration comes from pmlog‐
366       conf(1), is to regenerate the configuration file and check for  changes
367       whenever  pmlogger(1) is started from pmlogger_check.  If the PMDA con‐
368       figuration is stable,  this  is  not  necessary,  and  setting  $PMLOG‐
369       GER_CHECK_SKIP_LOGCONF to yes disables the regeneration and checking.
370

SEE ALSO

372       egrep(1), PCPIntro(1), pmconfig(1), pmlc(1), pmlogconf(1), pmlogger(1),
373       pmlogger_daily_report(1),    pmlogger_merge(1),    pmlogmv(1),    pmlo‐
374       grewrite(1), pmnewlog(1), pmsocks(1), xz(1) and cron(8).
375
376
377
378Performance Co-Pilot                  PCP                    PMLOGGER_CHECK(1)
Impressum