1PMLOGGER_DAILY(1) General Commands Manual PMLOGGER_DAILY(1)
2
6 pmlogger_daily - administration of Performance Co-Pilot archive log
7 files
8
10 $PCP_BINADM_DIR/pmlogger_daily [-DEfKMNoprRVzZ?] [-c control] [-k
11 time] [-l logfile] [-m addresses] [-s size] [-t want] [-x time] [-X
12 program] [-Y regex]
13
15 pmlogger_daily and the related pmlogger_check(1) tools along with asso‐
16 ciated control files (see pmlogger.control(5)) may be used to create a
17 customized regime of administration and management for historical ar‐
18 chives of performance data within the Performance Co-Pilot (see PCPIn‐
19 tro(1)) infrastructure.
20 pmlogger_daily is intended to be run once per day, preferably in the
22 early morning, as soon after midnight as practicable. Its task is to
23 aggregate, rotate and perform general housekeeping one or more sets of
24 PCP archives.
25 To accommodate the evolution of PMDAs and changes in production logging
27 environments, pmlogger_daily is integrated with pmlogrewrite(1) to al‐
28 low optional and automatic rewriting of archives before merging. If
29 there are global rewriting rules to be applied across all archives men‐
30 tioned in the control file(s), then create the directory
31 $PCP_SYSCONF_DIR/pmlogrewrite and place any pmlogrewrite(1) rewriting
32 rules in this directory. For rewriting rules that are specific to only
33 one family of archives, use the directory name from the control file(s)
34 - i.e. the fourth field - and create a file, or a directory, or a sym‐
35 bolic link named pmlogrewrite within this directory and place the re‐
36 quired rewriting rule(s) in the pmlogrewrite file or in files within
37 the pmlogrewrite subdirectory. pmlogger_daily will choose rewriting
38 rules from the archive directory if they exist, else rewriting rules
39 from $PCP_SYSCONF_DIR/pmlogrewrite if that directory exists, else no
40 rewriting is attempted.
41 As an alternate mechanism, if the file $PCP_LOG_DIR/pmlogger/.Nee‐
43 dRewrite exists when pmlogger_daily starts then this is treated the
44 same as specifying -R on the command line and $PCP_LOG_DIR/pmlog‐
45 ger/.NeedRewrite will be removed once all the rewriting has been done.
46
48 -c control, --control=control
49 Both pmlogger_daily and pmlogger_check(1) are controlled by PCP
50 logger control file(s) that specifies the pmlogger instances to be
51 managed. The default control file is $PCP_PMLOGGERCONTROL_PATH,
52 but an alternate may be specified using the -c option. If the di‐
53 rectory $PCP_PMLOGGERCONTROL_PATH.d (or control.d from the -c op‐
54 tion) exists, then the contents of any additional control files
55 therein will be appended to the main control file (which must ex‐
56 ist).
57 -D, --noreport
59 Do not perform the conditional pmlogger_daily_report(1) processing
60 as described below.
61 -E, --expunge
63 This option causes pmlogger_daily to pass the -E flag to pmlog‐
64 ger_merge(1) in order to expunge metrics with metadata inconsis‐
65 tencies and continue rather than fail. This is intended for auto‐
66 mated daily log rotation where it is highly desirable for unat‐
67 tended daily archive merging, rewriting and compression to suc‐
68 ceed. For further details, see pmlogger_merge(1) and description
69 for the -x flag in pmlogextract(1).
70 -f, --force
72 This option forces pmlogger_daily to attempt compression actions.
73 Using this option in production is not recommended.
74 -k time, --discard=time
76 After some period, old PCP archives are discarded. time is a time
77 specification in the syntax of find-filter(1), so DD[:HH[:MM]].
78 The optional HH (hours) and MM (minutes) parts are 0 if not speci‐
79 fied. By default the time is 14:0:0 or 14 days, but may be
80 changed using this option.
81 Some special values are recognized for the time, namely 0 to keep
83 no archives beyond the the ones being currently written by pmlog‐
84 ger(1), and forever or never to prevent any archives being dis‐
85 carded.
86 The time can also be set using the $PCP_CULLAFTER variable, set in
88 either the environment or in a control file. If both $PCP_CUL‐
89 LAFTER and -k specify different values for time then the environ‐
90 ment variable value is used and a warning is issued, i.e. if
91 $PCP_CULLAFTER is set in the control file, it overrides -k given
92 on the command line.
93 Note that the semantics of time are that it is measured from the
95 time of last modification of each archive, and not from the origi‐
96 nal archive creation date. This has subtle implications for com‐
97 pression (see below) - the compression process results in the cre‐
98 ation of new archive files which have new modification times. In
99 this case, the time period (re)starts from the time of compres‐
100 sion.
101 -K When this option is specified for pmlogger_daily then only the
103 compression tasks are attempted, so no pmlogger rotation, no
104 culling, no rewriting, etc. When -K is used and a period of 0 is
105 in effect (from -x on the command line or $PCP_COMPRESSAFTER in
106 the environment or via the control file) this is intended for en‐
107 vironments where compression of archives is desired before the
108 scheduled daily processing happens. To achieve this, once pmlog‐
109 ger_check(1) has completed regular processing, it calls pmlog‐
110 ger_daily with just the -K option. Provided $PCP_COMPRESSAFTER is
111 set to 0 along with any other required compression options to
112 match the scheduled invocation of pmlogger_daily, then this will
113 compress all volumes except the ones being currently written by
114 pmlogger(1). If $PCP_COMPRESSAFTER is set to a value greater than
115 zero, then manually running pmlogger_daily with the -x option may
116 be used to compress volumes that are younger than the $PCP_COM‐
117 PRESSAFTER time. This may be used to reclaim filesystem space by
118 compressing volumes earlier than they would have otherwise been
119 compressed. Note that since the default value of $PCP_COMPRES‐
120 SAFTER is 0 days, the -x option has no effect unless the control
121 file has been edited and $PCP_COMPRESSAFTER has been set to a
122 value greater than 0.
123 -l file, --logfile=file
125 In order to ensure that mail is not unintentionally sent when
126 these scripts are run from cron(8) or systemd(1) diagnostics are
127 always sent to log files. By default, this file is
128 $PCP_LOG_DIR/pmlogger/pmlogger_daily.log but this can be changed
129 using the -l option. If this log file already exists when the
130 script starts, it will be renamed with a .prev suffix (overwriting
131 any log file saved earlier) before diagnostics are generated to
132 the log file. The -l and -t options cannot be used together.
133 -m addresses, --mail=addresses
135 Use of this option causes pmlogger_daily to construct a summary of
136 the ``notices'' file entries which were generated in the last 24
137 hours, and e-mail that summary to the set of space-separated ad‐
138 dresses. This daily summary is stored in the file
139 $PCP_LOG_DIR/NOTICES.daily, which will be empty when no new ``no‐
140 tices'' entries were made in the previous 24 hour period.
141 -M This option may be used to disable archive merging (or renaming)
143 and rewriting (-M implies -r). This is most useful in cases where
144 the archives are being incrementally copied to a remote reposi‐
145 tory, e.g. using rsync(1). Merging, renaming and rewriting all
146 risk an increase in the synchronization load, especially immedi‐
147 ately after pmlogger_daily has run, so -M may be useful in these
148 cases.
149 -N, --showme
151 This option enables a ``show me'' mode, where the programs actions
152 are echoed, but not executed, in the style of ``make -n''. Using
153 -N in conjunction with -V maximizes the diagnostic capabilities
154 for debugging.
155 -o By default all possible archives will be merged. This option re‐
157 instates the old behaviour in which only yesterday's archives will
158 be considered as merge candidates. In the special case where only
159 a single input archive needs to be merged, pmlogmv(1) is used to
160 rename the archive, otherwise pmlogger_merge(1) is used to merge
161 all of the archives for a single host and a single day into a new
162 PCP archive and the individual archives are removed.
163 -p If this option is specified for pmlogger_daily then the status of
165 the daily processing is polled and if the daily pmlogger(1) rota‐
166 tion, culling, rewriting, compressing, etc. has not been done in
167 the last 24 hours then it is done now. The intent is to have pm‐
168 logger_daily called regularly with the -p option (at 30 mins past
169 the hour, every hour in the default cron(8) set up) to ensure
170 daily processing happens as soon as possible if it was missed at
171 the regularly scheduled time (which is 00:10 by default), e.g. if
172 the system was down or suspended at that time. With this option
173 pmlogger_daily simply exits if the previous day's processing has
174 already been done. Note that this option is not used on platforms
175 supporting systemd(1) because the pmlogger_daily.timer service
176 unit specifies a timer setting with Persistent=true. The -K and
177 -p options to pmlogger_daily are mutually exclusive.
178 -r, --norewrite
180 This command line option acts as an override and prevents all ar‐
181 chive rewriting with pmlogrewrite(1) independent of the presence
182 of any rewriting rule files or directories.
183 -R, --rewriteall
185 Sometimes PMDA changes require all archives to be rewritten, not
186 just the ones involved in any current merging. This is required
187 for example after a PCP upgrade where a new version of an existing
188 PMDA has revised metadata. The -R command line forces this uni‐
189 versal-style of rewriting. The -R option to pmlogger_daily is mu‐
190 tually exclusive with both the -r and -M options.
191 -s size, --rotate=size
193 If the PCP ``notices'' file ($PCP_LOG_DIR/NOTICES) is larger than
194 20480 bytes, pmlogger_daily will rename the file with a ``.old''
195 suffix, and start a new ``notices'' file. The rotate threshold
196 may be changed from 20480 to size bytes using the -s option.
197 -t period
199 To assist with debugging or diagnosing intermittent failures the
200 -t option may be used. This will turn on very verbose tracing
201 (-VV) and capture the trace output in a file named
202 $PCP_LOG_DIR/pmlogger/daily.datestamp.trace, where datestamp is
203 the time pmlogger_daily was run in the format YYYYMMDD.HH.MM. In
204 addition, the period argument will ensure that trace files created
205 with -t will be kept for period days and then discarded.
206 -V, --verbose
208 The output from the cron execution of the scripts may be extended
209 using the -V option to the scripts which will enable verbose trac‐
210 ing of their activity. By default the scripts generate no output
211 unless some error or warning condition is encountered. A second
212 -V increases the verbosity. Using -N in conjunction with -V maxi‐
213 mizes the diagnostic capabilities for debugging.
214 -x time, --compress-after=time
216 Archive data files can optionally be compressed after some period
217 to conserve disk space. This is particularly useful for large
218 numbers of pmlogger processes under the control of pmlogger_daily.
219 time is a time specification in the syntax of find-filter(1), so
221 DD[:HH[:MM]]. The optional HH (hours) and MM (minutes) parts are
222 0 if not specified.
223 Some special values are recognized for the time, namely 0 to apply
225 compression as soon as possible, and forever or never to prevent
226 any compression being done.
227 If transparent_decompress is enabled when libpcp was built (can be
229 checked with the pmconfig(1) -L option), then the default behav‐
230 iour is compression ``as soon as possible''. Otherwise the de‐
231 fault behaviour is to not compress files (which matches the his‐
232 torical default behaviour in earlier PCP releases).
233 The time can also be set using the $PCP_COMPRESSAFTER variable,
235 set in either the environment or in a control file. If both
236 $PCP_COMPRESSAFTER and -x specify different values for time then
237 the environment variable value is used and a warning is issued.
238 For important other detailed notes concerning volume compression,
239 see the -K and -k options (above).
240 -X program, --compressor=program
242 This option specifies the program to use for compression - by de‐
243 fault this is xz(1). The environment variable $PCP_COMPRESS may
244 be used as an alternative mechanism to define program. If both
245 $PCP_COMPRESS and -X specify different compression programs then
246 the environment variable value is used and a warning is issued.
247 -Y regex, --regex=regex
249 This option allows a regular expression to be specified causing
250 files in the set of files matched for compression to be omitted -
251 this allows only the data file to be compressed, and also prevents
252 the program from attempting to compress it more than once. The
253 default regex is ".(index|Z|gz|bz2|zip|xz|lzma|lzo|lz4)$" - such
254 files are filtered using the -v option to egrep(1). The environ‐
255 ment variable $PCP_COMPRESSREGEX may be used as an alternative
256 mechanism to define regex. If both $PCP_COMPRESSREGEX and -Y
257 specify different values for regex then the environment variable
258 value is used and a warning is issued.
259 -z This option causes pmlogger_daily to not ``re-exec'', see pmlog‐
261 ger(1), when it would otherwise choose to do so and is intended
262 only for QA testing.
263 -Z This option causes pmlogger_daily to ``re-exec'', see pmlogger(1),
265 whenever that is possible and is intended only for QA testing.
266 -?, --help
268 Display usage message and exit.
269
271 Additionally pmlogger_daily supports the following ``hooks'' to allow
272 auxiliary operations to be performed at key points in the daily pro‐
273 cessing of the archives. These callbacks are controlled via variables
274 that may be set in the environment or via the control file.
275 Note that merge callbacks and autosaving described below are not en‐
277 abled when only compression tasks are being attempted, i.e. when -K
278 command line option is used.
279 All of the callback script execution and the autosave file moving will
281 be executed as the non-privileged user ``pcp'' and group ``pcp'', so
282 appropriate permissions may need to have been set up in advance.
283 $PCP_MERGE_CALLBACK
285 As each day's archive is created by merging and before any com‐
286 pression takes place, if $PCP_MERGE_CALLBACK is defined, then it
287 is assumed to be a script that will be called with one argument
288 being the name of the archive (stripped of any suffixes), so some‐
289 thing of the form /some/directory/path/YYYYMMDD. The script needs
290 to be either a full pathname, or something that will be found on
291 the shell's $PATH . The callback script will be run in the fore‐
292 ground, so pmlogger_daily will wait for it to complete.
293 If the control file contains more than one $PCP_MERGE_CALLBACK
295 specification then these will be run serially in the order they
296 appear in the control file. If $PCP_MERGE_CALLBACK is defined in
297 the environment when pmlogger_daily is run, this is treated as
298 though this option was the first in the control file, i.e. it will
299 be run before any merge callbacks mentioned in the control file.
300 If the pcp-zeroconf packages is installed, then a special merge
302 callback is added to call pmlogger_daily_report(1) first, before
303 any other merge callback options. Refer to pmlogger_daily_re‐
304 port(1) for an explanation of the pcp-zeroconf requirements.
305 If pmlogger_daily is in ``catch up'' mode (more than one day's
307 worth of archives need to be combined) then each call back is exe‐
308 cuted once for each day's archive that is generated.
309 A typical use might be to produce daily reports from the PCP ar‐
311 chive which needs to wait until the archive has been created, but
312 is more efficient if it is done before any potential compression
313 of the archive.
314 $PCP_COMPRESS_CALLBACK
316 If pmlogger_daily is run with -x 0 or $PCP_COMPRESSAFTER=0, then
317 compression is done immediately after merging. As each day's ar‐
318 chive is compressed, if $PCP_COMPRESS_CALLBACK is defined, then it
319 is assumed to be a script that will be called with one argument
320 being the name of the archive (stripped of any suffixes), so some‐
321 thing of the form /some/directory/path/YYYYMMDD. The script needs
322 to be either a full pathname, or something that will be found on
323 the shell's $PATH . The callback script will be run in the fore‐
324 ground, so pmlogger_daily will wait for it to complete.
325 If the control file contains more than one $PCP_COMPRESS_CALLBACK
327 specification then these will be run serially in the order they
328 appear in the control file. If $PCP_COMPRESS_CALLBACK is defined
329 in the environment when pmlogger_daily is run, this is treated as
330 though this option was the first in the control file, i.e. it will
331 be run first.
332 If pmlogger_daily is in ``catch up'' mode (more than one day's
334 worth of archives need to be compressed) then each call back is
335 executed once for each day's archive that is compressed.
336 A typical use might be to keep recent archives in uncompressed
338 form for efficient querying, but move the older archives to some
339 other storage location once the compression has been done.
340 $PCP_AUTOSAVE_DIR
342 Once the merging and possible compression has been done by pmlog‐
343 ger_daily, if $PCP_AUTOSAVE_DIR is defined then all of the physi‐
344 cal files that make up one day's archive will be moved (autosaved)
345 to the directory specified by $PCP_AUTOSAVE_DIR.
346 The basename of the archive is used to set the reserved words
348 DATEYYYY (year), DATEMM (month) and DATEDD (day) and these (along
349 with LOCALHOSTNAME) may appear literally in $PCP_AUTOSAVE_DIR, and
350 will be substituted at execution time to generate the destination
351 directory name. For example:
352 $PCP_AUTOSAVE_DIR=/gpfs/LOCALHOSTNAME/DATEYYYY/DATEMM-DATEDD
353 Note that these ``date'' reserved words correspond to the date on
355 which the archive data was collected, not the date that pmlog‐
356 ger_daily was run.
357 If $PCP_AUTOSAVE_DIR (after LOCALHOSTNAME and ``date'' substitu‐
359 tion) does not exist then pmlogger_daily will attempt to create it
360 (along with any parent directories that do not exist). Just be
361 aware that this directory creation runs under the uid of the user
362 ``pcp'', so directories along the path to $PCP_AUTOSAVE_DIR may
363 need to be writeable by this non-root user.
364 By ``move'' the archives we mean a paranoid checksum-copy-check‐
366 sum-remove (using the -c option for pmlogmv(1)) that will bail if
367 the copy fails or the checksums do not match (the archives are im‐
368 portant so we cannot risk something like a full filesystem or a
369 permissions issue messing with the copy process).
370 If pmlogger_daily is in ``catch up'' mode (more than one day's
372 worth of archives need to be combined) then the archives for more
373 than one day could be copied in this step.
374 A typical use might be to create PCP archives on a local filesys‐
376 tem initially, then once all the data for a single day has been
377 collected and merged, migrate that day's archive to a shared
378 filesystem or a remote filesystem. This may allow automatic
379 backup to off-site storage and/or reduce the number of I/O opera‐
380 tions and filesystem metadata operations on the (potentially
381 slower) non-local filesystem.
382
384 Refer to pmlogger.control(5) for a description of the contol file(s)
385 that are used to control which pmlogger instances and which archives
386 are managed by pmlogger_check and pmlogger_daily(1).
387
389 $PCP_VAR_DIR/config/pmlogger/config.default
390 default pmlogger configuration file location for the local primary
391 logger, typically generated automatically by pmlogconf(1).
392 $PCP_ARCHIVE_DIR/<hostname>
394 default location for archives of performance information collected
395 from the host hostname
396 $PCP_ARCHIVE_DIR/<hostname>/lock
398 transient lock file to guarantee mutual exclusion during pmlogger
399 administration for the host hostname - if present, can be safely
400 removed if neither pmlogger_daily nor pmlogger_check(1) are run‐
401 ning
402 $PCP_ARCHIVE_DIR/<hostname>/Latest
404 PCP archive folio created by mkaf(1) for the most recently
405 launched archive containing performance metrics from the host
406 hostname
407 $PCP_LOG_DIR/NOTICES
409 PCP ``notices'' file used by pmie(1) and friends
410 $PCP_LOG_DIR/pmlogger/pmlogger_daily.log
412 if the previous execution of pmlogger_daily produced any output it
413 is saved here. The normal case is no output in which case the
414 file does not exist.
415 $PCP_ARCHIVE_DIR/<hostname>/SaveLogs
417 if this directory exists, then the log file from the -l argument
418 of a newly launched pmlogger(1) for hostname will be linked into
419 this directory with the name archive.log where archive is the
420 basename of the associated pmlogger(1) PCP archive files. This
421 allows the log file to be inspected at a later time, even if sev‐
422 eral pmlogger(1) instances for hostname have been launched in the
423 interim. Because the PCP archive management tools run under the
424 uid of the user ``pcp'', $PCP_ARCHIVE_DIR/<hostname>/SaveLogs typ‐
425 ically needs to be owned by the user ``pcp''.
426 $PCP_LOG_DIR/pmlogger/.NeedRewrite
428 if this file exists, then this is treated as equivalent to using
429 -R on the command line and the file will be removed once all
430 rewriting has been done.
431
433 Environment variables with the prefix PCP_ are used to parameterize the
434 file and directory names used by PCP. On each installation, the file
435 /etc/pcp.conf contains the local values for these variables. The
436 $PCP_CONF variable may be used to specify an alternative configuration
437 file, as described in pcp.conf(5).
438
440 Earlier versions of pmlogger_daily used find(1) to locate files for
441 compressing or culling and the -k and -x options took only integer val‐
442 ues to mean ``days''. The semantics of this was quite loose given that
443 find(1) offers different precision and semantics across platforms.
444 The current implementation of pmlogger_daily uses find-filter(1) which
446 provides high precision intervals and semantics that are relative to
447 the time of execution and are consistent across platforms.
448
450 egrep(1), find-filter(1), PCPIntro(1), pmconfig(1), pmlc(1), pmlog‐
451 conf(1), pmlogctl(1), pmlogextract(1), pmlogger(1), pmlogger_check(1),
452 pmlogger_daily_report(1), pmlogger_merge(1), pmlogmv(1), pmlo‐
453 grewrite(1), systemd(1), xz(1) and cron(8).
454Performance Co-Pilot PCP PMLOGGER_DAILY(1)