1PMLOGGER_CHECK(1) General Commands Manual PMLOGGER_CHECK(1)
2
6 pmlogger_check, pmlogger_daily - administration of Performance Co-Pilot
7 archive log files
8
10 $PCP_BINADM_DIR/pmlogger_check [-CNsTV] [-c control] [-l logfile]
11 $PCP_BINADM_DIR/pmlogger_daily [-KMNoprRV] [-c control] [-k discard]
12 [-l logfile] [-m addresses] [-s size] [-t want] [-x compress] [-X pro‐
13 gram] [-Y regex]
14
16 These shell scripts and associated control files may be used to create
17 a customized regime of administration and management for Performance
18 Co-Pilot (see PCPintro(1)) archive log files.
19 pmlogger_daily is intended to be run once per day, preferably in the
21 early morning, as soon after midnight as practicable. Its task is to
22 aggregate, rotate and perform general housekeeping one or more sets of
23 PCP archives.
24 After some period, old PCP archives are discarded. This period is 14
26 days by default, but may be changed using the -k option. Some special
27 values are recognized for the period (discard), namely 0 to keep no ar‐
28 chives beyond the current one, and forever or never to prevent any ar‐
29 chives being discarded. Note that the semantics of discard are that it
30 is measured from the time of last modification of each archive, and not
31 from the current day. This has subtle implications for compression
32 (see below) - the compression process results in the creation of new
33 archive files which have new modification times. In this case, the
34 discard period (re)starts from the time of compression.
35 Archive data files can optionally be compressed after some period to
37 conserve disk space. This is particularly useful for large numbers of
38 pmlogger processes under the control of pmlogger_check. If transpar‐
39 ent_decompress is enabled when libpcp was built (can be checked with
40 pmconfig -L), then the default behaviour is compression ``as soon as
41 possible'' otherwise the default behaviour is to not compress files
42 (which matches the historical default behaviour in earlier PCP
43 releases).
44 The -x option controls compression and compress specifies the number of
46 days after which to compress archive data files and metadata files. If
47 compress is 0 then compression will be applied as soon as possible. If
48 compress is never or forever then no compression will be done. The
49 environment variable PCP_COMPRESSAFTER may be used as an alternative
50 mechanism to define compress. If both PCP_COMPRESSAFTER and -x specify
51 different values for compress then the environment variable value is
52 used and a warning is issued.
53 The -X option specifies the program to use for compression - by default
55 this is xz(1). The environment variable PCP_COMPRESS may be used as an
56 alternative mechanism to define program. If both PCP_COMPRESS and -X
57 specify different compression programs then the environment variable
58 value is used and a warning is issued.
59 Use of the -Y option allows a regular expression to be specified caus‐
61 ing files in the set of files matched for compression to be omitted -
62 this allows only the data file to be compressed, and also prevents the
63 program from attempting to compress it more than once. The default
64 regex is ".(index|Z|gz|bz2|zip|xz|lzma|lzo|lz4)$" - such files are fil‐
65 tered using the -v option to egrep(1). The environment variable
66 PCP_COMPRESSREGEX may be used as an alternative mechanism to define
67 regex. If both PCP_COMPRESSREGEX and -Y specify different values for
68 regex then the environment variable value is used and a warning is
69 issued.
70 To accommodate the evolution of PMDAs and changes in production logging
72 environments, pmlogger_daily is integrated with pmlogrewrite(1) to
73 allow optional and automatic rewriting of archives before merging. If
74 there are global rewriting rules to be applied across all archives men‐
75 tioned in the control file(s), then create the directory
76 $PCP_SYSCONF_DIR/pmlogrewrite and place any pmlogrewrite(1) rewriting
77 rules in this directory. For rewriting rules that are specific to only
78 one family of archives, use the directory name from the control file(s)
79 - i.e. the fourth field - and create a file, or a directory, or a sym‐
80 bolic link named pmlogrewrite within this directory and place the
81 required rewriting rule(s) in the pmlogrewrite file or in files within
82 the pmlogrewrite subdirectory. pmlogger_daily will choose rewriting
83 rules from the archive directory if they exist, else rewriting rules
84 from $PCP_SYSCONF_DIR/pmlogrewrite if that directory exists, else no
85 rewriting is attempted.
86 The -r command line option acts as an over-ride and prevents all ar‐
88 chive rewriting with pmlogrewrite(1) independent of the presence of any
89 rewriting rule files or directories.
90 Sometimes PMDA changes require all archives to be rewritten, not just
92 the ones involved in any current merging. This is required for example
93 after a PCP upgrade where a new version of an existing PMDA has revised
94 metadata. The -R command line forces this universal-style of rewrit‐
95 ing.
96 The -R option to pmlogger_daily is mutually exclusive with both the -r
98 and -M options.
99 As an alternate mechanism, if the file $PCP_LOG_DIR/pmlogger/.Nee‐
101 dRewrite exists when pmlogger_daily starts then this is treated the
102 same as specifying -R on the command line and $PCP_LOG_DIR/pmlog‐
103 ger/.NeedRewrite will be removed once all the rewriting has been done.
104 By default all possible archives will be merged. The -o option rein‐
106 states the old behaviour in which only yesterday's archives will be
107 considered as merge candidates.
108 In the special case where only a single input archive needs to be
110 merged, pmlogmv(1) is used to rename the archive, otherwise pmlog‐
111 ger_merge(1) is used to merge all of the archives for a single host and
112 a single day into a new PCP archive and the individual archives are
113 removed.
114 The -M option may be used to disable archive merging (or renaming) and
116 rewriting (-M implies -r). This is most useful in cases where the ar‐
117 chives are being incrementally copied to a remote repository, e.g.
118 using rsync(1). Merging, renaming and rewriting all risk an increase
119 in the synchronization load, especially immediately after pmlog‐
120 ger_daily has run, so -M may be useful in these cases.
121 To assist with debugging or diagnosing intermittent failures the -t
123 option may be used. This will turn on very verbose tracing (-VV) and
124 capture the trace output in a file named $PCP_LOG_DIR/pmlog‐
125 ger/daily.datestamp.trace, where datestamp is the time pmlogger_daily
126 was run in the format YYYYMMDD.HH.MM. In addition, the want argument
127 will ensure that trace files created with -t will be kept for want days
128 and then discarded.
129 In addition, if the PCP ``notices'' file ($PCP_LOG_DIR/NOTICES) is
131 larger than 20480 bytes, pmlogger_daily will rename the file with a
132 ``.old'' suffix, and start a new ``notices'' file. The rotate thresh‐
133 old may be changed from 20480 to size bytes using the -s option.
134 Use of the -m option causes pmlogger_daily to construct a summary of
136 the ``notices'' file entries which were generated in the last 24 hours,
137 and e-mail that summary to the set of space-separated addresses. This
138 daily summary is stored in the file $PCP_LOG_DIR/NOTICES.daily, which
139 will be empty when no new ``notices'' entries were made in the previous
140 24 hour period.
141 If the -K option is specified for pmlogger_daily then only the compres‐
143 sion tasks are attempted, so no pmlogger(1) rotation, no culling, no
144 rewriting, etc. When -K is used and a compress value of 0 is in effect
145 (from -x on the command line or PCP_COMPRESSAFTER in the environment or
146 via the control file) this is intended for environments where compres‐
147 sion of archives is desired before the scheduled daily processing hap‐
148 pens. To achieve this, once pmlogger_check has completed regular pro‐
149 cessing, it calls pmlogger_daily with just the -K option. Provided
150 PCP_COMPRESSAFTER is set to 0 along with any other required compression
151 options to match the scheduled invocation of pmlogger_daily, then this
152 will compress all volumes except the ones being currently written by
153 pmlogger(1).
154 If the -p option is specified for pmlogger_daily then the status of the
156 daily processing is polled and if the daily pmlogger(1) rotation,
157 culling, rewriting, compressing, etc. has not been done in the last 24
158 hours then it is done now. The intent is to have pmlogger_daily called
159 regularly with the -p option (at 30 mins past the hour, every hour in
160 the default cron(8) set up) to ensure daily processing happens as soon
161 as possible if it was missed at the regularly scheduled time (which is
162 00:10 by default), e.g. if the system was down or suspended at that
163 time.
164 With the -p option, pmlogger_daily simply exits if the previous day's
166 processing has already been done.
167 The -K and -p options to pmlogger_daily are mutually exclusive.
169 The script $PCP_BINADM_DIR/pmlogger_daily could be copied and modified
171 to implement a site-specific procedure for end-of-week and/or end-of-
172 month management for a set of PCP archives.
173 pmlogger_check may be run at any time, and is intended to check that
175 the desired set of pmlogger(1) processes are running, and if not to re-
176 launch any failed loggers. Use of the -s option provides the reverse
177 functionality, allowing the set of pmlogger processes to be cleanly
178 shutdown. Use of the -C option queries the system service runlevel
179 information for pmlogger, and uses that to determine whether to start
180 processes.
181 The -T option provides a terser form of output for pmlogger_check that
183 is most suitable for a pmlogger ``farm'' where many instances of pmlog‐
184 ger are expected to be running.
185 Using -N option invokes the scripts in a ``show me'' or ``dry run''
187 mode where the tasks that would be performed are reported, but no
188 changes are made. This is typically used for debugging in combination
189 with one (verbose) or two (very verbose) -V options.
190 Both pmlogger_daily and pmlogger_check are controlled by PCP logger
192 control file(s) that specifies the pmlogger instances to be managed.
193 The default control file is $PCP_PMLOGGERCONTROL_PATH, but an alternate
194 may be specified using the -c option. If the directory $PCP_PMLOGGER‐
195 CONTROL_PATH.d (or control.d from the -c option) exists, then the con‐
196 tents of any additional control files therein will be appended to the
197 main control file (which must exist).
198 Warning: The $PCP_PMLOGGERCONTROL_PATH and $PCP_PMLOGGERCONTROL_PATH.d
200 files must not be writable by any user other than root.
201 The control file(s) should be customized according to the following
203 rules that define for the current version (1.1) of the control file
204 format.
205 1. Lines beginning with a ``#'' are comments.
207 2. Lines beginning with a ``$'' are assumed to be assignments to envi‐
208 ronment variables in the style of sh(1), and all text following the
209 ``$'' will be eval'ed by the script reading the control file, and
210 the corresponding variable exported into the environment. This is
211 particularly useful to set and export variables into the environ‐
212 ment of the administrative scripts, e.g.
213 $ PMCD_CONNECT_TIMEOUT=20
214 3. There must be a version line in the initial control file of the
215 form:
216 $ version=1.1
217 4. There should be one line in the control file(s) for each pmlogger
218 instance of the form:
219 host y|n y|n directory args
221 5. Fields within a line of the control file(s) are usually separated
223 by one or more spaces or tabs (although refer to the description of
224 the directory field for some important exceptions).
225 6. The first field is the name of the host that is the source of the
226 performance metrics for this pmlogger instance.
227 7. The second field indicates if this is a primary pmlogger instance
228 (y) or not (n). Since the primary logger must run on the local
229 host, and there may be at most one primary logger for a particular
230 host, this field can be y for at most one pmlogger instance, in
231 which case the host name must be the name of the local host.
232 8. The third field indicates if this pmlogger instance needs to be
233 started under the control of pmsocks(1) to connect to a pmcd
234 through a firewall (y or n).
235 9. The fourth field is a directory name. All files associated with
236 this pmlogger instance will be created in this directory, and this
237 will be the current directory for the execution of any programs
238 required in the maintenance of those archives. A useful convention
239 is that primary logger archives for the local host with hostname
240 myhost are maintained in the directory $PCP_LOG_DIR/pmlogger/myhost
241 (this is where the default pmlogger start-up script in
242 $PCP_RC_DIR/pcp will create the archives), while archives for the
243 remote host mumble are maintained in $PCP_LOG_DIR/pmlogger/mumble.
244 10. The directory field may contain embedded shell syntax that will be
245 evaluated by sh(1) to produce the real directory name to be used.
246 The allowed constructs are:
247 · Any text (including white space) enclosed with $( and ).
248 · Any text (including white space) enclosed with ` and ` (back
249 quotes).
250 · Any text (including white space) enclosed with " and " (double
251 quotes).
252 · Any word containing a $ (assumed to introduce an environment
253 variable name).
254 11. All other fields are interpreted as arguments to be passed to
255 pmlogger(1) and/or pmnewlog(1). Most typically this would be the
256 -c option.
257 The following sample control lines specify a primary logger on the
259 local host (bozo), and non-primary loggers to collect and log perfor‐
260 mance metrics from the hosts wobbly and boing.
261 $version=1.1
263 bozo y n $PCP_LOG_DIR/pmlogger/bozo -c config.default
264 wobbly n n "/store/wobbly/$(date +%Y)" -c ./wobbly.config
265 boing n n $PCP_LOG_DIR/pmlogger/boing -c ./pmlogger.config
266 Typical crontab(5) entries for periodic execution of pmlogger_daily and
268 pmlogger_check are given in $PCP_SYSCONF_DIR/pmlogger/crontab (unless
269 installed by default in /etc/cron.d already) and shown below.
270 # daily processing of archive logs
272 14 0 * * * $PCP_BINADM_DIR/pmlogger_daily
273 # every 30 minutes, check pmlogger instances are running
274 25,55 * * * * $PCP_BINADM_DIR/pmlogger_check
275 In order to ensure that mail is not unintentionally sent when these
277 scripts are run from cron(8) diagnostics are always sent to a log file.
278 By default, this file is $PCP_LOG_DIR/pmlogger/pmlogger_daily.log or
279 $PCP_LOG_DIR/pmlogger/pmlogger_check.log but this can be changed using
280 the -l option. If this log file already exists when the script starts,
281 it will be renamed with a .prev suffix (overwriting any log file saved
282 earlier) before diagnostics are generated to the log file. The -l and
283 -t options cannot be used together.
284 The output from the cron execution of the scripts may be extended using
286 the -V option to the scripts which will enable verbose tracing of their
287 activity. By default the scripts generate no output unless some error
288 or warning condition is encountered.
289
292 $PCP_PMLOGGERCONTROL_PATH
293 the PCP logger control file
294 Warning: this file must not be writable by any user other
295 than root.
296 $PCP_PMLOGGERCONTROL_PATH.d
298 optional directory containing additional PCP logger control
299 files, typically one per host
300 Warning: the files herein must not be writable by any user
301 other than root.
302 $PCP_SYSCONF_DIR/pmlogger/crontab
304 sample crontab for automated script execution by $PCP_USER
305 (or root). Exists only if the platform does not support the
306 /etc/cron.d mechanism.
307 $PCP_VAR_DIR/config/pmlogger/config.default
309 default pmlogger configuration file location for the local
310 primary logger, typically generated automatically by pmlog‐
311 conf(1).
312 $PCP_LOG_DIR/pmlogger/hostname
314 default location for archives of performance information col‐
315 lected from the host hostname
316 $PCP_LOG_DIR/pmlogger/hostname/lock
318 transient lock file to guarantee mutual exclusion during
319 pmlogger administration for the host hostname - if present,
320 can be safely removed if neither pmlogger_daily nor pmlog‐
321 ger_check are running
322 $PCP_LOG_DIR/pmlogger/hostname/Latest
324 PCP archive folio created by mkaf(1) for the most recently
325 launched archive containing performance metrics from the host
326 hostname
327 $PCP_LOG_DIR/NOTICES
329 PCP ``notices'' file used by pmie(1) and friends
330 $PCP_LOG_DIR/pmlogger/pmlogger_check.log
332 if the previous execution of pmlogger_check produced any out‐
333 put it is saved here. The normal case is no output in which
334 case the file does not exist.
335 $PCP_LOG_DIR/pmlogger/pmlogger_daily.log
337 if the previous execution of pmlogger_daily produced any out‐
338 put it is saved here. The normal case is no output in which
339 case the file does not exist.
340 $PCP_LOG_DIR/pmlogger/hostname/SaveLogs
342 if this directory exists, then the log file from the -l argu‐
343 ment of a newly launched pmlogger(1) for hostname will be
344 linked into this directory with the name archive.log where
345 archive is the basename of the associated pmlogger(1) PCP ar‐
346 chive files. This allows the log file to be inspected at a
347 later time, even if several pmlogger(1) instances for host‐
348 name have been launched in the interim. Because the cron-
349 driven PCP archive management scripts run under the uid of
350 the user ``pcp'', $PCP_LOG_DIR/pmlogger/hostname/SaveLogs
351 typically needs to be owned by the user ``pcp''.
352 $PCP_LOG_DIR/pmlogger/.NeedRewrite
354 if this file exists, then this is treated as equivalent to
355 using -R on the command line and the file will be removed
356 once all rewriting has been done.
357
359 Environment variables with the prefix PCP_ are used to parameterize the
360 file and directory names used by PCP. On each installation, the file
361 /etc/pcp.conf contains the local values for these variables. The
362 $PCP_CONF variable may be used to specify an alternative configuration
363 file, as described in pcp.conf(5).
364
366 egrep(1), PCPIntro(1), pmconfig(1), pmlc(1), pmlogconf(1), pmlogger(1),
367 pmlogger_daily_report(1), pmlogger_merge(1), pmlogmv(1), pmlo‐
368 grewrite(1), pmnewlog(1), pmsocks(1), xz(1) and cron(8).
369Performance Co-Pilot PCP PMLOGGER_CHECK(1)