1PMDASTATSD(1)               General Commands Manual              PMDASTATSD(1)
2
3
4

NAME

6       pmdastatsd - StatsD protocol performance metric domain agent (PMDA)
7

SYNOPSIS

9       $PCP_PMDAS_DIR/statsd/pmdastatsd [-Z maximum udp packet size] [-P port]
10       [-v] [-g] [-o debug output filename] [-s] [-r parser  type]  [-a  port]
11       [-z maximum of unprocessed packets]
12

DESCRIPTION

14       StatsD is simple, text-based UDP protocol for receiving monitoring data
15       of applications in architecture client-server.  pmdastatsd is  an  Per‐
16       formance  Metrics  Domain  Agent  that collects StatsD data, aggregates
17       them and makes them available to any Performance Co-Pilot client, which
18       is ideal for easily tracking stats in your application.
19
20       The  statsd  PMDA  supports Counter, Gauge and Duration (with instances
21       for minimum, maximum,  median,  average,  90th  percentile,  95th  per‐
22       centile,  99th  percentile, count and standard deviation) metric types,
23       offers multiple parsing options: Ragel  or  handwritten/custom  parser,
24       offers  multiple  aggregating  options  for duration metric type: basic
25       histogram or HDR histogram, supports custom form  of  labels,  logging,
26       exports multiple metrics about itself and may be configured either with
27       an ini file or command line options.
28

CONFIGURATION

30       A brief description of the pmdastatsd command line options follows:
31
32       -Z, --max-udp=<value>
33            Maximum allowed packet size, any larger then this will  be  thrown
34            away.  Default: 1472
35
36       -P, --port=<value>
37            Which  port  agent  will  listen to for incoming trafic.  Default:
38            8125
39
40       -v, --verbose
41            Verbosity level.  Prints info about agent execution into  logfile.
42            Valid values are 0-2. 0 = Default value, shows config information,
43            read socket state, and first 100 dropped messages. 1 = Shows  PMNS
44            and  related  information. 2 = Most detailed verbosity level, also
45            shows dropped  messages  above  100.   All  levels  include  those
46            belows.
47
48       -o, --debug-output-filename=<value>
49            You  can send USR1 signal that 'asks' agent to output basic infor‐
50            mation     about     all     aggregated     metric     into      a
51            $PCP_LOG_DIR/pmcd/statsd_{name} file.  Default: 0
52
53       -s, --version
54            Flag  controlling  whether  or not to log current agent version on
55            start.  Default: 0
56
57       -p, --parser-type=<value>
58            Flag specifying which parser to use for incoming packets; Basic  =
59            0, Ragel = 1.  Default: 0
60
61       -a, --duration-aggregation-type=<value>
62            Flag  specifying which aggregation scheme to use for duration met‐
63            rics; basic histogram = 0, HDR histogram = 1.  Default: 1
64
65       -z, -max-unprocessed-packets=<value>
66            Maximum size of packet queue that the agent will save  in  memory.
67            There  are 2 queues: one for packets that are waiting to be parsed
68            and one for parsed packets before they are  aggregated.   Default:
69            2048
70
71       The   agent   also   looks   for   a   pmdastatsd.ini   file   in   the
72       $PCP_PMDAS_DIR/statsd directory.  There, the same options may be speci‐
73       fied, albeit with slightly different names as follows:
74
75            max_udp_packet_size=<value>
76            port=<value>
77            verbose=<value>
78            debug=<value>
79            debug_output_filename=<value>
80            version=<value>
81            parser_type=<value>
82            duration_aggregation_type=<value>
83            max_unprocessed_packets=<value>
84
85       Should  an option be specified in both pmdastatsd.ini and command line,
86       then the latter takes precedence.  Most of the time you  will  want  to
87       configure the agent with an ini file, as the agent should never be exe‐
88       cuted directly.
89
90       Location of the log file.  By default, a log file named  statsd.log  is
91       written in the current directory of pmcd(1) when pmdastatsd is started,
92       i.e.  $PCP_LOG_DIR/pmcd.  If the log file cannot be created or  is  not
93       writable, output is written to standard error instead.
94

INSTALLATION

96       If you want to install the pmdastatsd, do the following as root:
97
98            # cd $PCP_PMDAS_DIR/statsd
99            # ./Install
100
101       To uninstall, do the following as root:
102
103            # cd $PCP_PMDAS_DIR/statsd
104            # ./Remove
105
106       pmdastatsd  is  launched  by  pmcd(1)  and  should  never  be  executed
107       directly.  The Install and Remove scripts notify pmcd(1) when the agent
108       is installed or removed.
109

USAGE

111       Once  started, pmdastatsd will listen on specified port for any content
112       in a form of:
113
114           <metricname>:<value>|<type>
115
116       There may be multiple such messages in single datagram, split by a new‐
117       line character, so this:
118
119           <metricname>:<value>|<type>\n<metricname>:<value>|<type>
120
121       is valid as well.  Constraints for each of the above variables are:
122
123           <metricname> = [a-z][a-zA-Z0-9_.]*
124           <value> = described further in each metric type subsection
125           <type> = One of the following: c , g or ms
126
127       If  verbose  logging  is turned on, agent will log every message parsed
128       and related failures.
129
130       All recorded metrics will, if parsed and  aggregated  successfully,  be
131       made available under statsd.*  namespace.
132
133   1 Counter metric
134       Stores  metrics  as  simple  counters,  adding  any  incoming values to
135       already existing ones.
136
137           <metricname>:<value>|c
138
139       Where value is positive number.
140
141       Example:
142
143       After aggregating following messages:
144
145           metric:20|c
146           metric:10|c
147           metric:3.3|c
148
149       Value available to PCP will be:
150
151           # pminfo --fetch statsd.metric
152
153               inst [0 or '/'] value 33.3
154
155   2 Gauge metric
156       Stores metrics as modifiable values, with  an  option  to  either  set,
157       increment or decrement values.
158
159           <metricname>:<value>|g
160
161       Where value can be in a form of:
162
163           '-{value}',  when  negative  value is supplied agent will substract
164           value stored with the value passed
165
166           '+{value}', when positive value with a leading plus  sign  is  sup‐
167           plied, the agent will add the passed value to the value stored
168
169           '{value}',  when  a value without any leading sign is supplied, the
170           agent will set the metric to the passed value.
171
172       Initial value for metric of gauge type is 0.
173
174       Example:
175
176       After aggregating following messages:
177
178           metric:20|g
179           metric:+10|g
180           metric:-3.3|g
181
182       Value available to PCP will be:
183
184           # pminfo --fetch statsd.metric
185
186               inst [0 or '/'] value 26.7
187
188   3 Duration metric
189       Aggregates values either via HDR histogram or simply stores all  values
190       and then calculates instances from all values received.
191
192           <metricname>:<value>|ms
193
194       Where value is a positive number.
195
196       Example:
197
198       With  larger message count, the values may vary based on selected dura‐
199       tion aggregation scheme.
200
201       After aggregating following messages:
202
203           metric:10|ms
204           metric:20|ms
205
206       Values available to PCP will be:
207
208           # pminfo --fetch statsd.metric
209
210               inst[0 or '/min'] value 10
211               inst[1 or '/max'] value 20
212               inst[2 or '/median'] value 10
213               inst[3 or '/average'] value 15
214               inst[4 or '/percentile90'] value 20
215               inst[5 or '/percentile95'] value 20
216               inst[6 or '/percentile99'] value 20
217               inst[7 or '/count'] value 2
218               inst[8 or '/std_deviation'] value 5
219
220       Note:
221
222       Once you send given metricname with specified type, the agent  will  no
223       longer aggregate any messages with same.  metricname but different type
224       and will throw them away.
225
226   4 Labels
227       StatsD datagrams may also contain key:value pairs separated  by  commas
228       like so:
229
230           metric,tagX=X,tagW=W:5|c
231
232           OR
233
234           metric:5|ms|#tagX:X,tagW:W
235
236       Where tagX is a key, X is a value and tagW is a key, W is a value.
237
238       Both key and value of such a pair are [a-ZA-Z0-9_.]{1,}.
239
240       Both  formats  are  interchangeable  and you may combine them together.
241       When key is not unique, right most value takes  precendence.   This  is
242       valid:
243
244           metric,tagX=1:5|c|#tagX:2
245
246       Pair with key tagX will have value of 2.
247
248       You  may use these labels to map specific values to some PCP instances.
249       PCP labels are also assigned to these PCP instances.  Pairs are ordered
250       by key in resulting instance name and label descriptor.
251
252       Single label:
253
254           metric,tagX=X:5|c
255
256       Such  a  payload  would  map to PCP as follows (non-related labels were
257       ommited):
258
259           # pminfo --fetch --labels statsd.metric
260
261               inst [0 or '/tagX=X'] value 5
262               inst [0 or '/tagX=X'] labels {'tagX':'X'}
263
264       As shown earlier you may also send payload with multiple labels.   When
265       multiple  labels  are supplied they are split in instance name by '::'.
266       Example:
267
268           metric,tagX=X,tagW=W:5|c
269
270       This resolves to:
271
272           # pminfo --fetch --labels statsd.metric
273
274               inst [0 or '/tagX=X::tagW=W'] value 5
275               inst [0 or '/tagX=X::tagW=W'] labels {'tagX':'X','tagW':'W'}
276
277       Note:
278
279       Be mindful of the fact  that  duration  metric  type  already  maps  to
280       instances  even  without  any  labels.  Sending labeled value to such a
281       metric creates another 9 (as there are that many hardcoded) instances.
282
283       Example:
284
285           metric:200|ms
286           metric:100|ms
287           metric,target=cpu2:10|ms
288           metric,target=cpu2:100|ms
289           metric,target=cpu2:1000|ms
290
291       Creates 18 instances.   Duration  data  type  and  label  name  compose
292       instance name in following manner:
293
294           # pminfo --fetch --labels statsd.metric
295
296               ...
297               inst [10 or '/max::target=cpu0'] value 1000
298               inst [10 or '/max::target=cpu0'] labels {'target':'cpu0'}
299               ...
300
301   5 Hardcoded stats
302       Agent also exports metrics about itself:
303
304       statsd.pmda.received
305            Number of datagrams that the agent has received
306
307       statsd.pmda.parsed
308            Number of datagrams that were successfully parsed
309
310       statsd.pmda.dropped
311            Number of datagrams that were dropped
312
313       statsd.pmda.aggregated
314            Number of datagrams that were aggregated
315
316       statsd.pmda.metrics_tracked
317            This  metric has 3 instances.  counter - Number of tracked counter
318            metrics.  gauge - Number of tracked  gauge  metrics.   duration  -
319            Number  of  tracked  duration  metrics.  total - Number of tracked
320            metrics total.
321
322       statsd.pmda.settings.max_udp_packet_size
323            Maximum UDP packet size
324
325       statsd.pmda.settings.max_unprocessed_packets
326            Maximum size of unprocessed packets Q
327
328       statsd.pmda.settings.verbose
329            Verbosity flag
330
331       statsd.pmda.settings.debug
332            Debug flag
333
334       statsd.pmda.settings.debug_output_filename
335            Debug output filename
336
337       statsd.pmda.settings.port
338            Port that is listened to
339
340       statsd.pmda.settings.parser_type
341            Used parser type
342
343       statsd.pmda.settings.duration_aggregation_type
344            Used duration aggregation type
345
346       These names are blocklisted for user usage.   No  messages  with  these
347       names  will processed.  While not yet reserved, the whole statsd.pmda.*
348       namespace is not recommended to use for user metrics.
349

FILES

351       $PCP_PMCDCONF_PATH
352                 command line options used to launch pmdastatsd
353       $PCP_PMDAS_DIR/statsd/Install
354                 installation script for the pmdastatsd agent
355       $PCP_PMDAS_DIR/statsd/Remove
356                 undo installation script for the pmdastatsd agent
357       $PCP_LOG_DIR/pmcd/statsd.log
358                 default log file for error  messages  and  other  information
359                 from pmdastatsd
360

PCP ENVIRONMENT

362       Environment variables with the prefix PCP_ are used to parameterize the
363       file and directory names used by PCP.  On each installation,  the  file
364       /etc/pcp.conf  contains  the  local  values  for  these variables.  The
365       $PCP_CONF variable may be used to specify an alternative  configuration
366       file, as described in pcp.conf(5).
367

SEE ALSO

369       PCPIntro(1), pmcd(1), pcp.conf(5) and pcp.env(5).
370
371
372
373Performance Co-Pilot                  PCP                        PMDASTATSD(1)
Impressum