1COLLECTD-PYTHON(5) collectd COLLECTD-PYTHON(5)
2
6 collectd-python - Documentation of collectd's "python plugin"
7
9 LoadPlugin python
10 # ...
11 <Plugin python>
12 ModulePath "/path/to/your/python/modules"
13 LogTraces true
14 Interactive false
15 Import "spam"
16 <Module spam>
18 spam "wonderful" "lovely"
19 </Module>
20 </Plugin>
21
23 The "python plugin" embeds a Python-interpreter into collectd and
24 provides an interface to collectd's plugin system. This makes it
25 possible to write plugins for collectd in Python. This is a lot more
26 efficient than executing a Python-script every time you want to read a
27 value with the "exec plugin" (see collectd-exec(5)) and provides a lot
28 more functionality, too.
29 The minimum required Python version is 2.6.
31
33 LoadPlugin Plugin
34 Loads the Python plugin Plugin.
35 Encoding Name
37 The default encoding for Unicode objects you pass to collectd. If
38 you omit this option it will default to ascii on Python 2. On
39 Python 3 it will always be utf-8, as this function was removed, so
40 this will be silently ignored. These defaults are hardcoded in
41 Python and will ignore everything else, including your locale.
42 ModulePath Name
44 Prepends Name to sys.path. You won't be able to import any scripts
45 you wrote unless they are located in one of the directories in this
46 list. Please note that it only has effect on plugins loaded after
47 this option. You can use multiple ModulePath lines to add more than
48 one directory.
49 LogTraces bool
51 If a Python script throws an exception it will be logged by
52 collectd with the name of the exception and the message. If you set
53 this option to true it will also log the full stacktrace just like
54 the default output of an interactive Python interpreter. This does
55 not apply to the CollectError exception, which will never log a
56 stacktrace. This should probably be set to false most of the time
57 but is very useful for development and debugging of new modules.
58 Interactive bool
60 This option will cause the module to launch an interactive Python
61 interpreter that reads from and writes to the terminal. Note that
62 collectd will terminate right after starting up if you try to run
63 it as a daemon while this option is enabled so make sure to start
64 collectd with the -f option.
65 The collectd module is not imported into the interpreter's globals.
67 You have to do it manually. Be sure to read the help text of the
68 module, it can be used as a reference guide during coding.
69 This interactive session will behave slightly differently from a
71 daemonized collectd script as well as from a normal Python
72 interpreter:
73 • 1. collectd will try to import the readline module to give you
75 a decent way of entering your commands. The daemonized collectd
76 won't do that.
77 • 2. Python will be handling SIGINT. Pressing Ctrl+C will usually
79 cause collectd to shut down. This would be problematic in an
80 interactive session, therefore Python will be handling it in
81 interactive sessions. This allows you to use Ctrl+C to
82 interrupt Python code without killing collectd. This also means
83 you can catch KeyboardInterrupt exceptions which does not work
84 during normal operation.
85 To quit collectd send EOF (press Ctrl+D at the beginning of a
87 new line).
88 • 3. collectd handles SIGCHLD. This means that Python won't be
90 able to determine the return code of spawned processes with
91 system(), popen() and subprocess. This will result in Python
92 not using external programs like less to display help texts.
93 You can override this behavior with the PAGER environment
94 variable, e.g. export PAGER=less before starting collectd.
95 Depending on your version of Python this might or might not
96 result in an OSError exception which can be ignored.
97 If you really need to spawn new processes from Python you can
99 register an init callback and reset the action for SIGCHLD to
100 the default behavior. Please note that this will break the exec
101 plugin. Do not even load the exec plugin if you intend to do
102 this!
103 There is an example script located in
105 contrib/python/getsigchld.py to do this. If you import this
106 from collectd.conf SIGCHLD will be handled normally and
107 spawning processes from Python will work as intended.
108 Import Name
110 Imports the python script Name and loads it into the collectd
111 python process. If your python script is not found, be sure its
112 directory exists in python's sys.path. You can prepend to the
113 sys.path using the ModulePath configuration option.
114 <Module Name> block
116 This block may be used to pass on configuration settings to a
117 Python module. The configuration is converted into an instance of
118 the Config class which is passed to the registered configuration
119 callback. See below for details about the Config class and how to
120 register callbacks.
121 The name identifies the callback.
123
125 There are a lot of places where strings are sent from collectd to
126 Python and from Python to collectd. How exactly this works depends on
127 whether byte or unicode strings or Python2 or Python3 are used.
128 Python2 has str, which is just bytes, and unicode. Python3 has str,
130 which is a unicode object, and bytes.
131 When passing strings from Python to collectd all of these object are
133 supported in all places, however str should be used if possible. These
134 strings must not contain a NUL byte. Ignoring this will result in a
135 TypeError exception. If a byte string was used it will be used as is
136 by collectd. If a unicode object was used it will be encoded using the
137 default encoding (see above). If this is not possible Python will raise
138 a UnicodeEncodeError exception.
139 When passing strings from collectd to Python the behavior depends on
141 the Python version used. Python2 will always receive a str object.
142 Python3 will usually receive a str object as well, however the original
143 string will be decoded to unicode using the default encoding. If this
144 fails because the string is not a valid sequence for this encoding a
145 bytes object will be returned instead.
146
148 Writing your own plugins is quite simple. collectd manages plugins by
149 means of dispatch functions which call the appropriate callback
150 functions registered by the plugins. Any plugin basically consists of
151 the implementation of these callback functions and initializing code
152 which registers the functions with collectd. See the section "EXAMPLES"
153 below for a really basic example. The following types of callback
154 functions are known to collectd (all of them are optional):
155 configuration functions
157 These are called during configuration if an appropriate Module
158 block has been encountered. It is called once for each Module block
159 which matches the name of the callback as provided with the
160 register_config method - see below.
161 Python thread support has not been initialized at this point so do
163 not use any threading functions here!
164 init functions
166 These are called once after loading the module and before any calls
167 to the read and write functions. It should be used to initialize
168 the internal state of the plugin (e. g. open sockets, ...). This is
169 the earliest point where you may use threads.
170 read functions
172 These are used to collect the actual data. It is called once per
173 interval (see the Interval configuration option of collectd).
174 Usually it will call plugin_dispatch_values to dispatch the values
175 to collectd which will pass them on to all registered write
176 functions. If this function throws any kind of exception the plugin
177 will be skipped for an increasing amount of time until it returns
178 normally again.
179 write functions
181 These are used to write the dispatched values. It is called once
182 for every value that was dispatched by any plugin.
183 flush functions
185 These are used to flush internal caches of plugins. It is usually
186 triggered by the user only. Any plugin which caches data before
187 writing it to disk should provide this kind of callback function.
188 log functions
190 These are used to pass messages of plugins or the daemon itself to
191 the user.
192 notification function
194 These are used to act upon notifications. In general, a
195 notification is a status message that may be associated with a data
196 instance. Usually, a notification is generated by the daemon if a
197 configured threshold has been exceeded (see the section "THRESHOLD
198 CONFIGURATION" in collectd.conf(5) for more details), but any
199 plugin may dispatch notifications as well.
200 shutdown functions
202 These are called once before the daemon shuts down. It should be
203 used to clean up the plugin (e.g. close sockets, ...).
204 Any function (except log functions) may throw an exception in case of
206 errors. The exception will be passed on to the user using collectd's
207 logging mechanism. If a log callback throws an exception it will be
208 printed to standard error instead.
209 See the documentation of the various register_ methods in the section
211 "FUNCTIONS" below for the number and types of arguments passed to each
212 callback function. This section also explains how to register callback
213 functions with collectd.
214 To enable a module, copy it to a place where Python can find it (i. e.
216 a directory listed in sys.path) just as any other Python plugin and add
217 an appropriate Import option to the configuration file. After
218 restarting collectd you're done.
219
221 The following complex types are used to pass values between the Python
222 plugin and collectd:
223 CollectdError
225 This is an exception. If any Python script raises this exception it
226 will still be treated like an error by collectd but it will be logged
227 as a warning instead of an error and it will never generate a
228 stacktrace.
229 class CollectdError(Exception)
231 Basic exception for collectd Python scripts. Throwing this exception
233 will not cause a stacktrace to be logged, even if LogTraces is enabled
234 in the config.
235 Signed
237 The Signed class is just a long. It has all its methods and behaves
238 exactly like any other long object. It is used to indicate if an
239 integer was or should be stored as a signed or unsigned integer object.
240 class Signed(long)
242 This is a long by another name. Use it in meta data dicts to choose the
244 way it is stored in the meta data.
245 Unsigned
247 The Unsigned class is just a long. It has all its methods and behaves
248 exactly like any other long object. It is used to indicate if an
249 integer was or should be stored as a signed or unsigned integer object.
250 class Unsigned(long)
252 This is a long by another name. Use it in meta data dicts to choose the
254 way it is stored in the meta data.
255 Config
257 The Config class is an object which keeps the information provided in
258 the configuration file. The sequence of children keeps one entry for
259 each configuration option. Each such entry is another Config instance,
260 which may nest further if nested blocks are used.
261 class Config(object)
263 This represents a piece of collectd's config file. It is passed to
265 scripts with config callbacks (see register_config) and is of little
266 use if created somewhere else.
267 It has no methods beyond the bare minimum and only exists for its data
269 members.
270 Data descriptors defined here:
272 parent
274 This represents the parent of this node. On the root node of the
275 config tree it will be None.
276 key This is the keyword of this item, i.e. the first word of any given
278 line in the config file. It will always be a string.
279 values
281 This is a tuple (which might be empty) of all value, i.e. words
282 following the keyword in any given line in the config file.
283 Every item in this tuple will be either a string, a float or a
285 boolean, depending on the contents of the configuration file.
286 children
288 This is a tuple of child nodes. For most nodes this will be empty.
289 If this node represents a block instead of a single line of the
290 config file it will contain all nodes in this block.
291 PluginData
293 This should not be used directly but it is the base class for both
294 Values and Notification. It is used to identify the source of a value
295 or notification.
296 class PluginData(object)
298 This is an internal class that is the base for Values and Notification.
300 It is pretty useless by itself and was therefore not exported to the
301 collectd module.
302 Data descriptors defined here:
304 host
306 The hostname of the host this value was read from. For dispatching
307 this can be set to an empty string which means the local hostname
308 as defined in collectd.conf.
309 plugin
311 The name of the plugin that read the data. Setting this member to
312 an empty string will insert "python" upon dispatching.
313 plugin_instance
315 Plugin instance string. May be empty.
316 time
318 This is the Unix timestamp of the time this value was read. For
319 dispatching values this can be set to zero which means "now". This
320 means the time the value is actually dispatched, not the time it
321 was set to 0.
322 type
324 The type of this value. This type has to be defined in your
325 types.db. Attempting to set it to any other value will raise a
326 TypeError exception. Assigning a type is mandatory, calling
327 dispatch without doing so will raise a RuntimeError exception.
328 type_instance
330 Type instance string. May be empty.
331 Values
333 A Value is an object which features a sequence of values. It is based
334 on the PluginData type and uses its members to identify the values.
335 class Values(PluginData)
337 A Values object used for dispatching values to collectd and receiving
339 values from write callbacks.
340 Method resolution order:
342 Values
344 PluginData
345 object
346 Methods defined here:
348 dispatch([type][, values][, plugin_instance][, type_instance][,
350 plugin][, host][, time][, interval]) -> None.
351 Dispatch this instance to the collectd process. The object has
352 members for each of the possible arguments for this method. For a
353 detailed explanation of these parameters see the member of the same
354 same.
355 If you do not submit a parameter the value saved in its member will
357 be submitted. If you do provide a parameter it will be used
358 instead, without altering the member.
359 write([destination][, type][, values][, plugin_instance][,
361 type_instance][, plugin][, host][, time][, interval]) -> None.
362 Write this instance to a single plugin or all plugins if
363 "destination" is omitted. This will bypass the main collectd
364 process and all filtering and caching. Other than that it works
365 similar to "dispatch". In most cases "dispatch" should be used
366 instead of "write".
367 Data descriptors defined here:
369 interval
371 The interval is the timespan in seconds between two submits for the
372 same data source. This value has to be a positive integer, so you
373 can't submit more than one value per second. If this member is set
374 to a non-positive value, the default value as specified in the
375 config file will be used (default: 10).
376 If you submit values more often than the specified interval, the
378 average will be used. If you submit less values, your graphs will
379 have gaps.
380 values
382 These are the actual values that get dispatched to collectd. It has
383 to be a sequence (a tuple or list) of numbers. The size of the
384 sequence and the type of its content depend on the type member your
385 types.db file. For more information on this read the types.db(5)
386 manual page.
387 If the sequence does not have the correct size upon dispatch a
389 RuntimeError exception will be raised. If the content of the
390 sequence is not a number, a TypeError exception will be raised.
391 meta
393 These are the meta data for this Value object. It has to be a
394 dictionary of numbers, strings or bools. All keys must be strings.
395 int and <long> objects will be dispatched as signed integers unless
396 they are between 2**63 and 2**64-1, which will result in a unsigned
397 integer. You can force one of these storage classes by using the
398 classes collectd.Signed and collectd.Unsigned. A meta object
399 received by a write callback will always contain Signed or Unsigned
400 objects.
401 Notification
403 A notification is an object defining the severity and message of the
404 status message as well as an identification of a data instance by means
405 of the members of PluginData on which it is based.
406 class Notification(PluginData) The Notification class is a wrapper
408 around the collectd notification. It can be used to notify other
409 plugins about bad stuff happening. It works similar to Values but has a
410 severity and a message instead of interval and time. Notifications can
411 be dispatched at any time and can be received with
412 register_notification.
413 Method resolution order:
415 Notification
417 PluginData
418 object
419 Methods defined here:
421 dispatch([type][, message][, plugin_instance][, type_instance][,
423 plugin][, host][, time][, severity][, meta]) -> None. Dispatch a
424 notification.
425 Dispatch this instance to the collectd process. The object has
426 members for each of the possible arguments for this method. For a
427 detailed explanation of these parameters see the member of the same
428 same.
429 If you do not submit a parameter the value saved in its member will
431 be submitted. If you do provide a parameter it will be used
432 instead, without altering the member.
433 Data descriptors defined here:
435 message
437 Some kind of description of what's going on and why this
438 Notification was generated.
439 severity
441 The severity of this notification. Assign or compare to
442 NOTIF_FAILURE, NOTIF_WARNING or NOTIF_OKAY.
443 meta
445 These are the meta data for the Notification object. It has to be
446 a dictionary of numbers, strings or bools. All keys must be
447 strings. int and long objects will be dispatched as signed integers
448 unless they are between 2**63 and 2**64-1, which will result in a
449 unsigned integer. One of these storage classes can be forced by
450 using the classes collectd.Signed and collectd.Unsigned. A meta
451 object received by a notification callback will always contain
452 Signed or Unsigned objects.
453
455 The following functions provide the C-interface to Python-modules.
456 register_*(callback[, data][, name]) -> identifier
458 There are eight different register functions to get callback for
459 eight different events. With one exception all of them are called
460 as shown above.
461 • callback is a callable object that will be called every time
463 the event is triggered.
464 • data is an optional object that will be passed back to the
466 callback function every time it is called. If you omit this
467 parameter no object is passed back to your callback, not even
468 None.
469 • name is an optional identifier for this callback. The default
471 name is python.module. module is taken from the __module__
472 attribute of your callback function. Every callback needs a
473 unique identifier, so if you want to register the same callback
474 multiple times in the same module you need to specify a name
475 here. Otherwise it's safe to ignore this parameter.
476 • identifier is the full identifier assigned to this callback.
478 These functions are called in the various stages of the daemon (see
480 the section "WRITING YOUR OWN PLUGINS" above) and are passed the
481 following arguments:
482 register_config
484 The only argument passed is a Config object. See above for the
485 layout of this data type. Note that you cannot receive the
486 whole config files this way, only Module blocks inside the
487 Python configuration block. Additionally you will only receive
488 blocks where your callback identifier matches python.blockname.
489 register_init
491 The callback will be called without arguments.
492 register_read(callback[, interval][, data][, name]) -> identifier
494 This function takes an additional parameter: interval. It
495 specifies the time between calls to the callback function.
496 The callback will be called without arguments.
498 register_shutdown
500 The callback will be called without arguments.
501 register_write
503 The callback function will be called with one argument passed,
504 which will be a Values object. For the layout of Values see
505 above. If this callback function throws an exception the next
506 call will be delayed by an increasing interval.
507 register_flush
509 Like register_config is important for this callback because it
510 determines what flush requests the plugin will receive.
511 The arguments passed are timeout and identifier. timeout
513 indicates that only data older than timeout seconds is to be
514 flushed. identifier specifies which values are to be flushed.
515 register_log
517 The arguments are severity and message. The severity is an
518 integer and small for important messages and high for less
519 important messages. The least important level is LOG_DEBUG, the
520 most important level is LOG_ERR. In between there are (from
521 least to most important): LOG_INFO, LOG_NOTICE, and
522 LOG_WARNING. message is simply a string without a newline at
523 the end.
524 If this callback throws an exception it will not be logged. It
526 will just be printed to sys.stderr which usually means silently
527 ignored.
528 register_notification
530 The only argument passed is a Notification object. See above
531 for the layout of this data type.
532 unregister_*(identifier) -> None
534 Removes a callback or data-set from collectd's internal list of
535 callback functions. Every register_* function has an unregister_*
536 function. identifier is either the string that was returned by the
537 register function or a callback function. The identifier will be
538 constructed in the same way as for the register functions.
539 get_dataset(name) -> definition
541 Returns the definition of a dataset specified by name. definition
542 is a list of tuples, each representing one data source. Each tuple
543 has 4 values:
544 name
546 A string, the name of the data source.
547 type
549 A string that is equal to either of the variables
550 DS_TYPE_COUNTER, DS_TYPE_GAUGE, DS_TYPE_DERIVE or
551 DS_TYPE_ABSOLUTE.
552 min A float or None, the minimum value.
554 max A float or None, the maximum value.
556 flush(plugin[, timeout][, identifier]) - None
558 Flush one or all plugins. timeout and the specified identifiers are
559 passed on to the registered flush-callbacks. If omitted, the
560 timeout defaults to "-1". The identifier defaults to None. If the
561 plugin argument has been specified, only named plugin will be
562 flushed.
563 error, warning, notice, info, debug(message)
565 Log a message with the specified severity.
566
568 Any Python module will start similar to:
569 import collectd
571 A very simple read function might look like:
573 import random
575 def read(data=None):
577 vl = collectd.Values(type='gauge')
578 vl.plugin='python.spam'
579 vl.dispatch(values=[random.random() * 100])
580 A very simple write function might look like:
582 def write(vl, data=None):
584 for i in vl.values:
585 print "%s (%s): %f" % (vl.plugin, vl.type, i)
586 To register those functions with collectd:
588 collectd.register_read(read)
590 collectd.register_write(write)
591 See the section "CLASSES" above for a complete documentation of the
593 data types used by the read, write and match functions.
594
596 • collectd is heavily multi-threaded. Each collectd thread accessing
597 the Python plugin will be mapped to a Python interpreter thread.
598 Any such thread will be created and destroyed transparently and on-
599 the-fly.
600 Hence, any plugin has to be thread-safe if it provides several
602 entry points from collectd (i. e. if it registers more than one
603 callback or if a registered callback may be called more than once
604 in parallel).
605 • The Python thread module is initialized just before calling the
607 init callbacks. This means you must not use Python's threading
608 module prior to this point. This includes all config and possibly
609 other callback as well.
610 • The python plugin exports the internal API of collectd which is
612 considered unstable and subject to change at any time. We try hard
613 to not break backwards compatibility in the Python API during the
614 life cycle of one major release. However, this cannot be
615 guaranteed at all times. Watch out for warnings dispatched by the
616 python plugin after upgrades.
617
619 • Not all aspects of the collectd API are accessible from Python.
620 This includes but is not limited to filters.
621
623 collectd(1), collectd.conf(5), collectd-perl(5), collectd-exec(5),
624 types.db(5), python(1),
625
627 The "python plugin" has been written by Sven Trenkel
628 <collectd at semidefinite.de>.
629 This manpage has been written by Sven Trenkel
631 <collectd at semidefinite.de>. It is based on the collectd-perl(5)
632 manual page by Florian Forster <octo at collectd.org> and Sebastian
633 Harl <sh at tokkee.org>.
6345.12.0 2020-09-03 COLLECTD-PYTHON(5)