1COLLECTD-PYTHON(5)                 collectd                 COLLECTD-PYTHON(5)
2
3
4

NAME

6       collectd-python - Documentation of collectd's "python plugin"
7

SYNOPSIS

9         LoadPlugin python
10         # ...
11         <Plugin python>
12           ModulePath "/path/to/your/python/modules"
13           LogTraces true
14           Interactive false
15           Import "spam"
16
17           <Module spam>
18             spam "wonderful" "lovely"
19           </Module>
20         </Plugin>
21

DESCRIPTION

23       The "python plugin" embeds a Python-interpreter into collectd and
24       provides an interface to collectd's plugin system. This makes it
25       possible to write plugins for collectd in Python. This is a lot more
26       efficient than executing a Python-script every time you want to read a
27       value with the "exec plugin" (see collectd-exec(5)) and provides a lot
28       more functionality, too.
29
30       The minimum required Python version is 2.6.
31

CONFIGURATION

33       LoadPlugin Plugin
34           Loads the Python plugin Plugin.
35
36       Encoding Name
37           The default encoding for Unicode objects you pass to collectd. If
38           you omit this option it will default to ascii on Python 2. On
39           Python 3 it will always be utf-8, as this function was removed, so
40           this will be silently ignored.  These defaults are hardcoded in
41           Python and will ignore everything else, including your locale.
42
43       ModulePath Name
44           Prepends Name to sys.path. You won't be able to import any scripts
45           you wrote unless they are located in one of the directories in this
46           list. Please note that it only has effect on plugins loaded after
47           this option. You can use multiple ModulePath lines to add more than
48           one directory.
49
50       LogTraces bool
51           If a Python script throws an exception it will be logged by
52           collectd with the name of the exception and the message. If you set
53           this option to true it will also log the full stacktrace just like
54           the default output of an interactive Python interpreter. This does
55           not apply to the CollectError exception, which will never log a
56           stacktrace.  This should probably be set to false most of the time
57           but is very useful for development and debugging of new modules.
58
59       Interactive bool
60           This option will cause the module to launch an interactive Python
61           interpreter that reads from and writes to the terminal. Note that
62           collectd will terminate right after starting up if you try to run
63           it as a daemon while this option is enabled so make sure to start
64           collectd with the -f option.
65
66           The collectd module is not imported into the interpreter's globals.
67           You have to do it manually. Be sure to read the help text of the
68           module, it can be used as a reference guide during coding.
69
70           This interactive session will behave slightly differently from a
71           daemonized collectd script as well as from a normal Python
72           interpreter:
73
74           ·   1. collectd will try to import the readline module to give you
75               a decent way of entering your commands. The daemonized collectd
76               won't do that.
77
78           ·   2. Python will be handling SIGINT. Pressing Ctrl+C will usually
79               cause collectd to shut down. This would be problematic in an
80               interactive session, therefore Python will be handling it in
81               interactive sessions. This allows you to use Ctrl+C to
82               interrupt Python code without killing collectd. This also means
83               you can catch KeyboardInterrupt exceptions which does not work
84               during normal operation.
85
86               To quit collectd send EOF (press Ctrl+D at the beginning of a
87               new line).
88
89           ·   3. collectd handles SIGCHLD. This means that Python won't be
90               able to determine the return code of spawned processes with
91               system(), popen() and subprocess. This will result in Python
92               not using external programs like less to display help texts.
93               You can override this behavior with the PAGER environment
94               variable, e.g. export PAGER=less before starting collectd.
95               Depending on your version of Python this might or might not
96               result in an OSError exception which can be ignored.
97
98               If you really need to spawn new processes from Python you can
99               register an init callback and reset the action for SIGCHLD to
100               the default behavior. Please note that this will break the exec
101               plugin. Do not even load the exec plugin if you intend to do
102               this!
103
104               There is an example script located in
105               contrib/python/getsigchld.py  to do this. If you import this
106               from collectd.conf SIGCHLD will be handled normally and
107               spawning processes from Python will work as intended.
108
109       Import Name
110           Imports the python script Name and loads it into the collectd
111           python process. If your python script is not found, be sure its
112           directory exists in python's sys.path. You can prepend to the
113           sys.path using the ModulePath configuration option.
114
115       <Module Name> block
116           This block may be used to pass on configuration settings to a
117           Python module.  The configuration is converted into an instance of
118           the Config class which is passed to the registered configuration
119           callback. See below for details about the Config class and how to
120           register callbacks.
121
122           The name identifies the callback.
123

STRINGS

125       There are a lot of places where strings are sent from collectd to
126       Python and from Python to collectd. How exactly this works depends on
127       whether byte or unicode strings or Python2 or Python3 are used.
128
129       Python2 has str, which is just bytes, and unicode. Python3 has str,
130       which is a unicode object, and bytes.
131
132       When passing strings from Python to collectd all of these object are
133       supported in all places, however str should be used if possible. These
134       strings must not contain a NUL byte. Ignoring this will result in a
135       TypeError exception.  If a byte string was used it will be used as is
136       by collectd. If a unicode object was used it will be encoded using the
137       default encoding (see above). If this is not possible Python will raise
138       a UnicodeEncodeError exception.
139
140       When passing strings from collectd to Python the behavior depends on
141       the Python version used. Python2 will always receive a str object.
142       Python3 will usually receive a str object as well, however the original
143       string will be decoded to unicode using the default encoding. If this
144       fails because the string is not a valid sequence for this encoding a
145       bytes object will be returned instead.
146

WRITING YOUR OWN PLUGINS

148       Writing your own plugins is quite simple. collectd manages plugins by
149       means of dispatch functions which call the appropriate callback
150       functions registered by the plugins. Any plugin basically consists of
151       the implementation of these callback functions and initializing code
152       which registers the functions with collectd. See the section "EXAMPLES"
153       below for a really basic example. The following types of callback
154       functions are known to collectd (all of them are optional):
155
156       configuration functions
157           These are called during configuration if an appropriate Module
158           block has been encountered. It is called once for each Module block
159           which matches the name of the callback as provided with the
160           register_config method - see below.
161
162           Python thread support has not been initialized at this point so do
163           not use any threading functions here!
164
165       init functions
166           These are called once after loading the module and before any calls
167           to the read and write functions. It should be used to initialize
168           the internal state of the plugin (e. g. open sockets, ...). This is
169           the earliest point where you may use threads.
170
171       read functions
172           These are used to collect the actual data. It is called once per
173           interval (see the Interval configuration option of collectd).
174           Usually it will call plugin_dispatch_values to dispatch the values
175           to collectd which will pass them on to all registered write
176           functions. If this function throws any kind of exception the plugin
177           will be skipped for an increasing amount of time until it returns
178           normally again.
179
180       write functions
181           These are used to write the dispatched values. It is called once
182           for every value that was dispatched by any plugin.
183
184       flush functions
185           These are used to flush internal caches of plugins. It is usually
186           triggered by the user only. Any plugin which caches data before
187           writing it to disk should provide this kind of callback function.
188
189       log functions
190           These are used to pass messages of plugins or the daemon itself to
191           the user.
192
193       notification function
194           These are used to act upon notifications. In general, a
195           notification is a status message that may be associated with a data
196           instance.  Usually, a notification is generated by the daemon if a
197           configured threshold has been exceeded (see the section "THRESHOLD
198           CONFIGURATION" in collectd.conf(5) for more details), but any
199           plugin may dispatch notifications as well.
200
201       shutdown functions
202           These are called once before the daemon shuts down. It should be
203           used to clean up the plugin (e.g. close sockets, ...).
204
205       Any function (except log functions) may throw an exception in case of
206       errors. The exception will be passed on to the user using collectd's
207       logging mechanism. If a log callback throws an exception it will be
208       printed to standard error instead.
209
210       See the documentation of the various register_ methods in the section
211       "FUNCTIONS" below for the number and types of arguments passed to each
212       callback function. This section also explains how to register callback
213       functions with collectd.
214
215       To enable a module, copy it to a place where Python can find it (i. e.
216       a directory listed in sys.path) just as any other Python plugin and add
217       an appropriate Import option to the configuration file. After
218       restarting collectd you're done.
219

CLASSES

221       The following complex types are used to pass values between the Python
222       plugin and collectd:
223
224   CollectdError
225       This is an exception. If any Python script raises this exception it
226       will still be treated like an error by collectd but it will be logged
227       as a warning instead of an error and it will never generate a
228       stacktrace.
229
230        class CollectdError(Exception)
231
232       Basic exception for collectd Python scripts.  Throwing this exception
233       will not cause a stacktrace to be logged, even if LogTraces is enabled
234       in the config.
235
236   Signed
237       The Signed class is just a long. It has all its methods and behaves
238       exactly like any other long object. It is used to indicate if an
239       integer was or should be stored as a signed or unsigned integer object.
240
241        class Signed(long)
242
243       This is a long by another name. Use it in meta data dicts to choose the
244       way it is stored in the meta data.
245
246   Unsigned
247       The Unsigned class is just a long. It has all its methods and behaves
248       exactly like any other long object. It is used to indicate if an
249       integer was or should be stored as a signed or unsigned integer object.
250
251        class Unsigned(long)
252
253       This is a long by another name. Use it in meta data dicts to choose the
254       way it is stored in the meta data.
255
256   Config
257       The Config class is an object which keeps the information provided in
258       the configuration file. The sequence of children keeps one entry for
259       each configuration option. Each such entry is another Config instance,
260       which may nest further if nested blocks are used.
261
262        class Config(object)
263
264       This represents a piece of collectd's config file. It is passed to
265       scripts with config callbacks (see register_config) and is of little
266       use if created somewhere else.
267
268       It has no methods beyond the bare minimum and only exists for its data
269       members.
270
271       Data descriptors defined here:
272
273       parent
274           This represents the parent of this node. On the root node of the
275           config tree it will be None.
276
277       key This is the keyword of this item, i.e. the first word of any given
278           line in the config file. It will always be a string.
279
280       values
281           This is a tuple (which might be empty) of all value, i.e. words
282           following the keyword in any given line in the config file.
283
284           Every item in this tuple will be either a string, a float or a
285           boolean, depending on the contents of the configuration file.
286
287       children
288           This is a tuple of child nodes. For most nodes this will be empty.
289           If this node represents a block instead of a single line of the
290           config file it will contain all nodes in this block.
291
292   PluginData
293       This should not be used directly but it is the base class for both
294       Values and Notification. It is used to identify the source of a value
295       or notification.
296
297        class PluginData(object)
298
299       This is an internal class that is the base for Values and Notification.
300       It is pretty useless by itself and was therefore not exported to the
301       collectd module.
302
303       Data descriptors defined here:
304
305       host
306           The hostname of the host this value was read from. For dispatching
307           this can be set to an empty string which means the local hostname
308           as defined in collectd.conf.
309
310       plugin
311           The name of the plugin that read the data. Setting this member to
312           an empty string will insert "python" upon dispatching.
313
314       plugin_instance
315           Plugin instance string. May be empty.
316
317       time
318           This is the Unix timestamp of the time this value was read. For
319           dispatching values this can be set to zero which means "now". This
320           means the time the value is actually dispatched, not the time it
321           was set to 0.
322
323       type
324           The type of this value. This type has to be defined in your
325           types.db.  Attempting to set it to any other value will raise a
326           TypeError exception.  Assigning a type is mandatory, calling
327           dispatch without doing so will raise a RuntimeError exception.
328
329       type_instance
330           Type instance string. May be empty.
331
332   Values
333       A Value is an object which features a sequence of values. It is based
334       on the PluginData type and uses its members to identify the values.
335
336        class Values(PluginData)
337
338       A Values object used for dispatching values to collectd and receiving
339       values from write callbacks.
340
341       Method resolution order:
342
343       Values
344       PluginData
345       object
346
347       Methods defined here:
348
349       dispatch([type][, values][, plugin_instance][, type_instance][,
350       plugin][, host][, time][, interval]) -> None.
351           Dispatch this instance to the collectd process. The object has
352           members for each of the possible arguments for this method. For a
353           detailed explanation of these parameters see the member of the same
354           same.
355
356           If you do not submit a parameter the value saved in its member will
357           be submitted. If you do provide a parameter it will be used
358           instead, without altering the member.
359
360       write([destination][, type][, values][, plugin_instance][,
361       type_instance][, plugin][, host][, time][, interval]) -> None.
362           Write this instance to a single plugin or all plugins if
363           "destination" is omitted. This will bypass the main collectd
364           process and all filtering and caching. Other than that it works
365           similar to "dispatch". In most cases "dispatch" should be used
366           instead of "write".
367
368       Data descriptors defined here:
369
370       interval
371           The interval is the timespan in seconds between two submits for the
372           same data source. This value has to be a positive integer, so you
373           can't submit more than one value per second. If this member is set
374           to a non-positive value, the default value as specified in the
375           config file will be used (default: 10).
376
377           If you submit values more often than the specified interval, the
378           average will be used. If you submit less values, your graphs will
379           have gaps.
380
381       values
382           These are the actual values that get dispatched to collectd. It has
383           to be a sequence (a tuple or list) of numbers. The size of the
384           sequence and the type of its content depend on the type member your
385           types.db file. For more information on this read the types.db(5)
386           manual page.
387
388           If the sequence does not have the correct size upon dispatch a
389           RuntimeError exception will be raised. If the content of the
390           sequence is not a number, a TypeError exception will be raised.
391
392       meta
393           These are the meta data for this Value object.  It has to be a
394           dictionary of numbers, strings or bools. All keys must be strings.
395           int and <long> objects will be dispatched as signed integers unless
396           they are between 2**63 and 2**64-1, which will result in a unsigned
397           integer.  You can force one of these storage classes by using the
398           classes collectd.Signed and collectd.Unsigned. A meta object
399           received by a write callback will always contain Signed or Unsigned
400           objects.
401
402   Notification
403       A notification is an object defining the severity and message of the
404       status message as well as an identification of a data instance by means
405       of the members of PluginData on which it is based.
406
407       class Notification(PluginData) The Notification class is a wrapper
408       around the collectd notification.  It can be used to notify other
409       plugins about bad stuff happening. It works similar to Values but has a
410       severity and a message instead of interval and time.  Notifications can
411       be dispatched at any time and can be received with
412       register_notification.
413
414       Method resolution order:
415
416       Notification
417       PluginData
418       object
419
420       Methods defined here:
421
422       dispatch([type][, message][, plugin_instance][, type_instance][,
423       plugin][, host][, time][, severity][, meta]) -> None.  Dispatch a
424       notification.
425           Dispatch this instance to the collectd process. The object has
426           members for each of the possible arguments for this method. For a
427           detailed explanation of these parameters see the member of the same
428           same.
429
430           If you do not submit a parameter the value saved in its member will
431           be submitted. If you do provide a parameter it will be used
432           instead, without altering the member.
433
434       Data descriptors defined here:
435
436       message
437           Some kind of description of what's going on and why this
438           Notification was generated.
439
440       severity
441           The severity of this notification. Assign or compare to
442           NOTIF_FAILURE, NOTIF_WARNING or NOTIF_OKAY.
443
444       meta
445           These are the meta data for the Notification object.  It has to be
446           a dictionary of numbers, strings or bools. All keys must be
447           strings. int and long objects will be dispatched as signed integers
448           unless they are between 2**63 and 2**64-1, which will result in a
449           unsigned integer.  One of these storage classes can be forced by
450           using the classes collectd.Signed and collectd.Unsigned. A meta
451           object received by a notification callback will always contain
452           Signed or Unsigned objects.
453

FUNCTIONS

455       The following functions provide the C-interface to Python-modules.
456
457       register_*(callback[, data][, name]) -> identifier
458           There are eight different register functions to get callback for
459           eight different events. With one exception all of them are called
460           as shown above.
461
462           ·   callback is a callable object that will be called every time
463               the event is triggered.
464
465           ·   data is an optional object that will be passed back to the
466               callback function every time it is called. If you omit this
467               parameter no object is passed back to your callback, not even
468               None.
469
470           ·   name is an optional identifier for this callback. The default
471               name is python.module. module is taken from the __module__
472               attribute of your callback function. Every callback needs a
473               unique identifier, so if you want to register the same callback
474               multiple times in the same module you need to specify a name
475               here. Otherwise it's safe to ignore this parameter.
476
477           ·   identifier is the full identifier assigned to this callback.
478
479           These functions are called in the various stages of the daemon (see
480           the section "WRITING YOUR OWN PLUGINS" above) and are passed the
481           following arguments:
482
483           register_config
484               The only argument passed is a Config object. See above for the
485               layout of this data type.  Note that you cannot receive the
486               whole config files this way, only Module blocks inside the
487               Python configuration block. Additionally you will only receive
488               blocks where your callback identifier matches python.blockname.
489
490           register_init
491               The callback will be called without arguments.
492
493           register_read(callback[, interval][, data][, name]) -> identifier
494               This function takes an additional parameter: interval. It
495               specifies the time between calls to the callback function.
496
497               The callback will be called without arguments.
498
499           register_shutdown
500               The callback will be called without arguments.
501
502           register_write
503               The callback function will be called with one argument passed,
504               which will be a Values object. For the layout of Values see
505               above.  If this callback function throws an exception the next
506               call will be delayed by an increasing interval.
507
508           register_flush
509               Like register_config is important for this callback because it
510               determines what flush requests the plugin will receive.
511
512               The arguments passed are timeout and identifier. timeout
513               indicates that only data older than timeout seconds is to be
514               flushed. identifier specifies which values are to be flushed.
515
516           register_log
517               The arguments are severity and message. The severity is an
518               integer and small for important messages and high for less
519               important messages. The least important level is LOG_DEBUG, the
520               most important level is LOG_ERR. In between there are (from
521               least to most important): LOG_INFO, LOG_NOTICE, and
522               LOG_WARNING. message is simply a string without a newline at
523               the end.
524
525               If this callback throws an exception it will not be logged. It
526               will just be printed to sys.stderr which usually means silently
527               ignored.
528
529           register_notification
530               The only argument passed is a Notification object. See above
531               for the layout of this data type.
532
533       unregister_*(identifier) -> None
534           Removes a callback or data-set from collectd's internal list of
535           callback functions. Every register_* function has an unregister_*
536           function.  identifier is either the string that was returned by the
537           register function or a callback function. The identifier will be
538           constructed in the same way as for the register functions.
539
540       get_dataset(name) -> definition
541           Returns the definition of a dataset specified by name. definition
542           is a list of tuples, each representing one data source. Each tuple
543           has 4 values:
544
545           name
546               A string, the name of the data source.
547
548           type
549               A string that is equal to either of the variables
550               DS_TYPE_COUNTER, DS_TYPE_GAUGE, DS_TYPE_DERIVE or
551               DS_TYPE_ABSOLUTE.
552
553           min A float or None, the minimum value.
554
555           max A float or None, the maximum value.
556
557       flush(plugin[, timeout][, identifier]) - None
558           Flush one or all plugins. timeout and the specified identifiers are
559           passed on to the registered flush-callbacks. If omitted, the
560           timeout defaults to "-1". The identifier defaults to None. If the
561           plugin argument has been specified, only named plugin will be
562           flushed.
563
564       error, warning, notice, info, debug(message)
565           Log a message with the specified severity.
566

EXAMPLES

568       Any Python module will start similar to:
569
570         import collectd
571
572       A very simple read function might look like:
573
574         import random
575
576         def read(data=None):
577           vl = collectd.Values(type='gauge')
578           vl.plugin='python.spam'
579           vl.dispatch(values=[random.random() * 100])
580
581       A very simple write function might look like:
582
583         def write(vl, data=None):
584           for i in vl.values:
585             print "%s (%s): %f" % (vl.plugin, vl.type, i)
586
587       To register those functions with collectd:
588
589         collectd.register_read(read)
590         collectd.register_write(write)
591
592       See the section "CLASSES" above for a complete documentation of the
593       data types used by the read, write and match functions.
594

CAVEATS

596       ·   collectd is heavily multi-threaded. Each collectd thread accessing
597           the Python plugin will be mapped to a Python interpreter thread.
598           Any such thread will be created and destroyed transparently and on-
599           the-fly.
600
601           Hence, any plugin has to be thread-safe if it provides several
602           entry points from collectd (i. e. if it registers more than one
603           callback or if a registered callback may be called more than once
604           in parallel).
605
606       ·   The Python thread module is initialized just before calling the
607           init callbacks.  This means you must not use Python's threading
608           module prior to this point. This includes all config and possibly
609           other callback as well.
610
611       ·   The python plugin exports the internal API of collectd which is
612           considered unstable and subject to change at any time. We try hard
613           to not break backwards compatibility in the Python API during the
614           life cycle of one major release.  However, this cannot be
615           guaranteed at all times. Watch out for warnings dispatched by the
616           python plugin after upgrades.
617

KNOWN BUGS

619       ·   Not all aspects of the collectd API are accessible from Python.
620           This includes but is not limited to filters.
621

SEE ALSO

623       collectd(1), collectd.conf(5), collectd-perl(5), collectd-exec(5),
624       types.db(5), python(1),
625

AUTHOR

627       The "python plugin" has been written by Sven Trenkel
628       <collectd at semidefinite.de>.
629
630       This manpage has been written by Sven Trenkel
631       <collectd at semidefinite.de>.  It is based on the collectd-perl(5)
632       manual page by Florian Forster <octo at collectd.org> and Sebastian
633       Harl <sh at tokkee.org>.
634
635
636
6375.9.1.599.gbb5cfbc                2020-03-08                COLLECTD-PYTHON(5)
Impressum