1BABELTRACE2-INTRO(7)          Babeltrace 2 manual         BABELTRACE2-INTRO(7)
2
3
4

NAME

6       babeltrace2-intro - Introduction to Babeltrace 2
7

DESCRIPTION

9       This manual page is an introduction to the Babeltrace 2 project.
10
11       The “WHAT IS BABELTRACE 2?” section describes the parts of the project
12       and shows the major changes from Babeltrace 1 to Babeltrace 2 while the
13       “BABELTRACE 2 CONCEPTS” section defines the core concepts of
14       Babeltrace 2.
15
16       The “TRACE PROCESSING GRAPH REPRESENTATION” section shows how some
17       concepts are visually represented in other Babeltrace 2 manual pages.
18

WHAT IS BABELTRACE 2?

20       Babeltrace 2 is an open-source software project of which the purpose is
21       to process or convert traces (see
22       <https://en.wikipedia.org/wiki/Tracing_(software)>).
23
24       The Babeltrace 2 project includes the following parts:
25
26       Babeltrace 2 library (libbabeltrace2)
27           A shared library with a C API.
28
29           With libbabeltrace2, you can programmatically create plugins and
30           component classes, build and run trace processing graphs, and more
31           (see the “BABELTRACE 2 CONCEPTS” section for more details about
32           those concepts).
33
34           All the other Babeltrace 2 parts rely on this library.
35
36       babeltrace2 command-line program
37           A command-line interface which uses libbabeltrace2 to load plugins,
38           create a trace processing graph, create components, connect their
39           ports correctly, and run the graph.
40
41           You can also use babeltrace2 to list the available plugins or to
42           query an object from a component class.
43
44           See babeltrace2(1).
45
46       Babeltrace 2 Python bindings
47           A Python 3 package (bt2) which offers a Pythonic interface of
48           libbabeltrace2.
49
50           You can perform the same operations which are available in
51           libbabeltrace2 with the Python bindings, but more conveniently and
52           with less code. However, the Python bindings are less performant
53           than libbabeltrace2.
54
55       Babeltrace 2 project’s plugins
56           The Babeltrace 2 plugins shipped with the project.
57
58           Those plugins are not special in that they only rely on
59           libbabeltrace2 and you don’t need them to use libbabeltrace2,
60           babeltrace2(1), or the Python bindings. However, the project’s
61           plugins provide many widely used trace format encoders/decoders as
62           well as common trace processing graph utilities.
63
64           The Babeltrace 2 project’s plugins are:
65
66           ctf
67               Common Trace Format (see <https://diamon.org/ctf/>) (CTF)
68               input/output, including the LTTng live source.
69
70               See babeltrace2-plugin-ctf(7).
71
72           lttng-utils
73               Graph utilities specific to LTTng (see <https://lttng.org/>)
74               traces.
75
76               See babeltrace2-plugin-lttng-utils(7).
77
78           text
79               Plain text input/output.
80
81               See babeltrace2-plugin-text(7).
82
83           utils
84               Common graph utilities (muxer, trimmer, counter, dummy sink).
85
86               See babeltrace2-plugin-utils(7).
87
88   Changes since Babeltrace 1
89       This manual page is an introduction to Babeltrace 2, a rewrite of
90       Babeltrace 1 with a focus on extensibility, flexibility, and
91       interoperability.
92
93       Babeltrace 1 exists since 2010.
94
95       You can install both projects on the same file system as there are no
96       file name conflicts.
97
98       The major improvements brought by Babeltrace 2 are:
99
100       General
101
102           ·   Full plugin support: any user can distribute a Babeltrace 2
103               plugin and, as long as libbabeltrace2 finds it, any application
104               linked to libbabeltrace2 can load it and use it.
105
106               Plugins are not just trace format encoders and decoders: they
107               package source, filter, and sink component classes so that you
108               can connect specialized, reusable components together in a
109               trace processing graph to create a customized trace conversion
110               or analysis device.
111
112               This modular strategy is much like how the FFmpeg (see
113               <https://www.ffmpeg.org/>), GStreamer (see
114               <https://gstreamer.freedesktop.org/>), and DirectShow (see
115               <https://en.wikipedia.org/wiki/DirectShow>) projects approach
116               media stream processing.
117
118           ·   All the parts of the Babeltrace 2 project run on the major
119               operating systems, including Windows and macOS.
120
121           ·   Some component classes, such as sink.text.pretty (similar to
122               the text output format of babeltrace(1)) and sink.text.details,
123               can write color codes to the standard output when it’s
124               connected to a color-enabled terminal.
125
126               The Babeltrace 2 log, printed to the standard output, can also
127               be colorized.
128
129       Command-line interface
130
131           ·   Whereas you can convert traces from one format to another with
132               Babeltrace 1’s CLI tool, babeltrace(1), you can also execute a
133               custom trace manipulation task with babeltrace2(1) thanks to
134               the babeltrace2-run(1) command.
135
136           ·   The babeltrace2-convert(1) command features an automatic source
137               component discovery algorithm to find the best suited
138               components to create for a given non-option argument (file or
139               directory path, or custom string like an LTTng live (see
140               <https://lttng.org>) URL).
141
142               For example:
143
144                   $ babeltrace2 /path/to/ctf/trace
145
146                   $ babeltrace2 net://localhost/host/myhost/my-session
147
148       CTF (see <https://diamon.org/ctf/>) input/output
149
150           ·   The source.ctf.fs component class, which is more or less the
151               equivalent of Babeltrace 1’s ctf input format, has features not
152               found in Babeltrace 1:
153
154               ·   The component handles many trace quirks which are the
155                   results of known tracer bugs and corner cases (LTTng-UST,
156                   LTTng-modules, and barectf (see
157                   <https://github.com/efficios/barectf>)), making it possible
158                   to decode malformed packets.
159
160               ·   The component merges CTF traces sharing the same UUID into
161                   a single, logical trace.
162
163                   This feature supports LTTng 2.11’s tracing session rotation
164                   trace chunks.
165
166           ·   With a sink.ctf.fs component, you can create CTF traces on the
167               file system.
168
169               With babeltrace2(1), you can use the --output-format=ctf and
170               --output options to create an implicit sink.ctf.fs component.
171
172               For example:
173
174                   $ babeltrace2 /path/to/input/trace \
175                                 --output-format=ctf --output=trace-dir
176
177       LTTng live (see <https://lttng.org>) input
178
179           ·   The babeltrace(1) command exits successfully when it cannot
180               find an LTTng live (--input-format=lttng-live option) tracing
181               session.
182
183               The session-not-found-action initialization parameter controls
184               what a source.ctf.lttng-live message iterator does when it
185               cannot find the remote tracing session.
186
187               If the action is end, the message iterator does like
188               babeltrace(1) and simply ends successfully.
189
190               If the action is continue (the default), the message iterator
191               never ends: it keeps on trying until the tracing session
192               exists, indeed subscribing to the session.
193
194       Library
195
196           ·   libbabeltrace2 shares nothing with libbabeltrace.
197
198               The Babeltrace 2 library C API has features such as:
199
200               ·   A single header file.
201
202               ·   Function precondition and postcondition checking.
203
204               ·   Object-oriented model with shared and unique objects.
205
206               ·   Strict C typing and const correctness.
207
208               ·   User-extensible classes.
209
210               ·   Rich, thread-safe error reporting.
211
212               ·   Per-component and per-subsystem logging levels.
213
214               ·   Trace intermediate representation (IR) objects to make the
215                   API trace-format-agnostic.
216
217               ·   A versioned protocol for message interchange between
218                   components to enable forward and backward compatibility.
219
220           ·   You can build the library in developer mode to enable an
221               extensive set of function precondition and postcondition
222               checks.
223
224               The developer mode can help detect programming errors early
225               when you develop a Babeltrace 2 plugin or an application using
226               libbabeltrace2.
227
228               See the project’s README for build-time requirements and
229               detailed build instructions.
230

BABELTRACE 2 CONCEPTS

232       This section defines the main concepts of the Babeltrace 2 project.
233
234       These concepts translate into types and functions in libbabeltrace2 and
235       its Python bindings, but also as command-line actions and options in
236       the babeltrace2 program. The other Babeltrace 2 manual pages assume
237       that you are familiar with the following definitions.
238
239       Some Babeltrace 2 concepts are interdependent: it is normal to jump
240       from one definition to another to understand the big picture.
241
242       Component class
243           A reusable class which you can instantiate as one or more
244           components within a trace processing graph.
245
246           There are three types of component classes used to create the three
247           types of components: source, filter, and sink.
248
249           A component class implements methods, one of which is an
250           initialization method, or constructor, to create a component. You
251           pass initialization parameters to this method to customize the
252           created component. For example, the initialization method of the
253           source.ctf.fs component class accepts a mandatory inputs parameter
254           which is an array of file system path(s) to the CTF trace(s). It
255           also accepts an optional clock-class-offset-ns parameter which is
256           an offset, in nanoseconds, to add to all the clock classes
257           (descriptors of stream clocks) found in the traces’s metadata.
258
259           A component class can have a description and a help text.
260
261       Component
262           A node within a trace processing graph.
263
264           There are three types of components:
265
266           Source component
267               An input component which produces messages.
268
269               Examples: CTF files input, log file input, LTTng live input,
270               random event generator.
271
272           Filter component
273               An intermediate component which can transform the messages it
274               consumes, augment them, sort them, discard them, or create new
275               ones.
276
277               Examples: filter which removes messages based on an expression,
278               filter which adds debugging information to selected events,
279               message muxer, trace trimmer.
280
281           Sink component
282               An output component which consumes messages and usually writes
283               them to one or more formatted files.
284
285               Examples: log file output, CTF files output, pretty-printed
286               plain text output.
287
288           Components are connected together within a trace processing graph
289           through their ports. Source components have output ports, sink
290           components have input ports, and filter components have both.
291
292           A component is the instance of a component class. The terms
293           component and component class instance are equivalent.
294
295           Within a trace processing graph, each component has a unique name.
296           This is not the name of its component class, but an instance name.
297           If human is a component class name, than Nancy and John could be
298           component names.
299
300           Once a graph is configured (the first time it runs), you cannot add
301           components to it for the remaining graph’s lifetime.
302
303       Port
304           A connection point, on a component, from which are sent or where
305           are received messages when the trace processing graph runs.
306
307           An output port is from where messages are sent. An input port is
308           where messages are received. Source components have output ports,
309           sink components have input ports, and filter components have both.
310
311           You can only connect an output port to a single input port.
312
313           All ports do not need to be connected.
314
315           A filter or sink component receiving messages from its input ports
316           is said to consume messages.
317
318           The link between an output port and input port is a connection.
319
320           Once a graph is configured (the first time it runs), you cannot
321           connect ports for the remaining graph’s lifetime.
322
323       Connection
324           The link between an output port and an input port through which
325           messages flow when a trace processing graph runs.
326
327       Message iterator
328           An iterator on an input port of which the returned elements are
329           messages.
330
331           A component or another message iterator can create many message
332           iterators on a single input port, before or while the trace
333           processing graph runs.
334
335       Message
336           The element of a message iterator.
337
338           Messages flow from output ports to input ports.
339
340           A source component message iterator produces messages, while a sink
341           component consumes them. A filter component message iterator can
342           both consume and produce messages.
343
344           The main types of messages are:
345
346           Event
347               A trace event record within a packet or within a stream.
348
349           Packet beginning
350               The beginning of a packet within a stream.
351
352               A packet is a conceptual container of events.
353
354           Packet end
355               The end of a packet within a stream.
356
357           Stream beginning
358               The beginning of a stream.
359
360               A stream is a conceptual container of packets and/or events.
361
362               Usually, a given source component’s output port sends packet
363               and event messages which belong to a single stream, but it’s
364               not required.
365
366           Stream end
367               The end of a stream.
368
369           Discarded events
370               A count of discarded events within a given time interval for a
371               given stream.
372
373           Discarded packets
374               A count of discarded packets within a given time interval for a
375               given stream.
376
377       Trace processing graph
378           A filter graph (see <https://en.wikipedia.org/wiki/Filter_graph>)
379           where nodes are components and messages flow from output ports to
380           input ports.
381
382           You can build a trace processing graph with libbabeltrace2, with
383           the Babeltrace 2 Python bindings, or with the babeltrace2-run(1)
384           and babeltrace2-convert(1) CLI commands.
385
386           When a trace processing graph runs, the sink components consume
387           messages from their input ports, making all the graph’s message
388           iterators work one message at a time to perform the trace
389           conversion or analysis duty.
390
391       Plugin
392           A container, or package, of component classes as a shared library
393           or Python module.
394
395           Each component class within a plugin has a type (source, filter, or
396           sink) and a name. The type and name pair is unique within a given
397           plugin.
398
399           libbabeltrace2 can load a plugin (.so, .dll, or .py file) at run
400           time: the result is a plugin object in which you can find a
401           specific component class and instantiate it within a trace
402           processing graph as a component.
403
404           The babeltrace2 program uses the
405           COMP-CLS-TYPE.PLUGIN-NAME.COMP-CLS-NAME format to identify a
406           specific component class within a specific plugin.  COMP-CLS-TYPE
407           is either source (or src), filter (or flt), or sink.
408
409           You can list the available Babeltrace 2 plugins with the
410           babeltrace2-list-plugins(1) command.
411
412       Query
413           An operation with which you can get a named object from a component
414           class, possibly with custom query parameters.
415
416           The plain text metadata stream of a CTF trace and the available
417           LTTng live sessions of a given LTTng relay daemon are examples of
418           query objects.
419
420           You can use libbabeltrace2, the Babeltrace 2 Python bindings, or
421           the babeltrace2-query(1) CLI command to query a component class’s
422           object.
423

TRACE PROCESSING GRAPH REPRESENTATION

425       In the Babeltrace 2 manual pages, a component is represented with a
426       box. The box has the component class type, plugin name, and component
427       class name at the top. Just below, between square brackets, is its
428       component name within the trace processing graph. Each port is
429       represented with an @ symbol on the border(s) of the component box with
430       its name inside the box. Output ports are on the box’s right border
431       while input ports are on the box’s left border.
432
433       For example, here’s a source component box:
434
435           +------------+
436           | src.ctf.fs |
437           |  [my-src]  |
438           |            |
439           |    stream0 @
440           |    stream1 @
441           |    stream2 @
442           +------------+
443
444       This one is an instance of the source.ctf.fs component class named my-
445       src. It has three output ports named stream0, stream1, and stream2.
446
447       A trace processing graph is represented with multiple component boxes
448       connected together. The connections are arrows from output ports to
449       input ports.
450
451       For example, here’s a simple conversion graph:
452
453           +------------+    +-----------------+    +------------------+
454           | src.ctf.fs |    | flt.utils.muxer |    | sink.text.pretty |
455           |    [ctf]   |    |     [muxer]     |    |      [text]      |
456           |            |    |                 |    |                  |
457           |    stream0 @--->@ in0         out @--->@ in               |
458           |    stream1 @--->@ in1             |    +------------------+
459           |    stream2 @--->@ in2             |
460           +------------+    @ in3             |
461                             +-----------------+
462
463       Note that input port in3 of component muxer is not connected in this
464       example.
465
466       Sometimes, we symbolically represent other resources which are consumed
467       from or produced by components. In this case, arrows are used, but they
468       do not go to or from port symbols (@), except for messages. For
469       example, in the graph above, the ctf source component consumes a CTF
470       trace and the text sink component prints plain text to the terminal, so
471       here’s a more complete diagram:
472
473           CTF trace
474             |
475             |   +------------+    +-----------------+    +------------------+
476             |   | src.ctf.fs |    | flt.utils.muxer |    | sink.text.pretty |
477             '-->|    [ctf]   |    |     [muxer]     |    |      [text]      |
478                 |            |    |                 |    |                  |
479                 |    stream0 @--->@ in0         out @--->@ in               |
480                 |    stream1 @--->@ in1             |    +-----+------------+
481                 |    stream2 @--->@ in2             |          |
482                 +------------+    @ in3             |          '--> Terminal
483                                   +-----------------+
484
485       Here’s another example of a more complex graph which splits a specific
486       stream using some criteria:
487
488           +------------+    +-----------------+    +------------------+
489           | src.ctf.fs |    | flt.utils.muxer |    | sink.text.pretty |
490           |  [ctf-in]  |    |     [muxer]     |    |      [text]      |
491           |            |    |                 |    |                  |
492           |    stream0 @--->@ in0         out @--->@ in               |
493           |    stream1 @--->@ in1             |    +------------------+
494           |    stream2 @-.  @ in2             |
495           +------------+ |  +-----------------+      +-------------+
496                          |                           | sink.ctf.fs |
497                          |                           |  [ctf-out0] |
498                          |  +-------------------+    |             |
499                          |  | flt.some.splitter | .->@ in          |
500                          |  |     [splitter]    | |  +-------------+
501                          |  |                   | |
502                          '->@ in              A @-'  +-------------+
503                             |                 B @-.  | sink.ctf.fs |
504                             +-------------------+ |  |  [ctf-out1] |
505                                                   |  |             |
506                                                   '->@ in          |
507                                                      +-------------+
508

BUGS

510       If you encounter any issue or usability problem, please report it on
511       the Babeltrace bug tracker (see
512       <https://bugs.lttng.org/projects/babeltrace>).
513

RESOURCES

515       The Babeltrace project shares some communication channels with the
516       LTTng project (see <https://lttng.org/>).
517
518       ·   Babeltrace website (see <https://babeltrace.org/>)
519
520       ·   Mailing list (see <https://lists.lttng.org>) for support and
521           development: lttng-dev@lists.lttng.org
522
523       ·   IRC channel (see <irc://irc.oftc.net/lttng>): #lttng on
524           irc.oftc.net
525
526       ·   Bug tracker (see <https://bugs.lttng.org/projects/babeltrace>)
527
528       ·   Git repository (see <https://git.efficios.com/?p=babeltrace.git>)
529
530       ·   GitHub project (see <https://github.com/efficios/babeltrace>)
531
532       ·   Continuous integration (see
533           <https://ci.lttng.org/view/Babeltrace/>)
534
535       ·   Code review (see <https://review.lttng.org/q/project:babeltrace>)
536

AUTHORS

538       The Babeltrace 2 project is the result of hard work by many regular
539       developers and occasional contributors.
540
541       The current project maintainer is Jérémie Galarneau
542       <mailto:jeremie.galarneau@efficios.com>.
543
545       This manual page is part of the Babeltrace 2 project.
546
547       Babeltrace is distributed under the MIT license (see
548       <https://opensource.org/licenses/MIT>).
549

SEE ALSO

551       babeltrace2(1)
552
553
554
555Babeltrace 2.0.1               14 September 2019          BABELTRACE2-INTRO(7)
Impressum