1LTTNG-CONCEPTS(7) LTTng Manual LTTNG-CONCEPTS(7)
2
6 lttng-concepts - LTTng concepts
7
9 This manual page documents the concepts of LTTng.
10 Many other LTTng manual pages refer to this one so that you can
12 understand what are the various LTTng objects and how they relate to
13 each other.
14 The concepts of LTTng 2.13.4 are:
16 • Instrumentation point, event rule, and event
18 • Trigger
20 • Recording session
22 • Tracing domain
24 • Channel and ring buffer
26 • Recording event rule and event record
28
30 An instrumentation point is a point, within a piece of software, which,
31 when executed, creates an LTTng event.
32 LTTng offers various types of instrumentation; see the “Instrumentation
34 point types” section below to learn about them.
35 An event rule is a set of conditions to match a set of events.
37 When LTTng creates an event E, an event rule ER is said to match E
39 when E satisfies all the conditions of ER. This concept is similar to a
40 regular expression which matches a set of strings.
41 When an event rule matches an event, LTTng emits the event, therefore
43 attempting to execute one or more actions.
44 Important
46 The event creation and emission processes are documentation
47 concepts to help understand the journey from an instrumentation
48 point to the execution of actions.
49 The actual creation of an event can be costly because LTTng needs
51 to evaluate the arguments of the instrumentation point.
52 In practice, LTTng implements various optimizations for the Linux
54 kernel and user space tracing domains (see the “TRACING DOMAIN”
55 section below) to avoid actually creating an event when the tracer
56 knows, thanks to properties which are independent from the event
57 payload and current context, that it would never emit such an
58 event. Those properties are:
59 • The instrumentation point type (see the “Instrumentation point
61 types” section below).
62 • The instrumentation point name.
64 • The instrumentation point log level.
66 • For a recording event rule (see the “RECORDING EVENT RULE AND
68 EVENT RECORD” section below):
69 • The status of the rule itself.
71 • The status of the channel (see the “CHANNEL AND RING
73 BUFFER” section below).
74 • The activity of the recording session (started or stopped;
76 see the “RECORDING SESSION” section below).
77 • Whether or not the process for which LTTng would create the
79 event is allowed to record events (see lttng-track(1)).
80 In other words: if, for a given instrumentation point IP, the LTTng
82 tracer knows that it would never emit an event, executing IP
83 represents a simple boolean variable check and, for a Linux kernel
84 recording event rule, a few process attribute checks.
85 As of LTTng 2.13.4, there are two places where you can find an event
87 rule:
88 Recording event rule
90 A specific type of event rule of which the action is to record the
91 matched event as an event record.
92 See the “RECORDING EVENT RULE AND EVENT RECORD” section below.
94 Create or enable a recording event rule with the lttng-enable-
96 event(1) command.
97 List the recording event rules of a specific recording session
99 and/or channel with the lttng-list(1) and lttng-status(1) commands.
100 “Event rule matches” trigger condition (since LTTng 2.13)
102 When the event rule of the trigger condition matches an event,
103 LTTng can execute user-defined actions such as sending an LTTng
104 notification, starting a recording session, and more.
105 See lttng-add-trigger(1) and lttng-event-rule(7).
107 For LTTng to emit an event E, E must satisfy all the basic conditions
109 of an event rule ER, that is:
110 • The instrumentation point from which LTTng creates E has a specific
112 type.
113 See the “Instrumentation point types” section below.
115 • A pattern matches the name of E while another pattern doesn’t.
117 • The log level of the instrumentation point from which LTTng
119 creates E is at least as severe as some value, or is exactly some
120 value.
121 • The fields of the payload of E and the current context fields
123 satisfy a filter expression.
124 A recording event rule has additional, implicit conditions to satisfy.
126 See the “RECORDING EVENT RULE AND EVENT RECORD” section below to learn
127 more.
128 Instrumentation point types
130 As of LTTng 2.13.4, the available instrumentation point types are,
131 depending on the tracing domain (see the “TRACING DOMAIN” section
132 below):
133 Linux kernel
135 LTTng tracepoint
137 A statically defined point in the source code of the kernel
138 image or of a kernel module using the LTTng-modules macros.
139 List the available Linux kernel tracepoints with lttng list
141 --kernel. See lttng-list(1) to learn more.
142 Linux kernel system call
144 Entry, exit, or both of a Linux kernel system call.
145 List the available Linux kernel system call instrumentation
147 points with lttng list --kernel --syscall. See lttng-list(1) to
148 learn more.
149 Linux kprobe
151 A single probe dynamically placed in the compiled kernel code.
152 When you create such an instrumentation point, you set its
154 memory address or symbol name.
155 Linux user space probe
157 A single probe dynamically placed at the entry of a compiled
158 user space application/library function through the kernel.
159 When you create such an instrumentation point, you set:
161 With the ELF method
163 Its application/library path and its symbol name.
164 With the USDT method
166 Its application/library path, its provider name, and its
167 probe name.
168 “USDT” stands for SystemTap User-level Statically Defined
170 Tracing, a DTrace-style marker.
171 As of LTTng 2.13.4, LTTng only supports USDT probes which are
173 NOT reference-counted.
174 Linux kretprobe
176 Entry, exit, or both of a Linux kernel function.
177 When you create such an instrumentation point, you set the
179 memory address or symbol name of its function.
180 User space
182 LTTng tracepoint
184 A statically defined point in the source code of a C/C++
185 application/library using the LTTng-UST macros.
186 List the available Linux kernel tracepoints with lttng list
188 --userspace. See lttng-list(1) to learn more.
189 java.util.logging, Apache log4j, and Python
191 Java or Python logging statement
193 A method call on a Java or Python logger attached to an
194 LTTng-UST handler.
195 List the available Java and Python loggers with lttng list
197 --jul, lttng list --log4j, and lttng list --python. See lttng-
198 list(1) to learn more.
199
201 A trigger associates a condition to one or more actions.
202 When the condition of a trigger is satisfied, LTTng attempts to execute
204 its actions.
205 As of LTTng 2.13.4, the available trigger conditions and actions are:
207 Conditions
209 • The consumed buffer size of a given recording session (see the
211 “RECORDING SESSION” section below) becomes greater than some
212 value.
213 • The buffer usage of a given channel (see the “CHANNEL AND RING
215 BUFFER” section below) becomes greater than some value.
216 • The buffer usage of a given channel becomes less than some
218 value.
219 • There’s an ongoing recording session rotation (see the
221 “Recording session rotation” section below).
222 • A recording session rotation becomes completed.
224 • An event rule matches an event.
226 As of LTTng 2.13.4, this is the only available condition when
228 you add a trigger with the lttng-add-trigger(1) command. The
229 other ones are available through the liblttng-ctl C API.
230 Actions
232 • Send a notification to a user application.
234 • Start a given recording session, like lttng-start(1) would do.
236 • Stop a given recording session, like lttng-stop(1) would do.
238 • Archive the current trace chunk of a given recording session
240 (rotate), like lttng-rotate(1) would do.
241 • Take a snapshot of a given recording session, like lttng-
243 snapshot(1) would do.
244 A trigger belongs to a session daemon (see lttng-sessiond(8)), not to a
246 specific recording session. For a given session daemon, each Unix user
247 has its own, private triggers. Note, however, that the root Unix user
248 may, for the root session daemon:
249 • Add a trigger as another Unix user.
251 • List all the triggers, regardless of their owner.
253 • Remove a trigger which belongs to another Unix user.
255 For a given session daemon and Unix user, a trigger has a unique name.
257 Add a trigger to a session daemon with the lttng-add-trigger(1)
259 command.
260 List the triggers of your Unix user (or of all users if your Unix user
262 is root) with the lttng-list-triggers(1) command.
263 Remove a trigger with the lttng-remove-trigger(1) command.
265
267 A recording session (named “tracing session” prior to LTTng 2.13) is a
268 stateful dialogue between you and a session daemon (see lttng-
269 sessiond(8)) for everything related to event recording.
270 Everything that you do when you control LTTng tracers to record events
272 happens within a recording session. In particular, a recording session:
273 • Has its own name, unique for a given session daemon.
275 • Has its own set of trace files, if any.
277 • Has its own state of activity (started or stopped).
279 An active recording session is an implicit recording event rule
281 condition (see the “RECORDING EVENT RULE AND EVENT RECORD” section
282 below).
283 • Has its own mode (local, network streaming, snapshot, or live).
285 See the “Recording session modes” section below to learn more.
287 • Has its own channels (see the “CHANNEL AND RING BUFFER” section
289 below) to which are attached their own recording event rules.
290 • Has its own process attribute inclusion sets (see lttng-track(1)).
292 Those attributes and objects are completely isolated between different
294 recording sessions.
295 A recording session is like an ATM session: the operations you do on
297 the banking system through the ATM don’t alter the data of other users
298 of the same system. In the case of the ATM, a session lasts as long as
299 your bank card is inside. In the case of LTTng, a recording session
300 lasts from the lttng-create(1) command to the lttng-destroy(1) command.
301 A recording session belongs to a session daemon (see lttng-
303 sessiond(8)). For a given session daemon, each Unix user has its own,
304 private recording sessions. Note, however, that the root Unix user may
305 operate on or destroy another user’s recording session.
306 Create a recording session with the lttng-create(1) command.
308 List the recording sessions of the connected session daemon with the
310 lttng-list(1) command.
311 Start and stop a recording session with the lttng-start(1) and lttng-
313 stop(1) commands.
314 Save and load a recording session with the lttng-save(1) and lttng-
316 load(1) commands.
317 Archive the current trace chunk of (rotate) a recording session with
319 the lttng-rotate(1) command.
320 Destroy a recording session with the lttng-destroy(1) command.
322 Current recording session
324 When you run the lttng-create(1) command, LTTng creates the
325 $LTTNG_HOME/.lttngrc file if it doesn’t exist ($LTTNG_HOME defaults to
326 $HOME).
327 $LTTNG_HOME/.lttngrc contains the name of the current recording
329 session.
330 When you create a new recording session with the create command, LTTng
332 updates the current recording session.
333 The following lttng(1) commands select the current recording session if
335 you don’t specify one:
336 • lttng-add-context(1)
338 • lttng-clear(1)
340 • lttng-destroy(1)
342 • lttng-disable-channel(1)
344 • lttng-disable-event(1)
346 • lttng-disable-rotation(1)
348 • lttng-enable-channel(1)
350 • lttng-enable-event(1)
352 • lttng-enable-rotation(1)
354 • lttng-regenerate(1)
356 • lttng-rotate(1)
358 • lttng-save(1)
360 • lttng-snapshot(1)
362 • lttng-start(1)
364 • lttng-status(1)
366 • lttng-stop(1)
368 • lttng-track(1)
370 • lttng-untrack(1)
372 • lttng-view(1)
374 Set the current recording session manually with the lttng-set-
376 session(1) command, without having to edit the .lttngrc file.
377 Recording session modes
379 LTTng offers four recording session modes:
380 Local mode
382 Write the trace data to the local file system.
383 Network streaming mode
385 Send the trace data over the network to a listening relay daemon
386 (see lttng-relayd(8)).
387 Snapshot mode
389 Only write the trace data to the local file system or send it to a
390 listening relay daemon (lttng-relayd(8)) when LTTng takes a
391 snapshot.
392 LTTng forces all the channels (see the “CHANNEL AND RING BUFFER”
394 section below) to be created to be configured to be snapshot-ready.
395 LTTng takes a snapshot of such a recording session when:
397 • You run the lttng-snapshot(1) command.
399 • LTTng executes a snapshot-session trigger action (see the
401 “TRIGGER” section above).
402 Live mode
404 Send the trace data over the network to a listening relay daemon
405 (see lttng-relayd(8)) for live reading.
406 An LTTng live reader (for example, babeltrace2(1)) can connect to
408 the same relay daemon to receive trace data while the recording
409 session is active.
410 Recording session rotation
412 A recording session rotation is the action of archiving the current
413 trace chunk of the recording session to the file system.
414 Once LTTng archives a trace chunk, it does NOT manage it anymore: you
416 can read it, modify it, move it, or remove it.
417 An archived trace chunk is a collection of metadata and data stream
419 files which form a self-contained LTTng trace. See the “Trace chunk
420 naming” section below to learn how LTTng names a trace chunk archive
421 directory.
422 The current trace chunk of a given recording session includes:
424 • The stream files which LTTng already wrote to the file system, and
426 which are not part of a previously archived trace chunk, since the
427 most recent event amongst:
428 • The first time the recording session was started, either with
430 the lttng-start(1) command or with a start-session trigger
431 action (see the “TRIGGER” section above).
432 • The last rotation, performed with:
434 • An lttng-rotate(1) command.
436 • A rotation schedule previously set with lttng-enable-
438 rotation(1).
439 • An executed rotate-session trigger action (see the
441 “TRIGGER” section above).
442 • The content of all the non-flushed sub-buffers of the channels of
444 the recording session.
445 Trace chunk archive naming
447 A trace chunk archive is a subdirectory of the archives subdirectory
448 within the output directory of a recording session (see the --output
449 option of the lttng-create(1) command and of lttng-relayd(8)).
450 A trace chunk archive contains, through tracing domain and possibly
452 UID/PID subdirectories, metadata and data stream files.
453 A trace chunk archive is, at the same time:
455 • A self-contained LTTng trace.
457 • A member of a set of trace chunk archives which form the complete
459 trace of a recording session.
460 In other words, an LTTng trace reader can read both the recording
462 session output directory (all the trace chunk archives), or a single
463 trace chunk archive.
464 When LTTng performs a recording session rotation, it names the
466 resulting trace chunk archive as such, relative to the output directory
467 of the recording session:
468 archives/BEGIN-END-ID
470 BEGIN
472 Date and time of the beginning of the trace chunk archive with the
473 ISO 8601-compatible YYYYmmddTHHMMSS±HHMM form, where YYYYmmdd is
474 the date and HHMMSS±HHMM is the time with the time zone offset from
475 UTC.
476 Example: 20171119T152407-0500
478 END
480 Date and time of the end of the trace chunk archive with the
481 ISO 8601-compatible YYYYmmddTHHMMSS±HHMM form, where YYYYmmdd is
482 the date and HHMMSS±HHMM is the time with the time zone offset from
483 UTC.
484 Example: 20180118T152407+0930
486 ID
488 Unique numeric identifier of the trace chunk within its recording
489 session.
490 Trace chunk archive name example:
492 archives/20171119T152407-0500-20171119T151422-0500-3
494
496 A tracing domain identifies a type of LTTng tracer.
497 A tracing domain has its own properties and features.
499 There are currently five available tracing domains:
501 ┌──────────────────┬─────────────────────┬──────────────────┐
503 │Tracing domain │ “Event rule │ Option for other │
504 │ │ matches” trigger │ CLI commands │
505 │ │ condition option │ │
506 ├──────────────────┼─────────────────────┼──────────────────┤
507 │ │ │ │
508 │Linux kernel │ --type option │ --kernel │
509 │ │ starts with kernel: │ │
510 ├──────────────────┼─────────────────────┼──────────────────┤
511 │ │ │ │
512 │User space │ --type option │ --userspace │
513 │ │ starts with user: │ │
514 ├──────────────────┼─────────────────────┼──────────────────┤
515 │ │ │ │
516 │java.util.logging │ --type option │ --jul │
517 │(JUL) │ starts with jul: │ │
518 ├──────────────────┼─────────────────────┼──────────────────┤
519 │ │ │ │
520 │Apache log4j │ --type option │ --log4j │
521 │ │ starts with log4j: │ │
522 ├──────────────────┼─────────────────────┼──────────────────┤
523 │ │ │ │
524 │Python │ --type option │ --python │
525 │ │ starts with python: │ │
526 └──────────────────┴─────────────────────┴──────────────────┘
527 You must specify a tracing domain to target a type of LTTng tracer when
529 using some lttng(1) commands to avoid ambiguity. For example, because
530 the Linux kernel and user space tracing domains support named
531 tracepoints as instrumentation points (see the “INSTRUMENTATION POINT,
532 EVENT RULE, AND EVENT” section above), you need to specify a tracing
533 domain when you create an event rule because both tracing domains could
534 have tracepoints sharing the same name.
535 You can create channels (see the “CHANNEL AND RING BUFFER” section
537 below) in the Linux kernel and user space tracing domains. The other
538 tracing domains have a single, default channel.
539
541 A channel is an object which is responsible for a set of ring buffers.
542 Each ring buffer is divided into multiple sub-buffers. When a recording
544 event rule (see the “RECORDING EVENT RULE AND EVENT RECORD” section
545 below) matches an event, LTTng can record it to one or more sub-buffers
546 of one or more channels.
547 When you create a channel with the lttng-enable-channel(1) command, you
549 set its final attributes, that is:
550 • Its buffering scheme.
552 See the “Buffering scheme” section below.
554 • What to do when there’s no space left for a new event record
556 because all sub-buffers are full.
557 See the “Event record loss mode” section below.
559 • The size of each ring buffer and how many sub-buffers a ring buffer
561 has.
562 See the “Sub-buffer size and count” section below.
564 • The size of each trace file LTTng writes for this channel and the
566 maximum count of trace files.
567 See the “Maximum trace file size and count” section below.
569 • The periods of its read, switch, and monitor timers.
571 See the “Timers” section below.
573 • For a Linux kernel channel: its output type (mmap(2) or splice(2)).
575 See the --output option of the lttng-enable-channel(1) command.
577 • For a user space channel: the value of its blocking timeout.
579 See the --blocking-timeout option of the lttng-enable-channel(1)
581 command.
582 Note that the lttng-enable-event(1) command can automatically create a
584 default channel with sane defaults when no channel exists for the
585 provided tracing domain.
586 A channel is always associated to a tracing domain (see the “TRACING
588 DOMAIN” section below). The java.util.logging (JUL), log4j, and Python
589 tracing domains each have a default channel which you can’t configure.
590 A channel owns recording event rules.
592 List the channels of a given recording session with the lttng-list(1)
594 and lttng-status(1) commands.
595 Disable an enabled channel with the lttng-disable-channel(1) command.
597 Buffering scheme
599 A channel has at least one ring buffer per CPU. LTTng always records an
600 event to the ring buffer dedicated to the CPU which emits it.
601 The buffering scheme of a user space channel determines what has its
603 own set of per-CPU ring buffers:
604 Per-user buffering (--buffers-uid option of the lttng-enable-channel(1)
606 command)
607 Allocate one set of ring buffers (one per CPU) shared by all the
608 instrumented processes of:
609 If your Unix user is root
611 Each Unix user.
612 Otherwise
614 Your Unix user.
615 Per-process buffering (--buffers-pid option of the lttng-enable-
617 channel(1) command)
618 Allocate one set of ring buffers (one per CPU) for each
619 instrumented process of:
620 If your Unix user is root
622 All Unix users.
623 Otherwise
625 Your Unix user.
626 The per-process buffering scheme tends to consume more memory than the
628 per-user option because systems generally have more instrumented
629 processes than Unix users running instrumented processes. However, the
630 per-process buffering scheme ensures that one process having a high
631 event throughput won’t fill all the shared sub-buffers of the same Unix
632 user, only its own.
633 The buffering scheme of a Linux kernel channel is always to allocate a
635 single set of ring buffers for the whole system. This scheme is similar
636 to the per-user option, but with a single, global user “running” the
637 kernel.
638 Event record loss mode
640 When LTTng emits an event, LTTng can record it to a specific, available
641 sub-buffer within the ring buffers of specific channels. When there’s
642 no space left in a sub-buffer, the tracer marks it as consumable and
643 another, available sub-buffer starts receiving the following event
644 records. An LTTng consumer daemon eventually consumes the marked
645 sub-buffer, which returns to the available state.
646 In an ideal world, sub-buffers are consumed faster than they are
648 filled. In the real world, however, all sub-buffers can be full at some
649 point, leaving no space to record the following events.
650 By default, LTTng-modules and LTTng-UST are non-blocking tracers: when
652 there’s no available sub-buffer to record an event, it’s acceptable to
653 lose event records when the alternative would be to cause substantial
654 delays in the execution of the instrumented application. LTTng
655 privileges performance over integrity; it aims at perturbing the
656 instrumented application as little as possible in order to make the
657 detection of subtle race conditions and rare interrupt cascades
658 possible.
659 Since LTTng 2.10, the LTTng user space tracer, LTTng-UST, supports a
661 blocking mode. See the --blocking-timeout of the lttng-enable-
662 channel(1) command to learn how to use the blocking mode.
663 When it comes to losing event records because there’s no available
665 sub-buffer, or because the blocking timeout of the channel is reached,
666 the event record loss mode of the channel determines what to do. The
667 available event record loss modes are:
668 Discard mode
670 Drop the newest event records until a sub-buffer becomes available.
671 This is the only available mode when you specify a blocking
673 timeout.
674 With this mode, LTTng increments a count of lost event records when
676 an event record is lost and saves this count to the trace. A trace
677 reader can use the saved discarded event record count of the trace
678 to decide whether or not to perform some analysis even if trace
679 data is known to be missing.
680 Overwrite mode
682 Clear the sub-buffer containing the oldest event records and start
683 writing the newest event records there.
684 This mode is sometimes called flight recorder mode because it’s
686 similar to a flight recorder
687 <https://en.wikipedia.org/wiki/Flight_recorder>: always keep a
688 fixed amount of the latest data. It’s also similar to the roll mode
689 of an oscilloscope.
690 Since LTTng 2.8, with this mode, LTTng writes to a given sub-buffer
692 its sequence number within its data stream. With a local, network
693 streaming, or live recording session (see the “Recording session
694 modes” section above), a trace reader can use such sequence numbers
695 to report lost packets. A trace reader can use the saved discarded
696 sub-buffer (packet) count of the trace to decide whether or not to
697 perform some analysis even if trace data is known to be missing.
698 With this mode, LTTng doesn’t write to the trace the exact number
700 of lost event records in the lost sub-buffers.
701 Which mechanism you should choose depends on your context: prioritize
703 the newest or the oldest event records in the ring buffer?
704 Beware that, in overwrite mode, the tracer abandons a whole sub-buffer
706 as soon as a there’s no space left for a new event record, whereas in
707 discard mode, the tracer only discards the event record that doesn’t
708 fit.
709 Set the event record loss mode of a channel with the --discard and
711 --overwrite options of the lttng-enable-channel(1) command.
712 There are a few ways to decrease your probability of losing event
714 records. The “Sub-buffer size and count” section below shows how to
715 fine-tune the sub-buffer size and count of a channel to virtually stop
716 losing event records, though at the cost of greater memory usage.
717 Sub-buffer size and count
719 A channel has one or more ring buffer for each CPU of the target
720 system.
721 See the “Buffering scheme” section above to learn how many ring buffers
723 of a given channel are dedicated to each CPU depending on its buffering
724 scheme.
725 Set the size of each sub-buffer the ring buffers of a channel contain
727 with the --subbuf-size option of the lttng-enable-channel(1) command.
728 Set the number of sub-buffers each ring buffer of a channel contains
730 with the --num-subbuf option of the lttng-enable-channel(1) command.
731 Note that LTTng switching the current sub-buffer of a ring buffer
733 (marking a full one as consumable and switching to an available one for
734 LTTng to record the next events) introduces noticeable CPU overhead.
735 Knowing this, the following list presents a few practical situations
736 along with how to configure the sub-buffer size and count for them:
737 High event throughput
739 In general, prefer large sub-buffers to lower the risk of losing
740 event records.
741 Having larger sub-buffers also ensures a lower sub-buffer switching
743 frequency (see the “Timers” section below).
744 The sub-buffer count is only meaningful if you create the channel
746 in overwrite mode (see the “Event record loss mode” section above):
747 in this case, if LTTng overwrites a sub-buffer, then the other
748 sub-buffers are left unaltered.
749 Low event throughput
751 In general, prefer smaller sub-buffers since the risk of losing
752 event records is low.
753 Because LTTng emits events less frequently, the sub-buffer
755 switching frequency should remain low and therefore the overhead of
756 the tracer shouldn’t be a problem.
757 Low memory system
759 If your target system has a low memory limit, prefer fewer first,
760 then smaller sub-buffers.
761 Even if the system is limited in memory, you want to keep the
763 sub-buffers as large as possible to avoid a high sub-buffer
764 switching frequency.
765 Note that LTTng uses CTF <https://diamon.org/ctf/> as its trace format,
767 which means event record data is very compact. For example, the average
768 LTTng kernel event record weights about 32 bytes. Therefore, a
769 sub-buffer size of 1 MiB is considered large.
770 The previous scenarios highlight the major trade-off between a few
772 large sub-buffers and more, smaller sub-buffers: sub-buffer switching
773 frequency vs. how many event records are lost in overwrite mode.
774 Assuming a constant event throughput and using the overwrite mode, the
775 two following configurations have the same ring buffer total size:
776 Two sub-buffers of 4 MiB each
778 Expect a very low sub-buffer switching frequency, but if LTTng ever
779 needs to overwrite a sub-buffer, half of the event records so far
780 (4 MiB) are definitely lost.
781 Eight sub-buffers of 1 MiB each
783 Expect four times the tracer overhead of the configuration above,
784 but if LTTng needs to overwrite a sub-buffer, only the eighth of
785 event records so far (1 MiB) are definitely lost.
786 In discard mode, the sub-buffer count parameter is pointless: use two
788 sub-buffers and set their size according to your requirements.
789 Maximum trace file size and count
791 By default, trace files can grow as large as needed.
792 Set the maximum size of each trace file that LTTng writes of a given
794 channel with the --tracefile-size option of the lttng-enable-channel(1)
795 command.
796 When the size of a trace file reaches the fixed maximum size of the
798 channel, LTTng creates another file to contain the next event records.
799 LTTng appends a file count to each trace file name in this case.
800 If you set the trace file size attribute when you create a channel, the
802 maximum number of trace files that LTTng creates is unlimited by
803 default. To limit them, use the --tracefile-count option of lttng-
804 enable-channel(1). When the number of trace files reaches the fixed
805 maximum count of the channel, LTTng overwrites the oldest trace file.
806 This mechanism is called trace file rotation.
807 Important
809 Even if you don’t limit the trace file count, always assume that
810 LTTng manages all the trace files of the recording session.
811 In other words, there’s no safe way to know if LTTng still holds a
813 given trace file open with the trace file rotation feature.
814 The only way to obtain an unmanaged, self-contained LTTng trace
816 before you destroy the recording session is with the recording
817 session rotation feature (see the “Recording session rotation”
818 section above), which is available since LTTng 2.11.
819 Timers
821 Each channel can have up to three optional timers:
822 Switch timer
824 When this timer expires, a sub-buffer switch happens: for each ring
825 buffer of the channel, LTTng marks the current sub-buffer as
826 consumable and switches to an available one to record the next
827 events.
828 A switch timer is useful to ensure that LTTng consumes and commits
830 trace data to trace files or to a distant relay daemon (lttng-
831 relayd(8)) periodically in case of a low event throughput.
832 Such a timer is also convenient when you use large sub-buffers (see
834 the “Sub-buffer size and count” section above) to cope with a
835 sporadic high event throughput, even if the throughput is otherwise
836 low.
837 Set the period of the switch timer of a channel, or disable the
839 timer altogether, with the --switch-timer option of the lttng-
840 enable-channel(1) command.
841 Read timer
843 When this timer expires, LTTng checks for full, consumable
844 sub-buffers.
845 By default, the LTTng tracers use an asynchronous message mechanism
847 to signal a full sub-buffer so that a consumer daemon can consume
848 it.
849 When such messages must be avoided, for example in real-time
851 applications, use this timer instead.
852 Set the period of the read timer of a channel, or disable the timer
854 altogether, with the --read-timer option of the lttng-enable-
855 channel(1) command.
856 Monitor timer
858 When this timer expires, the consumer daemon samples some channel
859 statistics to evaluate the following trigger conditions:
860 1. The consumed buffer size of a given recording session becomes
862 greater than some value.
863 2. The buffer usage of a given channel becomes greater than some
865 value.
866 3. The buffer usage of a given channel becomes less than some
868 value.
869 If you disable the monitor timer of a channel C:
871 • The consumed buffer size value of the recording session of C
873 could be wrong for trigger condition type 1: the consumed
874 buffer size of C won’t be part of the grand total.
875 • The buffer usage trigger conditions (types 2 and 3) for C will
877 never be satisfied.
878 See the “TRIGGER” section above to learn more about triggers.
880 Set the period of the monitor timer of a channel, or disable the
882 timer altogether, with the --monitor-timer option of the lttng-
883 enable-channel(1) command.
884
886 A recording event rule is a specific type of event rule (see the
887 “INSTRUMENTATION POINT, EVENT RULE, AND EVENT” section above) of which
888 the action is to serialize and record the matched event as an event
889 record.
890 Set the explicit conditions of a recording event rule when you create
892 it with the lttng-enable-event(1) command. A recording event rule also
893 has the following implicit conditions:
894 • The recording event rule itself is enabled.
896 A recording event rule is enabled on creation.
898 • The channel to which the recording event rule is attached is
900 enabled.
901 A channel is enabled on creation.
903 See the “CHANNEL AND RING BUFFER” section above.
905 • The recording session of the recording event rule is active
907 (started).
908 A recording session is inactive (stopped) on creation.
910 See the “RECORDING SESSION” section above.
912 • The process for which LTTng creates an event to match is allowed to
914 record events.
915 All processes are allowed to record events on recording session
917 creation.
918 Use the lttng-track(1) and lttng-untrack(1) commands to select
920 which processes are allowed to record events based on specific
921 process attributes.
922 You always attach a recording event rule to a channel, which belongs to
924 a recording session, when you create it.
925 When a recording event rule ER matches an event E, LTTng attempts to
927 serialize and record E to one of the available sub-buffers of the
928 channel to which E is attached.
929 When multiple matching recording event rules are attached to the same
931 channel, LTTng attempts to serialize and record the matched event once.
932 In the following example, the second recording event rule is redundant
933 when both are enabled:
934 $ lttng enable-event --userspace hello:world
936 $ lttng enable-event --userspace hello:world --loglevel=INFO
937 List the recording event rules of a specific recording session and/or
939 channel with the lttng-list(1) and lttng-status(1) commands.
940 Disable a recording event rule with the lttng-disable-event(1) command.
942 As of LTTng 2.13.4, you cannot remove a recording event rule: it exists
944 as long as its recording session exists.
945
947 • LTTng project website <https://lttng.org>
948 • LTTng documentation <https://lttng.org/docs>
950 • LTTng bug tracker <https://bugs.lttng.org>
952 • Git repositories <https://git.lttng.org>
954 • GitHub organization <https://github.com/lttng>
956 • Continuous integration <https://ci.lttng.org/>
958 • Mailing list <https://lists.lttng.org/> for support and
960 development: lttng-dev@lists.lttng.org
961 • IRC channel <irc://irc.oftc.net/lttng>: #lttng on irc.oftc.net
963
965 This program is part of the LTTng-tools project.
966 LTTng-tools is distributed under the GNU General Public License
968 version 2 <http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html>.
969 See the LICENSE <https://github.com/lttng/lttng-
970 tools/blob/master/LICENSE> file for details.
971
973 Special thanks to Michel Dagenais and the DORSAL laboratory
974 <http://www.dorsal.polymtl.ca/> at École Polytechnique de Montréal for
975 the LTTng journey.
976 Also thanks to the Ericsson teams working on tracing which helped us
978 greatly with detailed bug reports and unusual test cases.
979
981 lttng(1), lttng-relayd(8), lttng-sessiond(8)
982LTTng 2.13.4 14 June 2021 LTTNG-CONCEPTS(7)