1SECURE-LOGGING(7)       The secure logging manual page       SECURE-LOGGING(7)
2
3
4

NAME

6       secure-logging - Forward integrity and confidentiality for system logs
7

SYNOPSIS

9       $(slog --key-file  <host key file> --mac-file <MAC file> $RAWMSG)
10

DESCRIPTION

12       Secure logging is an extension to syslog-ng providing forward integrity
13       and confidentiality of system logs. It is implemented in form of a
14       module and is configured as a template in the syslog-ng configuration
15       file.
16
17       The main objective of the secure logging module is to provide tamper
18       evident logging, i.e. to adequately protect log records of an
19       information system and to provide a sensor indicating attack attempts.
20       The secure logging module achieves this by authentically encrypting
21       each log record with an individual cryptographic key used only once and
22       protecting the integrity of the whole log archive by a cryptographic
23       authentication code. Each attempt to tamper with either an individual
24       log record or the log archive itself will be immediately detected
25       during log archive verification. Therefore, an attacker can no longer
26       tamper with log records without being detected.
27
28       In order to use the log file created by the secure logging module for
29       analysis, the log file must first be decrypted and its integrity
30       verified. This is achieved with a command line utility that is part of
31       the secure logging module and is installed as part of the syslog-ng
32       package. This utility can easily be integrated into the import module
33       of existing analysis environments.
34
35       The secure logging environment uses a cryptographic key for encrypting
36       log entries. Each individual log entry is encrypted with its own key
37       which is immediately discarded after successful encryption in order to
38       provide forward integrity. An efficient algorithm generates the key for
39       the next log entry based the key used for encrypting the previous log
40       entry. The resulting chain of keys preserves forward integrity, i.e. a
41       potential attacker cannot deduce the previous key from the current key.
42
43       In order to ease log verification and analysis, a sequence number is
44       added to each log entry. This sequence number is also added to the
45       encryption key creating a one-to-one correspondence of the encryption
46       key with the log entry. In order to prevent truncation attacks or
47       deletion of individual log entries, a message authentication code (MAC)
48       is iteratively applied to the complete log archive and stored in a
49       separate file. It is used upon verification to check the integrity of
50       the whole log archive.
51
52       Here is an example of three short original log messages that will be
53       sent to a destination with secure logging enabled:
54
55           This is a log message
56           And here comes another log message
57           This is a log message with a longer text that is processed without any problems
58
59
60       In order to inspect the status of the secure logging environment, one
61       can check the sequence counter by querying the key file with the
62       slogkey utility like this:
63
64           user@host:~> slogkey --counter /etc/syslog-ng/host.key
65           counter=3
66
67
68       Counting starts at zero. This is why the counter is set to three after
69       processing three messages.
70
71       The output of the secure logging template for the three example
72       messages is shown below. One clearly sees the sequence number that was
73       prepended to each message. The colon indicates the end of the sequence
74       number and the start of the original message. As three message were
75       processed, the sequence counter of the key will be three.
76
77           AAAAAAAAAAA=:k3/dYpLsgO2tUJKSauo6dycIBzW6OTC3pyA9TP+7AnqFgEojBzgC2rcK4OPfRtr8yg==
78           AQAAAAAAAAA=:smw0ATISVgN+BYEu5d7OLBE7aQhHpK9Ro4MndmNgSVrqhcmRCBCj6DUnD6ku0Z29CKJ0N6LAJUgByX4Ev+g=
79           AgAAAAAAAAA=:5UVybnKL1EAbgC4CLfd8HpgurjREf4LEN61/yWHSD2hbXjRD4QmQdtbwguT1chzdItKSQASps9QRIvR5Jd4AHzHfqxI4aRgdUBcNbAq26nwUCg5vPWygjmbtQaxZgCJYkry8slxCigmbTVs=
80
81
82       The output of a successful verification run is shown below.
83
84           0000000000000000: This is a log message
85           0000000000000001: And here comes another log message
86           0000000000000002: This is a log message with a longer text that is processed without any problems
87
88
89       The original log messages have been successfully restored.
90       Additionally, the sequence counter is also prepended to the clear text
91       messages. This helps when in analyzing problems with a particular log
92       entry. As real log files will contain thousands of entries. the
93       sequence counter eases identification of erroneous entries.
94
95       Before the secure logging module can be used as part of an existing
96       syslog-ng installation, several preparatory activities need to be
97       performed.
98

KEY GENERATION

100       In order to bootstrap the system, an initial key k0 must be created and
101       installed on the log host before secure logging environment is started
102       for the first time.
103
104       The newly created host key k0 has its counter set to 0 indicating that
105       it represents the initial host key k0. This host key k0 must be kept
106       secret and not be disclosed to third parties. It will be required to
107       successfully decrypt and verify log archives processed by the secure
108       logging environment. As each log entry will be encrypted with its own
109       key, a new host key will be created after successful processing of a
110       log entry and will replace the previous key. Therefore, the initial
111       host key needs to be stored in a safe place before starting the secure
112       logging environment, as it will be deleted from the log host after
113       processing of the first log entry. The following steps must be done
114       before starting the secure logging environment. Steps 1 and 2 are
115       performed with the slogkey utility. See slogkey(1) for details on how
116       to generate a master key and to derive a host key from it. Step 3 and 4
117       depend on the actual deployment in a target environment.
118
119        1. Create a master key
120
121        2. Derive an initial host key k0 from a previously created master key
122
123        3. Store the initial host key k0 in a safe location outside of the log
124           host
125
126        4. Deploy the key k0 on the log host where the secure logging module
127           will be used
128

CONFIGURATION

130       Secure logging is configured by adding the corresponding statements to
131       the syslog-ng.conf file. See syslog-ng.conf(5) for information on how
132       to configure syslog-ng using the configuration file. Details can be
133       found in the The syslog-ng Administrator Guide[1].
134
135       Secure logging is implemented as a template and is configured
136       accordingly. Apart from the actual template configuration, no other
137       settings are required in order to activate secure logging. The secure
138       logging is activated by placing the following statement in the
139       configuration file
140
141       template("$(slog --key-file <host key file> --mac-file <MAC file>
142       $RAWMSG)\n");
143
144       where
145
146       slog
147           The name of the secure logging template function. This name can be
148           also be found by calling syslog-ng with the --module-registry
149           arguments and checking the template-func property of the secure
150           logging module in the corresponding output.
151
152       --key-file or -k
153           The host key. <host key file> is the full path of the file storing
154           the host key on the log host. If this arguments is not supplied or
155           does not point to a valid regular key file, syslog-ng will not
156           start and a display a corresponding error message.
157
158       --mac-file or -m
159           The MAC file. <MAC file> is the full path of the MAC file on the
160           log host. The file does not need to exist, as it will be
161           automatically created upon the initial start. If the path is not
162           correct, syslog-ng will not start and a display a corresponding
163           error message.
164
165       $RAWMSG
166           $RAWMSG provides access to the original log message received at the
167           source. This macro is only available if the store-raw-message flag
168           was set for the source. Otherwise, an empty string will be passed
169           to the secure logging template. If access to the original message
170           is not available, e.g. if the source does not support the
171           store-raw-message flag, then the $MSG macro can also be used. In
172           this case, however, the integrity guarantee provided by secure
173           logging is limited to the content that this macro provides and does
174           not protect the complete original message.
175
176       \n
177           \n is the line separator. This is important, as the secure logging
178           template expects log entries to be separated by a line separator.
179           When detecting a line separator, the log entry is regarded as
180           complete and is encrypted with the current host key. Therefore,
181           only a single line separator is allowed.
182
183       The secure logging template can be combined with any source or
184       destination within the following limitations:
185
186       •   Sources must be line-oriented. Secure logging uses a line separator
187           in order to distinguish between individual log entries. Sources
188           which provide data in a different format, e.g. in the form of raw
189           data obtained directly from a database system, cannot currently be
190           used with the secure logging template, as the separation of log
191           entries is not clearly defined for this type of data.
192
193       •   Only sources for which the store-raw-message flag is implemented
194           and set do benefit from the integrity guarantee provided by the
195           secure logging template. Secure logging aims at protecting the
196           integrity of complete log messages including all associated
197           meta-data, such as timestamps and host names.  syslog-ng parses the
198           log message into its internal format and provide easy access to
199           parts of a message through macros. While this is convenient when
200           rewriting log messages, it is not helpful for secure logging.
201           syslog-ng provides the store-raw-message flag which provides access
202           to a copy of the original log message after parsing. This is the
203           log message processed and protected by the secure logging template.
204           If the source does not support the store-raw-message flag, then the
205           $MSG macro can also be used. However, in this case the integrity
206           guarantee provided by secure logging is limited to the content that
207           this macro provides.
208
209       •   Log rotation of any kind cannot be used with destinations using
210           secure logging. The reason is that log rotate will overwrite, i.e.
211           delete previous log files. This destroys the cryptographic chain of
212           trust of the log entries making recovery impossible. In order to
213           allow for a an efficient handling of log files, the secure logging
214           environment features iterative verification. Using iterative
215           verification, a log file can be verified in steps. For this to
216           work, the log file must first be downloaded from the log host,
217           together with the corresponding host key and MAC file to a
218           verification host. After this download the log file can be safely
219           deleted from the log host. Verification is then performed on the
220           verification host using the iterative mode of the slogverify
221           utility. See slogverify(1) for details.
222
223       The following example configuration shows the use of the secure logging
224       template on a file destination.
225
226           #############################################################################
227           # Minimal syslog-ng.conf file with secure logging enabled. Encrypted log
228           # entries will be logged to a single file called /var/log/messages.slog
229           #
230
231           @version: 3.35
232           @include "scl.conf"
233
234           source s_local {
235               system();
236               internal();
237           };
238
239           source s_network {
240               network(
241                   transport("udp")
242                   port(514)
243                   flags(store-raw-message)
244               );
245           };
246
247           # Secure logging template definition
248           template secure_logging {
249               template("$(slog --key-file /etc/syslog-ng/host.key --mac-file /etc/syslog-ng/mac.dat $RAWMSG)\n");
250           };
251
252           # This configures a secure logging destination
253           destination d_local {
254                file("/var/log/messages.slog" template(secure_logging));
255           };
256
257           log {
258               source(s_local);
259
260               # This source has the raw message flag set
261               source(s_network);
262
263               # This statement activates secure logging for this destination
264               destination(d_local);
265           };
266
267

LOG VERIFICATION

269       In order to analyze the log file created in a secure logging
270       environment, the log files must be decrypted and their integrity be
271       verified. Verification requires both the initial host key k0 and the
272       corresponding MAC file and is performed with the slogverify utility. It
273       is not normally performed on the log host where the secure logging
274       environment is producing log data. In a typical deployment, log files
275       would be retrieved from the log host and transferred to a central log
276       collector where verification it performed. As verification requires the
277       use of the initial host key k0, it should only be performed in a
278       trusted environment.
279
280   Normal mode
281       In normal mode, a complete log archive is verified at once. In a
282       typical environment, this would mean retrieving a log file together
283       with is MAC file from a log host and retrieving the corresponding
284       initial key k0 form a safe location and supplying them to the
285       slogverify utility. A typical call sequence for verification in normal
286       mode would look like this
287
288       slogverify --key-file host0.key --mac-file mac.dat
289       /var/log/messages.slog /var/log/verified/messages
290
291       with
292
293       host0.key
294           The initial host key k0. Supplying the initial key k0 is enough for
295           decrypting all log entries, as the key derivation algorithm is able
296           to generate the necessary keys for all subsequent log entries based
297           on the initial key k0.
298
299       mac.dat
300           The MAC file from the log host.
301
302       /var/log/messages.slog
303           The file containing the encrypted log entries as retrieved from a
304           log host.
305
306       /var/log/verified/messages
307           The file receiving the plain text log after decryption.
308
309       Log files may become large and not fit into system memory. Verification
310       is therefore performed in chunks. Each part of the log file is
311       transferred to an internal buffer on which verification is performed.
312       After the buffer has been processed, the next chunk is fetched. An
313       optional buffer argument can be supplied to the slogverify utility in
314       order to change the default buffer size of 1000 log entries to a number
315       suitable for the system on which the verification is performed, for
316       example
317
318       slogverify --key-file host.key --mac-file mac.dat
319       /var/log/messages.slog /var/log/verified/messages 8000
320
321       See slogverify(1) for details on verification in normal mode.
322
323   Iterative mode
324       Verification in normal mode may not be suitable for some application
325       scenarios. Many log hosts use log rotation in order to preserve storage
326       space. In log rotation, a threshold for the maximum amount of storage
327       space and the number of generations is defined for different type of
328       log files. When either storage space is exhausted or the number of
329       generations is reached, the oldest log file will be overwritten by new
330       incoming log data. This procedure is not possible in secure logging
331       environment, as overwriting, i.e. deleting a log file would break the
332       cryptographic chain that is established between the log entries. This
333       comes as no surprise, as one of the main objectives of secure logging
334       is to protect against deletion of log entries by a potential attacker.
335
336       In order allow for a procedure similar to log rotation, the secure
337       logging environment features an iterative mode. In iterative mode, log
338       files can be split into different files and each of these files can be
339       verified separately. Care must be taken when performing verification in
340       iterative mode, as each of the different log files needs to be
341       accompanied by a copy of the host key and the MAC files present on the
342       system at the time of retrieval. A typical usage scenario for the
343       iterative mode would look like this:
344
345        1. Define a storage threshold for the secure logging file destination.
346           In this example we assume 500MB.
347
348        2. Let the secure logging environment produce log data that is written
349           to the destination until 500MB are reached.
350
351        3. Stop the secure logging environment and retrieve the log file, the
352           host key and the MAC files from the log host.
353
354        4. Delete the log file on the log host but leave host key and MAC file
355           untouched.
356
357        5. Restart the secure logging environment.
358
359        6. Perform verification in iterative mode with the log file, the host
360           key and the MAC just retrieved.
361
362       Steps 2-6 have to repeated each time the log file reaches a size of 50
363       MB. Assuming that the log file parts will be named after the iteration,
364       e.g. log.1, log.2, log.3, etc. and a similar convention is applied to
365       the host keys and MAC files, a typical call sequence for the validation
366       of a log file part in iterative mode after three iterations will look
367       like this:
368
369       slogverify --iterative --prev-key-file host.key.2 --prev-mac-file
370       mac.dat.2 --mac-file mac.dat /var/log/messages.slog.3
371       /var/log/verified/messages.3
372
373       with
374
375       host.key.2
376           The host key from the previous iteration. In this example, this is
377           the second iteration.
378
379       mac.dat.2
380           The MAC file from the previous iteration. In the example,
381           verification is performed during the third iteration, so the MAC
382           file from the second iteration is required.
383
384       mac.dat
385           The current MAC file from the log host.
386
387       /var/log/messages.slog.3
388           The file part containing the encrypted log entries as retrieved
389           from the log host during the third iteration.
390
391       /var/log/verified/messages.3
392           The file receiving the plain text log after decryption during the
393           third iteration.
394
395       In a real deployment, the above steps would typically be automated
396       using a scripting engine.
397
398       See slogverify(1) for details on verification in iterative mode.
399

FILES

401       /usr/bin/slogkey
402
403       /usr/bin/slogencrypt
404
405       /usr/bin/slogverify
406
407       /etc/syslog-ng.conf
408

SEE ALSO

410       syslog-ng.conf(5)
411
412       slogkey(1)
413
414       slogencrypt(1)
415
416       slogverify(1)
417
418           Note
419           For the detailed documentation of see The syslog-ng Administrator
420           Guide[1]
421
422           If you experience any problems or need help with syslog-ng, visit
423           the syslog-ng mailing list[2].
424
425           For news and notifications about of syslog-ng, visit the syslog-ng
426           blogs[3].
427
428           For specific information requests related to secure logging send a
429           mail to the Airbus Secure Logging Team <secure-logging@airbus.com>.
430

AUTHOR

432       This manual page was written by the Airbus Secure Logging Team
433       <secure-logging@airbus.com>.
434

NOTES

437        1. The syslog-ng Administrator Guide
438           https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/index.html
439
440        2. syslog-ng mailing list
441           https://lists.balabit.hu/mailman/listinfo/syslog-ng
442
443        3. syslog-ng blogs
444           https://syslog-ng.org/blogs/
445
446
447
4483.35                              11/15/2021                 SECURE-LOGGING(7)
Impressum