1PDBTOOL(1) The pdbtool manual page PDBTOOL(1)
2
6 pdbtool - An application to test and convert syslog-ng pattern database
7 rules
8
10 pdbtool [command] [options]
11
13 This manual page is only an abstract, for the complete documentation of
14 syslog-ng and pdbtool, see The syslog-ng Administrator Guide[1].
15 The syslog-ng application can match the contents of the log messages to
17 a database of predefined message patterns (also called patterndb). By
18 comparing the messages to the known patterns, syslog-ng is able to
19 identify the exact type of the messages, tag the messages, and sort
20 them into message classes. The message classes can be used to classify
21 the type of the event described in the log message. The functionality
22 of the pattern database is similar to that of the logcheck project, but
23 the syslog-ng approach is faster, scales better, and is much easier to
24 maintain compared to the regular expressions of logcheck.
25 The pdbtool application is a utility that can be used to:
27 · test messages, or specific rules
29 · convert an older pattern database to the latest database format
31 · merge pattern databases into a single file
33 · automatically create pattern databases from a large amount of log
35 messages
36 · dump the RADIX tree built from the pattern database (or a part of
38 it) to explore how the pattern matching works.
39
41 dictionary [options]
42 Lists every name-value pair that can be set by the rules of the pattern
44 database.
45 --dump-tags or -T
47 List the tags instead of the names of the name-value pairs.
48 --pdb <path-to-file> or -p <path-to-file>
50 Name of the pattern database file to use.
51 --program <programname> or -P <programname>
53 List only the name-value pairs that can be set for the messages of
54 the specified $PROGRAM application.
55
57 dump [options]
58 Display the RADIX tree built from the patterns. This shows how are the
60 patterns represented in syslog-ng and it might also help to track down
61 pattern-matching problems. The dump utility can dump the tree used for
62 matching the PROGRAM or the MESSAGE parts.
63 --debug or -d
65 Enable debug/diagnostic messages on stderr.
66 --pdb or -p
68 Name of the pattern database file to use.
69 --program or -P
71 Displays the RADIX tree built from the patterns belonging to the
72 ${PROGRAM} application.
73 --program-tree or -T
75 Display the ${PROGRAM} tree.
76 --verbose or -v
78 Enable verbose messages on stderr.
79 Example and sample output:
81 pdbtool dump -p patterndb.xml -P 'sshd'
83 'p'
85 'assword for'
86 @QSTRING:@
87 'from'
88 @QSTRING:@
89 'port '
90 @NUMBER:@ rule_id='fc49054e-75fd-11dd-9bba-001e6806451b'
91 ' ssh' rule_id='fc55cf86-75fd-11dd-9bba-001e6806451b'
92 '2' rule_id='fc4b7982-75fd-11dd-9bba-001e6806451b'
93 'ublickey for'
94 @QSTRING:@
95 'from'
96 @QSTRING:@
97 'port '
98 @NUMBER:@ rule_id='fc4d377c-75fd-11dd-9bba-001e6806451b'
99 ' ssh' rule_id='fc5441ac-75fd-11dd-9bba-001e6806451b'
100 '2' rule_id='fc44a9fe-75fd-11dd-9bba-001e6806451b'
101
104 match [options]
105 Use the match command to test the rules in a pattern database. The
107 command tries to match the specified message against the patterns of
108 the database, evaluates the parsers of the pattern, and also displays
109 which part of the message was parsed successfully. The command returns
110 with a 0 (success) or 1 (no match) return code and displays the
111 following information:
112 · the class assigned to the message (that is, system, violation, and
114 so on),
115 · the ID of the rule that matched the message, and
117 · the values of the parsers (if there were parsers in the matching
119 pattern).
120 The match command has the following options:
122 --color-out or -c
124 Color the terminal output to highlight the part of the message that
125 was successfully parsed.
126 --debug or -d
128 Enable debug/diagnostic messages on stderr.
129 --debug-csv or -C
131 Print the debugging information returned by the --debug-pattern
132 option as comma-separated values.
133 --debug-pattern or -D
135 Print debugging information about the pattern matching. See also
136 the --debug-csv option.
137 --file=<filename-with-path> or -f
139 Process the messages of the specified log file with the pattern
140 database. This option allows to classify messages offline, and to
141 apply the pattern database to already existing logfiles. To read
142 the messages from the standard input (stdin), specify a hyphen (-)
143 character instead of a filename.
144 --filter=<filter-expression> or -F
146 Print only messages matching the specified syslog-ng filter
147 expression.
148 --message or -M
150 The text of the log message to match (only the ${MESSAGE} part
151 without the syslog headers).
152 --pdb or -p
154 Name of the pattern database file to use.
155 --program or -P
157 Name of the program to use, as contained in the ${PROGRAM} part of
158 the syslog message.
159 --template=<template-expression> or -T
161 A syslog-ng template expression that is used to format the output
162 messages.
163 --verbose or -v
165 Enable verbose messages on stderr.
166 Example: The following command checks if the patterndb.xml file
168 recognizes the Accepted publickey for myuser from 127.0.0.1 port 59357
169 ssh2 message:
170 pdbtool match -p patterndb.xml -P sshd -M "Accepted publickey for myuser from 127.0.0.1 port 59357 ssh2"
172 The following example applies the sshd.pdb pattern database file to the
174 log messages stored in the /var/log/messages file, and displays only
175 the messages that received a useracct tag.
176 pdbtool match -p sshd.pdb \
178 –file /var/log/messages \
179 –filter ‘tags(“usracct”);’
180
182 merge [options]
183 Use the merge command to combine separate pattern database files into a
185 single file (pattern databases are usually stored in separate files per
186 applications to simplify maintenance). If a file uses an older database
187 format, it is automatically updated to the latest format (V3). See the
188 The syslog-ng Administrator Guide[1] for details on the different
189 pattern database versions.
190 --debug or -d
192 Enable debug/diagnostic messages on stderr.
193 --directory or -D
195 The directory that contains the pattern database XML files to be
196 merged.
197 --glob or -G
199 Specify filenames to be merged using a glob pattern, for example,
200 using wildcards. For details on glob patterns, see man glob. This
201 pattern is applied only to the filenames, and not on directory
202 names.
203 --pdb or -p
205 Name of the output pattern database file.
206 --recursive or -r
208 Merge files from subdirectories as well.
209 --verbose or -v
211 Enable verbose messages on stderr.
212 Example:
214 pdbtool merge --recursive --directory /home/me/mypatterns/ --pdb /var/lib/syslog-ng/patterndb.xml
216 Currently it is not possible to convert a file without merging, so if
218 you only want to convert an older pattern database file to the latest
219 format, you have to copy it into an empty directory.
220
222 patternize [options]
223 Automatically create a pattern database from a log file containing a
225 large number of log messages. The resulting pattern database is printed
226 to the standard output (stdout). The pdbtool patternize command uses a
227 data clustering technique to find similar log messages and replacing
228 the differing parts with @ESTRING:: @ parsers. For details on pattern
229 databases and message parsers, see the The syslog-ng Administrator
230 Guide[1]. The patternize command is available only in version 3.2 and
231 later.
232 --debug or -d
234 Enable debug/diagnostic messages on stderr.
235 --file=<path> or -f
237 The logfile containing the log messages to create patterns from. To
238 receive the log messages from the standard input (stdin), use -.
239 --iterate-outliers or -o
241 Recursively iterate on the log lines to cover as many log messages
242 with patterns as possible.
243 --named-parsers or -n
245 The number of example log messages to include in the pattern
246 database for every pattern. Default value: 1
247 --no-parse or -p
249 Do not parse the input file, treat every line as the message part
250 of a log message.
251 --samples=<number-of-samples>
253 Include a generated name in the parsers, for example,
254 .dict.string1, .dict.string2, and so on.
255 --support=<number> or -S
257 A pattern is added to the output pattern database if at least the
258 specified percentage of log messages from the input logfile match
259 the pattern. For example, if the input logfile contains 1000 log
260 messages and the --support=3.0 option is used, a pattern is created
261 only if the pattern matches at least 3 percent of the log messages
262 (that is, 30 log messages). If patternize does not create enough
263 patterns, try to decrease the support value.
264 Default value: 4.0
266 --verbose or -v
268 Enable verbose messages on stderr.
269 Example:
271 pdbtool patternize --support=2.5 --file=/var/log/messages
273
275 test [options]
276 Use the test command to validate a pattern database XML file. Note that
278 you must have the xmllint application installed. The test command is
279 available only in version 3.2 and later.
280 --color-out or -c
282 Enable coloring in terminal output.
283 --debug or -d
285 Enable debug/diagnostic messages on stderr.
286 --debug or -D
288 Print debugging information on non-matching patterns.
289 --rule-id or -r
291 Test only the patterndb rule (specified by its rule id) against its
292 example.
293 --validate
295 Validate a pattern database XML file.
296 --verbose or -v
298 Enable verbose messages on stderr.
299 Example:
301 pdbtool test --validate /home/me/mypatterndb.pdb
303
305 /usr/local/
306 /usr/local/etc/syslog-ng.conf
308
310 The syslog-ng Administrator Guide[1]
311 syslog-ng.conf(5)
313 syslog-ng(8)
315 Note
317 For the detailed documentation of see The 3.17 Administrator
318 Guide[2]
319 If you experience any problems or need help with syslog-ng, visit
321 the syslog-ng mailing list[3].
322 For news and notifications about of syslog-ng, visit the syslog-ng
324 blogs[4].
325
327 This manual page was written by the Balabit Documentation Team
328 <documentation@balabit.com>.
329
332 1. The syslog-ng Administrator Guide
333 https://www.balabit.com/support/documentation/
334 2. The 3.17 Administrator Guide
336 https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/index.html
337 3. syslog-ng mailing list
339 https://lists.balabit.hu/mailman/listinfo/syslog-ng
340 4. syslog-ng blogs
342 https://syslog-ng.org/blogs/
3433.17 08/10/2018 PDBTOOL(1)