1NFDUMP(1) BSD General Commands Manual NFDUMP(1)
3
5 nfdump — flow display and analysis program
6
8 nfdump -r flowpath [-w outfile] [-f filterfile] [-C config] [-R filelist]
9 [-M dirlist] [-O order] [-t timewin] [-c num] [-a]
10 [-A aggregation] [-b] [-B] [-I] [-D nameserver] [-G geoDB]
11 [-s statistic] [-n num] [-o format] [-6] [-q] [-N] [-i ident]
12 [-v flowfile] [-E flowfile] [-x flowfile] [-z=<compress>]
13 [-J compress] [-X] [-Z] [-T] [-V] [-h] [filter]
14
16 nfdump reads the flow data from one or more binary files, created by any
17 nfdump collector nfcapd, nfpcapd and sfcapd. It processes and lists the
18 flows in many different output formats and can create a wide range of
19 statistics.
20 nfdump has a very powerful flow filter to process flows. The filter syn‐
22 tax is very similar to tcpdump, but adapted and extended for flow filter‐
23 ing. A flow filter may also contain arrays of many thousand IP addresses
24 etc. to search for specific records.
25 nfdump can aggreagte flows according to a user defined number of ele‐
27 ments. This masks certain elements and allows to sum up flow records
28 matching the same values.
29 The combination of flow filtering and aggregation as input for any flow
31 statistics allows complex flow processing. Pre-filtered and aggregated
32 flow data may also be written back into a binary flow file, which again
33 may be processed with nfdump
34 nfdump can enrich the listing of flows with geo location information and
36 AS information, unless AS information is already available in the flow
37 records. IP addresses can be tagged with a two letter country code, or
38 with a longer location label containing the geographic region, country
39 and city. The geo location and AS information is retrieved from the op‐
40 tional geoDB database, created by the geolookup program from the nfdump
41 tools. geolookup uses the Maxmind database GeoDB or GeoLite2 to create a
42 binary lookup database for nfdump Please check the geolooup(1) man page
43 for more details.
44 The options are as follows:
46 -r flowpath
48 Reads flow records from this path. flowpath may be a single
49 file, or a directory containing any number of flow files or sub
50 directories. All files are processed in the order, as listed by
51 the OS.
52 -w outfile
54 Writes all processed records into outfile instead of printing.
55 The flowfile is a binary flow file and may be processed again
56 with nfdump This can be useful to limit flows according to a flow
57 filter and/or specific flow aggregation.
58 -f filterfile
60 Reads the flow filter from filterfile. This can be useful for
61 very long or structured filters, with comments and long lists.
62 Note: Any filter specified directly on the command line takes
63 precedence over the filterfile.
64 -C config
66 Read more options from file config. nfdump tries to read by de‐
67 fault %prefix/etc/nfdump.config. This may be overwritten by the
68 environment valiable NFCONF which again may be overwritten by
69 this option -C. In order to prevent reading any config file, even
70 if it would exist set -C none. A config file is not required, but
71 may be handy for often used output formats etc.
72 -O order
74 Sets an output order for records to be printed as text output.
75 This order applies after all records processing, such as filter‐
76 ing, and aggregation and before printing.
77 flows Sort according to the number of flows
78 packets Sort according to (in)packets
79 ipkg Same as packets
80 opkg Sort according to output packets
81 bytes Sort according to (in)bytes
82 ibyte Same as bytes
83 obyte Sort according to output bytes
84 pps Sort according to (in)packets per second
85 ipps Same as ipps
86 opps Sort according to out packets per second
87 bps Sort according to (in)bytes per second
88 ibps Same as bps
89 obps Sort according to output bytes per second
90 bpp Sort according to (in)bytes per packet
91 ibpp Same as bpp
92 obpp Sort according to output packets
93 tstart Sort according to start time of flow - former -m
94 tend Sort according to end time of flows
95 duration Sort according to duration of flows
96 -t timewin
98 Set time window to process flows. This option is considered
99 legacy andmay be replaced with a filter primitive in future
100 rleases. The time window is specified as:
101 YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss]. Any parts of the time
102 spec may be omitted e.g YYYY/MM/dd expands to
103 YYYY/MM/dd.00:00:00-infinity and processes all flow from a given
104 day onwards. The time window may also be specified as +/- n. In
105 this case it is relative to the beginning or end of all flows.
106 +10 means the first 10 seconds of all flows, -10 means the last
107 10 seconds of all flows.
108 -c num Limit the number of records to be processed to the first num
110 records, which passwd the filter.
111 -a Aggregate flow records. The default aggregation is done at con‐
113 nection level by taking the 5-tuple protocol, srcip, dstip,
114 srcport and dstport. This way of aggregation may be overwritten
115 by option -A
116 -A aggregation
118 Sets the list of elements in a flow record to be aggregated.
119 aggregation is a ',' separated list of any number of v9/ipfix el‐
120 ements. The following elements are accepted:
121 proto IP protocol
122 srcip Source IP address
123 dstip Destination IP address
124 srcip4/net IPv4 source IP address with applied netmask
125 srcip6/net IPv6 source IP address with applied netmask
126 dstip4/net IPv4 destination IP address with applied netmask
127 dstip6/net IPv6 destination IP address with applied netmask
128 srcnet Apply netmask srcmask in netflow record for source IP
129 dstnet Apply netmask dstmask in netflow record for dest IP
130 srcport Source port
131 dstport Destination port
132 srcmask Source mask
133 dstmask Destination mask
134 srcvlan Source vlan label
135 dstvlan Destination vlan label
136 srcas Source AS number
137 dstas Destination AS number
138 nextas BGP Next AS
139 prevas BGP Previous AS
140 inif SNMP input interface number
141 outif SNMP output interface number
142 next IP next hop
143 bgpnext BGP next hop
144 insrcmac In source MAC address
145 outdstmac out destination MAC address
146 indstmac In destination MAC address
147 outsrcmac Out source MAC address
148 tos Source type of service
149 srctos Source type of Service
150 dsttos Destination type of Service
151 mpls1 MPLS label 1
152 mpls2 MPLS label 2
153 mpls3 MPLS label 3
154 mpls4 MPLS label 4
155 mpls5 MPLS label 5
156 mpls6 MPLS label 6
157 mpls7 MPLS label 7
158 mpls8 MPLS label 8
159 mpls9 MPLS label 9
160 mpls10 MPLS label 10
161 router IP address of exporting router
162 odid observation domain ID
163 opid observation point ID
164 xsrcip X-late source IP address, if compiled with NSEL sup‐
165 port
166 xdstip X-late destination IP address, if compiled with NSEL
167 support
168 xsrcport X-late source port, if compiled with NSEL support
169 xdstport X-late destination port, if compiled with NSEL sup‐
170 port
171 nfdump automatically compiles the appropriate output format for
173 the selected aggregation elements unless an explicit output for‐
174 mat -o is given. The automatic output format is identical to
175 -o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl'
177 where <fields> represents the selected aggregation tags.
179 -b Aggregate flow records as bidirectional flows. This automatically
181 implies -a. Aggregation is done on connection level by taking
182 the 5-tuple protocol, srcip, dstip, srcport and dstport The re‐
183 verse order applies for the corresponding reverse flow. Input and
184 output packets/bytes are counted and reported separately. Both
185 flows are merged into a single record with corresponding input
186 and output counters. An appropriate output format is selected au‐
187 tomatically, which may be overwritten by any -o format option.
188 -B Similar to option -b but tries to guess the correct client to
190 server direction. Automagically swaps flows if src port is < dst
191 port for TCP and UDP flows and src port < 1024 and dst port >
192 1024. Some exporters do not really care sending the flows in
193 proper order. It's considered to be a conveniency option.
194 -I Print flow statistics of a single file or the summary of all the
196 files specified by -r flowpath.
197 -g Print for each flow file given by -r flowpath a one line summary,
199 which can be easily used by gnu plot.
200 -D nameserver
202 Sets the nameserver to translate hostnames into IP addresses in
203 filter expressions. See filter below for more details.
204 -G geoDB
206 Use geoDB as geo lookup DB for geo location and AS lookups.
207 nfdump tries to read the environment variable NFGEODB for the
208 path of geoDB. The option -G overwrites NFGEODB. In order to pre‐
209 vent reading any geoDB file, even if it would exist set -G none.
210 -s statistic [:p [/orderby]]
212 Generate the Top N flow record or flow element statistic. By op‐
213 tionally adding :p to statistic, the statistic is additionally
214 split up into the transport layer protocols. By default the
215 statistic is transport protocol independent. Each statistic may
216 be ordered by the optional parameter orderby This can be flows,
217 packets, bytes, pps, bps or bpp. You may specify more than one
218 orderby option, which results in the same statistic but ordered
219 differently. If no orderby is given, the statistic is ordered by
220 flows. You can specify as many -s flow element statistics as
221 needed on the command line for the same run.
222 statistic can be:
224 record aggregated netflow records.
226 srcip source IP addresses
227 dstip destination IP addresses
228 ip any (src or dst) IP addresses
229 nhip next hop IP addresses
230 nhbip BGP next hop IP addresses
231 router exporting router IP address
232 srcport source ports
233 dstport destination ports
234 port any (source or destination) ports
235 tos type of service - default src
236 srctos src type of service
237 dsttos dst type of service
238 dir flow directions ingress/egress
239 srcas source AS numbers
240 dstas destination AS numbers
241 srcgeo 2 letter geo source country code
242 dstgeo 2 letter geo destination country code
243 as any (source or destination) AS numbers
244 inif input interface
245 outif output interface
246 if any interface
247 inam input interface name
248 onam output interface name
249 srcmask src mask
250 dstmask dst mask
251 srcvlan src vlan label
252 dstvlan dst vlan label
253 vlan any vlan label
254 insrcmac input src MAC address
255 outdstmac output dst MAC address
256 indstmac input dst MAC address
257 outsrcmac output src MAC address
258 srcmac any src MAC address
259 dstmac any dst MAC address
260 inmac any input MAC address
261 outmac any output MAC address
262 mask any mask
263 proto IP protocols
264 mpls1 MPLS label 1
265 mpls2 MPLS label 2
266 mpls3 MPLS label 3
267 mpls4 MPLS label 4
268 mpls5 MPLS label 5
269 mpls6 MPLS label 6
270 mpls7 MPLS label 7
271 mpls8 MPLS label 8
272 mpls9 MPLS label 9
273 mpls10 MPLS label 10
274 sysid Internal SysID of exporter
275 nbar nbar ID
276 ja3 ja3 hashes
277 odid observation domain ID
278 opid observation point ID
279 vrf/ivrf ingress vrf
280 evrf egress vrf
281 ivrfnam ingress vrf name
282 evrfnam egress vrf name
283 NSEL/ASA statistics
285 event NSEL/ASA event
286 xevent NSEL/ASA extended event
287 xsrcip NSEL/ASA translated src IP address
288 xsrcport NSEL/ASA translated src port
289 xdstip NSEL/ASA translated dst IP address
290 xdstport NSEL/ASA translated dst port
291 iacl NSEL/ASA ingress ACL
292 iace NSEL/ASA ingress ACE
293 ixace NSEL/ASA ingress xACE
294 eacl NSEL/ASA egress ACL
295 eace NSEL/ASA egress ACE
296 exace NSEL/ASA egress xACE
297 NAT statistics
299 nevent NAT event
300 nsrcip NAT src IP address
301 nsrcport NAT src port
302 ndstip NAT dst IP address
303 ndstport NAT dst port
304 % nfdump -s srcip -s ip/flows/bytes -s record/bytes
306 -n num Set the number of records to be printed to num. This option ap‐
308 plies to -s statistics as well as to ordered output -O -or
309 -aggregated -records -a The default is set to 10 for statistics
310 and unlimited for the other use cases. To disable the limit, set
311 num to 0.
312 -o format
314 Sets the output format to print flow records. has many different
315 output formats already predefined. format may be one of the
316 options below:
317 raw Print the full flow record on multiple lines. This
319 prints all available information.
320 fmt: user Print the flow records according the format user. This
321 is a very flexible and powerful way to format flow
322 records. See the section OUTPUT below for more details
323 on how to compile your own format.
324 json Print full record as a separate json object.
325 csv Legacy .csv format - will get removed in future re‐
326 leases. Please use json instead.
327 pipe Legacy '|' separated format - will get removed in fu‐
328 ture releases. Please use json instead.
329 Already predefined fmt formats:
331 line Print each flow on one line. Default format.
333 long Print each flow on one line with more details
334 biline Same as line, but for bi-directional flows
335 bilong Same as long, but for bi-directional flows
336 gline Same as line, but add country code to IPs. If a geoDB
337 file is supplied this is the default output format
338 glong Same as long, but add country code to IPs
339 extended Print each flow on one line with even more details.
340 nsel Print format for NSEL event records. Default format if
341 NSEL/NAT support has been compiled in.
342 nel Print format for NAT event records.
343 The nfdump config file may contain additional formats. If you
345 want to add new formats or change existing ones, check the config
346 file.
347 IPv6 addresses are printed condensed in any fmt defined format to
349 prevent cluttering the output with large blank blocks. A con‐
350 densed IPV6 uses max 16 characters. If it is longer, then the
351 middle part of the IP is cut out and replaced be "..". For pre‐
352 viewing an output, this fits most needs. For a listing with the
353 full IPV6 addresses add option -6.
354 -6 Print full length of IPv6 addresses in output instead of con‐
356 densed.
357 -q Quiet mode. Suppress the header line and the statistics at the
359 bottom of text outputs.
360 -N Print plain numbers in output without scaling. Easier for output
362 parsing with 3rd party tools.
363 -i ident
365 Change the ident label in the file, specified by -r to ident
366 -v flowfile
368 Verify the consistency of flowfile and print the file parameters
369 and number of records.
370 -E flowfile
372 Print the exporter and sampler list if found in flowfile. Addi‐
373 tional statistics per exporter are printed with number of flows,
374 packets and sequence errors.
375 -x flowfile
377 This options works on nfdump version 1.6.x files only and may get
378 removed in future. Scans and prints extension maps located in
379 flowfile
380 -z=lzo Compress flow files with LZO1X-1 compression. Fastest compres‐
382 sion.
383 -z=lz4 Compress flow files with LZ4 compression. Fast and efficient.
385 -z=bz2 Compress flow files with bz2 compression. Slow but most effi‐
387 cient. May be used for archiving files or if you are really short
388 of spce.
389 -J compress
391 Change compression for any number of files given by option -r
392 flowpath Set compress to 0 for no compression or to any of: 1 or
393 LZO, 2 or BZ2, 3 or LZ4. This option may be used for archiving
394 flow files and changing the compression to use less disk space.
395 -X Compiles the filter syntax and dumps the filter engine table to
397 stdout. This is for debugging purpose only.
398 -Z Check filter syntax and exit. Sets the return value accordingly.
400 -R filelist
402 Select a range of files. This option is mainly used by old NfSen
403 and documented here as legacy option.
404 /any/dir Read recursively all files in directory dir.
405 /dir/file Read all files beginning with file.
406 /dir/file1:file2 Read all files from file1 to file2.
407 When using in combination with a sub hierarchy:
408 /dir/sub1/sub2/file1:sub3/sub4/file2 Read all files from
409 sub1/sub2/file1 sub3/sub4/file2 iterating over all required hier‐
410 archy levels. Note: files are read in alphabetical order.
411 -M dirlist
413 Read the same file hierarchy from multiple directories. This op‐
414 tion is mainly used by old NfSen and documented here as legacy
415 option. Example: /any/path/to/dir1:dir2:dir3 etc. and will be
416 expanded to the directories: /any/path/to/dir1, /any/path/to/dir2
417 and /any/path/to/dir3. Any number of colon separated directories
418 may be given. The files to read are specified by -r or -R and are
419 expected to exist in all the given directories. The options -r
420 and -R must not contain any directories when used in combination
421 with -M.
422 -T Tag IP addresses with a prepending cntrl-A character, to allow
424 output parsers to hook in. This option is mainly used by old Nf‐
425 Sen and documented here as legacy option.
426 -V Print nfdump version and exit.
428 -h Print help text on stdout with all options and exit.
430 filter selects, which records will be further processed. If no filter is
432 given, all records will be processed. Otherwise, only those flows match‐
433 ing the filter will be processed. Any IP address in a filter may be spec‐
434 ified as IPv4 or IPv6.
435 The filter syntax is similar to tcpdump but adapted and extended for flow
437 records. The filter can be either specified on the command line after
438 all options or in a separate file. It can span several lines. Anything
439 after a '#' is treated as a comment and ignored to the end of the line.
440 There is virtually no limit in the length of the filter expression. All
441 keywords are case insensitive.
442 A single filter primitive filters a single element of a flow record. A
444 filter consists of one or more primitives, which are linked together:
445 expr and expr
447 expr or expr
448 not expr and (expr)
449 Possible filter primitives:
451 @include file Expands the content of file into the current filter
452 count comp number True if the comparison with the record counter
454 matches number Each record gets assigned a record
455 number at the time it is read from file. Therefore
456 this record number is not unique and may change, de‐
457 pending on the order files are read.
458 ident string True if the record ident field matches string. This
459 filter can be used to filter out different sources.
460 inet
462 ipv4 True if source and destination IP of a record are
463 IPv4 IPs.
464 inet6
466 ipv6 True if source and destination IP of a record are
467 IPv6 IPs.
468 proto protocol True if the record protocol field matches protocol.
470 protocol can be a symbolic name such as tcp, udp,
471 icmp, ah, esp, ipip, and many more or a protocol num‐
472 ber, such as 6, 17 for protocol tcp and udp.
473 tun proto protocol True if the record tunnel protocol field matches
475 protocol. protocol may be a symbolic name or protocol
476 number.
477 ip ipaddr
479 src ip ipaddr
480 dst ip ipaddr True if the respective IP field of the record matches
481 ipaddr. ipaddr may be an IPv4 or IPv6 address or a
482 symbolic hostname. In this case a DNS lookup resolves
483 the hostname to one or more IP addresses. If more
484 than one IP results, all IPs are chained together in
485 an or chain. (IP or IP or IP). If ip is not speci‐
486 fied with src or dst the source or destination IP may
487 match.
488 host ipaddr host is just a synonym for ip (See above)
489 ip in [iplist]
491 src in ip [iplist]
492 dst ip [iplist] True if the respective IP field of the record is in
493 iplist. iplist is a space or ',' separated list of IP
494 addresses or networks in CIDR notation. This is the
495 preferred way to search in large list of IP addresses
496 and networks and is much more efficient than to chain
497 all IP addresses together. (IP1 or IP2 or IP3). The
498 iplist may contain several hundreds to thousand IPs
499 and/or networks. For just a few IPs use an or chain,
500 otherwise use an iplist If ip is not specified with
501 src or dst the source or destination IP may match.
502 net network netmask
504 src net network netmask
505 dst net network netmask
506 net network/netbits
507 src net network/netbits
508 dst net network/netbits
509 True if the respective IP field of the record matches
510 the network if the corresponding netmask or netbits
511 are applied to the IP address. If net is not speci‐
512 fied with src or dst the source or destination IP may
513 match.
514 geo geoloc
516 src geo geocode
517 dst geo geocode True, if the 2-letter country code resolved by ge‐
518 olookup of the source or destination IP address
519 matches geocode. This filter works only, if a valid
520 geoDB is specified. See geo location option above.
521 The 2-letter country code corresponds to the maxmin
522 DB definitions. if geo is not specified with src or
523 dst the source or destination geo location code may
524 match.
525 tunip ipaddr
527 src tunip ipaddr
528 dst tunip ipaddr True if the respective tunnel IP field of the record
529 matches ipaddr. If tunip is not specified with src
530 or dst the source or destination tunnel IP may match.
531 port comp num
533 src port comp num
534 dst port comp num True if the comparison of the respective port field
535 matches num See comp for the comparator details. If
536 port is not specified with src or dst the source or
537 destination port may match.
538 port in [portlist]
540 src port in [portlist]
541 dst port in [portlist]
542 True if the respective port field of the record is in
543 portlist. portlist is a space or ',' separated list
544 of port numbers. This is the preferred way to search
545 in large list of port numbers and is much more effi‐
546 cient than to chain all ports together. (PORT1 or
547 PORT2 or PORT3). portlist may contain several hun‐
548 dreds to thousand of port numbers. If port is not
549 specified with src or dst the source or destination
550 port may match.
551 icmp-type num
553 icmp-code num True if the respective icmp field of the record
554 matches num. This automatically implies proto icmp.
555 engine-type num
557 engine-id num
558 sysid num True if the respective fields of the record matches
559 num engine type and ID are set by the exporting de‐
560 vice, sysid refers to the nfdump collector internal
561 assigned number. See also option -E above.
562 if num
564 in if num
565 out if num True if the respective interface fields of the record
566 matches num. This ID may correspond to the SNMP ID of
567 the interface but depends on the exporter. If if is
568 not specified with in or out the input or output in‐
569 terface may match.
570 as comp num
572 src as comp num
573 dst as comp num
574 prev as comp num
575 next as comp num True if the comparison of the respective AS fields
576 matches nfdump supports 32-bit AS numbers every
577 where. Without or the source or destination AS may
578 match. See comp for the comparator details.
579 as in [aslist]
581 src as in [aslist]
582 dst as in [aslist]
583 prev as in [aslist]
584 next as in [aslist]
585 True if the respective AS field of the record is in
586 aslist. aslist is a space or ',' separated list of AS
587 numbers. This is the preferred way to search in
588 large list of AS numbers and is much more efficient
589 than to chain all ports together. aslist may contain
590 several hundreds to thousand of AS numbers. If as is
591 not specified with src, dst, prev or next the source
592 or destination AS may match.
593 mask bits
595 src mask bits
596 dst mask bits True if the respective mask bit field of the record
597 matches bits If mask is not specified with src or dst
598 the source or destination mask bits may match.
599 vlan num
601 src vlan num
602 dst vlan num True if the respective vlan field of the record
603 matches num If vlan is not specified with src or dst
604 the source or destination vlan may match.
605 flags tcpflags True if the respective tcp flags field of the record
607 matches any of the given tcpflags. tcpflags is a
608 string combination of all flags to be tested:
609 A ACK.
610 S SYN.
611 F FIN.
612 R Reset.
613 P Push.
614 U Urgent.
615 X All flags on.
616 The order of the flags within tcpflags is not rele‐
617 vant. Flags not mentioned are treated as don't care.
618 In order to get those flows with only the SYN flag
619 set, use the syntax
620 flags S and not flags AFRPU
622 router ip ipaddr True if the ip address of the sending router matches
624 ipaddr as valid IPv4/IPv6 address.
625 next ip ipaddr True if the field next-ip of the record matches
627 ipaddr as valid IPv4/IPv6 address.
628 bgpnext ip ipaddr True if the field bgpnext-ip of the record matches
630 ipaddr as valid IPv4/IPv6 address.
631 mac macaddr
633 in mac macaddr
634 in src mac macaddr
635 in dst mac macaddr
636 out mac macaddr
637 out src mac macaddr
638 out dst mac macaddr
639 True if the respective mac address field of the
640 record matches macaddr By prepending mac with any
641 combination of a direction specifier as defined by
642 CISCO v9 the test is limited to those mac addresses
643 only. Otherwise multiple matches are possible. With‐
644 out any specifiers any mac address is tested against
645 macaddr
646 mpls labelN comp number
648 True if the comparison of the mpls label N with N as
649 mpls label number 1..10 matches number Filters ac‐
650 cording a specific number in the mpls label stack.
651 mpls eos comp number
653 True if the comparison of the end of stack mpls label
654 matches number
655 mpls expN comp number
657 True if the comparison of the experimental bits 0..7
658 of mpls label N with N as mpls label number 1..10
659 matches number
660 packets comp num
662 in packets comp num
663 out packets comp num
664 True if the comparison of the packet counter in the
665 flow record matches num. num may contain any valid
666 scaling factor such as k, m, g Example: packets > 1k.
667 For a single flow packets and in packets is equiva‐
668 lent and describes the number of packets from source
669 to destination. In case of a bi-directional flow
670 (sent by an exporter or combined by option --B ) the
671 packet counter for the reverse flow can be tested
672 with out packet
673 bytes comp num
675 in bytes comp num
676 out bytes comp num True if the comparison of the byte counter in the
677 flow record matches num. num may contain any valid
678 scaling factor such as k, m, g Example: bytes > 1k
679 bytes and in bytes is equivalent and describes the
680 number of bytes from source to destination. In case
681 of a bi-directional flow (sent by an exporter or com‐
682 bined by option --B ) the byte counter for the re‐
683 verse flow can be tested with out bytes
684 flows comp num True if the comparison of the flow counter in the
686 flow record matches num. num may contain any valid
687 scaling factor such as k, m, g For each received
688 flow, the flow counter is set to 1, unless the ex‐
689 porter sends this information. If multiple flows are
690 aggregated, this counter is increased respectively.
691 tos num True if the type of service field of the flow record
693 matches num
694 flowdir direction True, if the flow direction field in the flow record
696 matches direction. direction may be ingress, egress,
697 0 for ingress, or 1 for egress
698 duration comp time True if the calculated duration of a flow (tend -
700 tstart) compares to time. The duration is specified
701 in msec (milliseconds)
702 pps comp num True if the calculated value of in-packets/duration
704 (packets per second) compares with the number num.
705 num may contain any valid scaling factor such as k,
706 m, g
707 bps comp num True if the calculated value of 8*in-bytes/duration
709 (bits per second) compares with the number num. num
710 may contain any valid scaling factor such as k, m, g
711 bpp comp num True if the calculated value of in-bytes/in-packets
713 (bytes per packet) compares with the number num. num
714 may contain any valid scaling factor such as k, m, g
715 observation domain id comp number
717 observation point id comp number
718 True if the comparison of the observation domain ID
719 or point ID field respectively matches number
720 payload filters Some exporters, such as yaf or the nfdump collector
722 nfpcap can send payload data along the netflow
723 information. If such payloads are sent it can be fil‐
724 tered according the filter primitives below:
725 payload content 'string'
727 True if the string string is found in the payload
728 data. string must be quoted with single or double
729 quotes: 'string', “string”
730 payload regex 'regex'
732 payload regex 'regex' flags
733 True if regex matches the payload data. regex
734 searches over the full payload length. A ' ' byte
735 does not stop the match process. regex must be
736 quoted with single or double quotes: 'regex' or
737 “regex” The regex engine understands the following
738 reduced syntax:
739 (...) subexpressions/capture ranges
741 | the "or" operator
742 ^and $ anchors
743 [...] and [^...] character classes
744 ?, *, +, simple quantifiers
745 *?, +?, ?? lazy quantifiers
746 {<num>}, {<num1>,<num2>} complex quantifiers
747 flags are optional can be:
749 m multiline
750 i case insensitive matching
751 s
752 payload ja3 md5string
754 True, if the payload contains the start of an SSL/TLS
755 handshake and the calculated jas value of the hand‐
756 shake matches md5string
757 payload ja3 defined
759 True, if the payload contains the start of an SSL/TLS
760 handshake and a valid ja3 value can be calculated.
761 Useful to mask out all flow records with no SSL/TLS
762 traffic in order to generate a -s ja3 statistic.
763 OpenBSD pflog implemented elements
765 pf action action True, if the respective pflog action field compares
767 to one of pass, block, scrub, noscrub, nat, nonat,
768 binat, nobinat, rdr, nordr, synblock, defer, match,
769 divert, rt, afrt
770 pf reason reason True, if the respective pflog reason field compares
771 to one of match, bad-offset, fragment, short,
772 normalize, memory, bad-timestamp, congestion,
773 ip-option, proto-cksum, state-mismatch, state-insert,
774 state-limit, src-limit, synproxy, translate, no-route
775 pf rule ruleNr True, if the respective pflog rule number field
776 matches ruleNr
777 pf dir in|out True, if the respective pflog rule direction field
778 matches in or out
779 pf interface interfaceName
780 True, if the respective pflog rule interface name
781 field matches the string interfaceName
782 nprobe implemented elements
784 client latency comp time
786 server latency comp time
787 True, if the respective latency field in the flow
788 record compares to time. time is specified in msec.
789 CISCO ASA, network security event logging (NSEL) and NAT event logging
791 (NEL) specific filters:
792 NSEL specific filters:
793 asa event event True if the NSEL event type of an event record
795 matches event which may be: ignore, create, term,
796 delete, deny
797 asa event comp number
799 True if the comparison of the NSEL event type of an
800 event records matches number as a number.
801 asa event denied reason
803 True if the event denied type of an event records
804 matches reason which may be ingress, egress,
805 interface, nosyn
806 asa xevent comp num
808 True, if the comparison of the extended event field
809 of the event record matches num
810 xip ipaddr
812 src xip ipaddr
813 dst xip ipaddr True, if the field of the translated source or desti‐
814 nation IP address matches ipaddr if xip is specified
815 without src or dst both IP addresses may match.
816 xport ipaddr
818 src xport ipaddr
819 dst xport ipaddr True, if the field of the translated source or desti‐
820 nation IP address matches ipaddr if xport is speci‐
821 fied without src or dst both ports may match.
822 xnet network/mask
824 src xnet network/mask
825 dst xnet network/mask
826 True if the translated source or destination IP ad‐
827 dress matches network if mask mask is applied. if
828 xnet is specified without src or dst both IP ad‐
829 dresses may match.
830 ingress ACL comp number
832 ingress ACE comp number
833 ingress XACE comp number
834 True if the comparison of the respective ingress
835 field matches number
836 egress ACL comp number
838 True if the comparison of the egress field matches
839 number
840 NEL specific filters:
842 nat event event True if the NEL event type of an event record matches
843 event. event may be add, delete
844 nat event comp number
846 True if the comparison of the NEL event type of an
847 event records matches number as a number.
848 nip ipaddr
850 src nip ipaddr
851 dst nip ipaddr True, if the field of the nat source or destination
852 IP address matches ipaddr if nip is specified without
853 src or dst both IP addresses may match.
854 It Cm nport Ar number
856 src nport number
857 dst nport number True, if the field of the nat source or destination
858 port matches number if nip is specified without src
859 or dst both ports may match.
860 ingress vrf number True, if the field of the ingess vrf field of the
862 event record matches number
863 pblock start comp number
865 pblock step comp number
866 pblock end comp number
867 True if the comparison of the start, step or end of
868 the NAT port block in the event record matches number
869 port in pblock
870 src port in pblock
871 dst port in pblock True, if the source or destination port field matches
872 the NAT port block range
873 comp Many filter elements support the comparison with a
875 number. The following comparators are supported for
876 each of those filters: =, ==, >, <, >=, <= To prevent
877 collisions with bash interpretation, alternative com‐
878 parators are available: EQ, LT, GT, LE, GE If comp is
879 omitted, '==' is assumed.
880
882 This section describes how output formats are compiled. nfdump has a lot
883 of already pre-defined output formats such as raw, json, csv etc. One
884 line formats as described for option -o can be compiled from various ele‐
885 ments of a flow record. As a flow record can contains man different ele‐
886 ments it is often useful to compile an output format for specific needs.
887 Format description
889 The output format is specified by -o “fmt: string” string contains the
890 field tags to be printed as well as other characters if needed. A tag
891 starts with a % sign followed by the field name. tags are separated by
892 spaces from other tags. Characters or other strings, not starting with a
893 % sign are copied literally to the output.
894 Example:
896 -o “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
897 This is the definition of the predined format line. It adds the elements
899 tstart duration protocol source IP address/port followed by the literal
900 characters -> and destination ip address/port packets, bytes, flows
901 counter. Depending on the task, different output formats are required to
902 see the required fields of a flow record. You can either extend a prede‐
903 fined format or specify a new one at the command line.
904 Example: Extend the predefined format long with the the IP address of the
906 sending router
907 -o “fmt:%long %ra”
908 Predefined formats can be extended by simply add their name with a % sign
910 somewhere in the format string. As described under the output option -o
911 Format definition
913 nfdump has already many formats predefined. Most of the time, these for‐
914 mat are good enough. Sometimes you may need different formats, which can
915 be compiled as described above. In order to prevent adding the same often
916 used output format each time you run nfdump a new output format may be
917 define in the config file nfdump.conf The file nfdump.conf.dist contains
918 the definition of the already hard coded formats. These may be uncom‐
919 mented and changed according to the specific needs. New formats may be
920 added using the following syntax:
921 fmt.newname = “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
923 with newname any new or existing definition of output formats. Existing
925 formats are overwritten with the new definition.
926 Tag definition
928 The following list contains all tags, which are available to compile the
929 output format:
930 %<format>
932 Inserts the predefined format at this position. e.g.
933 %line
934 %cnt Record counter. record numbers are assigned dynamically
935 assigned while reading read from file.
936 %nfv Netflow version.
937 %ts Start Time - first seen
938 %tfs First seen - identical to %ts
939 %tsr Start Time, but in fractional seconds since the epoch
940 (1970-01-01) UNIX format.
941 %te End Time - last seen
942 %ter End Time, in fractional seconds
943 %tr Time the flow was received by the collector
944 %trr Time the flow was received, in fractional seconds
945 %td Duration of flow. Displayed in ddHHMMSS.msec
946 %pr Transort protocol
947 %exp Exporter ID
948 %eng Engine Type/ID
949 %lbl Flowlabel
950 %sa Source Address
951 %da Destination Address
952 %sap Source Address:Port
953 %dap Destination Address:Port
954 %gsap Source Address(country code):Port
955 %gdap Destination Address(country code):Port
956 %sp Source Port
957 %dp Destination Port
958 %it ICMP-type
959 %ic ICMP-code
960 %sn Source Network, mask applied
961 %dn Destination Network, mask applied
962 %nh Next-hop IP Address
963 %nhb BGP Next-hop IP Address
964 %ra Router IP Address
965 %sas Source AS
966 %das Destination AS
967 %nas Next AS
968 %pas Previous AS
969 %in Input Interface num
970 %out Output Interface num
971 %pkt Packets - default input
972 %ipkt Input Packets
973 %opkt Output Packets
974 %byt Bytes - default input
975 %ibyt Input Bytes
976 %obyt Output Bytes
977 %fl Flows
978 %flg TCP Flags
979 %tos Tos - default src
980 %stos Src Tos
981 %dtos Dst Tos
982 %dir Direction: ingress, egress
983 %smk Src mask
984 %dmk Dst mask
985 %fwd Forwarding Status
986 %svln Src vlan label
987 %dvln Dst vlan label
988 %ismc Input Src Mac Addr
989 %odmc Output Dst Mac Addr
990 %idmc Input Dst Mac Addr
991 %osmc Output Src Mac Addr
992 %mpls1 MPLS label 1
993 %mpls2 MPLS label 2
994 %mpls3 MPLS label 3
995 %mpls4 MPLS label 4
996 %mpls5 MPLS label 5
997 %mpls6 MPLS label 6
998 %mpls7 MPLS label 7
999 %mpls8 MPLS label 8
1000 %mpls9 MPLS label 9
1001 %mpls10 MPLS label 10
1002 %mpls MPLS labels 1-10
1003 %bps bps - bits per second
1004 %pps pps - packets per second
1005 %bpp bps - Bytes per package
1006 %sc src IP 2 letter country code
1007 %dc dst IP 2 letter country code
1008 %sloc src IP geo location info
1009 %dloc dst IP geo location info
1010 %sasn src AS organisation name
1011 %dasn dst AS organisation name
1012 %n new line char \n
1013 %ipl input payload
1014 %opl output payload
1015 %nbid nbar ID
1016 %ja3 ja3 hash
1017 %sni sni name in tls handshake
1018 %nbnam nbar name
1019 %odid observation domainID
1020 %opid observation pointID
1021 OpenBSD pflog specific formats
1023 %pfifn pflog interface name
1024 %pfact pflog action
1025 %pfrea pflog reason
1026 %pfdir pflog direction
1027 %pfrule pflog rule nr
1028 NSEL specific formats
1030 %nfc NSEL connection ID
1031 %evt NSEL event
1032 %xevt NSEL extended event
1033 %sgt NSEL Source security group tag
1034 %msec NSEL event time in msec
1035 %iacl NSEL ingress ACL
1036 %eacl NSEL egress ACL
1037 %xsa NSEL XLATE src IP address
1038 %xda NSEL XLATE dst IP address
1039 %xsp NSEL XLATE src port
1040 %xdp NSEL SLATE dst port
1041 %xsap Xlate Source Address:Port
1042 %xdap Xlate Destination Address:Port
1043 %uname NSEL user name
1044 NEL/NAT specific formats
1046 %nevt NAT event - same as %evt
1047 %ivrf NAT ingress VRF ID
1048 %evrf NAT egress VRF ID
1049 %nsa NAT src IP address
1050 %nda NAT dst IP address
1051 %nsp NAT src port
1052 %ndp NAT dst port
1053 %pbstart NAT pool block start
1054 %pbend NAT pool block end
1055 %pbstep NAT pool block step
1056 %pbsize NAT pool block size
1057 Nprobe formats
1059 %cl Client latency
1060 %sl Server latency
1061 %al Application latency
1062
1064 nfdump processes files created by any previous version of nfdump 1.6.x
1065 with some limitations for versions < 1.6.17. In order to convert flow
1066 files to the new 1.7.x binary format use the following command to
1067 read//write files:
1068 % nfdump -r oldfile -w newfile
1070 Print a statistic about the top 20 IP addresses, once sorted by flows and
1072 once by bytes
1073 % nfdump -r flowfile -s ip/flows/bytes -n 20
1075 Print two statistics, one about the source IP and one about the destina‐
1077 tion IP address limited to flow with either source or destination port
1078 443
1079 % nfdump -r flowfile -s srcip/bytes -s dstip/bytes -n 20 'port 443'
1081 Print a statistic about the IP pairs, which exchanged most traffic.
1083 % nfdump -r flowfile -s record/bytes -A srcip,dstip
1085 Print all flows in raw format with a HTTP header in the payload even if
1087 flow is not on port 80.
1088 % nfdump -r flowfile -o raw “payload regex 'GET|POST'”
1090 Print a statistic about all ja3 md5 sums for those flows, which a valid
1092 ja3 can be calculated
1093 % nfdump -r flowfile -s ja5 -n 0 'payload ja3 defined'
1095 Aggregate all flows and write the result back to a binary file, sorted by
1097 the start time
1098 % nfdump -r flowfile -a -Otstart -w newfile
1100
1102 nfdump returns 0 on success and 255 if processing failed.
1103
1105 https://www.iana.org/assignments/ipfix/ipfix.xhtml
1106 https://www.cisco.com/en/US/technologies/tk648/tk362/technolo‐
1108 gies_white_paper09186a00800a3db9.html
1109 nfcapd(1) nfpcapd(1) sfcapd(1) geolookup(1)
1111
1113 No software without bugs! Please report any bugs back to me.
1114BSD December 20, 2023 BSD