1NFDUMP(1) BSD General Commands Manual NFDUMP(1)
3
5 nfdump — flow display and analysis program
6
8 nfdump -r flowpath [-w outfile] [-f filterfile] [-C config] [-R filelist]
9 [-M dirlist] [-O order] [-t timewin] [-c num] [-a]
10 [-A aggregation] [-b] [-B] [-I] [-D nameserver] [-G geoDB]
11 [-s statistic] [-n num] [-o format] [-6] [-q] [-N] [-i ident]
12 [-v flowfile] [-E flowfile] [-x flowfile] [-z] [-y] [-j] [-J num]
13 [-X] [-Z] [-T] [-V] [-h] [filter]
14
16 nfdump reads the flow data from one or more binary files, created by any
17 nfdump collector nfcapd, nfpcapd and sfcapd. It processes and lists the
18 flows in many different output formats and can create a wide range of
19 statistics.
20 nfdump has a very powerful flow filter to process flows. The filter syn‐
22 tax is very similar to tcpdump, but adapted and extended for flow filter‐
23 ing. A flow filter may also contain arrays of many thousand IP addresses
24 etc. to search for specific records.
25 nfdump can aggreagte flows according to a user defined number of ele‐
27 ments. This masks certain elements and allows to sum up flow records
28 matching the same values.
29 The combination of flow filtering and aggregation as input for any flow
31 statistics allows complex flow processing. Pre-filtered and aggregated
32 flow data may also be written back into a binary flow file, which again
33 may be processed with nfdump
34 nfdump can enrich the listing of flows with geo location information and
36 AS information, unless AS information is already available in the flow
37 records. IP addresses can be tagged with a two letter country code, or
38 with a longer location label containing the geographic region, country
39 and city. The geo location and AS information is retrieved from the op‐
40 tional geoDB database, created by the geolookup program from the nfdump
41 tools. geolookup uses the Maxmind database GeoDB or GeoLite2 to create a
42 binary lookup database for nfdump Please check the geolooup(1) man page
43 for more details.
44 The options are as follows:
46 -r flowpath
48 Reads flow records from this path. flowpath may be a single
49 file, or a directory containing any number of flow files or sub
50 directories. All files are processed in the order, as listed by
51 the OS.
52 -w outfile
54 Writes all processed records into outfile instead of printing.
55 The flowfile is a binary flow file and may be processed again
56 with nfdump This can be useful to limit flows according to a flow
57 filter and/or specific flow aggregation.
58 -f filterfile
60 Reads the flow filter from filterfile. This can be useful for
61 very long or structured filters, with comments and long lists.
62 Note: Any filter specified directly on the command line takes
63 precedence over the filterfile.
64 -C config
66 Read more options from file config. nfdump tries to read by de‐
67 fault %prefix/etc/nfdump.config. This may be overwritten by the
68 environment valiable NFCONF which again may be overwritten by
69 this option -C. In order to prevent reading any config file, even
70 if it would exist set -C none. A config file is not required, but
71 may be handy for often used output formats etc.
72 -O order
74 Sets an output order for records to be printed as text output.
75 This order applies after all records processing, such as filter‐
76 ing, and aggregation and before printing.
77 flows Sort according to the number of flows
78 packets Sort according to (in)packets
79 ipkg Same as packets
80 opkg Sort according to output packets
81 bytes Sort according to (in)bytes
82 ibyte Same as bytes
83 obyte Sort according to output bytes
84 pps Sort according to (in)packets per second
85 ipps Same as ipps
86 opps Sort according to out packets per second
87 bps Sort according to (in)bytes per second
88 ibps Same as bps
89 obps Sort according to output bytes per second
90 bpp Sort according to (in)bytes per packet
91 ibpp Same as bpp
92 obpp Sort according to output packets
93 tstart Sort according to start time of flow - former -m
94 tend Sort according to end time of flows
95 duration Sort according to duration of flows
96 -t timewin
98 Set time window to process flows. This option is considered
99 legacy andmay be replaced with a filter primitive in future
100 rleases. The time window is specified as:
101 YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss]. Any parts of the time
102 spec may be omitted e.g YYYY/MM/dd expands to
103 YYYY/MM/dd.00:00:00-infinity and processes all flow from a given
104 day onwards. The time window may also be specified as +/- n. In
105 this case it is relative to the beginning or end of all flows.
106 +10 means the first 10 seconds of all flows, -10 means the last
107 10 seconds of all flows.
108 -c num Limit the number of records to be processed to the first num
110 records, which passwd the filter.
111 -a Aggregate flow records. The default aggregation is done at con‐
113 nection level by taking the 5-tuple protocol, srcip, dstip,
114 srcport and dstport. This way of aggregation may be overwritten
115 by option -A
116 -A aggregation
118 Sets the list of elements in a flow record to be aggregated.
119 aggregation is a ',' separated list of any number of v9/ipfix el‐
120 ements. The following elements are accepted:
121 proto IP protocol
122 srcip Source IP address
123 dstip Destination IP address
124 srcip4/net IPv4 source IP address with applied netmask
125 srcip6/net IPv6 source IP address with applied netmask
126 dstip4/net IPv4 destination IP address with applied netmask
127 dstip6/net IPv6 destination IP address with applied netmask
128 srcnet Apply netmask srcmask in netflow record for source IP
129 dstnet Apply netmask dstmask in netflow record for dest IP
130 srcport Source port
131 dstport Destination port
132 srcmask Source mask
133 dstmask Destination mask
134 srcvlan Source vlan label
135 dstvlan Destination vlan label
136 srcas Source AS number
137 dstas Destination AS number
138 nextas BGP Next AS
139 prevas BGP Previous AS
140 inif SNMP input interface number
141 outif SNMP output interface number
142 next IP next hop
143 bgpnext BGP next hop
144 insrcmac In source MAC address
145 outdstmac out destination MAC address
146 indstmac In destination MAC address
147 outsrcmac Out source MAC address
148 tos Source type of service
149 srctos Source type of Service
150 dsttos Destination type of Service
151 mpls1 MPLS label 1
152 mpls2 MPLS label 2
153 mpls3 MPLS label 3
154 mpls4 MPLS label 4
155 mpls5 MPLS label 5
156 mpls6 MPLS label 6
157 mpls7 MPLS label 7
158 mpls8 MPLS label 8
159 mpls9 MPLS label 9
160 mpls10 MPLS label 10
161 router IP address of exporting router
162 odid observation domain ID
163 opid observation point ID
164 xsrcip X-late source IP address, if compiled with NSEL sup‐
165 port
166 xdstip X-late destination IP address, if compiled with NSEL
167 support
168 xsrcport X-late source port, if compiled with NSEL support
169 xdstport X-late destination port, if compiled with NSEL sup‐
170 port
171 nfdump automatically compiles the appropriate output format for
173 the selected aggregation elements unless an explicit output for‐
174 mat -o is given. The automatic output format is identical to
175 -o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl'
177 where <fields> represents the selected aggregation tags.
179 -b Aggregate flow records as bidirectional flows. This automatically
181 implies -a. Aggregation is done on connection level by taking
182 the 5-tuple protocol, srcip, dstip, srcport and dstport The re‐
183 verse order applies for the corresponding reverse flow. Input and
184 output packets/bytes are counted and reported separately. Both
185 flows are merged into a single record with corresponding input
186 and output counters. An appropriate output format is selected au‐
187 tomatically, which may be overwritten by any -o format option.
188 -B Similar to option -b but tries to guess the correct client to
190 server direction. Automagically swaps flows if src port is < dst
191 port for TCP and UDP flows and src port < 1024 and dst port >
192 1024. Some exporters do not really care sending the flows in
193 proper order. It's considered to be a conveniency option.
194 -I Print flow statistics of a single file or the summary of all the
196 files specified by -r flowpath.
197 -g Print for each flow file given by -r flowpath a one line summary,
199 which can be easily used by gnu plot.
200 -D nameserver
202 Sets the nameserver to translate hostnames into IP addresses in
203 filter expressions. See filter below for more details.
204 -G geoDB
206 Use geoDB as geo lookup DB for geo location and AS lookups.
207 nfdump tries to read the environment variable NFGEODB for the
208 path of geoDB. The option -G overwrites NFGEODB. In order to pre‐
209 vent reading any geoDB file, even if it would exist set -G none.
210 -s statistic [:p [/orderby]]
212 Generate the Top N flow record or flow element statistic. By op‐
213 tionally adding :p to statistic, the statistic is additionally
214 split up into the transport layer protocols. By default the
215 statistic is transport protocol independent. Each statistic may
216 be ordered by the optional parameter orderby This can be flows,
217 packets, bytes, pps, bps or bpp. You may specify more than one
218 orderby option, which results in the same statistic but ordered
219 differently. If no orderby is given, the statistic is ordered by
220 flows. You can specify as many -s flow element statistics as
221 needed on the command line for the same run.
222 statistic can be:
224 record aggregated netflow records.
226 srcip source IP addresses
227 dstip destination IP addresses
228 ip any (src or dst) IP addresses
229 nhip next hop IP addresses
230 nhbip BGP next hop IP addresses
231 router exporting router IP address
232 srcport source ports
233 dstport destination ports
234 port any (source or destination) ports
235 tos type of service - default src
236 srctos src type of service
237 dsttos dst type of service
238 dir flow directions ingress/egress
239 srcas source AS numbers
240 dstas destination AS numbers
241 srcgeo 2 letter geo source country code
242 dstgeo 2 letter geo destination country code
243 as any (source or destination) AS numbers
244 inif input interface
245 outif output interface
246 if any interface
247 inam input interface name
248 onam output interface name
249 srcmask src mask
250 dstmask dst mask
251 srcvlan src vlan label
252 dstvlan dst vlan label
253 vlan any vlan label
254 insrcmac input src MAC address
255 outdstmac output dst MAC address
256 indstmac input dst MAC address
257 outsrcmac output src MAC address
258 srcmac any src MAC address
259 dstmac any dst MAC address
260 inmac any input MAC address
261 outmac any output MAC address
262 mask any mask
263 proto IP protocols
264 mpls1 MPLS label 1
265 mpls2 MPLS label 2
266 mpls3 MPLS label 3
267 mpls4 MPLS label 4
268 mpls5 MPLS label 5
269 mpls6 MPLS label 6
270 mpls7 MPLS label 7
271 mpls8 MPLS label 8
272 mpls9 MPLS label 9
273 mpls10 MPLS label 10
274 sysid Internal SysID of exporter
275 nbar nbar ID
276 ja3 ja3 hashes
277 odid observation domain ID
278 opid observation point ID
279 vrf/ivrf ingress vrf
280 evrf egress vrf
281 ivrfnam ingress vrf name
282 evrfnam egress vrf name
283 NSEL/ASA statistics
285 event NSEL/ASA event
286 xevent NSEL/ASA extended event
287 xsrcip NSEL/ASA translated src IP address
288 xsrcport NSEL/ASA translated src port
289 xdstip NSEL/ASA translated dst IP address
290 xdstport NSEL/ASA translated dst port
291 iacl NSEL/ASA ingress ACL
292 iace NSEL/ASA ingress ACE
293 ixace NSEL/ASA ingress xACE
294 eacl NSEL/ASA egress ACL
295 eace NSEL/ASA egress ACE
296 exace NSEL/ASA egress xACE
297 NAT statistics
299 nevent NAT event
300 nsrcip NAT src IP address
301 nsrcport NAT src port
302 ndstip NAT dst IP address
303 ndstport NAT dst port
304 % nfdump -s srcip -s ip/flows/bytes -s record/bytes
306 -n num Set the number of records to be printed to num. This option ap‐
308 plies to -s statistics as well as to ordered output -O -or
309 -aggregated -records -a The default is set to 10 for statistics
310 and unlimited for the other use cases. To disable the limit, set
311 num to 0.
312 -o format
314 Sets the output format to print flow records. has many different
315 output formats already predefined. format may be one of the
316 options below:
317 raw Print the full flow record on multiple lines. This
319 prints all available information.
320 fmt: user Print the flow records according the format user. This
321 is a very flexible and powerful way to format flow
322 records. See the section OUTPUT below for more details
323 on how to compile your own format.
324 json Print full record as a separate json object.
325 csv Legacy .csv format - will get removed in future re‐
326 leases. Please use json instead.
327 pipe Legacy '|' separated format - will get removed in fu‐
328 ture releases. Please use json instead.
329 Already predefined fmt formats:
331 line Print each flow on one line. Default format.
333 long Print each flow on one line with more details
334 biline Same as line, but for bi-directional flows
335 bilong Same as long, but for bi-directional flows
336 gline Same as line, but add country code to IPs. If a geoDB
337 file is supplied this is the default output format
338 glong Same as long, but add country code to IPs
339 extended Print each flow on one line with even more details.
340 nsel Print format for NSEL event records. Default format if
341 NSEL/NAT support has been compiled in.
342 nel Print format for NAT event records.
343 The nfdump config file may contain additional formats. If you
345 want to add new formats or change existing ones, check the config
346 file.
347 IPv6 addresses are printed condensed in any fmt defined format to
349 prevent cluttering the output with large blank blocks. A con‐
350 densed IPV6 uses max 16 characters. If it is longer, then the
351 middle part of the IP is cut out and replaced be "..". For pre‐
352 viewing an output, this fits most needs. For a listing with the
353 full IPV6 addresses add option -6.
354 -6 Print full length of IPv6 addresses in output instead of con‐
356 densed.
357 -q Quiet mode. Suppress the header line and the statistics at the
359 bottom of text outputs.
360 -N Print plain numbers in output without scaling. Easier for output
362 parsing with 3rd party tools.
363 -i ident
365 Change the ident label in the file, specified by -r to ident
366 -v flowfile
368 Verify the consistency of flowfile and print the file parameters
369 and number of records.
370 -E flowfile
372 Print the exporter and sampler list if found in flowfile. Addi‐
373 tional statistics per exporter are printed with number of flows,
374 packets and sequence errors.
375 -x flowfile
377 This options works on nfdump version 1.6.x files only and may get
378 removed in future. Scans and prints extension maps located in
379 flowfile
380 -z Compress flow files with LZO1X-1 compression. Fastest compres‐
382 sion.
383 -y Compress flow files with LZ4 compression. Fast and efficient.
385 -j Compress flow files with bz2 compression. Slow but most effi‐
387 cient. May be used for archiving files or if you are really short
388 of spce.
389 -J num Change compression for any number of files given by option -r
391 flowpath num: 0 uncompress, 1: LZO1X-1, 2: bz2, 3: LZ4 compres‐
392 sion. This option may be used for archiving flow files and chang‐
393 ing the compression to use less disk space.
394 -X Compiles the filter syntax and dumps the filter engine table to
396 stdout. This is for debugging purpose only.
397 -Z Check filter syntax and exit. Sets the return value accordingly.
399 -R filelist
401 Select a range of files. This option is mainly used by old NfSen
402 and documented here as legacy option.
403 /any/dir Read recursively all files in directory dir.
404 /dir/file Read all files beginning with file.
405 /dir/file1:file2 Read all files from file1 to file2.
406 When using in combination with a sub hierarchy:
407 /dir/sub1/sub2/file1:sub3/sub4/file2 Read all files from
408 sub1/sub2/file1 sub3/sub4/file2 iterating over all required hier‐
409 archy levels. Note: files are read in alphabetical order.
410 -M dirlist
412 Read the same file hierarchy from multiple directories. This op‐
413 tion is mainly used by old NfSen and documented here as legacy
414 option. Example: /any/path/to/dir1:dir2:dir3 etc. and will be
415 expanded to the directories: /any/path/to/dir1, /any/path/to/dir2
416 and /any/path/to/dir3. Any number of colon separated directories
417 may be given. The files to read are specified by -r or -R and are
418 expected to exist in all the given directories. The options -r
419 and -R must not contain any directories when used in combination
420 with -M.
421 -T Tag IP addresses with a prepending cntrl-A character, to allow
423 output parsers to hook in. This option is mainly used by old Nf‐
424 Sen and documented here as legacy option.
425 -V Print nfdump version and exit.
427 -h Print help text on stdout with all options and exit.
429 filter selects, which records will be further processed. If no filter is
431 given, all records will be processed. Otherwise, only those flows match‐
432 ing the filter will be processed. Any IP address in a filter may be spec‐
433 ified as IPv4 or IPv6.
434 The filter syntax is similar to tcpdump but adapted and extended for flow
436 records. The filter can be either specified on the command line after
437 all options or in a separate file. It can span several lines. Anything
438 after a '#' is treated as a comment and ignored to the end of the line.
439 There is virtually no limit in the length of the filter expression. All
440 keywords are case insensitive.
441 A single filter primitive filters a single element of a flow record. A
443 filter consists of one or more primitives, which are linked together:
444 expr and expr
446 expr or expr
447 not expr and (expr)
448 Possible filter primitives:
450 @include file Expands the content of file into the current filter
451 count comp number True if the comparison with the record counter
453 matches number Each record gets assigned a record
454 number at the time it is read from file. Therefore
455 this record number is not unique and may change, de‐
456 pending on the order files are read.
457 ident string True if the record ident field matches string. This
458 filter can be used to filter out different sources.
459 inet
461 ipv4 True if source and destination IP of a record are
462 IPv4 IPs.
463 inet6
465 ipv6 True if source and destination IP of a record are
466 IPv6 IPs.
467 proto protocol True if the record protocol field matches protocol.
469 protocol can be a symbolic name such as tcp, udp,
470 icmp, ah, esp, ipip, and many more or a protocol num‐
471 ber, such as 6, 17 for protocol tcp and udp.
472 tun proto protocol True if the record tunnel protocol field matches
474 protocol. protocol may be a symbolic name or protocol
475 number.
476 ip ipaddr
478 src ip ipaddr
479 dst ip ipaddr True if the respective IP field of the record matches
480 ipaddr. ipaddr may be an IPv4 or IPv6 address or a
481 symbolic hostname. In this case a DNS lookup resolves
482 the hostname to one or more IP addresses. If more
483 than one IP results, all IPs are chained together in
484 an or chain. (IP or IP or IP). If ip is not speci‐
485 fied with src or dst the source or destination IP may
486 match.
487 host ipaddr host is just a synonym for ip (See above)
488 ip in [iplist]
490 src in ip [iplist]
491 dst ip [iplist] True if the respective IP field of the record is in
492 iplist. iplist is a space or ',' separated list of IP
493 addresses or networks in CIDR notation. This is the
494 preferred way to search in large list of IP addresses
495 and networks and is much more efficient than to chain
496 all IP addresses together. (IP1 or IP2 or IP3). The
497 iplist may contain several hundreds to thousand IPs
498 and/or networks. For just a few IPs use an or chain,
499 otherwise use an iplist If ip is not specified with
500 src or dst the source or destination IP may match.
501 net network netmask
503 src net network netmask
504 dst net network netmask
505 net network/netbits
506 src net network/netbits
507 dst net network/netbits
508 True if the respective IP field of the record matches
509 the network if the corresponding netmask or netbits
510 are applied to the IP address. If net is not speci‐
511 fied with src or dst the source or destination IP may
512 match.
513 geo geoloc
515 src geo geocode
516 dst geo geocode True, if the 2-letter country code resolved by ge‐
517 olookup of the source or destination IP address
518 matches geocode. This filter works only, if a valid
519 geoDB is specified. See geo location option above.
520 The 2-letter country code corresponds to the maxmin
521 DB definitions. if geo is not specified with src or
522 dst the source or destination geo location code may
523 match.
524 tunip ipaddr
526 src tunip ipaddr
527 dst tunip ipaddr True if the respective tunnel IP field of the record
528 matches ipaddr. If tunip is not specified with src
529 or dst the source or destination tunnel IP may match.
530 port comp num
532 src port comp num
533 dst port comp num True if the comparison of the respective port field
534 matches num See comp for the comparator details. If
535 port is not specified with src or dst the source or
536 destination port may match.
537 port in [portlist]
539 src port in [portlist]
540 dst port in [portlist]
541 True if the respective port field of the record is in
542 portlist. portlist is a space or ',' separated list
543 of port numbers. This is the preferred way to search
544 in large list of port numbers and is much more effi‐
545 cient than to chain all ports together. (PORT1 or
546 PORT2 or PORT3). portlist may contain several hun‐
547 dreds to thousand of port numbers. If port is not
548 specified with src or dst the source or destination
549 port may match.
550 icmp-type num
552 icmp-code num True if the respective icmp field of the record
553 matches num. This automatically implies proto icmp.
554 engine-type num
556 engine-id num
557 sysid num True if the respective fields of the record matches
558 num engine type and ID are set by the exporting de‐
559 vice, sysid refers to the nfdump collector internal
560 assigned number. See also option -E above.
561 if num
563 in if num
564 out if num True if the respective interface fields of the record
565 matches num. This ID may correspond to the SNMP ID of
566 the interface but depends on the exporter. If if is
567 not specified with in or out the input or output in‐
568 terface may match.
569 as comp num
571 src as comp num
572 dst as comp num
573 prev as comp num
574 next as comp num True if the comparison of the respective AS fields
575 matches nfdump supports 32-bit AS numbers every
576 where. Without or the source or destination AS may
577 match. See comp for the comparator details.
578 as in [aslist]
580 src as in [aslist]
581 dst as in [aslist]
582 prev as in [aslist]
583 next as in [aslist]
584 True if the respective AS field of the record is in
585 aslist. aslist is a space or ',' separated list of AS
586 numbers. This is the preferred way to search in
587 large list of AS numbers and is much more efficient
588 than to chain all ports together. aslist may contain
589 several hundreds to thousand of AS numbers. If as is
590 not specified with src, dst, prev or next the source
591 or destination AS may match.
592 mask bits
594 src mask bits
595 dst mask bits True if the respective mask bit field of the record
596 matches bits If mask is not specified with src or dst
597 the source or destination mask bits may match.
598 vlan num
600 src vlan num
601 dst vlan num True if the respective vlan field of the record
602 matches num If vlan is not specified with src or dst
603 the source or destination vlan may match.
604 flags tcpflags True if the respective tcp flags field of the record
606 matches any of the given tcpflags. tcpflags is a
607 string combination of all flags to be tested:
608 A ACK.
609 S SYN.
610 F FIN.
611 R Reset.
612 P Push.
613 U Urgent.
614 X All flags on.
615 The order of the flags within tcpflags is not rele‐
616 vant. Flags not mentioned are treated as don't care.
617 In order to get those flows with only the SYN flag
618 set, use the syntax
619 flags S and not flags AFRPU
621 router ip ipaddr True if the ip address of the sending router matches
623 ipaddr as valid IPv4/IPv6 address.
624 next ip ipaddr True if the field next-ip of the record matches
626 ipaddr as valid IPv4/IPv6 address.
627 bgpnext ip ipaddr True if the field bgpnext-ip of the record matches
629 ipaddr as valid IPv4/IPv6 address.
630 mac macaddr
632 in mac macaddr
633 in src mac macaddr
634 in dst mac macaddr
635 out mac macaddr
636 out src mac macaddr
637 out dst mac macaddr
638 True if the respective mac address field of the
639 record matches macaddr By prepending mac with any
640 combination of a direction specifier as defined by
641 CISCO v9 the test is limited to those mac addresses
642 only. Otherwise multiple matches are possible. With‐
643 out any specifiers any mac address is tested against
644 macaddr
645 mpls labelN comp number
647 True if the comparison of the mpls label N with N as
648 mpls label number 1..10 matches number Filters ac‐
649 cording a specific number in the mpls label stack.
650 mpls eos comp number
652 True if the comparison of the end of stack mpls label
653 matches number
654 mpls expN comp number
656 True if the comparison of the experimental bits 0..7
657 of mpls label N with N as mpls label number 1..10
658 matches number
659 packets comp num
661 in packets comp num
662 out packets comp num
663 True if the comparison of the packet counter in the
664 flow record matches num. num may contain any valid
665 scaling factor such as k, m, g Example: packets > 1k.
666 For a single flow packets and in packets is equiva‐
667 lent and describes the number of packets from source
668 to destination. In case of a bi-directional flow
669 (sent by an exporter or combined by option --B ) the
670 packet counter for the reverse flow can be tested
671 with out packet
672 bytes comp num
674 in bytes comp num
675 out bytes comp num True if the comparison of the byte counter in the
676 flow record matches num. num may contain any valid
677 scaling factor such as k, m, g Example: bytes > 1k
678 bytes and in bytes is equivalent and describes the
679 number of bytes from source to destination. In case
680 of a bi-directional flow (sent by an exporter or com‐
681 bined by option --B ) the byte counter for the re‐
682 verse flow can be tested with out bytes
683 flow comp num True if the comparison of the flow counter in the
685 flow record matches num. num may contain any valid
686 scaling factor such as k, m, g For each received
687 flow, the flow counter is set to 1, unless the ex‐
688 porter sends this information. If multiple flows are
689 aggregated, this counter is increased respectively.
690 tos num True if the type of service field of the flow record
692 matches num
693 flowdir direction True, if the flow direction field in the flow record
695 matches direction. direction may be ingress, egress,
696 0 for ingress, or 1 for egress
697 duration comp time True if the calculated duration of a flow (tend -
699 tstart) compares to time. The duration is specified
700 in msec (milliseconds)
701 pps comp num True if the calculated value of in-packets/duration
703 (packets per second) compares with the number num.
704 num may contain any valid scaling factor such as k,
705 m, g
706 bps comp num True if the calculated value of 8*in-bytes/duration
708 (bits per second) compares with the number num. num
709 may contain any valid scaling factor such as k, m, g
710 bpp comp num True if the calculated value of in-bytes/in-packets
712 (bytes per packet) compares with the number num. num
713 may contain any valid scaling factor such as k, m, g
714 observation domain id comp number
716 observation point id comp number
717 True if the comparison of the observation domain ID
718 or point ID field respectively matches number
719 payload filters Some exporters, such as yaf or the nfdump collector
721 nfpcap can send payload data along the netflow
722 information. If such payloads are sent it can be fil‐
723 tered according the filter primitives below:
724 payload content 'string'
726 True if the string string is found in the payload
727 data. string must be quoted with single or double
728 quotes: 'string', “string”
729 payload regex 'regex'
731 payload regex 'regex' flags
732 True if regex matches the payload data. regex
733 searches over the full payload length. A ' ' byte
734 does not stop the match process. regex must be
735 quoted with single or double quotes: 'regex' or
736 “regex” The regex engine understands the following
737 reduced syntax:
738 (...) subexpressions/capture ranges
740 | the "or" operator
741 ^and $ anchors
742 [...] and [^...] character classes
743 ?, *, +, simple quantifiers
744 *?, +?, ?? lazy quantifiers
745 {<num>}, {<num1>,<num2>} complex quantifiers
746 flags are optional can be:
748 m multiline
749 i case insensitive matching
750 s
751 payload ja3 md5string
753 True, if the payload contains the start of an SSL/TLS
754 handshake and the calculated jas value of the hand‐
755 shake matches md5string
756 payload ja3 defined
758 True, if the payload contains the start of an SSL/TLS
759 handshake and a valid ja3 value can be calculated.
760 Useful to mask out all flow records with no SSL/TLS
761 traffic in order to generate a -s ja3 statistic.
762 OpenBSD pflog implemented elements
764 pf action action True, if the respective pflog action field compares
766 to one of pass, block, scrub, noscrub, nat, nonat,
767 binat, nobinat, rdr, nordr, synblock, defer, match,
768 divert, rt, afrt
769 pf reason reason True, if the respective pflog reason field compares
770 to one of match, bad-offset, fragment, short,
771 normalize, memory, bad-timestamp, congestion,
772 ip-option, proto-cksum, state-mismatch, state-insert,
773 state-limit, src-limit, synproxy, translate, no-route
774 pf rule ruleNr True, if the respective pflog rule number field
775 matches ruleNr
776 pf dir in|out True, if the respective pflog rule direction field
777 matches in or out
778 pf interface interfaceName
779 True, if the respective pflog rule interface name
780 field matches the string interfaceName
781 nprobe implemented elements
783 client latency comp time
785 server latency comp time
786 True, if the respective latency field in the flow
787 record compares to time. time is specified in msec.
788 CISCO ASA, network security event logging (NSEL) and NAT event logging
790 (NEL) specific filters:
791 NSEL specific filters:
792 asa event event True if the NSEL event type of an event record
794 matches event which may be: ignore, create, term,
795 delete, deny
796 asa event comp number
798 True if the comparison of the NSEL event type of an
799 event records matches number as a number.
800 asa event denied reason
802 True if the event denied type of an event records
803 matches reason which may be ingress, egress,
804 interface, nosyn
805 asa xevent comp num
807 True, if the comparison of the extended event field
808 of the event record matches num
809 xip ipaddr
811 src xip ipaddr
812 dst xip ipaddr True, if the field of the translated source or desti‐
813 nation IP address matches ipaddr if xip is specified
814 without src or dst both IP addresses may match.
815 xport ipaddr
817 src xport ipaddr
818 dst xport ipaddr True, if the field of the translated source or desti‐
819 nation IP address matches ipaddr if xport is speci‐
820 fied without src or dst both ports may match.
821 xnet network/mask
823 src xnet network/mask
824 dst xnet network/mask
825 True if the translated source or destination IP ad‐
826 dress matches network if mask mask is applied. if
827 xnet is specified without src or dst both IP ad‐
828 dresses may match.
829 ingress ACL comp number
831 ingress ACE comp number
832 ingress XACE comp number
833 True if the comparison of the respective ingress
834 field matches number
835 egress ACL comp number
837 True if the comparison of the egress field matches
838 number
839 NEL specific filters:
841 nat event event True if the NEL event type of an event record matches
842 event. event may be add, delete
843 nat event comp number
845 True if the comparison of the NEL event type of an
846 event records matches number as a number.
847 nip ipaddr
849 src nip ipaddr
850 dst nip ipaddr True, if the field of the nat source or destination
851 IP address matches ipaddr if nip is specified without
852 src or dst both IP addresses may match.
853 It Cm nport Ar number
855 src nport number
856 dst nport number True, if the field of the nat source or destination
857 port matches number if nip is specified without src
858 or dst both ports may match.
859 ingress vrf number True, if the field of the ingess vrf field of the
861 event record matches number
862 pblock start comp number
864 pblock step comp number
865 pblock end comp number
866 True if the comparison of the start, step or end of
867 the NAT port block in the event record matches number
868 port in pblock
869 src port in pblock
870 dst port in pblock True, if the source or destination port field matches
871 the NAT port block range
872 comp Many filter elements support the comparison with a
874 number. The following comparators are supported for
875 each of those filters: =, ==, >, <, >=, <= To prevent
876 collisions with bash interpretation, alternative com‐
877 parators are available: EQ, LT, GT, LE, GE If comp is
878 omitted, '==' is assumed.
879
881 This section describes how output formats are compiled. nfdump has a lot
882 of already pre-defined output formats such as raw, json, csv etc. One
883 line formats as described for option -o can be compiled from various ele‐
884 ments of a flow record. As a flow record can contains man different ele‐
885 ments it is often useful to compile an output format for specific needs.
886 Format description
888 The output format is specified by -o “fmt: string” string contains the
889 field tags to be printed as well as other characters if needed. A tag
890 starts with a % sign followed by the field name. tags are separated by
891 spaces from other tags. Characters or other strings, not starting with a
892 % sign are copied literally to the output.
893 Example:
895 -o “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
896 This is the definition of the predined format line. It adds the elements
898 tstart duration protocol source IP address/port followed by the literal
899 characters -> and destination ip address/port packets, bytes, flows
900 counter. Depending on the task, different output formats are required to
901 see the required fields of a flow record. You can either extend a prede‐
902 fined format or specify a new one at the command line.
903 Example: Extend the predefined format long with the the IP address of the
905 sending router
906 -o “fmt:%long %ra”
907 Predefined formats can be extended by simply add their name with a % sign
909 somewhere in the format string. As described under the output option -o
910 Format definition
912 nfdump has already many formats predefined. Most of the time, these for‐
913 mat are good enough. Sometimes you may need different formats, which can
914 be compiled as described above. In order to prevent adding the same often
915 used output format each time you run nfdump a new output format may be
916 define in the config file nfdump.conf The file nfdump.conf.dist contains
917 the definition of the already hard coded formats. These may be uncom‐
918 mented and changed according to the specific needs. New formats may be
919 added using the following syntax:
920 fmt.newname = “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
922 with newname any new or existing definition of output formats. Existing
924 formats are overwritten with the new definition.
925 Tag definition
927 The following list contains all tags, which are available to compile the
928 output format:
929 %<format>
931 Inserts the predefined format at this position. e.g.
932 %line
933 %cnt Record counter. record numbers are assigned dynamically
934 assigned while reading read from file.
935 %nfv Netflow version.
936 %ts Start Time - first seen
937 %tfs First seen - identical to %ts
938 %tsr Start Time, but in fractional seconds since the epoch
939 (1970-01-01) UNIX format.
940 %te End Time - last seen
941 %ter End Time, in fractional seconds
942 %tr Time the flow was received by the collector
943 %trr Time the flow was received, in fractional seconds
944 %td Duration of flow. Displayed in ddHHMMSS.msec
945 %pr Transort protocol
946 %exp Exporter ID
947 %eng Engine Type/ID
948 %lbl Flowlabel
949 %sa Source Address
950 %da Destination Address
951 %sap Source Address:Port
952 %dap Destination Address:Port
953 %gsap Source Address(country code):Port
954 %gdap Destination Address(country code):Port
955 %sp Source Port
956 %dp Destination Port
957 %it ICMP-type
958 %ic ICMP-code
959 %sn Source Network, mask applied
960 %dn Destination Network, mask applied
961 %nh Next-hop IP Address
962 %nhb BGP Next-hop IP Address
963 %ra Router IP Address
964 %sas Source AS
965 %das Destination AS
966 %nas Next AS
967 %pas Previous AS
968 %in Input Interface num
969 %out Output Interface num
970 %pkt Packets - default input
971 %ipkt Input Packets
972 %opkt Output Packets
973 %byt Bytes - default input
974 %ibyt Input Bytes
975 %obyt Output Bytes
976 %fl Flows
977 %flg TCP Flags
978 %tos Tos - default src
979 %stos Src Tos
980 %dtos Dst Tos
981 %dir Direction: ingress, egress
982 %smk Src mask
983 %dmk Dst mask
984 %fwd Forwarding Status
985 %svln Src vlan label
986 %dvln Dst vlan label
987 %ismc Input Src Mac Addr
988 %odmc Output Dst Mac Addr
989 %idmc Input Dst Mac Addr
990 %osmc Output Src Mac Addr
991 %mpls1 MPLS label 1
992 %mpls2 MPLS label 2
993 %mpls3 MPLS label 3
994 %mpls4 MPLS label 4
995 %mpls5 MPLS label 5
996 %mpls6 MPLS label 6
997 %mpls7 MPLS label 7
998 %mpls8 MPLS label 8
999 %mpls9 MPLS label 9
1000 %mpls10 MPLS label 10
1001 %mpls MPLS labels 1-10
1002 %bps bps - bits per second
1003 %pps pps - packets per second
1004 %bpp bps - Bytes per package
1005 %sc src IP 2 letter country code
1006 %dc dst IP 2 letter country code
1007 %sloc src IP geo location info
1008 %dloc dst IP geo location info
1009 %sasn src AS organisation name
1010 %dasn dst AS organisation name
1011 %n new line char \n
1012 %ipl input payload
1013 %opl output payload
1014 %nbid nbar ID
1015 %ja3 ja3 hash
1016 %sni sni name in tls handshake
1017 %nbnam nbar name
1018 %odid observation domainID
1019 %opid observation pointID
1020 OpenBSD pflog specific formats
1022 %pfifn pflog interface name
1023 %pfact pflog action
1024 %pfrea pflog reason
1025 %pfdir pflog direction
1026 %pfrule pflog rule nr
1027 NSEL specific formats
1029 %nfc NSEL connection ID
1030 %evt NSEL event
1031 %xevt NSEL extended event
1032 %sgt NSEL Source security group tag
1033 %msec NSEL event time in msec
1034 %iacl NSEL ingress ACL
1035 %eacl NSEL egress ACL
1036 %xsa NSEL XLATE src IP address
1037 %xda NSEL XLATE dst IP address
1038 %xsp NSEL XLATE src port
1039 %xdp NSEL SLATE dst port
1040 %xsap Xlate Source Address:Port
1041 %xdap Xlate Destination Address:Port
1042 %uname NSEL user name
1043 NEL/NAT specific formats
1045 %nevt NAT event - same as %evt
1046 %ivrf NAT ingress VRF ID
1047 %evrf NAT egress VRF ID
1048 %nsa NAT src IP address
1049 %nda NAT dst IP address
1050 %nsp NAT src port
1051 %ndp NAT dst port
1052 %pbstart NAT pool block start
1053 %pbend NAT pool block end
1054 %pbstep NAT pool block step
1055 %pbsize NAT pool block size
1056 Nprobe formats
1058 %cl Client latency
1059 %sl Server latency
1060 %al Application latency
1061
1063 nfdump processes files created by any previous version of nfdump 1.6.x
1064 with some limitations for versions < 1.6.17. In order to convert flow
1065 files to the new 1.7.x binary format use the following command to
1066 read//write files:
1067 % nfdump -r oldfile -w newfile
1069 Print a statistic about the top 20 IP addresses, once sorted by flows and
1071 once by bytes
1072 % nfdump -r flowfile -s ip/flows/bytes -n 20
1074 Print two statistics, one about the source IP and one about the destina‐
1076 tion IP address limited to flow with either source or destination port
1077 443
1078 % nfdump -r flowfile -s srcip/bytes -s dstip/bytes -n 20 'port 443'
1080 Print a statistic about the IP pairs, which exchanged most traffic.
1082 % nfdump -r flowfile -s record/bytes -A srcip,dstip
1084 Print all flows in raw format with a HTTP header in the payload even if
1086 flow is not on port 80.
1087 % nfdump -r flowfile -o raw “payload regex 'GET|POST'”
1089 Print a statistic about all ja3 md5 sums for those flows, which a valid
1091 ja3 can be calculated
1092 % nfdump -r flowfile -s ja5 -n 0 'payload ja3 defined'
1094 Aggregate all flows and write the result back to a binary file, sorted by
1096 the start time
1097 % nfdump -r flowfile -a -Otstart -w newfile
1099
1101 nfdump returns 0 on success and 255 if processing failed.
1102
1104 https://www.iana.org/assignments/ipfix/ipfix.xhtml
1105 https://www.cisco.com/en/US/technologies/tk648/tk362/technolo‐
1107 gies_white_paper09186a00800a3db9.html
1108 nfcapd(1) nfpcapd(1) sfcapd(1) geolookup(1)
1110
1112 No software without bugs! Please report any bugs back to me.
1113BSD December 17, 2023 BSD