1
2NFDUMP(1)                 BSD General Commands Manual                NFDUMP(1)
3

NAME

5     nfdump — flow display and analysis program
6

SYNOPSIS

8     nfdump -r flowpath [-w outfile] [-f filterfile] [-C config] [-R filelist]
9            [-M dirlist] [-O order] [-t timewin] [-c num] [-a]
10            [-A aggregation] [-b] [-B] [-I] [-D nameserver] [-G geoDB]
11            [-s statistic] [-n num] [-o format] [-6] [-q] [-N] [-i ident]
12            [-v flowfile] [-E flowfile] [-x flowfile] [-z] [-y] [-j] [-J num]
13            [-X] [-Z] [-T] [-V] [-h] [filter]
14

DESCRIPTION

16     nfdump reads the flow data from one or more binary files, created by any
17     nfdump collector nfcapd, nfpcapd and sfcapd.  It processes and lists the
18     flows in many different output formats and can create a wide range of
19     statistics.
20
21     nfdump has a very powerful flow filter to process flows. The filter syn‐
22     tax is very similar to tcpdump, but adapted and extended for flow filter‐
23     ing. A flow filter may also contain arrays of many thousand IP addresses
24     etc. to search for specific records.
25
26     nfdump can aggreagte flows according to a user defined number of ele‐
27     ments. This masks certain elements and allows to sum up flow records
28     matching the same values.
29
30     The combination of flow filtering and aggregation as input for any flow
31     statistics allows complex flow processing. Pre-filtered and aggregated
32     flow data may also be written back into a binary flow file, which again
33     may be processed with nfdump
34
35     nfdump can enrich the listing of flows with geo location information and
36     AS information, unless AS information is already available in the flow
37     records. IP addresses can be tagged with a two letter country code, or
38     with a longer location label containing the geographic region, country
39     and city.  The geo location and AS information is retrieved from the op‐
40     tional geoDB database, created by the geolookup program from the nfdump
41     tools.  geolookup uses the Maxmind database GeoDB or GeoLite2 to create a
42     binary lookup database for nfdump Please check the geolooup(1) man page
43     for more details.
44
45     The options are as follows:
46
47     -r flowpath
48             Reads flow records from this path.  flowpath may be a single
49             file, or a directory containing any number of flow files or sub
50             directories.  All files are processed in the order, as listed by
51             the OS.
52
53     -w outfile
54             Writes all processed records into outfile instead of printing.
55             The flowfile is a binary flow file and may be processed again
56             with nfdump This can be useful to limit flows according to a flow
57             filter and/or specific flow aggregation.
58
59     -f filterfile
60             Reads the flow filter from filterfile. This can be useful for
61             very long or structured filters, with comments and long lists.
62             Note: Any filter specified directly on the command line takes
63             precedence over the filterfile.
64
65     -C config
66             Read more options from file config. nfdump tries to read by de‐
67             fault %prefix/etc/nfdump.config. This may be overwritten by the
68             environment valiable NFCONF which again may be overwritten by
69             this option -C. In order to prevent reading any config file, even
70             if it would exist set -C none. A config file is not required, but
71             may be handy for often used output formats etc.
72
73     -O order
74             Sets an output order for records to be printed as text output.
75             This order applies after all records processing, such as filter‐
76             ing, and aggregation and before printing.
77             flows     Sort according to the number of flows
78             packets   Sort according to (in)packets
79             ipkg      Same as packets
80             opkg      Sort according to output packets
81             bytes     Sort according to (in)bytes
82             ibyte     Same as bytes
83             obyte     Sort according to output bytes
84             pps       Sort according to (in)packets per second
85             ipps      Same as ipps
86             opps      Sort according to out packets per second
87             bps       Sort according to (in)bytes per second
88             ibps      Same as bps
89             obps      Sort according to output bytes per second
90             bpp       Sort according to (in)bytes per packet
91             ibpp      Same as bpp
92             obpp      Sort according to output packets
93             tstart    Sort according to start time of flow - former -m
94             tend      Sort according to end time of flows
95             duration  Sort according to duration of flows
96
97     -t timewin
98             Set time window to process flows. This option is considered
99             legacy andmay be replaced with a filter primitiv in future
100             rleases. The time window is specified as:
101             YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss].  Any parts of the time
102             spec may be omitted e.g YYYY/MM/dd expands to
103             YYYY/MM/dd.00:00:00-infinity and processes all flow from a given
104             day onwards. The time window may also be specified as +/- n.  In
105             this case it is relativ to the beginning or end of all flows. +10
106             means the first 10 seconds of all flows, -10 means the last 10
107             seconds of all flows.
108
109     -c num  Limit the number of records to be processed to the first num
110             records, which passwd the filter.
111
112     -a      Aggregate flow records. The default aggregation is done at con‐
113             nection level by taking the 5-tuple protocol, srcip, dstip,
114             srcport and dstport. This way of aggregation may be overwritten
115             by option -A
116
117     -A aggregation
118             Sets the list of elements in a flow record to be aggregated.
119             aggregation is a ',' separated list of any number of v9/ipfix el‐
120             ements. The following elements are accepted:
121             proto       IP protocol
122             srcip       Source IP address
123             dstip       Destination IP address
124             srcip4/net  IPv4 source IP address with applied netmask
125             srcip6/net  IPv6 source IP address with applied netmask
126             dstip4/net  IPv4 destination IP address with applied netmask
127             dstip6/net  IPv6 destination IP address with applied netmask
128             srcnet      Apply netmask srcmask in netflow record for source IP
129             dstnet      Apply netmask dstmask in netflow record for dest IP
130             srcport     Source port
131             dstport     Destination port
132             srcmask     Source mask
133             dstmask     Destination mask
134             srcvlan     Source vlan label
135             dstvlan     Destination vlan label
136             srcas       Source AS number
137             dstas       Destination AS number
138             nextas      BGP Next AS
139             prevas      BGP Previous AS
140             inif        SNMP input interface number
141             outif       SNMP output interface number
142             next        IP next hop
143             bgpnext     BGP next hop
144             insrcmac    In source MAC address
145             outdstmac   out destination MAC address
146             indstmac    In destintation MAC address
147             outsrcmac   Out source MAC address
148             tos         Source type of service
149             srctos      Source type of Service
150             dsttos      Destination type of Service
151             mpls1       MPLS label 1
152             mpls2       MPLS label 2
153             mpls3       MPLS label 3
154             mpls4       MPLS label 4
155             mpls5       MPLS label 5
156             mpls6       MPLS label 6
157             mpls7       MPLS label 7
158             mpls8       MPLS label 8
159             mpls9       MPLS label 9
160             mpls10      MPLS label 10
161             router      IP address of exporting router
162             odid        observation domain ID
163             opid        observation point ID
164             xsrcip      X-late source IP address, if compiled with NSEL sup‐
165                         port
166             xdstip      X-late destination IP address, if compiled with NSEL
167                         support
168             xsrcport    X-late source port, if compiled with NSEL support
169             xdstport    X-late destination port, if compiled with NSEL sup‐
170                         port
171
172             nfdump automatically compiles the appropriate output format for
173             the selected aggregation elements unless an explicit output for‐
174             mat -o is given. The automatic output format is identical to
175
176                   -o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl'
177
178             where <fields> represents the selected aggregation tags.
179
180     -b      Aggregate flow records as bidirectional flows. This automatically
181             implies -a.  Aggregation is done on connection level by taking
182             the 5-tuple protocol, srcip, dstip, srcport and dstport The re‐
183             verse order applies for the corresponding reverse flow. Input and
184             output packets/bytes are counted and reported separately. Both
185             flows are merged into a single record with corresponding input
186             and output counters. An appropriate output format is selected au‐
187             tomatically, which may be overwritten by any -o format option.
188
189     -B      Similar to option -b but tries to guess the correct client to
190             server direction. Automagically swaps flows if src port is < dst
191             port for TCP and UDP flows and src port < 1024 and dst port >
192             1024.  Some exporters do not really care sending the flows in
193             proper order.  It's considered to be a conveniency option.
194
195     -I      Print flow statistics of a single file or the summary of all the
196             files specified by -r flowpath.
197
198     -g      Print for each flow file given by -r flowpath a one line summary,
199             which can be easily used by gnu plot.
200
201     -D nameserver
202             Sets the nameserver to translate hostnames into IP addresses in
203             filter expressions. See filter below for more details.
204
205     -G geoDB
206             Use geoDB as geo lookup DB for geo location and AS lookups.
207             nfdump tries to read the environment variable NFGEODB for the
208             path of geoDB. The option -G overwrites NFGEODB. In order to pre‐
209             vent reading any geoDB file, even if it would exist set -G none.
210
211     -s statistic [:p [/orderby]]
212             Generate the Top N flow record or flow element statistic. By op‐
213             tionally adding :p to statistic, the statistic is additionally
214             split up into the transport layer protocols.  By default the
215             statistic is transport protocol independent.  Each statistic may
216             be ordered by the optional parameter orderby This can be flows,
217             packets, bytes, pps, bps or bpp. You may specify more than one
218             orderby option, which results in the same statistic but ordered
219             differently. If no orderby is given, the statistic is ordered by
220             flows. You can specify as many -s flow element statistics as
221             needed on the command line for the same run.
222
223             statistic can be:
224
225             record     aggregated netflow records.
226             srcip      source IP addresses
227             dstip      destination IP addresses
228             ip         any (src or dst) IP addresses
229             nhip       next hop IP addresses
230             nhbip      BGP next hop IP addresses
231             router     exporting router IP address
232             srcport    source ports
233             dstport    destination ports
234             port       any (source or destination) ports
235             tos        type of service - default src
236             srctos     src type of service
237             dsttos     dst type of service
238             dir        flow directions ingress/egress
239             srcas      source AS numbers
240             dstas      destination AS numbers
241             srcgeo     2 letter geo source country code
242             dstgeo     2 letter geo destination country code
243             as         any (source or destination) AS numbers
244             inif       input interface
245             outif      output interface
246             if         any interface
247             inam       input interface name
248             onam       output interface name
249             srcmask    src mask
250             dstmask    dst mask
251             srcvlan    src vlan label
252             dstvlan    dst vlan label
253             vlan       any vlan label
254             insrcmac   input src MAC address
255             outdstmac  output dst MAC address
256             indstmac   input dst MAC address
257             outsrcmac  output src MAC address
258             srcmac     any src MAC address
259             dstmac     any dst MAC address
260             inmac      any input MAC address
261             outmac     any output MAC address
262             mask       any mask
263             proto      IP protocols
264             mpls1      MPLS label 1
265             mpls2      MPLS label 2
266             mpls3      MPLS label 3
267             mpls4      MPLS label 4
268             mpls5      MPLS label 5
269             mpls6      MPLS label 6
270             mpls7      MPLS label 7
271             mpls8      MPLS label 8
272             mpls9      MPLS label 9
273             mpls10     MPLS label 10
274             sysid      Internal SysID of exporter
275             nbar       nbar ID
276             ja3        ja3 hashes
277             odid       observation domain ID
278             opid       observation point ID
279             vrf/ivrf   ingress vrf
280             evrf       egress vrf
281             ivrfnam    ingress vrf name
282             evrfnam    egress vrf name
283
284                        NSEL/ASA statistics
285             event      NSEL/ASA event
286             xevent     NSEL/ASA extended event
287             xsrcip     NSEL/ASA translated src IP address
288             xsrcport   NSEL/ASA translated src port
289             xdstip     NSEL/ASA translated dst IP address
290             xdstport   NSEL/ASA translated dst port
291             iacl       NSEL/ASA ingress ACL
292             iace       NSEL/ASA ingress ACE
293             ixace      NSEL/ASA ingress xACE
294             eacl       NSEL/ASA egress ACL
295             eace       NSEL/ASA egress ACE
296             exace      NSEL/ASA egress xACE
297
298                        NAT statistics
299             nevent     NAT event
300             nsrcip     NAT src IP address
301             nsrcport   NAT src port
302             ndstip     NAT dst IP address
303             ndstport   NAT dst port
304
305                   % nfdump -s srcip -s ip/flows/bytes -s record/bytes
306
307     -n num  Set the number of records to be printed to num. This option ap‐
308             plies to -s statistics as well as to ordered output -O -or
309             -aggreated -records -a The default is set to 10 for statistics
310             and unlimited for the other use cases. To disable the limit, set
311             num to 0.
312
313     -o format
314             Sets the output format to print flow records.  has many different
315             output formats already predefined. format may be one of the
316             options below:
317
318             raw        Print the full flow record on multiple lines. This
319                        prints all available information.
320             fmt: user  Print the flow records according the format user. This
321                        is a very flexible and powerful way to format flow
322                        records. See the section OUTPUT below for more details
323                        on how to compile your own format.
324             json       Print full record as a separate json object.
325             csv        Legacy .csv format - will get removed in future re‐
326                        leases. Please use json instead.
327             pipe       Legacy '|' separated format - will get removed in fu‐
328                        ture releases. Please use json instead.
329
330             Already predefined fmt formats:
331
332             line      Print each flow on one line. Default format.
333             long      Print each flow on one line with more details
334             biline    Same as line, but for bi-directional flows
335             bilong    Same as long, but for bi-directional flows
336             gline     Same as line, but add country code to IPs. If a geoDB
337                       file is supplied this is the default output format
338             glong     Same as long, but add country code to IPs
339             extended  Print each flow on one line with even more details.
340             nsel      Print format for NSEL event records. Default format if
341                       NSEL/NAT support has been compiled in.
342             nel       Print format for NAT event records.
343
344             The nfdump config file may contain additional formats. If you
345             want to add new formats or change existing ones, check the config
346             file.
347
348             IPv6 addresses are printed condensed in any fmt defined format to
349             prevent cluttering the output with large blank blocks. A con‐
350             densed IPV6 uses max 16 characters. If it is longer, then the
351             middle part of the IP is cut out and replaced be "..". For pre‐
352             viewing an output, this fits most needs.  For a listing with the
353             full IPV6 addresses add option -6.
354
355     -6      Print full length of IPv6 addresses in output instead of con‐
356             densed.
357
358     -q      Quiet mode. Suppress the header line and the statistics at the
359             bottom of text outputs.
360
361     -N      Print plain numbers in output without scaling. Easier for output
362             parsing with 3rd party tools.
363
364     -i ident
365             Change the ident label in the file, specified by -r to ident
366
367     -v flowfile
368             Verify the consistency of flowfile and print the file parameters
369             and number of records.
370
371     -E flowfile
372             Print the exporter and sampler list if found in flowfile. Addi‐
373             tional statistics per exporter are printed with number of flows,
374             packets and sequence errors.
375
376     -x flowfile
377             This options works on nfdump version 1.6.x files only and may get
378             removed in future.  Scans and prints extension maps located in
379             flowfile
380
381     -z      Compress flow files with LZO1X-1 compression. Fastest compres‐
382             sion.
383
384     -y      Compress flow files with LZ4 compression. Fast and efficient.
385
386     -j      Compress flow files with bz2 compression. Slow but most effi‐
387             cient. May be used for archiving files or if you are really short
388             of spce.
389
390     -J num  Change compression for any number of files given by option -r
391             flowpath num: 0 uncompress, 1: LZO1X-1, 2: bz2, 3: LZ4 compres‐
392             sion. This option may be used for archiving flow files and chang‐
393             ing the compression to use less disk space.
394
395     -X      Compiles the filter syntax and dumps the filter engine table to
396             stdout. This is for debugging purpose only.
397
398     -Z      Check filter syntax and exit. Sets the return value accordingly.
399
400     -R filelist
401             Select a range of files. This option is mainly used by old NfSen
402             and documented here as legacy opton.
403             /any/dir          Read recursively all files in directory dir.
404             /dir/file         Read all files beginning with file.
405             /dir/file1:file2  Read all files from file1 to file2.
406             When using in combination with a sub hierarchy:
407             /dir/sub1/sub2/file1:sub3/sub4/file2 Read all files from
408             sub1/sub2/file1 sub3/sub4/file2 iterating over all required hier‐
409             archy levels.  Note: files are read in alphabetical order.
410
411     -M dirlist
412             Read the same file hierarchy from multiple directories. This op‐
413             tion is mainly used by old NfSen and documented here as legacy
414             option.  Example: /any/path/to/dir1:dir2:dir3 etc. and will be
415             expanded to the directories: /any/path/to/dir1, /any/path/to/dir2
416             and /any/path/to/dir3. Any number of colon separated directories
417             may be given. The files to read are specified by -r or -R and are
418             expected to exist in all the given directories. The options -r
419             and -R must not contain any directories when used in combination
420             with -M.
421
422     -T      Tag IP addresses with a prepending cntrl-A character, to allow
423             output parsers to hook in.  This option is mainly used by old Nf‐
424             Sen and documented here as legacy option.
425
426     -V      Print nfdump version and exit.
427
428     -h      Print help text on stdout with all options and exit.
429
430     filter selects, which records will be further processed. If no filter is
431     given, all records will be processed. Otherwise, only those flows match‐
432     ing the filter will be processed. Any IP address in a filter may be spec‐
433     ified as IPv4 or IPv6.
434
435     The filter syntax is similar to tcpdump but adapted and extended for flow
436     records.  The filter can be either specified on the command line after
437     all options or in a separate file. It can span several lines. Anything
438     after a '#' is treated as a comment and ignored to the end of the line.
439     There is virtually no limit in the length of the filter expression. All
440     keywords are case insensitive.
441
442     A single filter primitiv filters a single element of a flow record.  A
443     filter consists of one or more primitives, which are linked together:
444
445           expr and expr
446           expr or expr
447           not expr and (expr)
448
449     Possible filter primitives:
450     @include file       Expands the content of file into the current filter
451
452     count comp number   True if the comparison with the record counter
453                         matches number Each record gets assigned a record
454                         number at the time it is read from file. Therefore
455                         this record number is not unique and may change, de‐
456                         pending on the order files are read.
457     ident string        True if the record ident field matches string. This
458                         filter can be used to filter out different sources.
459
460     inet
461     ipv4                True if source and destination IP of a record are
462                         IPv4 IPs.
463
464     inet6
465     ipv6                True if source and destination IP of a record are
466                         IPv6 IPs.
467
468     proto protocol      True if the record protocol field matches protocol.
469                         protocol can be a symbolic name such as tcp, udp,
470                         icmp, ah, esp, ipip, and many more or a protocol num‐
471                         ber, such as 6, 17 for protocol tcp and udp.
472
473     tun proto protocol  True if the record tunnel protocol field matches
474                         protocol. protocol may be a symbolic name or protocol
475                         number.
476
477     ip ipaddr
478     src ip ipaddr
479     dst ip ipaddr       True if the respective IP field of the record matches
480                         ipaddr. ipaddr may be an IPv4 or IPv6 address or a
481                         symbolic hostname. In this case a DNS lookup resolves
482                         the hostname to one or more IP addresses. If more
483                         than one IP results, all IPs are chained together in
484                         an or chain. (IP or IP or IP).  If ip is not speci‐
485                         fied with src or dst the source or destination IP may
486                         match.
487     host ipaddr         host is just a synonym for ip (See above)
488
489     ip in [iplist]
490     src in ip [iplist]
491     dst ip [iplist]     True if the respective IP field of the record is in
492                         iplist. iplist is a space or ',' separated list of IP
493                         addresses or networks in CIDR notation.  This is the
494                         prefered way to search in large list of IP addresses
495                         and networks and is much more efficient than to chain
496                         all IP addresses together.  (IP1 or IP2 or IP3). The
497                         iplist may contain several hundreds to thousand IPs
498                         and/or networks. For just a few IPs use an or chain,
499                         otherwise use an iplist If ip is not specified with
500                         src or dst the source or destination IP may match.
501
502     net network netmask
503     src net network netmask
504     dst net network netmask
505     net network/netbits
506     src net network/netbits
507     dst net network/netbits
508                         True if the respective IP field of the record matches
509                         the network if the corresponding netmask or netbits
510                         are applied to the IP address.  If net is not speci‐
511                         fied with src or dst the source or destination IP may
512                         match.
513
514     geo geoloc
515     src geo geocode
516     dst geo geocode     True, if the 2-letter country code resolved by ge‐
517                         olookup of the source or destination IP address
518                         matches geocode. This filter works only, if a valid
519                         geoDB is specified. See geo location option above.
520                         The 2-letter country code corresponds to the maxmin
521                         DB definitions. if geo is not specified with src or
522                         dst the source or destination geo location code may
523                         match.
524
525     tunip ipaddr
526     src tunip ipaddr
527     dst tunip ipaddr    True if the respective tunnel IP field of the record
528                         matches ipaddr.  If tunip is not specified with src
529                         or dst the source or destination tunnel IP may match.
530
531     port comp num
532     src port comp num
533     dst port comp num   True if the comparison of the respective port field
534                         matches num See comp for the comparator details. If
535                         port is not specified with src or dst the source or
536                         destination port may match.
537
538     port in [portlist]
539     src port in [portlist]
540     dst port in [portlist]
541                         True if the respective port field of the record is in
542                         portlist. portlist is a space or ',' separated list
543                         of port numbers.  This is the prefered way to search
544                         in large list of port numbers and is much more effi‐
545                         cient than to chain all ports together. (PORT1 or
546                         PORT2 or PORT3).  portlist may contain several hun‐
547                         dreds to thousand of port numbers.  If port is not
548                         specified with src or dst the source or destination
549                         port may match.
550
551     icmp-type num
552     icmp-code num       True if the respective icmp field of the record
553                         matches num. This automatically implies proto icmp.
554
555     engine-type num
556     engine-id num
557     sysid num           True if the respective fields of the record matches
558                         num engine type and ID are set by the exporting de‐
559                         vice, sysid refers to the nfdump collector internal
560                         assigned number. See also option -E above.
561
562     if num
563     in if num
564     out if num          True if the respective interface fields of the record
565                         matches num. This ID may correspond to the SNMP ID of
566                         the interface but depends on the exporter.  If if is
567                         not specified with in or out the input or output in‐
568                         terface may match.
569
570     as comp num
571     src as comp num
572     dst as comp num
573     prev as comp num
574     next as comp num    True if the comparison of the respective AS fields
575                         matches nfdump supports 32-bit AS numbers every
576                         where.  Without or the source or destination AS may
577                         match. See comp for the comparator details.
578
579     as in [aslist]
580     src as in [aslist]
581     dst as in [aslist]
582     prev as in [aslist]
583     next as in [aslist]
584                         True if the respective AS field of the record is in
585                         aslist. aslist is a space or ',' separated list of AS
586                         numbers.  This is the prefered way to search in large
587                         list of AS numbers and is much more efficient than to
588                         chain all ports together.  aslist may contain several
589                         hundreds to thousand of AS numbers.  If as is not
590                         specified with src, dst, prev or next the source or
591                         destination AS may match.
592
593     mask bits
594     src mask bits
595     dst mask bits       True if the respective mask bit field of the record
596                         matches bits If mask is not specified with src or dst
597                         the source or destination mask bits may match.
598
599     vlan num
600     src vlan num
601     dst vlan num        True if the respective vlan field of the record
602                         matches num If vlan is not specified with src or dst
603                         the source or destination vlan may match.
604
605     flags tcpflags      True if the respective tcp flags field of the record
606                         matches any of the given tcpflags. tcpflags is a
607                         string combination of all flags to be tested:
608                               A    ACK.
609                               S    SYN.
610                               F    FIN.
611                               R    Reset.
612                               P    Push.
613                               U    Urgent.
614                               X    All flags on.
615                         The order of the flags within tcpflags is not rele‐
616                         vant. Flags not mentioned are treated as don't care.
617                         In order to get those flows with only the SYN flag
618                         set, use the syntax
619
620                               flags S and not flags AFRPU
621
622     router ip ipaddr    True if the ip address of the sending router matches
623                         ipaddr as valid IPv4/IPv6 address.
624
625     next ip ipaddr      True if the field next-ip of the record matches
626                         ipaddr as valid IPv4/IPv6 address.
627
628     bgpnext ip ipaddr   True if the field bgpnext-ip of the record matches
629                         ipaddr as valid IPv4/IPv6 address.
630
631     mac macaddr
632     in mac macaddr
633     in src mac macaddr
634     in dst mac macaddr
635     out mac macaddr
636     out src mac macaddr
637     out dst mac macaddr
638                         True if the respective mac address field of the
639                         record matches macaddr By prepending mac with any
640                         combination of a direction specifier as defined by
641                         CISCO v9 the test is limited to those mac addresses
642                         only. Otherwise multiple matches are possible. With‐
643                         out any specifiers any mac address is tested against
644                         macaddr
645
646     mpls labelN comp number
647                         True if the comparison of the mpls label N with N as
648                         mpls label number 1..10 matches number Filters ac‐
649                         cording a specific number in the mpls label stack.
650
651     mpls eos comp number
652                         True if the comparison of the end of stack mpls label
653                         matches number
654
655     mpls expN comp number
656                         True if the comparison of the experimental bits 0..7
657                         of mpls label N with N as mpls label number 1..10
658                         matches number
659
660     packets comp num
661     in packets comp num
662     out packets comp num
663                         True if the comparison of the packet counter in the
664                         flow record matches num. num may contain any valid
665                         scaling factor such as k, m, g Example: packets > 1k.
666                         For a single flow packets and in packets is equiva‐
667                         lent and describes the number of packets from source
668                         to destination.  In case of a bi-directional flow
669                         (sent by an exporter or combined by option --B ) the
670                         packet counter for the reverse flow can be tested
671                         with out packet
672
673     bytes comp num
674     in bytes comp num
675     out bytes comp num  True if the comparison of the byte counter in the
676                         flow record matches num. num may contain any valid
677                         scaling factor such as k, m, g Example: bytes > 1k
678                         bytes and in bytes is equivalent and describes the
679                         number of bytes from source to destination.  In case
680                         of a bi-directional flow (sent by an exporter or com‐
681                         bined by option --B ) the byte counter for the re‐
682                         verse flow can be tested with out bytes
683
684     flow comp num       True if the comparison of the flow counter in the
685                         flow record matches num. num may contain any valid
686                         scaling factor such as k, m, g For each received
687                         flow, the flow counter is set to 1, unless the ex‐
688                         porter sends this information. If multiple flows are
689                         aggregated, this counter is increased respectively.
690
691     tos num             True if the type of service field of the flow record
692                         matches num
693
694     flowdir direction   True, if the flow direction field in the flow record
695                         matches direction. direction may be ingress, egress,
696                         0 for ingress, or 1 for egress
697
698     duration comp time  True if the calculated duration of a flow (tend -
699                         tstart) compares to time. The duration is specified
700                         in msec (milliseconds)
701
702     pps comp num        True if the calculated value of in-packets/duration
703                         (packets per second) compares with the number num.
704                         num may contain any valid scaling factor such as k,
705                         m, g
706
707     bps comp num        True if the calculated value of 8*in-bytes/duration
708                         (bits per second) compares with the number num. num
709                         may contain any valid scaling factor such as k, m, g
710
711     bpp comp num        True if the calculated value of in-bytes/in-packets
712                         (bytes per packet) compares with the number num. num
713                         may contain any valid scaling factor such as k, m, g
714
715     observation domain id comp number
716     observation point id comp number
717                         True if the comparison of the observation domain ID
718                         or point ID field respectively matches number
719
720     payload filters     Some exporters, such as yaf or the nfdump collector
721                         nfpcap can send payload data along the netflow
722                         information. If such payloads are sent it can be fil‐
723                         tered according the filter primitives below:
724
725     payload content 'string'
726                         True if the string string is found in the payload
727                         data.  string must be quoted with single or double
728                         quotes: 'string', “string”
729
730     payload regex 'regex'
731     payload regex 'regex' flags
732                         True if regex matches the payload data.  regex
733                         searches over the full payload length. A ' ' byte
734                         does not stop the match process.  regex must be
735                         quoted with single or double quotes: 'regex' or
736                         “regex” The regex engine understands the following
737                         reduced syntax:
738
739                               (...) subexpressions/capture ranges
740                               | the "or" operator
741                               ^and $ anchors
742                               [...] and [^...] character classes
743                               ?, *, +, simple quantifiers
744                               *?, +?, ?? lazy quantifiers
745                               {<num>}, {<num1>,<num2>} complex quantifiers
746
747                         flags are otional can be:
748                               m multiline
749                               i case insensitive matching
750                               s
751
752     payload ja3 md5string
753                         True, if the payload contains the start of an SSL/TLS
754                         handshake and the calculated jas value of the hand‐
755                         shake matches md5string
756
757     payload ja3 defined
758                         True, if the payload contains the start of an SSL/TLS
759                         handshake and a valid ja3 value can be calculated.
760                         Useful to mask out all flow records with no SSL/TLS
761                         traffic in order to generate a -s ja3 statistic.
762
763     nprobe implemented elements
764
765     client latency comp time
766     server latency comp time
767                         True, if the respective latency field in the flow
768                         record compares to time. time is specified in msec.
769
770     CISCO ASA, network security event logging (NSEL) and NAT event logging
771                         (NEL) specific filters:
772     NSEL specific filters:
773
774     asa event event     True if the NSEL event type of an event record
775                         matches event which may be: ignore, create, term,
776                         delete, deny
777
778     asa event comp number
779                         True if the comparison of the NSEL event type of an
780                         event records matches number as a number.
781
782     asa event denied reason
783                         True if the event denied type of an event records
784                         matches reason which may be ingress, egress,
785                         interface, nosyn
786
787     asa xevent comp num
788                         True, if the comparison of the extended event field
789                         of the event record matches num
790
791     xip ipaddr
792     src xip ipaddr
793     dst xip ipaddr      True, if the field of the translated source or desti‐
794                         nation IP address matches ipaddr if xip is specified
795                         without src or dst both IP addresses may match.
796
797     xport ipaddr
798     src xport ipaddr
799     dst xport ipaddr    True, if the field of the translated source or desti‐
800                         nation IP address matches ipaddr if xport is speci‐
801                         fied without src or dst both ports may match.
802
803     xnet network/mask
804     src xnet network/mask
805     dst xnet network/mask
806                         True if the translated source or destination IP ad‐
807                         dress matches network if mask mask is applied.  if
808                         xnet is specified without src or dst both IP ad‐
809                         dresses may match.
810
811     ingress ACL comp number
812     ingress ACE comp number
813     ingress XACE comp number
814                         True if the comparison of the respective ingress
815                         field matches number
816
817     egress ACL comp number
818                         True if the comparison of the egress field matches
819                         number
820
821     NEL specific filters:
822     nat event event     True if the NEL event type of an event record matches
823                         event. event may be add, delete
824
825     nat event comp number
826                         True if the comparison of the NEL event type of an
827                         event records matches number as a number.
828
829     nip ipaddr
830     src nip ipaddr
831     dst nip ipaddr      True, if the field of the nat source or destination
832                         IP address matches ipaddr if nip is specified without
833                         src or dst both IP addresses may match.
834
835                         It Cm nport Ar number
836     src nport number
837     dst nport number    True, if the field of the nat source or destination
838                         port matches number if nip is specified without src
839                         or dst both ports may match.
840
841     ingress vrf number  True, if the field of the ingess vrf field of the
842                         event record matches number
843
844     pblock start comp number
845     pblock step comp number
846     pblock end comp number
847                         True if the comparison of the start, step or end of
848                         the NAT port block in the event record matches number
849     port in pblock
850     src port in pblock
851     dst port in pblock  True, if the source or destination port field matches
852                         the NAT port block range
853
854     comp                Many filter elements support the comparison with a
855                         number.  The following comparators are supported for
856                         each of those filters: =, ==, >, <, >=, <= To prevent
857                         collisions with bash interpretion, alternative com‐
858                         parators are available: EQ, LT, GT, LE, GE If comp is
859                         omitted, '==' is assumed.
860

OUTPUT FORMAT

862     This section describes how output formats are compiled.  nfdump has a lot
863     of already pre-defined output formats such as raw, json, csv etc. One
864     line formats as described for option -o can be compiled from various ele‐
865     ments of a flow record. As a flow record can contains man different ele‐
866     ments it is often useful to compile an output format for spcific needs.
867
868   Format description
869     The output format is specified by -o “fmt: stringstring contains the
870     field tags to be printed as well as other characters if needed. A tag
871     starts with a % sign followed by the field name.  tags are separated by
872     spaces from other tags. Characters or other strings, not starting with a
873     % sign are copied literally to the output.
874
875     Example:
876           -o “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
877
878     This is the definition of the predined format line. It adds the elements
879     tstart duration protocol source IP address/port followed by the literal
880     characters -> and destination ip address/port packets, bytes, flows
881     counter. Depending on the task, different output formats are required to
882     see the required fields of a flow record. You can either extend a prede‐
883     fined format or specifiy a new one at the command line.
884
885     Example: Extend the predefined format long with the the IP address of the
886     sending router
887           -o “fmt:%long %ra”
888
889     Predefined formats can be extended by simply add their name with a % sign
890     somewhere in the format string. As described under the output option -o
891
892   Format definition
893     nfdump has already many formats predefined. Most of the time, these for‐
894     mat are good enough. Sometimes you may need different formats, which can
895     be compiled as described above. In order to prevent adding the same often
896     used output format each time you run nfdump a new output format may be
897     define in the config file nfdump.conf The file nfdump.conf.dist contains
898     the definition of the already hard coded formats. These may be uncom‐
899     mented and changed according to the specific needs. New formats may be
900     added using the following sytax:
901
902           fmt.newname = “fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
903
904     with newname any new or existing definition of output formats. Existing
905     formats are overwriten with the new definition.
906
907   Tag definition
908     The following list contains all tags, which are available to compile the
909     output format:
910
911           %<format>
912                     Inserts the predefined format at this position. e.g.
913                     %line
914           %cnt      Record counter. record numbers are assigned dynamically
915                     assigned while reading readed from file.
916           %nfv      Netflow version.
917           %ts       Start Time - first seen
918           %tfs      First seen - identical to %ts
919           %tsr      Start Time, but in fractional seconds since the epoch
920                     (1970-01-01) UNIX format.
921           %te       End Time - last seen
922           %ter      End Time, in fractional seconds
923           %tr       Time the flow was received by the collector
924           %trr      Time the flow was received, in fractional seconds
925           %td       Duration of flow. Displayed in ddHHMMSS.msec
926           %pr       Transort protocol
927           %exp      Exporter ID
928           %eng      Engine Type/ID
929           %lbl      Flowlabel
930           %sa       Source Address
931           %da       Destination Address
932           %sap      Source Address:Port
933           %dap      Destination Address:Port
934           %gsap     Source Address(country code):Port
935           %gdap     Destination Address(country code):Port
936           %sp       Source Port
937           %dp       Destination Port
938           %it       ICMP-type
939           %ic       ICMP-code
940           %sn       Source Network, mask applied
941           %dn       Destination Network, mask applied
942           %nh       Next-hop IP Address
943           %nhb      BGP Next-hop IP Address
944           %ra       Router IP Address
945           %sas      Source AS
946           %das      Destination AS
947           %nas      Next AS
948           %pas      Previous AS
949           %in       Input Interface num
950           %out      Output Interface num
951           %pkt      Packets - default input
952           %ipkt     Input Packets
953           %opkt     Output Packets
954           %byt      Bytes - default input
955           %ibyt     Input Bytes
956           %obyt     Output Bytes
957           %fl       Flows
958           %flg      TCP Flags
959           %tos      Tos - default src
960           %stos     Src Tos
961           %dtos     Dst Tos
962           %dir      Direction: ingress, egress
963           %smk      Src mask
964           %dmk      Dst mask
965           %fwd      Forwarding Status
966           %svln     Src vlan label
967           %dvln     Dst vlan label
968           %ismc     Input Src Mac Addr
969           %odmc     Output Dst Mac Addr
970           %idmc     Input Dst Mac Addr
971           %osmc     Output Src Mac Addr
972           %mpls1    MPLS label 1
973           %mpls2    MPLS label 2
974           %mpls3    MPLS label 3
975           %mpls4    MPLS label 4
976           %mpls5    MPLS label 5
977           %mpls6    MPLS label 6
978           %mpls7    MPLS label 7
979           %mpls8    MPLS label 8
980           %mpls9    MPLS label 9
981           %mpls10   MPLS label 10
982           %mpls     MPLS labels 1-10
983           %bps      bps - bits per second
984           %pps      pps - packets per second
985           %bpp      bps - Bytes per package
986           %sc       src IP 2 letter country code
987           %dc       dst IP 2 letter country code
988           %sloc     src IP geo location info
989           %dloc     dst IP geo location info
990           %n        new line char \n
991           %ipl      input payload
992           %opl      ouput payload
993           %nbid     nbar ID
994           %ja3      ja3 hash
995           %sni      sni name in tls handshake
996           %nbnam    nbar name
997           %odid     observation domainID
998           %opid     observation pointID
999
1000           NSEL specific formats
1001           %nfc      NSEL connection ID
1002           %evt      NSEL event
1003           %xevt     NSEL extended event
1004           %sgt      NSEL Source security group tag
1005           %msec     NSEL event time in msec
1006           %iacl     NSEL ingress ACL
1007           %eacl     NSEL egress ACL
1008           %xsa      NSEL XLATE src IP address
1009           %xda      NSEL XLATE dst IP address
1010           %xsp      NSEL XLATE src port
1011           %xdp      NSEL SLATE dst port
1012           %xsap     Xlate Source Address:Port
1013           %xdap     Xlate Destination Address:Port
1014           %uname    NSEL user name
1015
1016           NEL/NAT specific formats
1017           %nevt     NAT event - same as %evt
1018           %ivrf     NAT ingress VRF ID
1019           %evrf     NAT egress VRF ID
1020           %nsa      NAT src IP address
1021           %nda      NAT dst IP address
1022           %nsp      NAT src port
1023           %ndp      NAT dst port
1024           %pbstart  NAT pool block start
1025           %pbend    NAT pool block end
1026           %pbstep   NAT pool block step
1027           %pbsize   NAT pool block size
1028
1029           Nprobe formats
1030           %cl       Client latency
1031           %sl       Server latency
1032           %al       Application latency
1033

EXAMPLES

1035     nfdump processes files created by any previous version of nfdump 1.6.x
1036     with some limitations for versions < 1.6.17. In order to convert flow
1037     files to the new 1.7.x binary format use the following command to
1038     read//write files:
1039
1040           % nfdump -r oldfile -w newfile
1041
1042     Print a statistic about the top 20 IP adresses, once sorted by flows and
1043     once by bytes
1044
1045           % nfdump -r flowfile -s ip/flows/bytes -n 20
1046
1047     Print two statistics, one about the source IP and one about the destina‐
1048     tion IP address limited to flow with either source or destination port
1049     443
1050
1051           % nfdump -r flowfile -s srcip/bytes -s dstip/bytes -n 20 'port 443'
1052
1053     Print a statistic about the IP pairs, which exchanged most traffic.
1054
1055           % nfdump -r flowfile -s record/bytes -A srcip,dstip
1056
1057     Print all flows in raw format with a HTTP header in the payload even if
1058     flow is not on port 80.
1059
1060           % nfdump -r flowfile -o raw “payload regex 'GET|POST'”
1061
1062     Print a statistic about all ja3 md5 sums for those flows, which a valid
1063     ja3 can be calculated
1064
1065           % nfdump -r flowfile -s ja5 -n 0 'payload ja3 defined'
1066
1067     Aggregate all flows and write the result back to a binary file, sorted by
1068     the start time
1069
1070           % nfdump -r flowfile -a -Otstart -w newfile
1071

RETURN VALUES

1073     nfdump returns 0 on success and 255 if processing failed.
1074

SEE ALSO

1076     https://www.iana.org/assignments/ipfix/ipfix.xhtml
1077
1078     https://www.cisco.com/en/US/technologies/tk648/tk362/technolo
1079     gies_white_paper09186a00800a3db9.html
1080
1081     nfcapd(1) nfpcapd(1) sfcapd(1) geolookup(1)
1082

BUGS

1084     No software without bugs! Please report any bugs back to me.
1085
1086BSD                            February 6, 2023                            BSD
Impressum