1RACLUSTER.CONF(1) General Commands Manual RACLUSTER.CONF(1)
2
6 racluster.conf - racluster flow model definitions.
7
9 racluster.conf
10
12 Programs that perform flexible aggregation of argus data, such as
13 racluster(1) and radium(8), can be configured to aggregate using arbi‐
14 trary flow models. This configuration file provides a syntax for flow
15 matching and aggregation model assignments on a per flow basis, allow‐
16 ing for highly flexible aggregation strategies on a single argus
17 stream.
18 The configuration file is structured as a set of initialization vari‐
20 ables, and then followed by a collection of filter matching statements
21 and model definitions. The concept is that one identifies specific
22 Argus Flow Activity Records through specification of a Argus filter,
23 and if the filter matches, then that record will be modified using the
24 optional racluster aggregation model, and then aggregated with other
25 records that matched the filter. If the filter does not match, raclus‐
26 ter will "fall through" to the next filter.
27 The 'cont' keyword will cause racluster to continue with the current
29 argus record, to the next configuration item in the fall through con‐
30 figuration. When the 'cont' keyword is used, records will be counted
31 multiple times, so be cognizant of what your trying to accomplish.
32
35 The aggregation clients have a small number of options for controlling
36 specific aspects of aggregation function and output.
37
40 Racluster configurations can be named. This is important for ra*
41 aggregation programs that support multiple concurrent models at a time,
42 so you can tell them apart. This is completely optional.
43
46 Racluster, when it merges argus records together, adds a new aggrega‐
47 tion metric to the resulting record, which reports on the number of
48 records that were merged together and provides some additional statis‐
49 tical values that provide record arrival rates and mean record dura‐
50 tions. By setting this option to "no", you can have racluster() not
51 provide this metric. This is very important if you have multiple
52 aggregatation processes 'piped' in order to generate complex aggrega‐
53 tion reports. It is also useful when creating full-duplex records from
54 half-duplex merging operations.
55 RACLUSTER_REPORT_AGGREGATION=yes
57
61 All aggregation clients detect when a flow descriptor object would need
62 to be modified to conform to an aggregation result. If you merge two
63 records that have different source addresses, the merge process will
64 modify the source addresses. For all fields there are specific algo‐
65 rithms used to perserving as much data as possible so that the result
66 is 'PRESERVE'ing. For IP addresses, the algorithm preserves the long‐
67 est prefix match. This is valuable information when attempting to dis‐
68 cover trends.
69 However, some applications may want the resulting output to completely
71 conform to the new flow definitions. In other words, if you are aggre‐
72 gatating and the source address is not in the key, the algorithm will
73 zero out the source address prior to record merging.
74 To force racluster() like programs to convert flow descriptions to the
76 new flow model descriptors, set this option to "no".
77 RACLUSTER_PRESERVE_FIELDS=yes
79
83 When aggregating Argus records together, all aggregation clients have
84 the ability to autocorrect the assignment of flow initiator and
85 receiver. This is important for processing Argus records derived from
86 Cisco Netflow style flow monitors, and for Argus records that were gen‐
87 erated by long lived flows that have extended idle periods. Because it
88 is possible for ra* aggregation clients to receive half-duplex flow
89 records, or multiple flow records for the same long live flow, autocor‐
90 recting the argus records allows the client aggregation process to
91 match A -> B and B -> A records that belong to the same flow.
92 However, with certain flow aggregation models, the autocorrection logic
94 can cause aggregation mis-match, merging dissimilar flow together.
95 This is especially the case for complex aggregations where fields are
96 ignored only on specific record types. As a result, when providing
97 custom aggregation models, autocorrection is disabled by default.
98 If you would like to re-enable the autocorrection function, set this
100 variable to "yes";
101 RACLUSTER_AUTO_CORRECTION=no
103
107 When aggregating Argus records together, one metric in the records is
108 accumulated and processed to generate 'mean', 'max', 'min', metrics can
109 be provided as the basis for the aggregation statistics.
110 Records with different metrics cannot be aggregated together, as the
112 resulting statistics would be meaningless. If you are in doubt, remem‐
113 ber to use the '-M dsrs="-agr"' option to ensure that you generate your
114 desired result.
115 RACLUSTER_AGG_METRIC=dur
118
122 An aggregation configuration is composed of one or more aggregration
123 statements. An aggregation statement contains, on a single line, a
124 mandatory filter specification, and optionally a model specification, a
125 status timer and/or an idle timer value, a label and the keyword
126 'cont'. Specifications are formated as:
127 keyword'='["]value["]
129 Valid keywords are:
131 filter, model, label, status and idle.
132 Filter specifications are standard ra* client filters. See ra.1 for a
134 complete description.
135 Model specifications are identical to the -m option support provided by
137 racluster() on the command line.
138 Labels are free form meta-data strings, that will be added to records
140 that matched the filter statement at this point in the fall through
141 configureation.
142 Status and Idle timer values are specified in seconds and are usually
144 not quoted, although they can be.
145 Argus record flow descriptors are compared to the filter that matches
147 statements in sequential, or "fall through", order, much like existing
148 Access Control List definitions supported by routers, switches and
149 firewalls. These filters are conventional ra* filter statements, with
150 the maxiumum length being 1024 chars.
151 The model statement is used to modify the flow description of matching
153 Argus records. Records are aggregated based on the modified flow
154 descriptor that results from applying the flow model, and are only
155 merged with other records that also matched this filter, which should
156 eliminate problems with a independent
157 In each flow descriptor matching statement is a TimeOut period, which
159 is how long the aggregator will hold the flow cache before reporting
160 it, and an IdleTimeOut period, which is how long the aggregation
161 process will hold the flow in its cache, if there is no activity.
162 If a record doesn't match any statement in the configuration, then it
164 is aggregated based on its unmodified flow descriptor. This aggregates
165 flow reports from the same long lived flow. Generally, most configura‐
166 tions have a "catch all" filter to aggregate flows that don't match any
167 filter expression.
168
172 This configuration is not intended to do anything useful, it is
173 designed for demonstration purposes, only. With that said, lets get to
174 it.
175 filter="icmp"
177 filter="arp" model="proto daddr"
178 filter="tcp or udp and dst port lt 1025" model="saddr daddr proto dport" status=120 label="reserved"
179 filter="tcp or udp" model="saddr daddr proto sport dport" status=120 idle=3600
180 grep="Video" model="saddr daddr proto dport" status=120 idle=3600
181 filter="" model="saddr daddr proto" status=0 idle=3600 label="other flows"
182 All records are first tested as 'icmp' flows, then 'arp' and then 'tcp
185 or udp' flows. If the records do not match these filters, they will
186 match the 4th aggregation statement, as it has a null filter, which
187 matches everything.
188 If a record matches the 1st statement, it is aggregated using the
190 default aggregation model, which is the conventional 5 tuple flow
191 model. For icmp, this includes the src and destination addresses, the
192 proto field and the icmp type and code values. Because there is no
193 status or idle timer values specified, the aggregator will merge
194 records until EOF is encountered on the input stream, and output the
195 single record and its stats at the end of the run.
196 If a record matches the 2nd statement, i.e. it is an arp flow, then the
198 record flow descriptor is modified so that the protocol (arp) and the
199 requestors source address are the only fields preserved. This would
200 merge all the arp records requesting a specific IP addresss, and is
201 useful for reducing arp traffic in the output stream or file. The idea
202 is that you are interested in the response, not who made the request.
203 If a record matches the 3rd statement, i.e. it is a "tcp or udp" flow
205 with the destination port less than 1025, the flow model ignores the
206 soure port value, and merges records that match. The aggregator will
207 generate a 120 second status report on the resulting flow, so that when
208 there is activity, the aggregator will generate output that doesn't
209 exceed 120 seconds in duration. If the flow is idle for 3600 seconds,
210 the aggregator will
211 This is a very popular service oriented aggregation strategy, as it
213 preserves the client address and the server address, along with the
214 service port number, when its less than 1025, which are the Well Known
215 Ports, or Reserved port number space.
216 All flows that match the 3rd statement are labeled, with the free form
218 meta-data string, "reserved" as these flows use the reserved port
219 range. If there is already a lobel in the flow record, the string is
220 appeneded to the label, separated by a ':', unless the string is
221 already in the label.
222 If the flow is "udp or tcp" and the dst port is not less than 1025,
224 then the flow will match the 4th statement, and the aggregator will use
225 the default flow model, generating status records and timeing out.
226 The 5th statement, the "catch all" statement, specifies that the aggre‐
228 gator should preserve only the source and destination addresses, and
229 the protcol number. No status records will be generated, but if it
230 times out it will flush the record out. All flows are labeled as
231 "other".
232 This configuration strategy should provide support for any type of
234 aggregation methodology you can dream up, well at least most that you
235 will want to use.
236
239 Copyright (c) 2000-2016 QoSient. All rights reserved.
240
243 racluster(1)
244racluster.conf 3.0.8 07 November 2004 RACLUSTER.CONF(1)