1RASQLINSERT(1)              General Commands Manual             RASQLINSERT(1)
2
3
4

NAME

6       rasqlinsert - write argus(8) data into mysql database tables.
7

SYNOPSIS

9       rasqlinsert [raoptions] [-- filter-expression]
10
11

DESCRIPTION

13       Rasqlinsert writes argus data into a mysql database.
14
15       The  principal  function  of  rasqlinsert  is to insert and update flow
16       data attributes, into a MySQL database table. Using the same syntax and
17       strategies  for  all  other ra* programs, rasqlinsert creates databases
18       and database tables, based on the print specification on the either the
19       command-line or the .rarc file.
20
21       The  concept is that where a ra* program would print fields to standard
22       out in ascii, rasqlinsert will insert those fields into the database as
23       attributes.   The  flow key, as defined by the "-m fields" option, pro‐
24       vides the definition of any keys that would be used in the  schema.   A
25       "-m  none"  option,  will  remove the use of any DBMS keys for inserted
26       data, and is the method to use when inserting  streaming,  unprocessed,
27       primitive argus data into a database table.
28
29       The  schema  is important for database utility and performance. You can
30       use MySQL querys against  the  attributes  that  you  insert  into  the
31       tables,  such  searching  and  sorting  on  IP  addresses, time, packet
32       counts, etc....  While rasqlinsert does not limit you to the number  of
33       attributes (columns) per record you provide, the RDBMS performance will
34       quide you as to how many fields are useful.
35
36       Rasqlinsert by default, includes the actual binary  argus  'record'  in
37       the  schema,  and  inserts  and  updates the binary record when needed.
38       This enables a large number of fucnctions  that  extend  beyond  simple
39       RDBMS  schema's that are useful.  Adding the 'record' is expensive, and
40       some will elect to not use this feature.  This can be controlled  using
41       the  option  '-s  -record' as a print field option in the standard ra.1
42       command line.  When the 'record' attribute is present, rasql.1 can read
43       the  records directly from the database, to provide additional process‐
44       ing on the database table contents.
45
46       When keys are used, the database will enforce that any insertions  meet
47       the  relaitional  requirements,  i.e.  that  the  keys be unique.  This
48       requirement demands a sense of caching and key tracking, which rasqlin‐
49       sert is specifically designed to provide.
50
51       Rasqlinsert  by  default,  will append data to existing tables, without
52       checking the schema for consistency.  If your schema has keys, and  you
53       attempt  to  append  new  records to an existing table, there is a high
54       likelyhood for error, as rasqlinsert will attempt to  insert  a  record
55       that  collides with an existing flow key.  Use the "-M cache" option to
56       cause rasqlinsert to reference the table contents prior to  aggregation
57       and insertion.
58
59       The binary data rasqlinsert inserts by default, is read using rasql.
60
61

RASQLINSERT SPECIFIC OPTIONS

63       Rasqlinsert, like all ra based clients, supports a number of ra options
64       including filtering of input argus records through a terminating filter
65       expression.  Rasqlinsert(1) specific options are:
66
67       -M cache
68           This causes rasqlinsert to use the database table as its persistent
69           cache store. This mechanism is used  to  control  memory  use  when
70           dealing with large amounts of data and flow keys.
71
72       -M drop
73           This  causes  rasqlinsert  to  drop any pre-existing database table
74           that has the same name as the target table name, on startup.
75
76

INVOCATION

78       This invocation writes aggregated argus(8) data from the  file  into  a
79       database  table.  The standard 5-tuple fields, 'saddr daddr proto sport
80       dport' are used as keys for each entry. rasqlinsert will aggregate  all
81       the data prior to inserting the data into the database:
82
83          rasqlinsert -r file -w mysql://user@localhost/db/table
84
85       Because  aggregation  can require a lot of memory, rasqlinsert provides
86       an option '-M cache' to have rasqlinsert use the database table as  the
87       persistent  cache  store  for  the aggregation.  With this example, the
88       standard 5-tuple fields, rasqlinsert will  aggregate  data  over  short
89       spans  of  time as it reads the data from the file, and then commit the
90       data to the database.  If additional data  arrives  that  matches  that
91       unique flow, rasqlinsert will fetch the entry from the database, aggre‐
92       gate, and then update the data entry in the database.
93
94          rasqlinsert -M cache -r file -w mysql://user@localhost/db/table
95
96       rasqlinsert can provide the  same  function  for  streaming  data  read
97       directly  from  an  argus  data  source.   This  allows  rasqlinsert to
98       reassemble all status records for an individual  flow,  such  that  the
99       resulting  table  has  only a single entry for each communciation rela‐
100       tionship seen.
101
102          rasqlinsert -M cache -S argus -w mysql://user@localhost/db/table
103
104       This invocation writes argus(8) data from the file into a database  ta‐
105       ble, without aggregation, by specifying no relational key in the data.
106
107          rasqlinsert -m none -r file -w mysql://user@localhost/db/table
108
109       This  invocation  writes  argus(8) data from the stream into a database
110       table, without modification.
111
112          rasqlinsert -m none -S argus -w mysql://user@localhost/db/table
113
114       This invocation writes argus(8) data from the stream into a daily data‐
115       base table, without modification. rasqlinsert will generate table names
116       based on time and insert its data relative to the timestamps  found  in
117       the  flow records it processes.  In this specific example, "-M time 1d"
118       specifies daily tables.
119
120          rasqlinsert -m none -S argus -w mysql://user@localhost/db/table_%Y_%m_%d -M time 1d
121
122
124       Copyright (c) 2000-2016 QoSient. All rights reserved.
125
126

SEE ALSO

128       rasql(1), ra(1), rarc(5), argus(8),
129
130

AUTHORS

132       Carter Bullard (carter@qosient.com).
133
134
135
136rasqlinsert 3.0.8               12 August 2009                  RASQLINSERT(1)
Impressum