1QSFILTER2(1)                  qsfilter2 man page                  QSFILTER2(1)
2
3
4

NAME

6       qsfilter2  - an utility to generate mod_qos request line rules out from
7       existing access/audit log data.
8

SYNOPSIS

10       qsfilter2  -i  <path>  [-c  <path>]  [-d   <num>]   [-h]   [-b   <num>]
11       [-p|-s|-m|-o]  [-l  <len>]  [-n] [-e] [-u 'uni'] [-k <prefix>] [-t] [-f
12       <path>] [-v 0|1|2]
13

DESCRIPTION

15       mod_qos implements a request filter which validates each request  line.
16       The  module  supports  both,  negative and positive security model. The
17       QS_Deny* directives are used to specify request line patterns which are
18       not allowed to access the server (negative security model / deny list).
19       These rules are used to restrict  access  to  certain  resources  which
20       should  not  be  available to users or to protect the server from mali‐
21       cious patterns. The QS_Permit*  rules  implement  a  positive  security
22       model (allow list). These directives are used to define allowed request
23       line patterns. Request which do not match any of these patterns are not
24       allowed to access the server.
25
26       qsfilter2  is an audit log analyzer used to generate filter rules (perl
27       compatible regular expressions) which may be used by  mod_qos  to  deny
28       access  for  suspect  requests (QS_PermitUri rules). It parses existing
29       audit log files in order to generate request patterns covering all  al‐
30       lowed requests.
31

OPTIONS

33       -i <path>
34              Input  file containing request URIs. The URIs for this file have
35              to be extracted from the servers access logs. Each line  of  the
36              input  file  contains a request URI consisting of a path and and
37              query.
38                   Example:
39                     /aaa/index.do
40                     /aaa/edit?image=1.jpg
41                     /aaa/image/1.jpg
42                     /aaa/view?page=1
43                     /aaa/edit?document=1
44
45              These access log data must include current request URIs but also
46              request  lines from previous rule generation steps. It must also
47              include request lines which cover manually generated rules.  You
48              may  use  the  'qos-path' and 'qos-query' variables to create an
49              audit log containing  all  request  data  (path  and  query/body
50              data). Example: 'CustomLog audit_log %{qos-path}n%{qos-query}n'.
51              See also http://mod-qos.sourceforge.net#qsfiltersample about the
52              module settings.
53
54       -c <path>
55              mod_qos   configuration  file  defining  QS_DenyRequestLine  and
56              QS_PermitUri directives. qsfilter2 generates rules  from  access
57              log  data automatically. Manually generated rules (QS_PermitUri)
58              may be provided from this file. Note: each manual rule  must  be
59              represented  by a request URI in the input data (-i) in order to
60              make sure not to be deleted by the rule optimisation  algorithm.
61              QS_Deny*  rules  from this file are used to filter request lines
62              which should not be used for allow list rule generation.
63                   Example:
64                     # manually defined allow list rule:
65                     QS_PermitUri                  +view                  deny
66              "^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$"
67                     # filter unwanted request line patterns:
68                     QS_DenyRequestLine +printable deny ".*[\x00-\x19].*"
69
70
71
72       -d <num>
73              Depth  (sub  locations) of the path string which is defined as a
74              literal string. Default is 1.
75
76       -h     Always use a string representing the handler name  in  the  path
77              even the url does not have a query. See also -d option.
78
79       -b <num>
80              Replaces  url pattern by the regular expression when detecting a
81              base64/hex encoded string. Detecting sensibility is defined by a
82              numeric  value. You should use values higher than 5 (default) or
83              0 to disable this function.
84
85       -p     Represents query by pcre only (no literal strings).
86
87       -s     Uses one single pcre for the whole query string.
88
89       -m     Uses one pcre for multiple query values (recommended mode).
90
91       -o     Does not care the order of query parameters.
92
93       -l <len>
94              Outsizes the query length by the defined length  ({0,size+len}),
95              default is 10.
96
97       -n     Disables redundant rules elimination.
98
99       -e     Exit on error.
100
101       -u 'uni'
102              Enables  additional  decoding  methods. Use the same settings as
103              you have used for the QS_Decoding directive.
104
105       -k <prefix>
106              Prefix used to generate rule identifiers (QSF by default).
107
108       -t     Calculates the maximal latency per request  (worst  case)  using
109              the generated rules.
110
111       -f <path>
112              Filters  the input by the provided path (prefix) only processing
113              matching lines.
114
115       -v <level>
116              Verbose mode. (0=silent, 1=rule source, 2=detailed). Default  is
117              1.  Don't use rules you haven't checked the request data used to
118              generate it! Level 1 is highly recommended (as long as you don't
119              have created the log data using your own web crawler).
120

OUTPUT

122       The  output  of qsfilter2 is written to stdout. The output contains the
123       generated QS_PermitUri directives but also information about the source
124       which  has  been  used to generate these rules. It is very important to
125       check the validity of each request line which has been used  to  calcu‐
126       late  the  QS_PermitUri rules. Each request line which has been used to
127       generate a new rule is shown in the output prefixed by "ADD line  <line
128       number>:". These request lines should be stored and reused at any later
129       rule generation (add them to the URI input file). The  subsequent  line
130       shows  the  generated rule. At the end of data processing a list of all
131       generated QS_PermitUri rules is shown. These  directives  may  be  used
132       withn the configuration file used by mod_qos.
133

EXAMPLE

135         qsfilter2 -i loc.txt -c httpd.conf -m -e
136         ...
137         # ADD line 1: /aaa/index.do
138         # 003 ^(/[a-zA-Z0-9\-_]+)+[/]?\.?[a-zA-Z]{0,4}$
139         # ADD line 3: /aaa/view?page=1
140         # --- ^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$
141         # ADD line 4: /aaa/edit?document=1
142         # 004 ^[/a-zA-Z]+/edit\?((document)(=[0-9]*)*[&]?)*$
143         # ADD line 5: /aaa/edit?image=1.jpg
144         # 005 ^[/a-zA-Z]+/edit\?((image)(=[0-9\.a-zA-Z]*)*[&]?)*$
145         ...
146         QS_PermitUri     +QSF001    deny    "^[/a-zA-Z]+/edit\?((document|im‐
147       age)(=[0-9\.a-zA-Z]*)*[&]?)*$"
148         QS_PermitUri +QSF002 deny "^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$"
149         QS_PermitUri +QSF003 deny "^(/[a-zA-Z0-9\-_]+)+[/]?\.?[a-zA-Z]{0,4}$"
150
151

SEE ALSO

153       qsdt(1), qsexec(1), qsgeo(1), qsgrep(1),  qshead(1),  qslog(1),  qslog‐
154       ger(1),  qspng(1),  qsre(1),  qsrespeed(1), qsrotate(1), qssign(1), qs‐
155       tail(1)
156

AUTHOR

158       Pascal Buchbinder, http://mod-qos.sourceforge.net/
159
160
161
162mod_qos utilities 11.70          November 2021                    QSFILTER2(1)
Impressum