1QSFILTER2(1) qsfilter2 man page QSFILTER2(1)
2
6 qsfilter2 - an utility to generate mod_qos request line rules out from
7 existing access/audit log data.
8
10 qsfilter2 -i <path> [-c <path>] [-d <num>] [-h] [-b <num>]
11 [-p|-s|-m|-o] [-l <len>] [-n] [-e] [-u 'uni'] [-k <prefix>] [-t] [-f
12 <path>] [-v 0|1|2]
13
15 mod_qos implements a request filter which validates each request line.
16 The module supports both, negative and positive security model. The
17 QS_Deny* directives are used to specify request line patterns which are
18 not allowed to access the server (negative security model / blacklist).
19 These rules are used to restrict access to certain resources which
20 should not be available to users or to protect the server from mali‐
21 cious patterns. The QS_Permit* rules implement a positive security
22 model (whitelist). These directives are used to define allowed request
23 line patterns. Request which do not match any of thses patterns are not
24 allowed to access the server.
25 qsfilter2 is an audit log analyzer used to generate filter rules (perl
27 compatible regular expressions) which may be used by mod_qos to deny
28 access for suspect requests (QS_PermitUri rules). It parses existing
29 audit log files in order to generate request patterns covering all
30 allowed requests.
31
33 -i <path>
34 Input file containing request URIs. The URIs for this file have
35 to be extracted from the servers access logs. Each line of the
36 input file contains a request URI consiting of a path and and
37 query.
38 Example:
39 /aaa/index.do
40 /aaa/edit?image=1.jpg
41 /aaa/image/1.jpg
42 /aaa/view?page=1
43 /aaa/edit?document=1
44 These access log data must include current request URIs but also
46 request lines from previous rule generation steps. It must also
47 include request lines which cover manually generated rules. You
48 may use the 'qos-path' and 'qos-query' variables to create an
49 audit log containing all request data (path and query/body
50 data). Example: 'CustomLog audit_log %{qos-path}n%{qos-query}n'.
51 See also http://mod-qos.sourceforge.net#qsfiltersample about the
52 module settings.
53 -c <path>
55 mod_qos configuration file defining QS_DenyRequestLine and
56 QS_PermitUri directives. qsfilter2 generates rules from access
57 log data automatically. Manually generated rules (QS_PermitUri)
58 may be provided from this file. Note: each manual rule must be
59 represented by a request URI in the input data (-i) in order to
60 make sure not to be deleted by the rule optimisation algorithm.
61 QS_Deny* rules from this file are used to filter request lines
62 which should not be used for whitelist rule generation.
63 Example:
64 # manually defined whitelist rule:
65 QS_PermitUri +view deny
66 "^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$"
67 # filter unwanted request line patterns:
68 QS_DenyRequestLine +printable deny ".*[\x00-\x19].*"
69 -d <num>
73 Depth (sub locations) of the path string which is defined as a
74 literal string. Default is 1.
75 -h Always use a string representing the handler name in the path
77 even the url does not have a query. See also -d option.
78 -b <num>
80 Replaces url pattern by the regular expression when detecting a
81 base64/hex encoded string. Detecting sensibility is defined by a
82 numeric value. You should use values higher than 5 (default) or
83 0 to disable this function.
84 -p Repesents query by pcre only (no literal strings).
86 -s Uses one single pcre for the whole query string.
88 -m Uses one pcre for multipe query values (recommended mode).
90 -o Does not care the order of query parameters.
92 -l <len>
94 Outsizes the query length by the defined length ({0,size+len}),
95 default is 10.
96 -n Disables redundant rules elimination.
98 -e Exit on error.
100 -u 'uni'
102 Enables additional decoding methods. Use the same settings as
103 you have used for the QS_Decoding directive.
104 -k <prefix>
106 Prefix used to generate rule identifiers (QSF by default).
107 -t Calculates the maximal latency per request (worst case) using
109 the generated rules.
110 -f <path>
112 Filters the input by the provided path (prefix) only processing
113 matching lines.
114 -v <level>
116 Verbose mode. (0=silent, 1=rule source, 2=detailed). Default is
117 1. Don't use rules you haven't checked the request data used to
118 generate it! Level 1 is highly recommended (as long as you don't
119 have created the log data using your own web crawler).
120
122 The output of qsfilter2 is written to stdout. The output contains the
123 generated QS_PermitUri directives but also information about the source
124 which has been used to generate these rules. It is very important to
125 check the validity of each request line which has been used to calcu‐
126 late the QS_PermitUri rules. Each request line which has been used to
127 generate a new rule is shown in the output prefixed by "ADD line <line
128 number>:". These request lines should be stored and reused at any later
129 rule generation (add them to the URI input file). The subsequent line
130 shows the generated rule. At the end of data processing a list of all
131 generated QS_PermitUri rules is shown. These directives may be used
132 withn the configuration file used by mod_qos.
133
135 qsfilter2 -i loc.txt -c httpd.conf -m -e
136 ...
137 # ADD line 1: /aaa/index.do
138 # 003 ^(/[a-zA-Z0-9\-_]+)+[/]?\.?[a-zA-Z]{0,4}$
139 # ADD line 3: /aaa/view?page=1
140 # --- ^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$
141 # ADD line 4: /aaa/edit?document=1
142 # 004 ^[/a-zA-Z]+/edit\?((document)(=[0-9]*)*[&]?)*$
143 # ADD line 5: /aaa/edit?image=1.jpg
144 # 005 ^[/a-zA-Z]+/edit\?((image)(=[0-9\.a-zA-Z]*)*[&]?)*$
145 ...
146 QS_PermitUri +QSF001 deny "^[/a-zA-Z]+/edit\?((docu‐
147 ment|image)(=[0-9\.a-zA-Z]*)*[&]?)*$"
148 QS_PermitUri +QSF002 deny "^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$"
149 QS_PermitUri +QSF003 deny "^(/[a-zA-Z0-9\-_]+)+[/]?\.?[a-zA-Z]{0,4}$"
150
153 qsdt(1), qsexec(1), qsgeo(1), qsgrep(1), qshead(1), qslog(1), qslog‐
154 ger(1), qspng(1), qsre(1), qsrespeed(1), qsrotate(1), qssign(1),
155 qstail(1)
156
158 Pascal Buchbinder, http://mod-qos.sourceforge.net/
159mod_qos utilities 11.64 September 2019 QSFILTER2(1)