1QSLOG(1) qslog man page QSLOG(1)
2
6 qslog - collects request statistics from access log data.
7
9 qslog -f <format_string> -o <out_file> [-p[c|u[c]] [-v]] [-x [<num>]]
10 [-u <name>] [-m] [-c <path>]
11
13 qslog is a real time access log analyzer. It collects the data from
14 stdin. The output is written to the specified file every minute and
15 includes the following entries:
16 - requests per second (r/s)
17 - number of requests within measured time (req)
18 - bytes sent to the client per second (b/s)
19 - bytes received from the client per second (ib/s)
20 - repsonse status codes within the last minute (1xx,2xx,3xx,4xx,5xx)
21 - average response duration (av)
22 - average response duration in milliseconds (avms)
23 - distribution of response durations in seconds within the last
24 minute (<1s,1s,2s,3s,4s,5s,>5s)
25 - distribution of response durations faster than a second within the
26 last minute (0-49ms,50-99ms,100-499ms,500-999ms)
27 - number of established (new) connections within the measured time
28 (esco)
29 - average system load (sl)
30 - free memory (m) (not available for all platforms)
31 - number of client ip addresses seen withn the last 600 seconds (ip)
32 - number of different users seen withn the last 600 seconds (usr)
33 - number of events identified by the 'E' format character
34 - number of mod_qos events within the last minute (qV=create session,
35 qv=VIP IP,qS=session pass, qD=access denied, qK=connection closed,
36 qT=dynamic keep-alive, qL=request/response slow down, qs=serialized
37 request, qA=connection abort, qU=new user tracking cookie)
38
40 -f <format_string>
41 Defines the log data format and the positions of data elements
42 processed by this utility. See to the 'LogFormat' directive of
43 the httpd.conf file to see the format defintions of the servers
44 access log data.
45 qslog knows the following elements:
46 I defines the client ip address (%h)
47 R defines the request line (%r)
48 S defines HTTP response status code (%s)
49 B defines the transferred bytes (%b or %O)
50 i defines the received bytes (%I)
51 D defines the request duration in microseconds (%D)
52 t defines the request duration in milliseconds (may be used
53 instead of D)
54 T defines the request duration in seconds (may be used
55 instead of D or t) (%T)
56 k defines the number of keepalive requests on the connec‐
57 tion (%k)
58 U defines the user tracking id (%{mod_qos_user_id}e)
59 Q defines the mod_qos_ev event message (%{mod_qos_ev}e)
60 C defines the element for the detailed log (-c option),
61 e.g. "%U"
62 s arbitrary counter to add up (sum within a minute)
63 a arbitrary counter to build an average from (average per
64 request)
65 A arbitrary counter to build an average from (average per
66 request)
67 M arbitrary counter to measure the maximum value reached
68 (peak)
69 E comma separated list of event strings
70 c content type (%{content-type}o), available in -pc mode
71 only
72 m request method (GET/POST) (%m), available in -pc mode
73 only
74 . defines an element to ignore (unknown string)
75 -o <out_file>
78 Specifies the file to store the output to. stdout is used if
79 this option is not defined.
80 -p Used for post processing when reading the log data from a file
82 (cat/pipe). qslog is started using it's offline mode (extracting
83 the time stamps from the log lines) in order to process existing
84 log files. The option "-pc" may be used alternatively if you
85 want to gather request information per client (identified by IP
86 address (I) or user tracking id (U) showing how many request
87 each client has performed within the captured period of time).
88 "-pc" supports the format characters IURSBTtDkMEcm. The option
89 "-pu" collects statistics on a per URL level (supports format
90 characters RSTtD). "-puc" is very similar to "-pu" but cuts the
91 end (handler) of each URL.
92 -v Verbose mode.
94 -x [<num>]
96 Rotates the output file once a day (move). You may specify the
97 number of rotated files to keep. Default are 14.
98 -u <name>
100 Becomes another user, e.g. www-data.
101 -m Calculates free system memory every minute.
103 -c <path>
105 Enables the collection of log statitics for different request
106 types. 'path' specifies the necessary rule file. Each rule con‐
107 sists of a rule identifier and a regular expression to identify
108 a request seprarated by a colon, e.g., 01:^(/a)|(/c). The regu‐
109 lar expressions are matched against the log data element which
110 has been identified by the 'C' format character.
111
113 The following environment variables are known to qslog:
114 QSEVENTPATH=<path>
116 Defines a file containing a comma or new line separated list of
117 known event strings expected within the log filed identified by
118 the 'E' format character.
119 QSCOUNTERPATH=<path>
121 Defines a file containing a by new line separated list of rules
122 which reflect possible QS_ClientEventLimitCount directive set‐
123 tings (for simulation purpose / -pc option). The 'E' format
124 character defines the event string in the log to match (literal
125 string) the 'event1' and 'event2' event names against.
126 Rule syntax: <name>:<event1>-<n>*<event2>/<duration>=<limit>
128 'name' defines the name you have given to the rule entry
129 and is logged along with with the number of times the 'limit'
130 has been reached within the 'duration'.
131 'event1' defines the variable name (if found in 'E') to
132 increment the counter.
133 'event2' defines the variable name (if found in 'E') to
134 decrement the counter (and the parameter 'n' defines by how
135 much).
136 'duration' defines the measure interval (in seconds) used
137 for the QS_ClientEventLimitCount directive.
138 'limit' defines the threshold (number) defined for the
139 QS_ClientEventLimitCount directive.
140 Note: If the 'name' parameter is prefixed by 'STATUS', the rule
142 is applied against the HTTP status code 'S' and the 'event1'
143 string shall contain a list of relevant status codes separated
144 by an underscore (while 'event2' is ignored).
145
147 Configuration using pipped logging:
148 CustomLog "|/bin/qslog -f ISBDQ -x -o /var/logs/stat.csv" "%h %>s %b
150 %D %{mod_qos_ev}e"
151 Post processing:
153 LogFormat "%t %h \"%r\" %>s %b \"%{User-Agent}i\" %T"
155 cat access_log | /bin/qslog -f ..IRSB.T -o /var/logs/stat.csv -p
156
159 qsexec(1), qsfilter2(1), qsgeo(1), qsgrep(1), qshead(1), qslogger(1),
160 qspng(1), qsrotate(1), qssign(1), qstail(1)
161
163 Pascal Buchbinder, http://mod-qos.sourceforge.net/
164mod_qos utilities 11.55 June 2018 QSLOG(1)