1VARNISHNCSA(1) VARNISHNCSA(1)
2
6 varnishncsa - Display Varnish logs in Apache / NCSA combined log format
7
9 varnishncsa [-a] [-b] [-c] [-C] [-d] [-D] [-F <format>] [-f <format‐
10 file>] [-g <request|vxid>] [-h] [-L <limit>] [-n <dir>] [-P <file>] [-q
11 <query>] [-r <filename>] [-t <seconds|off>] [-V] [-w <filename>]
12
14 The varnishncsa utility reads varnishd(1) shared memory logs and
15 presents them in the Apache / NCSA "combined" log format.
16 Each log line produced is based on a single Request type transaction
18 gathered from the shared memory log. The Request transaction is then
19 scanned for the relevant parts in order to output one log line. To fil‐
20 ter the log lines produced, use the query language to select the appli‐
21 cable transactions. Non-request transactions are ignored.
22 The following options are available:
24 -a When writing output to a file, append to it rather than over‐
26 write it.
27 -b Log backend requests. If -c is not specified, then only backend
29 requests will trigger log lines.
30 -c Log client requests. This is the default. If -b is specified,
32 then -c is needed to also log client requests
33 -C Do all regular expression and string matching caseless.
35 -d Process log records at the head of the log and exit.
37 -D Daemonize.
39 -F <format>
41 Set the output log format string.
42 -f <formatfile>
44 Read output format from a file. Will read a single line from the
45 specified file, and use that line as the format.
46 -g <request|vxid>
48 The grouping of the log records. The default is to group by
49 vxid.
50 -h Print program usage and exit
52 -L <limit>
54 Sets the upper limit of incomplete transactions kept before the
55 oldest transaction is force completed. A warning record is syn‐
56 thesized when this happens. This setting keeps an upper bound on
57 the memory usage of running queries. Defaults to 1000 transac‐
58 tions.
59 -n <dir>
61 Specify the varnishd working directory (also known as instance
62 name) to get logs from. If -n is not specified, the host name is
63 used.
64 -P <file>
66 Write the process' PID to the specified file.
67 -q <query>
69 Specifies the VSL query to use.
70 -r <filename>
72 Read log in binary file format from this file. The file can be
73 created with varnishlog -w filename.
74 -t <seconds|off>
76 Timeout before returning error on initial VSM connection. If set
77 the VSM connection is retried every 0.5 seconds for this many
78 seconds. If zero the connection is attempted only once and will
79 fail immediately if unsuccessful. If set to "off", the connec‐
80 tion will not fail, allowing the utility to start and wait inde‐
81 finetely for the Varnish instance to appear. Defaults to 5 sec‐
82 onds.
83 -V Print version information and exit.
85 -w <filename>
87 Redirect output to file. The file will be overwritten unless the
88 -a option was specified. If the application receives a SIGHUP in
89 daemon mode the file will be reopened allowing the old one to be
90 rotated away. This option is required when running in daemon
91 mode.
92 --optstring
94 Print the optstring parameter to getopt(3) to help writing wrap‐
95 per scripts.
96
98 The default mode of varnishncsa is "client mode". In this mode, the
99 log will be similar to what a web server would produce in the absence
100 of varnish. Client mode can be explicitly selected by using -c.
101 If the -b switch is specified, varnishncsa will operate in "backend
103 mode". In this mode, requests generated by varnish to the backends
104 will be logged. Unless -c is also specified, client requests received
105 by varnish will be ignored.
106 When running varnishncsa in both backend and client mode, it is
108 strongly advised to include the format specifier %{Varnish:side}x to
109 distinguish between backend and client requests.
110 Client requests that results in a pipe (ie. return(pipe) in vcl), will
112 not generate logging in backend mode. This is because varnish is not
113 generating requests, but blindly passes on bytes in both directions.
114 However, a varnishncsa instance running in normal mode can see this
115 case by using the formatter %{Varnish:handling}x, which will be 'pipe'.
116 In backend mode, some of the fields in the format string get different
118 meanings. Most notably, the byte counting formatters (%b, %I, %O) con‐
119 siders varnish to be the client.
120 It is possible to keep two varnishncsa instances running, one in back‐
122 end mode, and one in client mode, logging to different files.
123
125 Specify the log format to use. If no format is specified the default
126 log format is used:
127 %h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"
129 Escape sequences \n and \t are supported.
131 Supported formatters are:
133 %b In client mode, size of response in bytes, excluding HTTP head‐
135 ers. In backend mode, the number of bytes received from the
136 backend, excluding HTTP headers. In CLF format, i.e. a '-'
137 rather than a 0 when no bytes are sent.
138 %D In client mode, time taken to serve the request, in microsec‐
140 onds. In backend mode, time from the request was sent to the
141 entire body had been received.
142 %H The request protocol. Defaults to HTTP/1.0 if not known.
144 %h Remote host. Defaults to '-' if not known. In backend mode this
146 is the IP of the backend server.
147 %I In client mode, total bytes received from client. In backend
149 mode, total bytes sent to the backend.
150 %{X}i The contents of request header X. If the header appears multiple
152 times in a single transaction, the last occurrence is used.
153 %l Remote logname. Always '-'.
155 %m Request method. Defaults to '-' if not known.
157 %{X}o The contents of response header X. If the header appears multi‐
159 ple times in a single transaction, the last occurrence is used.
160 %O In client mode, total bytes sent to client. In backend mode,
162 total bytes received from the backend.
163 %q The query string. Defaults to an empty string if not present.
165 %r The first line of the request. Synthesized from other fields, so
167 it may not be the request verbatim. See the NOTES section.
168 %s Status sent to the client. In backend mode, status received
170 from the backend.
171 %t In client mode, time when the request was received, in HTTP
173 date/time format. In backend mode, time when the request was
174 sent.
175 %{X}t In client mode, time when the request was received, in the for‐
177 mat specified by X. In backend mode, time when the request was
178 sent. The time specification format is the same as for strf‐
179 time(3).
180 %T In client mode, time taken to serve the request, in seconds. In
182 backend mode, time from the request was sent to the entire body
183 had been received.
184 %U The request URL without the query string. Defaults to '-' if not
186 known.
187 %u Remote user from auth.
189 %{X}x Extended variables. Supported variables are:
191 Varnish:time_firstbyte
193 Time from when the request processing starts until the
194 first byte is sent to the client, in seconds. For back‐
195 end mode: Time from the request was sent to the backend
196 to the entire header had been received.
197 Varnish:hitmiss
199 One of the 'hit' or 'miss' strings, depending on whether
200 the request was a cache hit or miss. Pipe, pass and synth
201 are considered misses.
202 Varnish:handling
204 One of the 'hit', 'miss', 'pass', 'pipe' or 'synth'
205 strings indicating how the request was handled.
206 Varnish:side
208 Backend or client side. One of two values, 'b' or 'c',
209 depending on where the request was made. In pure backend
210 or client mode, this field will be constant.
211 Varnish:vxid
213 The VXID of the varnish transaction.
214 VCL_Log:key
216 The value set by std.log("key:value") in VCL.
217 VSL:tag:record-prefix[field]
219 The value of the VSL entry for the given tag-record pre‐
220 fix-field combination. Tag is mandatory, the other compo‐
221 nents are optional.
222 The record prefix will limit the matches to those records
224 that have this prefix as the first part of the record
225 content followed by a colon.
226 The field will, if present, treat the log record as a
228 white space separated list of fields, and only the nth
229 part of the record will be matched against. Fields start
230 counting at 1 and run up to 255.
231 Defaults to '-' when the tag is not seen, the record pre‐
233 fix does not match or the field is out of bounds. If a
234 tag appears multiple times in a single transaction, the
235 first occurrence is used.
236
238 SIGHUP Rotate the log file (see -w option).
239 SIGUSR1
241 Flush any outstanding transactions.
242
244 The %r formatter is equivalent to "%m http://%{Host}i%U%q %H". This
245 differs from apache's %r behavior, equivalent to "%m %U%q %H". Fur‐
246 thermore, when using the %r formatter, if the Host header appears mul‐
247 tiple times in a single transaction, the first occurrence is used.
248
250 Log the second field of the Begin record, corresponding to the VXID of
251 the parent transaction:
252 varnishncsa -F "%{VSL:Begin[2]}x"
254 Log the entire Timestamp record associated with the processing length:
256 varnishncsa -F "%{VSL:Timestamp:Process}x"
258
260 varnishd(1) varnishlog(1) varnishstat(1) vsl(7)
261
263 The varnishncsa utility was developed by Poul-Henning Kamp in coopera‐
264 tion with Verdens Gang AS and Varnish Software AS. This manual page was
265 initially written by Dag-Erling Smørgrav <des@des.no>, and later
266 updated by Martin Blix Grydeland and Pål Hermunn Johansen.
267
269 This document is licensed under the same licence as Varnish itself. See
270 LICENCE for details.
271 · Copyright (c) 2006 Verdens Gang AS
273 · Copyright (c) 2006-2016 Varnish Software AS
275 VARNISHNCSA(1)