1VMOD_QUERYSTRING(3) VMOD_QUERYSTRING(3)
2
6 vmod_querystring - Varnish Query-String Module
7
9 This module is a tool for query-string filtering in Varnish Cache. It
10 works with application/x-www-form-urlencoded strings that are commonly
11 used on the web. Such a query-string is interpreted as a key/values
12 store encoded with the following syntax:
13 key=value&other=value1&other=value2
15 The query-string parsing is very lenient and will tolerate malformed
17 strings. Assuming that a legitimate client might build a malformed
18 query-string that would be empty or include a trailing & or spurious &s
19 in the middle, it may be a good idea not to fail such requests and
20 instead get rid of the noise that might misguide Varnish and hurt your
21 hit rate. Empty parameters are ignored, and parameters are considered
22 empty when their names are empty:
23 next=empty&&previous=empty&=no-name-means-empty&no-value-is-fine
25 Once cleaned:
27 next=empty&previous=empty&no-value-is-fine
28 And since this module works with query-strings, a filter's input is
30 assumed to be a URL. The query-string is then the part of the URL after
31 a question mark when there's one. The rest of the URL is always left
32 untouched by the filters. Proper encoding of the URL isn't checked and
33 only separators (?, & and =) are considered.
34 For example:
36 /path?query-string
38 If it doesn't make any difference to your back-end application, you may
40 also sort the parameters inside the query-string. It will make the
41 hashing more deterministic for Varnish, possibly improving your hit
42 rate even more.
43 new xfilter = filter(BOOL sort, ENUM match)
45 new xfilter = filter(BOOL sort=0, ENUM {name, param} match=name)
46 A filter is first set up in vcl_init and then used during transactions.
48 The setup includes the creation of the filter to which you add crite‐
49 ria. During transactions, you may apply the filter to URLs (like
50 req.url or a Location header) or extract the filtered query-string.
51 A filter may sort its parameters, but by default it will maintain
53 order. The filter constructor takes an optional sort parameter, you may
54 use its name to improve readability.
55 Example
57 import querystring;
59 sub vcl_init {
61 new qf = querystring.filter(sort = true);
62 qf.add_string("_"); # a timestamp used to bypass caches
63 qf.add_glob("utm_*"); # google analytics parameters
64 qf.add_regex("sess[0-9]+"); # anti-CSRF token
65 }
66 sub vcl_hash {
68 hash_data(qf.apply(req.url));
69 hash_data(req.http.host);
70 return (lookup);
71 }
72 It is also possible for a filter to either match parameters by name, or
74 by themselves. By default the criteria are tested against the name
75 only, leaving alone the contents starting at the equal sign when
76 there's one. For that the constructor takes another optional match
77 parameter that can take either name (default) or param.
78 Example
80 sub vcl_init {
82 new fr = querystring.filter(match = param);
83 fr.add_regex("^lang=fr(-..)?$");
84 }
85 sub vcl_recv {
87 if (fr.extract(req.url)) {
88 set req.backend_hint = www_fr;
89 }
90 }
91 VOID xfilter.add_string(STRING)
93 Description
94 Use the string parameter as an exact-match criterion.
95 VOID xfilter.add_glob(STRING)
97 Description
98 Use the string parameter as a GLOB pattern matching criterion.
99 The underlying fnmatch function may fail, in which case the
100 query-param is kept to avoid spurious filtering and the error is
101 logged.
102 VOID xfilter.add_regex(STRING)
104 Description
105 The string parameter is compiled to a regular expression that
106 will be used as the matching criterion. If the regular expres‐
107 sion is invalid, it will fail the vcl.load command and report
108 the error in the varnish-cli output.
109 STRING xfilter.apply(STRING, ENUM {keep, drop} mode=drop)
111 Description
112 The URL parameter is filtered against the filter's criteria,
113 resulting in a new URL with a clean (and possibly sorted)
114 query-string. Parameters matching the criteria may be kept or
115 dropped, according to the optional mode argument that can be
116 either drop (default) or keep.
117 Example
119 set req.url = myfilter.apply(req.url, mode = drop);
121 STRING xfilter.extract(STRING, ENUM {keep, drop} mode=drop)
123 Description
124 This method works exactly like .apply and discards all URL parts
125 but the query-string.
126 STRING clean(STRING)
128 Description
129 This is a shorthand function that works like applying a filter
130 with no criteria. It will keep all parameters and shave off the
131 empty ones.
132 Example
134 set req.url = querystring.clean(req.url);
136 STRING sort(STRING)
138 Description
139 This is a shorthand function that works like applying a sort‐
140 ing-enabled filter with no criteria. It will keep all parameters
141 and shave off the empty ones.
142 Example
144 set req.url = querystring.sort(req.url);
146 STRING remove(STRING)
148 Description
149 This is a shorthand function that works like applying a filter
150 with a catch-all criteria. It will return the given URL with its
151 query-string removed. For better efficiency, it is not backed by
152 an actual filter.
153 Example
155 set req.url = querystring.remove(req.url);
157
159 Copyright (C) 2012-2018 Dridi Boukelmoune. License GPLv3+: GNU GPL
160 version 3 or later <http://gnu.org/licenses/gpl.html>.
161 This is free software: you are free to change and redistribute it.
163 There is NO WARRANTY, to the extent permitted by law.
164
166 vcl(7), varnishd(1), varnish-cli(7), glob(7)
167 RFC 1866 Section 8.2.1, The form-urlencoded Media Type
169 RFC 3986 Section 3, Syntax Components
170 RFC 7234 Section 2, Overview of Cache Operation
171 VMOD_QUERYSTRING(3)