vmod_querystring(3)

1VMOD(QUERYSTRING)                                            VMOD(QUERYSTRING)
2
3
4

NAME

6       VMOD querystring - Varnish Query-String Module
7

DESCRIPTION

9       This  module  is a tool for query-string filtering in Varnish Cache. It
10       works with application/x-www-form-urlencoded strings that are  commonly
11       used  on  the  web.  Such a query-string is interpreted as a key/values
12       store encoded with the following syntax:
13
14          key=value&other=value1&other=value2
15
16       The query-string parsing is very lenient and  will  tolerate  malformed
17       strings.   Assuming  that  a  legitimate client might build a malformed
18       query-string that would be empty or include a trailing & or spurious &s
19       in  the  middle,  it  may  be a good idea not to fail such requests and
20       instead get rid of the noise that might misguide Varnish and hurt  your
21       hit  rate.  Empty parameters are ignored, and parameters are considered
22       empty when their names are empty:
23
24          next=empty&&previous=empty&=no-name-means-empty&no-value-is-fine
25
26          Once cleaned:
27          next=empty&previous=empty&no-value-is-fine
28
29       And since this module works with query-strings,  a  filter's  input  is
30       assumed to be a URL. The query-string is then the part of the URL after
31       a question mark when there's one. The rest of the URL  is  always  left
32       untouched by the filters.  Proper encoding of the URL isn't checked and
33       only separators (?, & and =) are considered.
34
35       For example:
36
37          /path?query-string
38
39       If it doesn't make any difference to your back-end application, you may
40       also sort the parameters inside the query-string and remove duplicates.
41       It will make the  hashing  more  deterministic  for  Varnish,  possibly
42       improving your hit rate even more.
43
44   new xfilter = querystring.filter(BOOL sort, BOOL uniq, ENUM match)
45          new xfilter = querystring.filter(
46             BOOL sort=0,
47             BOOL uniq=0,
48             ENUM {name, param} match=name
49          )
50
51       A filter is first set up in vcl_init and then used during transactions.
52       The setup includes the creation of the filter to which you  add  crite‐
53       ria.  During  transactions,  you  may  apply  the  filter to URLs (like
54       req.url or a Location header) or extract the filtered query-string.
55
56       A filter may sort its parameters,  but  by  default  it  will  maintain
57       order.  The filter constructor takes an optional sort argument, you may
58       use its name to improve readability.
59
60       Example
61
62                 import querystring;
63
64                 sub vcl_init {
65                     new qf = querystring.filter(sort = true);
66                     qf.add_string("_"); # a timestamp used to bypass caches
67                     qf.add_glob("utm_*"); # google analytics parameters
68                     qf.add_regex("sess[0-9]+"); # anti-CSRF token
69                 }
70
71                 sub vcl_hash {
72                     hash_data(qf.apply()); # implicitly on req.url
73                     hash_data(req.http.host);
74                     return (lookup);
75                 }
76
77       It is also possible for a filter to either match parameters by name, or
78       by  themselves.  By  default  the  criteria are tested against the name
79       only, leaving alone the  contents  starting  at  the  equal  sign  when
80       there's  one.  For  that  the  constructor takes another optional match
81       argument that can take either name (default) or param.
82
83       Example
84
85                 sub vcl_init {
86                     new fr = querystring.filter(match = param);
87                     fr.add_regex("^lang=fr(-..)?$");
88                 }
89
90                 sub vcl_recv {
91                     if (fr.extract()) {
92                         set req.backend_hint = www_fr;
93                     }
94                 }
95
96       Finally, a filter can drop  consecutive  duplicate  parameters  if  the
97       optional  uniq argument. Combined with a sort, this effectively removes
98       all the duplicates from the URL.  The  match  argument  determines  how
99       parameters are considered duplicates.
100
101       Since  most  of the time it is either req.url or bereq.url that is fil‐
102       tered, omitting the url argument  for  functions  that  take  one  will
103       default  to  one  of  them depending on whether the function was called
104       during a client or backend transaction.
105
106   VOID xfilter.add_string(STRING)
107       Description
108              Use the string argument as an exact-match criterion.
109
110   VOID xfilter.add_glob(STRING)
111       Description
112              Use the string argument as a GLOB  pattern  matching  criterion.
113              The  underlying  fnmatch  function  may  fail, in which case the
114              query-param is kept to avoid spurious filtering and the error is
115              logged.
116
117   VOID xfilter.add_regex(STRING)
118       Description
119              The  string  argument  is  compiled to a regular expression that
120              will be used as the matching criterion. If the  regular  expres‐
121              sion  is  invalid,  it will fail the vcl.load command and report
122              the error in the varnish-cli output.
123
124   STRING xfilter.apply([STRING url], ENUM mode)
125          STRING xfilter.apply([STRING url], ENUM {keep, drop} mode=drop)
126
127       Description
128              The url argument is filtered against the filter's criteria, pre‐
129              viously  added using the add_*() methods in vcl_init. The result
130              is a new URL with a clean (possibly  sorted  and  de-duplicated)
131              query-string.
132
133              Depending  on  the  optional mode argument the matching criteria
134              act as a black list for drop (default) or a white list for keep.
135              In  the  resulting URL the query-string will either contain only
136              the matching parameters or everything but them.
137
138       Example
139
140                 set req.url = myfilter.apply(mode = drop);
141
142   STRING xfilter.extract([STRING url], ENUM mode)
143          STRING xfilter.extract(
144                [STRING url],
145                ENUM {keep, drop} mode=drop
146          )
147
148       Description
149              This method works exactly like .apply and discards all URL parts
150              but the query-string.
151
152   STRING clean([STRING url])
153       Description
154              This  is  a shorthand function that works like applying a filter
155              with no criteria. It will keep all parameters and shave off  the
156              empty ones.
157
158       Example
159
160                 set req.url = querystring.clean();
161
162   STRING sort([STRING url], BOOL uniq=0)
163       Description
164              This  is  a  shorthand function that works like applying a sort‐
165              ing-enabled filter with no criteria  matching  full  parameters,
166              not  just their names. It will keep all parameters and shave off
167              the empty ones. If the uniq argument is true  duplicate  parame‐
168              ters are also removed.
169
170       Example
171
172                 set req.url = querystring.sort();
173
174   STRING remove([STRING url])
175       Description
176              This  is  a shorthand function that works like applying a filter
177              with a catch-all criteria. It will return the given URL with its
178              query-string removed. For better efficiency, it is not backed by
179              an actual filter.
180
181       Example
182
183                 set req.url = querystring.remove();
184

COPYRIGHT

186       Copyright (C) 2012-2018  Dridi Boukelmoune.  License  GPLv3+:  GNU  GPL
187       version 3 or later <http://gnu.org/licenses/gpl.html>.
188
189       This  is  free  software:  you  are free to change and redistribute it.
190       There is NO WARRANTY, to the extent permitted by law.
191

NAME

DESCRIPTION

COPYRIGHT

SEE ALSO