1VMOD_BLOB(3)                                                      VMOD_BLOB(3)
2
3
4

NAME

6       vmod_blob - utilities for the VCL blob type
7

SYNOPSIS

9          import blob [from "path"] ;
10
11          BLOB decode(ENUM decoding, INT length, STRING encoded)
12
13          STRING encode(ENUM encoding, ENUM case, BLOB blob)
14
15          STRING transcode(ENUM decoding, ENUM encoding, ENUM case, INT length, STRING encoded)
16
17          BOOL same(BLOB, BLOB)
18
19          BOOL equal(BLOB, BLOB)
20
21          INT length(BLOB)
22
23          BLOB sub(BLOB, BYTES length, BYTES offset)
24
25          new xblob = blob.blob(ENUM decoding, STRING encoded)
26
27             BLOB xblob.get()
28
29             STRING xblob.encode(ENUM encoding, ENUM case)
30

DESCRIPTION

32       This  VMOD  provides  utility  functions and an object for the VCL data
33       type BLOB, which may contain arbitrary data of any length.
34
35       Examples:
36
37          sub vcl_init {
38              # Create blob objects from encodings such as base64 or hex.
39              new myblob   = blob.blob(BASE64, "Zm9vYmFy");
40              new yourblob = blob.blob(encoded="666F6F", decoding=HEX);
41          }
42
43          sub vcl_deliver {
44              # The .get() method retrieves the BLOB from an object.
45              set resp.http.MyBlob-As-Hex
46                  = blob.encode(blob=myblob.get(), encoding=HEX);
47
48              # The .encode() method efficiently retrieves an encoding.
49              set resp.http.YourBlob-As-Base64 = yourblob.encode(BASE64);
50
51              # decode() and encode() functions convert blobs to text and
52              # vice versa at runtime.
53              set resp.http.Base64-Encoded
54                  = blob.encode(BASE64,
55                                blob=blob.decode(HEX,
56                                                 encoded=req.http.Hex-Encoded));
57          }
58
59          sub vcl_recv {
60              # transcode() converts from one encoding to another.
61              # case=UPPER specifies upper-case hex digits A-F.
62              set req.http.Hex-Encoded
63                  = blob.transcode(decoding=BASE64, encoding=HEX,
64                                   case=UPPER, encoded="YmF6");
65
66              # transcode() from URL to IDENTITY effects a URL decode.
67              set req.url = blob.transcode(encoded=req.url, decoding=URL);
68
69              # transcode() from IDENTITY to URL effects a URL encode.
70              set req.http.url_urlcoded
71                  = blob.transcode(encoded=req.url, encoding=URL);
72          }
73

ENCODING SCHEMES

75       Binary-to-text encoding schemes are specified by ENUMs  in  the  VMOD's
76       constructor,  methods and functions. Decodings convert a (possibly con‐
77       catenated) string into a blob, while encodings convert a  blob  into  a
78       string.
79
80       ENUM values for an encoding scheme can be one of:
81
82       · IDENTITY
83
84       · BASE64
85
86       · BASE64URL
87
88       · BASE64URLNOPAD
89
90       · HEX
91
92       · URL
93
94       Empty  strings  are  decoded into a "null blob" (of length 0), and con‐
95       versely a null blob is encoded as the empty string.
96
97       For encodings with HEX or URL, you may also specify a  case  ENUM  with
98       one  of  the  values  LOWER,  UPPER or DEFAULT to produce a string with
99       lower- or uppercase hex digits (in [a-f] or [A-F]). The  default  value
100       for case is DEFAULT, which for HEX and URL means the same as LOWER.
101
102       The  case  ENUM is not relevant for decodings; HEX or URL strings to be
103       decoded as BLOBs may have hex digits in either case, or in mixed case.
104
105       The case ENUM MUST be set to DEFAULT for the other  encodings  (BASE64*
106       and IDENTITY).  You cannot, for example, produce an uppercase string by
107       using the IDENTITY scheme with case=UPPER. To  change  the  case  of  a
108       string, use the toupper or tolower functions from vmod_std(3).
109
110   IDENTITY
111       The  simplest encoding converts between the BLOB and STRING data types,
112       leaving the contents byte-identical.
113
114       Note that a BLOB may contain a null byte at  any  position  before  its
115       end; if such a BLOB is decoded with IDENTITY, the resulting STRING will
116       have a null byte at that position. Since VCL strings, like  C  strings,
117       are  represented with a terminating null byte, the string will be trun‐
118       cated, appearing to contain less data than the original blob. For exam‐
119       ple:
120
121          # Decode from the hex encoding for "foo\0bar".
122          # The header will be seen as "foo".
123          set resp.http.Trunced-Foo1
124              = blob.encode(IDENTITY, blob=blob.decode(HEX,
125                                                       encoded="666f6f00626172"));
126
127       IDENTITY is the default encoding and decoding. So the above can also be
128       written as:
129
130          # Decode from the hex encoding for "foo\0bar".
131          # The header will be seen as "foo".
132          set resp.http.Trunced-Foo2
133            = blob.encode(blob=blob.decode(HEX, encoded="666f6f00626172"));
134
135       The case ENUM MUST be set to DEFAULT for IDENTITY encodings.
136
137   BASE64*
138       The base64 encoding schemes use 4 characters to encode 3  bytes.  There
139       are no newlines or maximal line lengths -- whitespace is not permitted.
140
141       The  BASE64  encoding  uses  the  alphanumeric characters, + and /; and
142       encoded strings are padded with the = character so that their length is
143       always a multiple of four.
144
145       The BASE64URL encoding also uses the alphanumeric characters, but - and
146       _ instead of + and /, so that an encoded string can be used safely in a
147       URL. This scheme also uses the padding character =.
148
149       The  BASE64URLNOPAD  encoding  uses  the same alphabet as BASE6URL, but
150       leaves out the padding. Thus the length of an encoding with this scheme
151       is not necessarily a multiple of four.
152
153       The  case ENUM MUST be set to DEFAULT for for all of the BASE64* encod‐
154       ings.
155
156   HEX
157       The HEX encoding scheme converts hex strings into blobs and vice versa.
158       For encodings, you may use the case ENUM to specify upper- or lowercase
159       hex digits A through f  (default  DEFAULT,  which  means  the  same  as
160       LOWER).  A prefix such as 0x is not used for an encoding and is illegal
161       for a decoding.
162
163       If a hex string to be decoded has  an  odd  number  of  digits,  it  is
164       decoded  as  if  a  0  is  prepended to it; that is, the first digit is
165       interpreted as representing the least significant nibble of  the  first
166       byte. For example:
167
168          # The concatenated string is "abcdef0", and is decoded as "0abcdef0".
169          set resp.http.First = "abc";
170          set resp.http.Second = "def0";
171          set resp.http.Hex-Decoded
172              = blob.encode(HEX, blob=blob.decode(HEX,
173                                 encoded=resp.http.First + resp.http.Second));
174
175   URL
176       The  URL  decoding  replaces  any  %<2-hex-digits>  substrings with the
177       binary value of the hexadecimal number after the % sign.
178
179       The URL encoding implements "percent encoding" as per RFC3986. The case
180       ENUM  determines the case of the hex digits, but does not affect alpha‐
181       betic characters that are not percent-encoded.
182
183   BLOB decode(ENUM decoding, INT length, STRING encoded)
184          BLOB decode(
185             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} decoding=IDENTITY,
186             INT length=0,
187             STRING encoded
188          )
189
190       Returns the BLOB derived from  the  string  encoded  according  to  the
191       scheme specified by decoding.
192
193       If  length  > 0, only decode the first length characters of the encoded
194       string. If length <= 0 or greater than the length of the  string,  then
195       decode the entire string. The default value of length is 0.
196
197       decoding defaults to IDENTITY.
198
199       Example:
200
201          blob.decode(BASE64, encoded="Zm9vYmFyYmF6");
202
203          # same with named parameters
204          blob.decode(encoded="Zm9vYmFyYmF6", decoding=BASE64);
205
206          # convert string to blob
207          blob.decode(encoded="foo");
208
209   STRING encode(ENUM encoding, ENUM case, BLOB blob)
210          STRING encode(
211             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} encoding=IDENTITY,
212             ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT,
213             BLOB blob
214          )
215
216       Returns a string representation of the BLOB blob as specified by encod‐
217       ing. case determines the case of hex digits for the HEX and URL  encod‐
218       ings, and is ignored for the other encodings.
219
220       encoding  defaults  to IDENTITY, and case defaults to DEFAULT.  DEFAULT
221       is interpreted as LOWER for the HEX  and  URL  encodings,  and  is  the
222       required value for the other encodings.
223
224       Example:
225
226          set resp.http.encode1
227              = blob.encode(HEX,
228                            blob=blob.decode(BASE64, encoded="Zm9vYmFyYmF6"));
229
230          # same with named parameters
231          set resp.http.encode2
232              = blob.encode(blob=blob.decode(encoded="Zm9vYmFyYmF6",
233                                                     decoding=BASE64),
234                                encoding=HEX);
235
236          # convert blob to string
237          set resp.http.encode3
238              = blob.encode(blob=blob.decode(encoded="foo"));
239
240   transcode(...)
241          STRING transcode(
242             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} decoding=IDENTITY,
243             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} encoding=IDENTITY,
244             ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT,
245             INT length=0,
246             STRING encoded
247          )
248
249       Translates  from  one encoding to another, by first decoding the string
250       encoded according to the scheme decoding, and then returning the encod‐
251       ing of the resulting blob according to the scheme encoding. case deter‐
252       mines the case of hex digits for the HEX  and  URL  encodings,  and  is
253       ignored for other encodings.
254
255       As  with  decode(): If length > 0, only decode the first length charac‐
256       ters of the encoded string, otherwise decode  the  entire  string.  The
257       default value of length is 0.
258
259       decoding  and  encoding  default  to  IDENTITY,  and  case  defaults to
260       DEFAULT. DEFAULT is interpreted as LOWER for the HEX and URL encodings,
261       and is the required value for the other encodings.
262
263       Example:
264
265          set resp.http.Hex2Base64-1
266               = blob.transcode(HEX, BASE64, encoded="666f6f");
267
268           # same with named parameters
269           set resp.http.Hex2Base64-2
270              = blob.transcode(encoded="666f6f",
271                                    encoding=BASE64, decoding=HEX);
272
273           # URL decode -- recall that IDENTITY is the default encoding.
274           set resp.http.urldecoded
275              = blob.transcode(encoded="foo%20bar", decoding=URL);
276
277           # URL encode
278           set resp.http.urlencoded
279               = blob.transcode(encoded="foo bar", encoding=URL);
280
281   BOOL same(BLOB, BLOB)
282       Returns true if and only if the two BLOB arguments are the same object,
283       i.e. they specify exactly the same region of memory, or both are empty.
284
285       If the BLOBs are both empty (length is 0 and/or the internal pointer is
286       NULL),  then  same() returns true. If any non-empty BLOB is compared to
287       an empty BLOB, then same() returns false.
288
289   BOOL equal(BLOB, BLOB)
290       Returns true if and only if the two BLOB arguments have equal  contents
291       (possibly in different memory regions).
292
293       As with same(): If the BLOBs are both empty, then equal() returns true.
294       If any non-empty BLOB is  compared  to  an  empty  BLOB,  then  equal()
295       returns false.
296
297   INT length(BLOB)
298       Returns the length of the BLOB.
299
300   BLOB sub(BLOB, BYTES length, BYTES offset=0)
301       Returns a new BLOB formed from length bytes of the BLOB argument start‐
302       ing at offset bytes from the start of its memory  region.  The  default
303       value of offset is 0B.
304
305       sub()  fails and returns NULL if the BLOB argument is empty, or if off‐
306       set + length requires more bytes than are available in the BLOB.
307
308   new xblob = blob.blob(ENUM decoding, STRING encoded)
309          new xblob = blob.blob(
310             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} decoding=IDENTITY,
311             STRING encoded
312          )
313
314       Creates an object that  contains  the  BLOB  derived  from  the  string
315       encoded according to the scheme decoding.
316
317       Example:
318
319          new theblob1 = blob.blob(BASE64, encoded="YmxvYg==");
320
321          # same with named arguments
322          new theblob2 = blob.blob(encoded="YmxvYg==", decoding=BASE64);
323
324          # string as a blob
325          new stringblob = blob.blob(encoded="bazz");
326
327   BLOB xblob.get()
328       Returns the BLOB created by the constructor.
329
330       Example:
331
332          set resp.http.The-Blob1 =
333              blob.encode(blob=theblob1.get());
334
335          set resp.http.The-Blob2 =
336              blob.encode(blob=theblob2.get());
337
338          set resp.http.The-Stringblob =
339              blob.encode(blob=stringblob.get());
340
341   STRING xblob.encode(ENUM encoding, ENUM case)
342          STRING xblob.encode(
343                ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} encoding=IDENTITY,
344                ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT
345          )
346
347       Returns  an  encoding  of BLOB created by the constructor, according to
348       the scheme encoding. case determines the case of hex digits for the HEX
349       and URL encodings, and MUST be set to DEFAULT for the other encodings.
350
351       Example:
352
353          # blob as text
354          set resp.http.The-Blob = theblob1.encode();
355
356          # blob as base64
357          set resp.http.The-Blob-b64 = theblob1.encode(BASE64);
358
359       For  any  blob  object,  encoding  ENC and case CASE, encodings via the
360       .encode() method and the encode() function are equal:
361
362          # Always true:
363          blob.encode(ENC, CASE, blob.get()) == blob.encode(ENC, CASE)
364
365       But the object method is more efficient --  the  encoding  is  computed
366       once and cached (with allocation in heap memory), and the cached encod‐
367       ing is retrieved on every subsequent call. The encode()  function  com‐
368       putes  the  encoding  on every call, allocating space for the string in
369       Varnish workspaces.
370
371       So if the data in a BLOB are fixed at VCL initialization time, so  that
372       its  encodings  will  always be the same, it is better to create a blob
373       object. The VMOD's functions should be used for data that are not known
374       until runtime.
375

ERRORS

377       The  encoders,  decoders  and  sub()  may fail if there is insufficient
378       space to create the new blob or string. Decoders may also fail  if  the
379       encoded  string  is an illegal format for the decoding scheme. Encoders
380       will fail for the IDENTITY and BASE64* encoding  schemes  if  the  case
381       ENUM is not set to DEFAULT.
382
383       If  any  of the VMOD's methods, functions or constructor fail, then VCL
384       failure is invoked, just as if return(fail) had been called in the  VCL
385       source. This means that:
386
387       · If  the blob object constructor fails, or if any methods or functions
388         fail during vcl_init, then the VCL program will fail to load, and the
389         VCC compiler will emit an error message.
390
391       · If  a  method  or  function fails in any other VCL subroutine besides
392         vcl_synth, then control is directed to vcl_synth. The response status
393         is  set to 503 with the reason string "VCL failed", and an error mes‐
394         sage will be written to the Varnish log using the tag VCL_Error.
395
396       · If the failure occurs during vcl_synth, then  vcl_synth  is  aborted.
397         The  response  line  "503  VCL failed" is returned, and the VCL_Error
398         message is written to the log.
399

LIMITATIONS

401       The VMOD allocates memory in various ways for new  blobs  and  strings.
402       The  blob  object  and  its  methods allocate memory from the heap, and
403       hence they are only limited by available virtual memory.
404
405       The encode(),  decode()  and  transcode()  functions  allocate  Varnish
406       workspace,  as  does  sub() for the newly created BLOB.  If these func‐
407       tions are failing, as indicated by "out of space" messages in the  Var‐
408       nish  log  (with the VCL_Error tag), then you will need to increase the
409       varnishd parameters workspace_client and/or workspace_backend.
410
411       The transcode() function also allocates space on the stack for a tempo‐
412       rary  BLOB.  If  this  function  causes stack overflow, you may need to
413       increase the varnishd parameter thread_pool_stack.
414

SEE ALSO

416       · varnishd(1)
417
418       · vcl(7)
419
420       · vmod_std(3)
421
423          This document is licensed under the same conditions as Varnish itself.
424          See LICENSE for details.
425
426          Authors: Nils Goroll <nils.goroll@uplex.de>
427                   Geoffrey Simmons <geoffrey.simmons@uplex.de>
428
429
430
431
432                                                                  VMOD_BLOB(3)
Impressum