vmod_blob(3)

1VMOD(BLOB)                                                          VMOD(BLOB)
2
3
4

NAME

6       VMOD blob - Utilities for the VCL blob type, encoding and decoding
7

SYNOPSIS

9          import blob [as name] [from "path"]
10
11          BLOB decode(ENUM decoding, INT length, STRING encoded)
12
13          STRING encode(ENUM encoding, ENUM case, BLOB blob)
14
15          STRING transcode(ENUM decoding, ENUM encoding, ENUM case, INT length, STRING encoded)
16
17          BOOL same(BLOB, BLOB)
18
19          BOOL equal(BLOB, BLOB)
20
21          INT length(BLOB)
22
23          BLOB sub(BLOB, BYTES length, BYTES offset)
24
25          new xblob = blob.blob(ENUM decoding, STRING encoded)
26
27              BLOB xblob.get()
28
29              STRING xblob.encode(ENUM encoding, ENUM case)
30

DESCRIPTION

32       This  VMOD  provides  utility  functions and an object for the VCL data
33       type BLOB, which may contain arbitrary data of any length.
34
35       Examples:
36
37          sub vcl_init {
38              # Create blob objects from encodings such as base64 or hex.
39              new myblob   = blob.blob(BASE64, "Zm9vYmFy");
40              new yourblob = blob.blob(encoded="666F6F", decoding=HEX);
41          }
42
43          sub vcl_deliver {
44              # The .get() method retrieves the BLOB from an object.
45              set resp.http.MyBlob-As-Hex
46                  = blob.encode(blob=myblob.get(), encoding=HEX);
47
48              # The .encode() method efficiently retrieves an encoding.
49              set resp.http.YourBlob-As-Base64 = yourblob.encode(BASE64);
50
51              # decode() and encode() functions convert blobs to text and
52              # vice versa at runtime.
53              set resp.http.Base64-Encoded
54                  = blob.encode(BASE64,
55                                blob=blob.decode(HEX,
56                                                 encoded=req.http.Hex-Encoded));
57          }
58
59          sub vcl_recv {
60              # transcode() converts from one encoding to another.
61              # case=UPPER specifies upper-case hex digits A-F.
62              set req.http.Hex-Encoded
63                  = blob.transcode(decoding=BASE64, encoding=HEX,
64                                   case=UPPER, encoded="YmF6");
65
66              # transcode() from URL to IDENTITY effects a URL decode.
67              set req.url = blob.transcode(encoded=req.url, decoding=URL);
68
69              # transcode() from IDENTITY to URL effects a URL encode.
70              set req.http.url_urlcoded
71                  = blob.transcode(encoded=req.url, encoding=URL);
72          }
73
74   ENCODING SCHEMES
75       Binary-to-text encoding schemes are specified by ENUMs  in  the  VMOD's
76       constructor,  methods and functions. Decodings convert a (possibly con‐
77       catenated) string into a blob, while encodings convert a  blob  into  a
78       string.
79
80       ENUM values for an encoding scheme can be one of:
81
82       · IDENTITY
83
84       · BASE64
85
86       · BASE64URL
87
88       · BASE64URLNOPAD
89
90       · HEX
91
92       · URL
93
94       Empty  strings  are  decoded into a "null blob" (of length 0), and con‐
95       versely a null blob is encoded as the empty string.
96
97       For encodings with HEX or URL, you may also specify a  case  ENUM  with
98       one  of  the  values  LOWER,  UPPER or DEFAULT to produce a string with
99       lower- or uppercase hex digits (in [a-f] or [A-F]). The  default  value
100       for case is DEFAULT, which for HEX and URL means the same as LOWER.
101
102       The  case  ENUM is not relevant for decodings; HEX or URL strings to be
103       decoded as BLOBs may have hex digits in either case, or in mixed case.
104
105       The case ENUM MUST be set to DEFAULT for the other  encodings  (BASE64*
106       and IDENTITY).  You cannot, for example, produce an uppercase string by
107       using the IDENTITY scheme with case=UPPER. To  change  the  case  of  a
108       string,   use   the   std.toupper()  or  std.tolower()  functions  from
109       vmod_std(3).
110
111   IDENTITY
112       The simplest encoding converts between the BLOB and STRING data  types,
113       leaving the contents byte-identical.
114
115       Note  that  a  BLOB  may contain a null byte at any position before its
116       end; if such a BLOB is decoded with IDENTITY, the resulting STRING will
117       have  a  null byte at that position. Since VCL strings, like C strings,
118       are represented with a terminating null byte, the string will be  trun‐
119       cated, appearing to contain less data than the original blob. For exam‐
120       ple:
121
122          # Decode from the hex encoding for "foo\0bar".
123          # The header will be seen as "foo".
124          set resp.http.Trunced-Foo1
125              = blob.encode(IDENTITY, blob=blob.decode(HEX,
126                                                       encoded="666f6f00626172"));
127
128       IDENTITY is the default encoding and decoding. So the above can also be
129       written as:
130
131          # Decode from the hex encoding for "foo\0bar".
132          # The header will be seen as "foo".
133          set resp.http.Trunced-Foo2
134            = blob.encode(blob=blob.decode(HEX, encoded="666f6f00626172"));
135
136       The case ENUM MUST be set to DEFAULT for IDENTITY encodings.
137
138   BASE64*
139       The  base64  encoding schemes use 4 characters to encode 3 bytes. There
140       are no newlines or maximal line lengths -- whitespace is not permitted.
141
142       The BASE64 encoding uses the alphanumeric  characters,  +  and  /;  and
143       encoded strings are padded with the = character so that their length is
144       always a multiple of four.
145
146       The BASE64URL encoding also uses the alphanumeric characters, but - and
147       _ instead of + and /, so that an encoded string can be used safely in a
148       URL. This scheme also uses the padding character =.
149
150       The BASE64URLNOPAD encoding uses the same  alphabet  as  BASE6URL,  but
151       leaves out the padding. Thus the length of an encoding with this scheme
152       is not necessarily a multiple of four.
153
154       The case ENUM MUST be set to DEFAULT for for all of the BASE64*  encod‐
155       ings.
156
157   HEX
158       The HEX encoding scheme converts hex strings into blobs and vice versa.
159       For encodings, you may use the case ENUM to specify upper- or lowercase
160       hex  digits  A  through  f  (default  DEFAULT,  which means the same as
161       LOWER).  A prefix such as 0x is not used for an encoding and is illegal
162       for a decoding.
163
164       If  a  hex  string  to  be  decoded  has an odd number of digits, it is
165       decoded as if a 0 is prepended to it;  that  is,  the  first  digit  is
166       interpreted  as  representing the least significant nibble of the first
167       byte. For example:
168
169          # The concatenated string is "abcdef0", and is decoded as "0abcdef0".
170          set resp.http.First = "abc";
171          set resp.http.Second = "def0";
172          set resp.http.Hex-Decoded
173              = blob.encode(HEX, blob=blob.decode(HEX,
174                                 encoded=resp.http.First + resp.http.Second));
175
176   URL
177       The URL decoding  replaces  any  %<2-hex-digits>  substrings  with  the
178       binary value of the hexadecimal number after the % sign.
179
180       The URL encoding implements "percent encoding" as per RFC3986. The case
181       ENUM determines the case of the hex digits, but does not affect  alpha‐
182       betic characters that are not percent-encoded.
183
184   BLOB decode(ENUM decoding, INT length, STRING encoded)
185          BLOB decode(
186             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} decoding=IDENTITY,
187             INT length=0,
188             STRING encoded
189          )
190
191       Returns  the  BLOB  derived  from  the  string encoded according to the
192       scheme specified by decoding.
193
194       If length > 0, only decode the first length characters of  the  encoded
195       string.  If  length <= 0 or greater than the length of the string, then
196       decode the entire string. The default value of length is 0.
197
198       decoding defaults to IDENTITY.
199
200       Example:
201
202          blob.decode(BASE64, encoded="Zm9vYmFyYmF6");
203
204          # same with named parameters
205          blob.decode(encoded="Zm9vYmFyYmF6", decoding=BASE64);
206
207          # convert string to blob
208          blob.decode(encoded="foo");
209
210   STRING encode(ENUM encoding, ENUM case, BLOB blob)
211          STRING encode(
212             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} encoding=IDENTITY,
213             ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT,
214             BLOB blob
215          )
216
217       Returns a string representation of the BLOB blob as specified by encod‐
218       ing.  case determines the case of hex digits for the HEX and URL encod‐
219       ings, and is ignored for the other encodings.
220
221       encoding defaults to IDENTITY, and case defaults to  DEFAULT.   DEFAULT
222       is  interpreted  as  LOWER  for  the  HEX and URL encodings, and is the
223       required value for the other encodings.
224
225       Example:
226
227          set resp.http.encode1
228              = blob.encode(HEX,
229                            blob=blob.decode(BASE64, encoded="Zm9vYmFyYmF6"));
230
231          # same with named parameters
232          set resp.http.encode2
233              = blob.encode(blob=blob.decode(encoded="Zm9vYmFyYmF6",
234                                                     decoding=BASE64),
235                                encoding=HEX);
236
237          # convert blob to string
238          set resp.http.encode3
239              = blob.encode(blob=blob.decode(encoded="foo"));
240
241   STRING transcode(ENUM decoding,  ENUM  encoding,  ENUM  case,  INT  length,
242       STRING encoded)
243          STRING transcode(
244             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} decoding=IDENTITY,
245             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} encoding=IDENTITY,
246             ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT,
247             INT length=0,
248             STRING encoded
249          )
250
251       Translates  from  one encoding to another, by first decoding the string
252       encoded according to the scheme decoding, and then returning the encod‐
253       ing of the resulting blob according to the scheme encoding. case deter‐
254       mines the case of hex digits for the HEX  and  URL  encodings,  and  is
255       ignored for other encodings.
256
257       As  with  blob.decode():  If  length  > 0, only decode the first length
258       characters of the encoded string, otherwise decode the  entire  string.
259       The default value of length is 0.
260
261       decoding  and  encoding  default  to  IDENTITY,  and  case  defaults to
262       DEFAULT. DEFAULT is interpreted as LOWER for the HEX and URL encodings,
263       and is the required value for the other encodings.
264
265       Example:
266
267          set resp.http.Hex2Base64-1
268               = blob.transcode(HEX, BASE64, encoded="666f6f");
269
270           # same with named parameters
271           set resp.http.Hex2Base64-2
272              = blob.transcode(encoded="666f6f",
273                                    encoding=BASE64, decoding=HEX);
274
275           # URL decode -- recall that IDENTITY is the default encoding.
276           set resp.http.urldecoded
277              = blob.transcode(encoded="foo%20bar", decoding=URL);
278
279           # URL encode
280           set resp.http.urlencoded
281               = blob.transcode(encoded="foo bar", encoding=URL);
282
283   BOOL same(BLOB, BLOB)
284       Returns true if and only if the two BLOB arguments are the same object,
285       i.e. they specify exactly the same region of memory, or both are empty.
286
287       If the BLOBs are both empty (length is 0 and/or the internal pointer is
288       NULL), then blob.same() returns true. If any non-empty BLOB is compared
289       to an empty BLOB, then blob.same() returns false.
290
291   BOOL equal(BLOB, BLOB)
292       Returns true if and only if the two BLOB arguments have equal  contents
293       (possibly in different memory regions).
294
295       As  with  blob.same():  If  the BLOBs are both empty, then blob.equal()
296       returns true. If any non-empty BLOB is compared to an empty BLOB,  then
297       blob.equal() returns false.
298
299   INT length(BLOB)
300       Returns the length of the BLOB.
301
302   BLOB sub(BLOB, BYTES length, BYTES offset=0)
303       Returns a new BLOB formed from length bytes of the BLOB argument start‐
304       ing at offset bytes from the start of its memory  region.  The  default
305       value of offset is 0B.
306
307       blob.sub()  fails and returns NULL if the BLOB argument is empty, or if
308       offset + length requires more bytes than are available in the BLOB.
309
310   new xblob = blob.blob(ENUM decoding, STRING encoded)
311          new xblob = blob.blob(
312             ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} decoding=IDENTITY,
313             STRING encoded
314          )
315
316       Creates an object that  contains  the  BLOB  derived  from  the  string
317       encoded according to the scheme decoding.
318
319       Example:
320
321          new theblob1 = blob.blob(BASE64, encoded="YmxvYg==");
322
323          # same with named arguments
324          new theblob2 = blob.blob(encoded="YmxvYg==", decoding=BASE64);
325
326          # string as a blob
327          new stringblob = blob.blob(encoded="bazz");
328
329   BLOB xblob.get()
330       Returns the BLOB created by the constructor.
331
332       Example:
333
334          set resp.http.The-Blob1 =
335              blob.encode(blob=theblob1.get());
336
337          set resp.http.The-Blob2 =
338              blob.encode(blob=theblob2.get());
339
340          set resp.http.The-Stringblob =
341              blob.encode(blob=stringblob.get());
342
343   STRING xblob.encode(ENUM encoding, ENUM case)
344          STRING xblob.encode(
345                ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX, URL} encoding=IDENTITY,
346                ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT
347          )
348
349       Returns  an  encoding  of BLOB created by the constructor, according to
350       the scheme encoding. case determines the case of hex digits for the HEX
351       and URL encodings, and MUST be set to DEFAULT for the other encodings.
352
353       Example:
354
355          # blob as text
356          set resp.http.The-Blob = theblob1.encode();
357
358          # blob as base64
359          set resp.http.The-Blob-b64 = theblob1.encode(BASE64);
360
361       For  any  blob.blob()  object,  encoding  and  case,  encodings via the
362       xblob.encode() method and the blob.encode() function are equal:
363
364          # Always true:
365          blob.encode(ENC, CASE, blob.get()) == blob.encode(ENC, CASE)
366
367       But the xblob.encode() object method is more efficient -- the  encoding
368       is  computed  once and cached (with allocation in heap memory), and the
369       cached  encoding  is  retrieved   on   every   subsequent   call.   The
370       blob.encode()  function computes the encoding on every call, allocating
371       space for the string in Varnish workspaces.
372
373       So if the data in a BLOB are fixed at VCL initialization time, so  that
374       its  encodings  will  always  be  the  same,  it  is better to create a
375       blob.blob() object. The VMOD's functions should be used for  data  that
376       are not known until runtime.
377

ERRORS

379       The encoders, decoders and blob.sub() may fail if there is insufficient
380       space to create the new blob or string. Decoders may also fail  if  the
381       encoded  string  is an illegal format for the decoding scheme. Encoders
382       will fail for the IDENTITY and BASE64* encoding  schemes  if  the  case
383       ENUM is not set to DEFAULT.
384
385       If  any  of the VMOD's methods, functions or constructor fail, then VCL
386       failure is invoked, just as if return(fail) had been called in the  VCL
387       source. This means that:
388
389       · If  the  blob.blob()`_ object constructor fails, or if any methods or
390         functions fail during ``vcl_init{}, then the VCL program will fail to
391         load, and the VCC compiler will emit an error message.
392
393       · If  a  method  or  function fails in any other VCL subroutine besides
394         vcl_synth{}, then control is directed to  vcl_synth{}.  The  response
395         status  is  set  to  503  with the reason string "VCL failed", and an
396         error message will be written to the vsl(7) using the tag VCL_Error.
397
398       · If  the  failure  occurs  during  vcl_synth{},  then  vcl_synth{}  is
399         aborted.  The  response  line  "503  VCL failed" is returned, and the
400         VCL_Error message is written to the log.
401

LIMITATIONS

403       The VMOD allocates memory in various ways for new  blobs  and  strings.
404       The  blob.blob()  object and its methods allocate memory from the heap,
405       and hence they are only limited by available virtual memory.
406
407       The blob.encode(), blob.decode() and blob.transcode()  functions  allo‐
408       cate  Varnish workspace, as does blob.sub() for the newly created BLOB.
409       If these functions are failing, as indicated by "out of space" messages
410       in  the  Varnish  log  (with  the VCL_Error tag), then you will need to
411       increase    the    varnishd    parameters    workspace_client    and/or
412       workspace_backend.
413
414       The  blob.transcode()  function also allocates space on the stack for a
415       temporary BLOB. If this function causes stack overflow, you may need to
416       increase the varnishd parameter thread_pool_stack.
417

COPYRIGHT

428          This document is licensed under the same conditions as Varnish itself.
429          See LICENSE for details.
430
431          Authors: Nils Goroll <nils.goroll@uplex.de>
432                   Geoffrey Simmons <geoffrey.simmons@uplex.de>
433
434
435
436
437                                       3                            VMOD(BLOB)

NAME

SYNOPSIS

DESCRIPTION

ERRORS

LIMITATIONS

SEE ALSO

COPYRIGHT