1VTC(7)                                                                  VTC(7)
2
3
4

NAME

6       VTC - Varnish Test Case Syntax
7

OVERVIEW

9       This  document  describes  the  syntax used by Varnish Test Cases files
10       (.vtc).  A  vtc  file  describe  a  scenario  with  different  scripted
11       HTTP-talking  entities,  and generally one or more Varnish instances to
12       test.
13

PARSING

15       A vtc file will be read word after word, with very little tokenization,
16       meaning  a syntax error won't be detected until the test actually reach
17       the relevant action in the test.
18
19       A parsing error will most of the time result in an assert  being  trig‐
20       gered.  If  this  happens,  please refer yourself to the related source
21       file and line number. However, this guide should  help  you  avoid  the
22       most common mistakes.
23
24   Words and strings
25       The parser splits words by detecting whitespace characters and a string
26       is a word, or a series of words on  the  same  line  enclosed  by  dou‐
27       ble-quotes  ("..."),  or,  for  multi-line  strings,  enclosed in curly
28       brackets ({...}).
29
30   Comments
31       The leading whitespaces of lines are ignored. Empty lines (or ones con‐
32       sisting only of whitespaces) are ignored too, as are the lines starting
33       with "#" that are comments.
34
35   Lines and commands
36       Test files take at most one command per line, with the  first  word  of
37       the  line being the command and the following ones being its arguments.
38       To continue over to a new line without breaking  the  argument  string,
39       you can escape the newline character (\n) with a backslash (\).
40

MACROS

42       When a string is processed, macro expansion is performed. Macros are in
43       the form ${<name>[,<args>...]}, they have a name  followed  by  an  op‐
44       tional  comma- or space-separated list of arguments. Leading and trail‐
45       ing spaces are ignored.
46
47       The macros ${foo,bar,baz} and ${ foo bar baz } are  equivalent.  If  an
48       argument  contains a space or a comma, arguments can be quoted. For ex‐
49       ample the macro ${foo,"bar,baz"} gives  one  argument  bar,baz  to  the
50       macro called foo.
51
52       Unless  documented  otherwise,  all macros are simple macros that don't
53       take arguments.
54
55   Built-in macros
56       ${bad_backend}
57              A socket address that will reliably never accept connections.
58
59       ${bad_ip}
60              An unlikely IPv4 address.
61
62       ${date}
63              The current date and time formatted for HTTP.
64
65       ${listen_addr}
66              The default listen address various components use, by default  a
67              random port on localhost.
68
69       ${localhost}
70              The first IP address that resolves to "localhost".
71
72       ${pwd} The working directory from which varnishtest was executed.
73
74       ${string,<action>[,<args>...]}
75              The  string  macro  is  the  entry point for text generation, it
76              takes a specialized action with each its own set of arguments.
77
78       ${string,repeat,<uint>,<str>}
79              Repeat uint times the string str.
80
81       ${testdir}
82              The directory containing the VTC script of the ongoing test case
83              execution.
84
85       ${tmpdir}
86              The dedicated working directory for the ongoing test case execu‐
87              tion, which happens to also be the  current  working  directory.
88              Useful when an absolute path to the working directory is needed.
89
90       ${topbuild}
91              Only  present when the -i option is used, to work on Varnish it‐
92              self instead of a regular installation.
93

SYNTAX

95   barrier
96       NOTE: This command is available everywhere commands are given.
97
98       Barriers allows you to  synchronize  different  threads  to  make  sure
99       events occur in the right order. It's even possible to use them in VCL.
100
101       First, it's necessary to declare the barrier:
102
103          barrier bNAME TYPE NUMBER [-cyclic]
104
105       With the arguments being:
106
107       bNAME  this is the name of the barrier, used to identify it when you'll
108              create sync points. It must start with 'b'.
109
110       TYPE   it can be "cond" (mutex) or "sock" (socket)  and  sets  internal
111              behavior. If you don't need VCL synchronization, use cond.
112
113       NUMBER number of sync point needed to go through the barrier.
114
115       -cyclic
116              if  present,  the barrier will reset itself and be ready for an‐
117              other round once gotten through.
118
119       Then, to add a sync point:
120
121          barrier bNAME sync
122
123       This will block the parent thread until the number of sync  points  for
124       bNAME reaches the NUMBER given in the barrier declaration.
125
126       If  you  wish to synchronize the VCL, you need to declare a "sock" bar‐
127       rier.  This will emit a macro definition named  "bNAME_sock"  that  you
128       can use in VCL (after importing the vtc vmod):
129
130          vtc.barrier_sync("${bNAME_sock}");
131
132       This  function  returns 0 if everything went well and is the equivalent
133       of barrier bNAME sync at the VTC top-level.
134
135   client/server
136       Client and server threads are fake HTTP entities used to test your Var‐
137       nish  and  VCL. They take any number of arguments, and the one that are
138       not recognized, assuming they don't start  with  '-',  are  treated  as
139       specifications, laying out the actions to undertake:
140
141          client cNAME [...]
142          server sNAME [...]
143
144       Clients  and  server  are identified by a string that's the first argu‐
145       ment, clients' names start with 'c' and servers' names start with 's'.
146
147       As the client and server commands share a good deal  of  arguments  and
148       specification  actions,  they  are grouped in this single section, spe‐
149       cific items will be explicitly marked as such.
150
151       SECTION: client-server.macros Macros and automatic behaviour
152
153       To make things easier in the general case, clients will connect by  de‐
154       fault  to a Varnish server called v1. To connect to a different Varnish
155       server, use '-connect ${vNAME_sock}'.
156
157       The -vcl+backend switch of the varnish command will  add  all  the  de‐
158       clared  servers as backends. Be careful though, servers will by default
159       listen to the 127.0.0.1 IP and will pick a random port, and  publish  3
160       macros:  sNAME_addr,  sNAME_port and sNAME_sock, but only once they are
161       started. For 'varnish -vcl+backend' to create the vcl with the  correct
162       values, the server must be started first.
163
164       SECTION: client-server.args Arguments
165
166       -start Start the thread in background, processing the last given speci‐
167              fication.
168
169       -wait  Block until the thread finishes.
170
171       -run (client only)
172              Equivalent to "-start -wait".
173
174       -repeat NUMBER
175              Instead of processing the specification only once, do it  NUMBER
176              times.
177
178       -keepalive
179              For repeat, do not open new connections but rather run all iter‐
180              ations in the same connection
181
182       -break (server only)
183              Stop the server.
184
185       -listen STRING (server only)
186              Dictate the listening socket for the server. STRING  is  of  the
187              form  "IP  PORT", or "/PATH/TO/SOCKET" for a Unix domain socket.
188              In the latter case, the path must begin with '/', and the server
189              must be able to create it.
190
191       -connect STRING (client only)
192              Indicate  the  server  to connect to. STRING is also of the form
193              "IP PORT", or "/PATH/TO/SOCKET". As  with  "server  -listen",  a
194              Unix domain socket is recognized when STRING begins with a '/'.
195
196       -dispatch (server only, s0 only)
197              Normally,  to keep things simple, server threads only handle one
198              connection at a time, but the -dispatch switch allows to  accept
199              any  number  of  connection  and handle them following the given
200              spec.
201
202              However, -dispatch is only allowed for the server name "s0".
203
204       -proxy1 STRING (client only)
205              Use the PROXY protocol version 1 for this connection. STRING  is
206              of the form "CLIENTIP:PORT SERVERIP:PORT".
207
208       -proxy2 STRING (client only)
209              Use  the PROXY protocol version 2 for this connection. STRING is
210              of the form "CLIENTIP:PORT SERVERIP:PORT".
211
212       SECTION: client-server.spec Specification
213
214       It's a string, either double-quoted "like this", but most of  the  time
215       enclosed  in  curly brackets, allowing multilining. Write a command per
216       line in it, empty line are ignored, and long line can be wrapped by us‐
217       ing a backslash. For example:
218
219          client c1 {
220              txreq -url /foo \
221                    -hdr "bar: baz"
222
223              rxresp
224          } -run
225
226       accept (server only)
227              Close the current connection, if any, and accept a new one. Note
228              that this new connection is HTTP/1.x.
229
230       chunked STRING
231              Send STRING as chunked encoding.
232
233       chunkedlen NUMBER
234              Do as chunked except that the string will be generated for  you,
235              with a length of NUMBER characters.
236
237       close (server only)
238              Close  the  connection. Note that if operating in HTTP/2 mode no
239              extra (GOAWAY) frame is sent, it's simply a TCP close.
240
241       expect STRING1 OP STRING2
242              Test if "STRING1 OP STRING2" is true,  and  if  not,  fails  the
243              test.   OP can be ==, <, <=, >, >= when STRING1 and STRING2 rep‐
244              resent numbers in which case it's an order operator. If  STRING1
245              and  STRING2 are meant as strings OP is a matching operator, ei‐
246              ther == (exact match) or ~ (regex match).
247
248              varnishtest will first try to resolve  STRING1  and  STRING2  by
249              looking  if  they  have special meanings, in which case, the re‐
250              solved value is use for the test. Note that this value can be  a
251              string representing a number, allowing for tests such as:
252
253                 expect req.http.x-num > 2
254
255              Here's the list of recognized strings, most should be obvious as
256              they either match VCL logic, or the txreq/txresp options:
257
258              • remote.ip
259
260              • remote.port
261
262              • remote.path
263
264              • req.method
265
266              • req.url
267
268              • req.proto
269
270              • resp.proto
271
272              • resp.status
273
274              • resp.reason
275
276              • resp.chunklen
277
278              • req.bodylen
279
280              • req.body
281
282              • resp.bodylen
283
284              • resp.body
285
286              • req.http.NAME
287
288              • resp.http.NAME
289
290       expect_close
291              Reads from the connection, expecting nothing to read but an EOF.
292
293       fatal|non_fatal
294              Control whether a failure of this entity should stop the test.
295
296       gunzip Gunzip the body in place.
297
298       recv NUMBER
299              Read NUMBER bytes from the connection.
300
301       rxchunk
302              Receive an HTTP chunk.
303
304       rxpri (server only)
305              Receive a preface. If valid set the server to HTTP/2, abort oth‐
306              erwise.
307
308       rxreq (server only)
309              Receive and parse a request's headers and body.
310
311       rxreqbody (server only)
312              Receive a request's body.
313
314       rxreqhdrs (server only)
315              Receive and parse a request's headers (but not the body).
316
317       rxresp [-no_obj] (client only)
318              Receive  and  parse a response's headers and body. If -no_obj is
319              present, only get the headers.
320
321       rxrespbody (client only)
322              Receive (part of) a response's body.
323
324       -max : max length of this receive, 0 for all
325
326       rxresphdrs (client only)
327              Receive and parse a response's headers.
328
329       send STRING
330              Push STRING on the connection.
331
332       send_n NUMBER STRING
333              Write STRING on the socket NUMBER times.
334
335       send_urgent STRING
336              Send string as TCP OOB urgent data. You will never need this.
337
338       sendhex STRING
339              Send bytes as described by STRING. STRING should consist of  hex
340              pairs possibly separated by whitespace or newlines. For example:
341              "0F EE a5    3df2".
342
343       settings -dectbl INT
344              Force internal HTTP/2 settings to certain values. Currently only
345              support setting the decoding table size.
346
347       shell  Same as for the top-level shell.
348
349       stream HTTP/2  introduces  the  concept of streams, and these come with
350              their own specification, and as it's quite big, have been  moved
351              to their own chapter.
352
353       SECTION: stream stream
354
355       (note: this section is at the top-level for easier navigation, but it's
356       part of the client/server specification)
357
358       Streams map roughly to a request in HTTP/2, a request is sent on stream
359       N,  the  response too, then the stream is discarded. The main exception
360       is the first stream, 0, that serves as coordinator.
361
362       Stream syntax follow the client/server one:
363
364          stream ID [SPEC] [ACTION]
365
366       ID is the HTTP/2 stream number, while SPEC describes what will be  done
367       in that stream.
368
369       Note  that, when parsing a stream action, if the entity isn't operating
370       in HTTP/2 mode, these spec is ran before:
371
372          txpri/rxpri # client/server
373          stream 0 {
374              txsettings
375              rxsettings
376              txsettings -ack
377              rxsettings
378              expect settings.ack == true
379          } -run
380
381       And HTTP/2 mode is then activated before parsing the specification.
382
383       SECTION: stream.actions Actions
384
385       -start Run the specification in a thread, giving back  control  immedi‐
386              ately.
387
388       -wait  Wait for the started thread to finish running the spec.
389
390       -run   equivalent to calling -start then -wait.
391
392       timeout NUMBER
393              Set the TCP timeout for this entity.
394
395       txpri (client only)
396              Send  an HTTP/2 preface ("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") and
397              set client to HTTP/2.
398
399       txreq|txresp [...]
400              Send a minimal request or response, but overload  it  if  neces‐
401              sary.
402
403              txreq is client-specific and txresp is server-specific.
404
405              The only thing different between a request and a response, apart
406              from who can send them is that the first line (request  line  vs
407              status line), so all the options are prety much the same.
408
409              -method STRING (txreq only)
410                     What method to use (default: "GET").
411
412              -req STRING (txreq only)
413                     Alias for -method.
414
415              -url STRING (txreq only)
416                     What location to use (default "/").
417
418              -proto STRING
419                     What   protocol   use  in  the  status  line.   (default:
420                     "HTTP/1.1").
421
422              -status NUMBER (txresp only)
423                     What status code to return (default 200).
424
425              -reason STRING (txresp only)
426                     What message to put in the status line (default: "OK").
427
428              These three switches can appear in any order but must  come  be‐
429              fore the following ones.
430
431              -nohost
432                     Don't include a Host header in the request.
433
434              -nolen Don't include a Content-Length header.
435
436              -hdr STRING
437                     Add  STRING  as  a  header,  it  must follow this format:
438                     "name: value". It can be called multiple times.
439
440              -hdrlen STRING NUMBER
441                     Add STRING as a header with NUMBER bytes of content.
442
443              You can then use the arguments related to the body:
444
445              -body STRING
446                     Input STRING as body.
447
448              -bodyfrom FILE
449                     Same as -body but content is read from FILE.
450
451              -bodylen NUMBER
452                     Generate and input a body that is NUMBER bytes-long.
453
454              -gziplevel NUMBER
455                     Set the gzip level (call it before any of the other  gzip
456                     switches).
457
458              -gzipresidual NUMBER
459                     Add extra gzip bits. You should never need it.
460
461              -gzipbody STRING
462                     Gzip STRING and send it as body.
463
464              -gziplen NUMBER
465                     Combine  -bodylen  and  -gzipbody:  generate  a string of
466                     length NUMBER, gzip it and send as body.
467
468       write_body STRING
469              Write the body of a request or a response to a  file.  By  using
470              the  shell  command, higher-level checks on the body can be per‐
471              formed (eg. XML, JSON, ...) provided that  such  checks  can  be
472              delegated to an external program.
473
474   delay
475       NOTE: This command is available everywhere commands are given.
476
477       Sleep  for  the number of seconds specified in the argument. The number
478       can include a fractional part, e.g. 1.5.
479
480   feature
481       Test that the required feature(s) for a test are  available,  and  skip
482       the  test otherwise; or change the interpretation of the test, as docu‐
483       mented below. feature takes any number of arguments from this list:
484
485       64bit  The environment is 64 bits
486
487       ipv4   127.0.0.1 works
488
489       ipv6   [::1] works
490
491       dns    DNS lookups are working
492
493       topbuild
494              The test has been started with '-i'
495
496       root   The test has been invoked by the root user
497
498       user_varnish
499              The varnish user is present
500
501       user_vcache
502              The vcache user is present
503
504       group_varnish
505              The varnish group is present
506
507       cmd <command-line>
508              A command line that should execute with a zero exit status
509
510       ignore_unknown_macro
511              Do not fail the test if a string of the form ${...} is not  rec‐
512              ognized as a macro.
513
514       persistent_storage
515              Varnish was built with the deprecated persistent storage.
516
517       coverage
518              Varnish was built with code coverage enabled.
519
520       asan   Varnish was built with the address sanitizer.
521
522       msan   Varnish was built with the memory sanitizer.
523
524       tsan   Varnish was built with the thread sanitizer.
525
526       ubsan  Varnish was built with the undefined behavior sanitizer.
527
528       sanitizer
529              Varnish was built with a sanitizer.
530
531       workspace_emulator
532              Varnish was built with its workspace emulator.
533
534       A  feature  name can be prefixed with an exclamation mark (!) to skip a
535       test if the feature is present.
536
537       Be careful with ignore_unknown_macro, because it may cause a test  with
538       a  misspelled  macro  to  fail silently. You should only need it if you
539       must run a test with strings of the form "${...}".
540
541   filewrite
542       Write strings to file
543          filewrite [-a] /somefile "Hello" " " "Worldn"
544
545       The -a flag opens the file in append mode.
546
547   haproxy
548       Define and interact with haproxy instances.
549
550       To define a haproxy server, you'll use this syntax:
551
552          haproxy hNAME -conf-OK CONFIG
553          haproxy hNAME -conf-BAD ERROR CONFIG
554          haproxy hNAME [-D] [-W] [-arg STRING] [-conf[+vcl] STRING]
555
556       The first haproxy  hNAME  invocation  will  start  the  haproxy  master
557       process  in  the  background, waiting for the -start switch to actually
558       start the child.
559
560       Arguments:
561
562       hNAME  Identify the HAProxy server with a string, it must  starts  with
563              'h'.
564
565       -conf-OK CONFIG
566
567              Run haproxy in '-c' mode to check config is OK
568                     stdout/stderr   should  contain  'Configuration  file  is
569                     valid' The exit code should be 0.
570
571       -conf-BAD ERROR CONFIG
572
573              Run haproxy in '-c' mode to check config is BAD.
574                     "ERROR"  should  be  part  of  the  diagnostics  on  std‐
575                     out/stderr.  The exit code should be 1.
576
577       -D     Run HAproxy in daemon mode.  If not given '-d' mode used.
578
579       -W     Enable HAproxy in Worker mode.
580
581       -S     Enable HAproxy Master CLI in Worker mode
582
583       -arg STRING
584              Pass an argument to haproxy, for example "-h simple_list".
585
586       -cli STRING
587              Specify the spec to be run by the command line interface (CLI).
588
589       -mcli STRING
590              Specify  the  spec to be run by the command line interface (CLI)
591              of the Master process.
592
593       -conf STRING
594              Specify the configuration to be loaded by this HAProxy instance.
595
596       -conf+backend STRING
597
598              Specify the configuration to be loaded by this HAProxy instance,
599                     all server instances will be automatically appended
600
601       -start Start this HAProxy instance.
602
603       -wait  Stop this HAProxy instance.
604
605       -expectexit NUMBER
606              Expect haproxy to exit(3) with this value
607
608   logexpect
609       Reads the VSL and looks for records matching a given specification.  It
610       will  process records trying to match the first pattern, and when done,
611       will continue processing, trying to match the following pattern.  If  a
612       pattern isn't matched, the test will fail.
613
614       logexpect threads are declared this way:
615
616          logexpect lNAME -v <id> [-g <grouping>] [-d 0|1] [-q query] \
617                  [vsl arguments] {
618                          expect <skip> <vxid> <tag> <regex>
619                          expect <skip> <vxid> <tag> <regex>
620                          fail add <vxid> <tag> <regex>
621                          fail clear
622                          abort
623                          ...
624                  } [-start|-wait|-run]
625
626       And once declared, you can start them, or wait on them:
627
628          logexpect lNAME <-start|-wait>
629
630       With:
631
632       lNAME  Name the logexpect thread, it must start with 'l'.
633
634       -v id  Specify the varnish instance to use (most of the time, id=v1).
635
636       -g <session|request|vxid|raw
637              Decide  how  records  are  grouped, see -g in man varnishlog for
638              more information.
639
640       -d <0|1>
641              Start processing log records at the head of the log  instead  of
642              the tail.
643
644       -q query
645              Filter  records  using a query expression, see man vsl-query for
646              more information. Multiple -q options are not supported.
647
648       -m     Also emit log records for misses (only for debugging)
649
650       -err   Invert the meaning of success. Usually called once to expect the
651              logexpect to fail
652
653       -start Start the logexpect thread in the background.
654
655       -wait  Wait for the logexpect thread to finish
656
657       -run   Equivalent to "-start -wait".
658
659       VSL arguments (similar to the varnishlog options):
660
661       -C     Use caseless regex
662
663       -i <taglist>
664              Include tags
665
666       -I <[taglist:]regex>
667              Include by regex
668
669       -T <seconds>
670              Transaction end timeout
671
672       expect specification:
673
674       skip: [uint|*|?]
675              Max number of record to skip
676
677       vxid: [uint|*|=]
678              vxid to match
679
680       tag: [tagname|*|=]
681              Tag to match against
682
683       regex: regular expression to match against (optional)
684
685       For  skip, vxid and tag, '*' matches anything, '=' expects the value of
686       the previous matched record. The '?' marker is equivalent to zero,  ex‐
687       pecting  a  match on the next record. The difference is that '?' can be
688       used when the order of individual consecutive logs is  not  determinis‐
689       tic.  In  other words, lines from a block of alternatives marked by '?'
690       can be matched in any order, but all need to match eventually.
691
692       fail specification:
693
694       add: Add to the fail list
695          Arguments are equivalent to expect, except for skip missing
696
697       clear: Clear the fail list
698
699       Any number of fail specifications can be active during execution  of  a
700       logexpect. All active fail specifications are matched against every log
701       line and, if any match, the logexpect fails immediately.
702
703       For a logexpect to end successfully, there must be no specs on the fail
704       list, so logexpects should always end with
705          expect <skip> <vxid> <tag> <termination-condition> fail clear
706
707       abort specification:
708
709       abort(3)  varnishtest, intended to help debugging of the VSL client li‐
710       brary itself.
711
712   process
713       Run a process with stdin+stdout on a pseudo-terminal and  stderr  on  a
714       pipe.
715
716       Output from the pseudo-terminal is copied verbatim to ${pNAME_out}, and
717       the -log/-dump/-hexdump flags will also put it in the vtc-log.
718
719       The pseudo-terminal is not in ECHO mode, but if the programs run set it
720       to  ECHO mode ("stty sane") any input sent to the process will also ap‐
721       pear in this stream because of the ECHO.
722
723       Output from the stderr-pipe is copied verbatim to ${pNAME_err}, and  is
724       always included in the vtc_log.
725
726          process pNAME SPEC [-log] [-dump] [-hexdump] [-expect-exit N]
727                 [-start]  [-run]  [-write  STRING]  [-writeln  STRING] [-kill
728                 STRING] [-stop] [-wait] [-close]
729
730       pNAME  Name of the process. It must start with 'p'.
731
732       SPEC   The command(s) to run in this process.
733
734       -hexdump
735              Log output with vtc_hexdump(). Must be before -start/-run.
736
737       -dump  Log output with vtc_dump(). Must be before -start/-run.
738
739       -log   Log output with VLU/vtc_log(). Must be before -start/-run.
740
741       -start Start the process.
742
743       -expect-exit N
744              Expect exit status N
745
746       -wait  Wait for the process to finish.
747
748       -run   Shorthand for -start -wait.
749
750              In most cases, if you just want to start a process and wait  for
751              it  to  finish, you can use the shell command instead.  The fol‐
752              lowing commands are equivalent:
753
754                 shell "do --something"
755
756                 process p1 "do --something" -run
757
758              However, you may use the the  process  variant  to  conveniently
759              collect the standard input and output without dealing with shell
760              redirections yourself. The shell command can also expect an  ex‐
761              pression  from either output, consider using it if you only need
762              to match one.
763
764       -kill STRING
765              Send a signal to the process. The argument  can  be  either  the
766              string  "TERM",  "INT", or "KILL" for SIGTERM, SIGINT or SIGKILL
767              signals, respectively, or a hyphen (-) followed  by  the  signal
768              number.
769
770              If  you  need to use other signal names, you can use the kill(1)
771              command directly:
772
773                 shell "kill -USR1 ${pNAME_pid}"
774
775              Note that SIGHUP usage is discouraged in test cases.
776
777       -stop  Shorthand for -kill TERM.
778
779       -winsz LIN COL
780              Change the terminal window size to LIN lines and COL columns.
781
782       -write STRING
783              Write a string to the process' stdin.
784
785       -writeln STRING
786              Same as -write followed by a newline (\n).
787
788       -writehex HEXSTRING
789              Same as -write but interpreted as hexadecimal bytes.
790
791       -need-bytes [+]NUMBER
792              Wait until at least NUMBER bytes have been  received  in  total.
793              If '+' is prefixed, NUMBER new bytes must be received.
794
795       -expect-text LIN COL PAT
796              Wait  for PAT to appear at LIN,COL on the virtual screen.  Lines
797              and columns are numbered 1...N LIN==0 means "on any line" COL==0
798              means "anywhere on the line"
799
800       -match-text LIN COL PAT
801              Wait for the PAT regular expression to match the text at LIN,COL
802              on the virtual screen.  Lines and  columns  are  numbered  1...N
803              LIN==0 means "on any line" COL==0 means "anywhere on the line"
804
805       -close Alias for "-kill HUP"
806
807       -screen_dump
808              Dump the virtual screen into vtc_log
809
810   setenv
811       Set or change an environment variable:
812
813          setenv FOO "bar baz"
814
815       The above will set the environment variable $FOO to the value provided.
816       There is also an -ifunset argument which will only set the value if the
817       the environment variable does not already exist:
818
819          setenv -ifunset FOO quux
820
821   shell
822       NOTE: This command is available everywhere commands are given.
823
824       Pass the string given as argument to a shell. If you have multiple com‐
825       mands to run, you can use curly  brackets  to  describe  a  multi-lines
826       script, eg:
827
828          shell {
829                  echo begin
830                  cat /etc/fstab
831                  echo end
832          }
833
834       By default a zero exit code is expected, otherwise the vtc will fail.
835
836       Notice  that the commandstring is prefixed with "exec 2>&1;" to combine
837       stderr and stdout back to the test process.
838
839       Optional arguments:
840
841       -err   Expect non-zero exit code.
842
843       -exit N
844              Expect exit code N instead of zero.
845
846       -expect STRING
847              Expect string to be found in stdout+err.
848
849       -match REGEXP
850              Expect regexp to match the stdout+err output.
851
852   Specification
853       The specification of a stream follows the exact same rules as one for a
854       client or a server.
855
856   txreq, txresp, txcont, txpush
857       These  four  commands  are about sending headers. txreq and txresp will
858       send HEADER frames; txcont will send CONTINUATION frames;  txpush  PUSH
859       frames.   The  only difference between txreq and txresp are the default
860       headers set by each of them.
861
862       -noadd Do not add default headers.  Useful  to  avoid  duplicates  when
863              sending default headers using -hdr, -idxHdr and -litIdxHdr.
864
865       -status INT (txresp)
866              Set the :status pseudo-header.
867
868       -url STRING (txreq, txpush)
869              Set the :path pseudo-header.
870
871       -method STRING (txreq, txpush)
872              Set the :method pseudo-header.
873
874       -req STRING (txreq, txpush)
875              Alias for -method.
876
877       -scheme STRING (txreq, txpush)
878              Set the :scheme pseudo-header.
879
880       -hdr STRING1 STRING2
881              Insert a header, STRING1 being the name, and STRING2 the value.
882
883       -idxHdr INT
884              Insert an indexed header, using INT as index.
885
886       -litIdxHdr inc|not|never INT huf|plain STRING
887              Insert an literal, indexed header. The first argument specify if
888              the header should be added to the table, shouldn't,  or  mustn't
889              be compressed if/when retransmitted.
890
891              INT is the idex of the header name to use.
892
893              The third argument informs about the Huffman encoding: yes (huf)
894              or no (plain).
895
896              The last term is the literal value of the header.
897
898       -litHdr inc|not|never huf|plain STRING1 huf|plain STRING2
899              Insert a  literal  header,  with  the  same  first  argument  as
900              -litIdxHdr.
901
902              The  second  and third terms tell what the name of the header is
903              and if it should be Huffman-encoded, while the last two  do  the
904              same regarding the value.
905
906       -body STRING (txreq, txresp)
907              Specify a body, effectively putting STRING into a DATA frame af‐
908              ter the HEADER frame is sent.
909
910       -bodyfrom FILE (txreq, txresp)
911              Same as -body but content is read from FILE.
912
913       -bodylen INT (txreq, txresp)
914              Do the same thing as -body but generate a string of  INT  length
915              for you.
916
917       -gzipbody STRING (txreq, txresp)
918              Gzip STRING and send it as body.
919
920       -gziplen NUMBER (txreq, txresp)
921              Combine -bodylen and -gzipbody: generate a string of length NUM‐
922              BER, gzip it and send as body.
923
924       -nostrend (txreq, txresp)
925              Don't set the END_STREAM flag automatically, making the peer ex‐
926              pect a body after the headers.
927
928       -nohdrend
929              Don't  set  the  END_HEADERS flag automatically, making the peer
930              expect more HEADER frames.
931
932       -dep INT (txreq, txresp)
933              Tell the peer that this content depends on the stream  with  the
934              INT id.
935
936       -ex (txreq, txresp)
937              Make the dependency exclusive (-dep is still needed).
938
939       -weight (txreq, txresp)
940              Set the weight for the dependency.
941
942       -promised INT (txpush)
943              The id of the promised stream.
944
945       -pad STRING / -padlen INT (txreq, txresp, txpush)
946              Add  string as padding to the frame, either the one you provided
947              with -pad, or one that is generated for you, of  length  INT  is
948              -padlen case.
949
950   txdata
951       By  default,  data  frames  are  empty. The receiving end will know the
952       whole body has been delivered thanks to the END_STREAM flag set in  the
953       last DATA frame, and txdata automatically set it.
954
955       -data STRING
956              Data to be embedded into the frame.
957
958       -datalen INT
959              Generate and INT-bytes long string to be sent in the frame.
960
961       -pad STRING / -padlen INT
962              Add  string as padding to the frame, either the one you provided
963              with -pad, or one that is generated for you, of  length  INT  is
964              -padlen case.
965
966       -nostrend
967              Don't  set  the  END_STREAM  flag, allowing to send more data on
968              this stream.
969
970   rxreq, rxresp
971       These are two convenience functions to receive headers and body  of  an
972       incoming  request  or  response.  The only difference is that rxreq can
973       only be by a server, and rxresp by a client.
974
975   rxhdrs
976       rxhdrs will expect one HEADER frame, then, depending on the  arguments,
977       zero or more CONTINUATION frame.
978
979       -all   Keep  waiting  for CONTINUATION frames until END_HEADERS flag is
980              seen.
981
982       -some INT
983              Retrieve INT - 1 CONTINUATION frames after the HEADER frame.
984
985   rxpush
986       This works like rxhdrs, expecting a PUSH frame and then  zero  or  more
987       CONTINUATION frames.
988
989       -all   Keep  waiting  for CONTINUATION frames until END_HEADERS flag is
990              seen.
991
992       -some INT
993              Retrieve INT - 1 CONTINUATION frames after the PUSH frame.
994
995   rxdata
996       Receiving data is done using the rxdata keywords and will retrieve  one
997       DATA  frame,  if you wish to receive more, you can use these two conve‐
998       nience arguments:
999
1000       -all   keep waiting for DATA frame until one sets the END_STREAM flag
1001
1002       -some INT
1003              retrieve INT DATA frames.
1004
1005       Receive a frame, any frame.
1006
1007   sendhex
1008       Push bytes directly on the wire. sendhex takes exactly one argument:  a
1009       string describing the bytes, in hex notation, with possible whitespaces
1010       between them. Here's an example:
1011
1012          sendhex "00 00 08 00 0900       8d"
1013
1014   rxgoaway
1015       Receive a GOAWAY frame.
1016
1017   gunzip
1018       Same as the gunzip command for HTTP/1.
1019
1020   rxping
1021       Receive a PING frame.
1022
1023   txping
1024       Send PING frame.
1025
1026       -data STRING
1027              specify the payload of the frame, with STRING  being  an  8-char
1028              string.
1029
1030       -ack   set the ACK flag.
1031
1032   rxprio
1033       Receive a PRIORITY frame.
1034
1035   txprio
1036       Send a PRIORITY frame
1037
1038       -stream INT
1039              indicate the id of the stream the sender stream depends on.
1040
1041       -ex    the  dependency  should be made exclusive (only this streams de‐
1042              pends on the parent stream).
1043
1044       -weight INT
1045              an 8-bits integer is used to balance  priority  between  streams
1046              depending on the same streams.
1047
1048   rxrst
1049       Receive a RST_STREAM frame.
1050
1051   txrst
1052       Send  a  RST_STREAM  frame.  By default, txrst will send a 0 error code
1053       (NO_ERROR).
1054
1055       -err STRING|INT
1056              Sets the error code to be sent. The argument can be  an  integer
1057              or  a  string  describing the error, such as NO_ERROR, or CANCEL
1058              (see rfc7540#11.4 for more strings).
1059
1060   rxsettings
1061       Receive a SETTINGS frame.
1062
1063   txsettings
1064       SETTINGS frames must be acknowledge, arguments are as follow  (most  of
1065       them are from  rfc7540#6.5.2):
1066
1067       -hdrtbl INT
1068              headers table size
1069
1070       -push BOOL
1071              whether push frames are accepted or not
1072
1073       -maxstreams INT
1074              maximum concurrent streams allowed
1075
1076       -winsize INT
1077              sender's initial window size
1078
1079       -framesize INT
1080              largest frame size authorized
1081
1082       -hdrsize INT
1083              maximum size of the header list authorized
1084
1085       -ack   set the ack bit
1086
1087   rxwinup
1088       Receive a WINDOW_UPDATE frame.
1089
1090   txwinup
1091       Transmit  a WINDOW_UPDATE frame, increasing the amount of credit of the
1092       connection (from stream 0) or of the stream (any other stream).
1093
1094       -size INT
1095              give INT credits to the peer.
1096
1097       write_body STRING
1098              Same as the write_body command for HTTP/1.
1099
1100   expect
1101       expect in stream works as it does in client or server, except that  the
1102       elements compared will be different.
1103
1104       Most  of  these  elements will be frame specific, meaning that the last
1105       frame received on that stream must of the correct type.
1106
1107       Here the list of keywords you can look at.
1108
1109   GOAWAY specific
1110       goaway.err
1111              The error code (as integer) of the GOAWAY frame.
1112
1113       goaway.laststream
1114              Last-Stream-ID
1115
1116       goaway.debug
1117              Debug data, if any.
1118
1119   PING specific
1120       ping.data
1121              The 8-bytes string of the PING frame payload.
1122
1123       ping.ack (PING)
1124              "true" if the ACK flag was set, "false" otherwise.
1125
1126   PRIORITY specific
1127       prio.stream
1128              The stream ID announced.
1129
1130       prio.exclusive
1131              "true" if the priority is exclusive, else "false".
1132
1133       prio.weight
1134              The dependency weight.
1135
1136   PUSH_PROMISE specific
1137       push.id
1138              The id of the promised stream.
1139
1140   RESET_STREAM specific
1141       rst.err
1142              The error code (as integer) of the RESET_STREAM frame.
1143
1144   SETTINGS specific
1145       settings.ack
1146              "true" if the ACK flag was set, else ""false.
1147
1148       settings.push
1149              "true" if the push settings was set to yes, "false"  if  set  to
1150              no, and <undef> if not present.
1151
1152       settings.hdrtbl
1153              Value of HEADER_TABLE_SIZE if set, <undef> otherwise.
1154
1155       settings.maxstreams
1156              Value of MAX_CONCURRENT_STREAMS if set, <undef> otherwise.
1157
1158       settings.winsize
1159              Value of INITIAL_WINDOW_SIZE if set, <undef> otherwise.
1160
1161       setting.framesize
1162              Value of MAX_FRAME_SIZE if set, <undef> otherwise.
1163
1164       settings.hdrsize
1165              Value of MAX_HEADER_LIST_SIZE if set, <undef> otherwise.
1166
1167   WINDOW_UPDATE specific
1168       winup.size
1169              The size of the upgrade given by the WINDOW_UPDATE frame.
1170
1171   Generic frame
1172       frame.data
1173              Payload of the last frame
1174
1175       frame.type
1176              Type of the frame, as integer.
1177
1178       frame.size
1179              Size of the frame.
1180
1181       frame.stream
1182              Stream  of  the  frame  (correspond to the one you are executing
1183              this from, obviously).
1184
1185       frame.padding (for DATA, HEADERS, PUSH_PROMISE frames)
1186              Number of padded bytes.
1187
1188   Request and response
1189       Note: it's possible to inspect a request or response while it is  still
1190       being construct (in-between two frames for example).
1191
1192       req.bodylen / resp.bodylen
1193              Length in bytes of the request/response so far.
1194
1195       req.body / resp.body
1196              Body of the request/response so far.
1197
1198       req.http.STRING / resp.http.STRING
1199              Value of the header STRING in the request/response.
1200
1201       req.status / resp.status
1202              :status pseudo-header's value.
1203
1204       req.url / resp.url
1205              :path pseudo-header's value.
1206
1207       req.method / resp.method
1208              :method pseudo-header's value.
1209
1210       req.authority / resp.authority
1211              :method pseudo-header's value.
1212
1213       req.scheme / resp.scheme
1214              :method pseudo-header's value.
1215
1216   Stream
1217       stream.window
1218              The  current  window  size of the stream, or, if on stream 0, of
1219              the connection.
1220
1221       stream.weight
1222              Weight of the stream
1223
1224       stream.dependency
1225              Id of the stream this one depends on.
1226
1227   Index tables
1228       tbl.dec.size / tbl.enc.size
1229              Size (bytes) of the decoding/encoding table.
1230
1231       tbl.dec.size / tbl.enc.maxsize
1232              Maximum size (bytes) of the decoding/encoding table.
1233
1234       tbl.dec.length / tbl.enc.length
1235              Number of headers in decoding/encoding table.
1236
1237       tbl.dec[INT].key / tbl.enc[INT].key
1238              Name of the header at index INT of the decoding/encoding table.
1239
1240       tbl.dec[INT].value / tbl.enc[INT].value
1241              Value of the header at index INT of the decoding/encoding table.
1242
1243   syslog
1244       Define and interact with syslog instances (for use with haproxy)
1245
1246       To define a syslog server, you'll use this syntax:
1247
1248          syslog SNAME
1249
1250       Arguments:
1251
1252       SNAME  Identify the syslog server with a string which must  start  with
1253              'S'.
1254
1255       -level STRING
1256              Set  the  default  syslog  priority level used by any subsequent
1257              "recv" command.  Any syslog dgram with a different level will be
1258              skipped  by  "recv" command. This default level value may be su‐
1259              perseded by "recv" command if supplied as first argument:  "recv
1260              <level>".
1261
1262       -start Start the syslog server thread in the background.
1263
1264       -repeat
1265
1266              Instead of processing the specification only once, do it
1267                     NUMBER times.
1268
1269       -bind  Bind the syslog socket to a local address.
1270
1271       -wait  Wait for that thread to terminate.
1272
1273       -stop  Stop the syslog server thread.
1274
1275   tunnel
1276       The  goal  of a tunnel is to help control the data transfer between two
1277       parties, for example to trigger socket timeouts in the middle of proto‐
1278       col  frames,  without  the  need  to change how both parties are imple‐
1279       mented.
1280
1281       A tunnel accepts a connection and then connects on behalf of the source
1282       to  the  desired destination. Once both connections are established the
1283       tunnel will transfer bytes unchanged between the  source  and  destina‐
1284       tion.  Transfer  can  be interrupted, usually with the help of synchro‐
1285       nization methods like barriers. Once the transfer is paused, it is pos‐
1286       sible to let a specific amount of bytes move in either direction.
1287
1288       SECTION: tunnel.args Arguments
1289
1290       -start Start the tunnel in background, processing the last given speci‐
1291              fication.
1292
1293       -start+pause
1294              Start the tunnel, but already paused.
1295
1296       -wait  Block until the thread finishes.
1297
1298       -listen STRING
1299              Dictate the listening socket for the server. STRING  is  of  the
1300              form "IP PORT", or "HOST PORT".
1301
1302              Listens by defaults to a local random port.
1303
1304       -connect STRING
1305              Indicate  the  server  to connect to. STRING is also of the form
1306              "IP PORT", or "HOST PORT".
1307
1308              Connects by default to a varnish instance called v1.
1309
1310       SECTION: tunnel.spec Specification
1311
1312       The specification contains a list of tunnel commands that can  be  com‐
1313       bined with barriers and delays. For example:
1314
1315          tunnel t1 {
1316              barrier b1 sync
1317              pause
1318              delay 1
1319              send 42
1320              barrier b2 sync
1321              resume
1322          } -start
1323
1324       If  one end of the tunnel is closed before the end of the specification
1325       the test case will fail. A specification that ends in  a  paused  state
1326       will implicitely resume the tunnel.
1327
1328   varnish
1329       Define and interact with varnish instances.
1330
1331       To define a Varnish server, you'll use this syntax:
1332
1333          varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING]
1334                  [-errvcl STRING STRING] [-jail STRING] [-proto PROXY]
1335
1336       The  first  varnish  vNAME  invocation  will  start the varnishd master
1337       process in the background, waiting for the -start  switch  to  actually
1338       start the child.
1339
1340       Types used in the description below:
1341
1342       PATTERN
1343              is  a  'glob'  style  pattern  (ie: fnmatch(3)) as used in shell
1344              filename expansion.
1345
1346       Arguments:
1347
1348       vNAME  Identify the Varnish server with a string, it must  starts  with
1349              'v'.
1350
1351       -arg STRING
1352              Pass an argument to varnishd, for example "-h simple_list".
1353
1354       -vcl STRING
1355              Specify  the VCL to load on this Varnish instance. You'll proba‐
1356              bly want to use multi-lines strings for this ({...}).
1357
1358       -vcl+backend STRING
1359              Do the exact same thing as -vcl, but adds the  definition  block
1360              of known backends (ie. already defined).
1361
1362       -errvcl STRING1 STRING2
1363              Load  STRING2  as VCL, expecting it to fail, and Varnish to send
1364              an error string matching STRING1
1365
1366       -jail STRING
1367              Look at man varnishd (-j) for more information.
1368
1369       -proto PROXY
1370              Have Varnish use the proxy protocol. Note that PROXY here is the
1371              actual string.
1372
1373       You  can  decide  to start the Varnish instance and/or wait for several
1374       events:
1375
1376          varnish vNAME [-start] [-wait] [-wait-running] [-wait-stopped]
1377
1378       -start Start the child process.
1379
1380              Once successfully started, the following  macros  are  available
1381              for the default listen address: ${vNAME_addr}, ${vNAME_port} and
1382              ${vNAME_sock}. Additional macros are  available,  including  the
1383              listen  address name for each address vNAME listens to, like for
1384              example: ${vNAME_a0_addr}.
1385
1386       -stop  Stop the child process.
1387
1388       -syntax
1389              Set the VCL syntax level for this command (default: 4.1)
1390
1391       -wait  Wait for that instance to terminate.
1392
1393       -wait-running
1394              Wait for the Varnish child process to be started.
1395
1396       -wait-stopped
1397              Wait for the Varnish child process to stop.
1398
1399       -cleanup
1400              Once Varnish is stopped, clean everything after it. This is only
1401              used in very few tests and you should never need it.
1402
1403       -expectexit NUMBER
1404              Expect varnishd to exit(3) with this value
1405
1406       Once  Varnish is started, you can talk to it (as you would through var‐
1407       nishadm) with these additional switches:
1408
1409          varnish vNAME [-cli STRING] [-cliok STRING] [-clierr STRING]
1410                        [-clijson STRING]
1411
1412       -cli  STRING|-cliok  STRING|-clierr  STATUS  STRING|-cliexpect   REGEXP
1413       STRING
1414              All  four of these will send STRING to the CLI, the only differ‐
1415              ence is what they expect the result to be. -cli  doesn't  expect
1416              anything,  -cliok expects 200, -clierr expects STATUS, and -cli‐
1417              expect expects the REGEXP to match the returned response.
1418
1419       -clijson STRING
1420              Send STRING to the CLI, expect success (CLIS_OK/200)  and  check
1421              that the response is parsable JSON.
1422
1423       It  is  also  possible to interact with its shared memory (as you would
1424       through tools like varnishstat) with additional switches:
1425
1426       -expect !PATTERN|PATTERN OP NUMBER|PATTERN OP PATTERN
1427              Look into the VSM and make sure the first VSC counter identified
1428              by  PATTERN has a correct value. OP can be ==, >, >=, <, <=. For
1429              example:
1430
1431                 varnish v1 -expect SM?.s1.g_space > 1000000
1432                 varnish v1 -expect cache_hit >= cache_hit_grace
1433
1434              In the ! form the test fails if a counter matches PATTERN.
1435
1436              The MAIN. namespace can be omitted from PATTERN.
1437
1438              The test takes up to 5 seconds before timing out.
1439
1440       -vsc PATTERN
1441              Dump VSC counters matching PATTERN.
1442
1443       -vsl_catchup
1444              Wait until the logging thread has idled to make  sure  that  all
1445              the generated log is flushed
1446
1447   varnishtest
1448       Alternate name for 'vtest', see above.
1449
1450   vtest
1451       This  should  be  the first command in your vtc as it will identify the
1452       test case with a short yet descriptive sentence. It takes  exactly  one
1453       argument, a string, eg:
1454
1455          vtest "Check that vtest is actually a valid command"
1456
1457       It will also print that string in the log.
1458

HISTORY

1460       This document has been written by Guillaume Quintard.
1461

SEE ALSO

1463varnishtest(1)
1464
1465vmod_vtc(3)
1466
1468       This document is licensed under the same licence as Varnish itself. See
1469       LICENCE for details.
1470
1471       • Copyright (c) 2006-2016 Varnish Software AS
1472
1473
1474
1475
1476                                                                        VTC(7)
Impressum