1VTC(7) VTC(7)
2
6 VTC - Varnish Test Case Syntax
7
9 This document describes the syntax used by Varnish Test Cases files
10 (.vtc). A vtc file describe a scenario with different scripted
11 HTTP-talking entities, and generally one or more Varnish instances to
12 test.
13
15 A vtc file will be read word after word, with very little tokenization,
16 meaning a syntax error won't be detected until the test actually reach
17 the relevant action in the test.
18 A parsing error will most of the time result in an assert being trig‐
20 gered. If this happens, please refer yourself to the related source
21 file and line number. However, this guide should help you avoid the
22 most common mistakes.
23 Words and strings
25 The parser splits words by detecting whitespace characters and a string
26 is a word, or a series of words on the same line enclosed by dou‐
27 ble-quotes ("..."), or, for multi-line strings, enclosed in curly
28 brackets ({...}).
29 Comments
31 The leading whitespaces of lines are ignored. Empty lines (or ones con‐
32 sisting only of whitespaces) are ignored too, as are the lines starting
33 with "#" that are comments.
34 Lines and commands
36 Test files take at most one command per line, with the first word of
37 the line being the command and the following ones being its arguments.
38 To continue over to a new line without breaking the argument string,
39 you can escape the newline character (\n) with a backslash (\).
40
42 When a string is processed, macro expansion is performed. Macros are in
43 the form ${<name>[,<args>...]}, they have a name followed by an op‐
44 tional comma- or space-separated list of arguments. Leading and trail‐
45 ing spaces are ignored.
46 The macros ${foo,bar,baz} and ${ foo bar baz } are equivalent. If an
48 argument contains a space or a comma, arguments can be quoted. For ex‐
49 ample the macro ${foo,"bar,baz"} gives one argument bar,baz to the
50 macro called foo.
51 Unless documented otherwise, all macros are simple macros that don't
53 take arguments.
54 Built-in macros
56 ${bad_backend}
57 A socket address that will reliably never accept connections.
58 ${bad_ip}
60 An unlikely IPv4 address.
61 ${date}
63 The current date and time formatted for HTTP.
64 ${listen_addr}
66 The default listen address various components use, by default a
67 random port on localhost.
68 ${localhost}
70 The first IP address that resolves to "localhost".
71 ${pwd} The working directory from which varnishtest was executed.
73 ${string,<action>[,<args>...]}
75 The string macro is the entry point for text generation, it
76 takes a specialized action with each its own set of arguments.
77 ${string,repeat,<uint>,<str>}
79 Repeat uint times the string str.
80 ${testdir}
82 The directory containing the VTC script of the ongoing test case
83 execution.
84 ${tmpdir}
86 The dedicated working directory for the ongoing test case execu‐
87 tion, which happens to also be the current working directory.
88 Useful when an absolute path to the working directory is needed.
89 ${topbuild}
91 Only present when the -i option is used, to work on Varnish it‐
92 self instead of a regular installation.
93
95 barrier
96 NOTE: This command is available everywhere commands are given.
97 Barriers allows you to synchronize different threads to make sure
99 events occur in the right order. It's even possible to use them in VCL.
100 First, it's necessary to declare the barrier:
102 barrier bNAME TYPE NUMBER [-cyclic]
104 With the arguments being:
106 bNAME this is the name of the barrier, used to identify it when you'll
108 create sync points. It must start with 'b'.
109 TYPE it can be "cond" (mutex) or "sock" (socket) and sets internal
111 behavior. If you don't need VCL synchronization, use cond.
112 NUMBER number of sync point needed to go through the barrier.
114 -cyclic
116 if present, the barrier will reset itself and be ready for an‐
117 other round once gotten through.
118 Then, to add a sync point:
120 barrier bNAME sync
122 This will block the parent thread until the number of sync points for
124 bNAME reaches the NUMBER given in the barrier declaration.
125 If you wish to synchronize the VCL, you need to declare a "sock" bar‐
127 rier. This will emit a macro definition named "bNAME_sock" that you
128 can use in VCL (after importing the vtc vmod):
129 vtc.barrier_sync("${bNAME_sock}");
131 This function returns 0 if everything went well and is the equivalent
133 of barrier bNAME sync at the VTC top-level.
134 client/server
136 Client and server threads are fake HTTP entities used to test your Var‐
137 nish and VCL. They take any number of arguments, and the one that are
138 not recognized, assuming they don't start with '-', are treated as
139 specifications, laying out the actions to undertake:
140 client cNAME [...]
142 server sNAME [...]
143 Clients and server are identified by a string that's the first argu‐
145 ment, clients' names start with 'c' and servers' names start with 's'.
146 As the client and server commands share a good deal of arguments and
148 specification actions, they are grouped in this single section, spe‐
149 cific items will be explicitly marked as such.
150 Arguments
152 -start Start the thread in background, processing the last given speci‐
153 fication.
154 -wait Block until the thread finishes.
156 -run (client only)
158 Equivalent to "-start -wait".
159 -repeat NUMBER
161 Instead of processing the specification only once, do it NUMBER
162 times.
163 -keepalive
165 For repeat, do not open new connections but rather run all iter‐
166 ations in the same connection
167 -break (server only)
169 Stop the server.
170 -listen STRING (server only)
172 Dictate the listening socket for the server. STRING is of the
173 form "IP PORT", or "/PATH/TO/SOCKET" for a Unix domain socket.
174 In the latter case, the path must begin with '/', and the server
175 must be able to create it.
176 -connect STRING (client only)
178 Indicate the server to connect to. STRING is also of the form
179 "IP PORT", or "/PATH/TO/SOCKET". As with "server -listen", a
180 Unix domain socket is recognized when STRING begins with a '/'.
181 -dispatch (server only, s0 only)
183 Normally, to keep things simple, server threads only handle one
184 connection at a time, but the -dispatch switch allows to accept
185 any number of connection and handle them following the given
186 spec.
187 However, -dispatch is only allowed for the server name "s0".
189 -proxy1 STRING (client only)
191 Use the PROXY protocol version 1 for this connection. STRING is
192 of the form "CLIENTIP:PORT SERVERIP:PORT".
193 -proxy2 STRING (client only)
195 Use the PROXY protocol version 2 for this connection. STRING is
196 of the form "CLIENTIP:PORT SERVERIP:PORT".
197 Macros and automatic behaviour
199 To make things easier in the general case, clients will connect by de‐
200 fault to a Varnish server called v1. To connect to a different Varnish
201 server, use '-connect ${vNAME_sock}'.
202 The -vcl+backend switch of the varnish command will add all the de‐
204 clared servers as backends. Be careful though, servers will by default
205 listen to the 127.0.0.1 IP and will pick a random port, and publish 3
206 macros: sNAME_addr, sNAME_port and sNAME_sock, but only once they are
207 started. For 'varnish -vcl+backend' to create the vcl with the correct
208 values, the server must be started first.
209 Specification
211 It's a string, either double-quoted "like this", but most of the time
212 enclosed in curly brackets, allowing multilining. Write a command per
213 line in it, empty line are ignored, and long line can be wrapped by us‐
214 ing a backslash. For example:
215 client c1 {
217 txreq -url /foo \
218 -hdr "bar: baz"
219 rxresp
221 } -run
222 accept (server only)
224 Close the current connection, if any, and accept a new one. Note
225 that this new connection is HTTP/1.x.
226 chunked STRING
228 Send STRING as chunked encoding.
229 chunkedlen NUMBER
231 Do as chunked except that the string will be generated for you,
232 with a length of NUMBER characters.
233 close (server only)
235 Close the connection. Note that if operating in HTTP/2 mode no
236 extra (GOAWAY) frame is sent, it's simply a TCP close.
237 expect STRING1 OP STRING2
239 Test if "STRING1 OP STRING2" is true, and if not, fails the
240 test. OP can be ==, <, <=, >, >= when STRING1 and STRING2 rep‐
241 resent numbers in which case it's an order operator. If STRING1
242 and STRING2 are meant as strings OP is a matching operator, ei‐
243 ther == (exact match) or ~ (regex match).
244 varnishtest will first try to resolve STRING1 and STRING2 by
246 looking if they have special meanings, in which case, the re‐
247 solved value is use for the test. Note that this value can be a
248 string representing a number, allowing for tests such as:
249 expect req.http.x-num > 2
251 Here's the list of recognized strings, most should be obvious as
253 they either match VCL logic, or the txreq/txresp options:
254 • remote.ip
256 • remote.port
258 • remote.path
260 • req.method
262 • req.url
264 • req.proto
266 • resp.proto
268 • resp.status
270 • resp.reason
272 • resp.chunklen
274 • req.bodylen
276 • req.body
278 • resp.bodylen
280 • resp.body
282 • req.http.NAME
284 • resp.http.NAME
286 expect_close
288 Reads from the connection, expecting nothing to read but an EOF.
289 fatal|non_fatal
291 Control whether a failure of this entity should stop the test.
292 gunzip Gunzip the body in place.
294 recv NUMBER
296 Read NUMBER bytes from the connection.
297 rxchunk
299 Receive an HTTP chunk.
300 rxpri (server only)
302 Receive a preface. If valid set the server to HTTP/2, abort oth‐
303 erwise.
304 rxreq (server only)
306 Receive and parse a request's headers and body.
307 rxreqbody (server only)
309 Receive a request's body.
310 rxreqhdrs (server only)
312 Receive and parse a request's headers (but not the body).
313 rxresp [-no_obj] (client only)
315 Receive and parse a response's headers and body. If -no_obj is
316 present, only get the headers.
317 rxrespbody (client only)
319 Receive (part of) a response's body.
320 -max : max length of this receive, 0 for all
322 rxresphdrs (client only)
324 Receive and parse a response's headers.
325 send STRING
327 Push STRING on the connection.
328 send_n NUMBER STRING
330 Write STRING on the socket NUMBER times.
331 send_urgent STRING
333 Send string as TCP OOB urgent data. You will never need this.
334 sendhex STRING
336 Send bytes as described by STRING. STRING should consist of hex
337 pairs possibly separated by whitespace or newlines. For example:
338 "0F EE a5 3df2".
339 settings -dectbl INT
341 Force internal HTTP/2 settings to certain values. Currently only
342 support setting the decoding table size.
343 shell Same as for the top-level shell.
345 stream HTTP/2 introduces the concept of streams, and these come with
347 their own specification, and as it's quite big, have been moved
348 to their own chapter.
349 timeout NUMBER
351 Set the TCP timeout for this entity.
352 txpri (client only)
354 Send an HTTP/2 preface ("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") and
355 set client to HTTP/2.
356 txreq|txresp [...]
358 Send a minimal request or response, but overload it if neces‐
359 sary.
360 txreq is client-specific and txresp is server-specific.
362 The only thing different between a request and a response, apart
364 from who can send them is that the first line (request line vs
365 status line), so all the options are prety much the same.
366 -method STRING (txreq only)
368 What method to use (default: "GET").
369 -req STRING (txreq only)
371 Alias for -method.
372 -url STRING (txreq only)
374 What location to use (default "/").
375 -proto STRING
377 What protocol use in the status line. (default:
378 "HTTP/1.1").
379 -status NUMBER (txresp only)
381 What status code to return (default 200).
382 -reason STRING (txresp only)
384 What message to put in the status line (default: "OK").
385 These three switches can appear in any order but must come be‐
387 fore the following ones.
388 -nohost
390 Don't include a Host header in the request. Also Implied
391 by the addition of a Host header with -hdr.
392 -nolen Don't include a Content-Length header. Also implied by
394 the addition of a Content-Length or Transfer-Encoding
395 header with -hdr.
396 -nodate
398 Don't include a Date header in the response. Also implied
399 by the addition of a Date header with -hdr.
400 -hdr STRING
402 Add STRING as a header, it must follow this format:
403 "name: value". It can be called multiple times.
404 -hdrlen STRING NUMBER
406 Add STRING as a header with NUMBER bytes of content.
407 You can then use the arguments related to the body:
409 -body STRING
411 Input STRING as body.
412 -bodyfrom FILE
414 Same as -body but content is read from FILE.
415 -bodylen NUMBER
417 Generate and input a body that is NUMBER bytes-long.
418 -gziplevel NUMBER
420 Set the gzip level (call it before any of the other gzip
421 switches).
422 -gzipresidual NUMBER
424 Add extra gzip bits. You should never need it.
425 -gzipbody STRING
427 Gzip STRING and send it as body.
428 -gziplen NUMBER
430 Combine -bodylen and -gzipbody: generate a string of
431 length NUMBER, gzip it and send as body.
432 write_body STRING
434 Write the body of a request or a response to a file. By using
435 the shell command, higher-level checks on the body can be per‐
436 formed (eg. XML, JSON, ...) provided that such checks can be
437 delegated to an external program.
438 delay
440 NOTE: This command is available everywhere commands are given.
441 Sleep for the number of seconds specified in the argument. The number
443 can include a fractional part, e.g. 1.5.
444 feature
446 Test that the required feature(s) for a test are available, and skip
447 the test otherwise; or change the interpretation of the test, as docu‐
448 mented below. feature takes any number of arguments from this list:
449 64bit The environment is 64 bits
451 ipv4 127.0.0.1 works
453 ipv6 [::1] works
455 dns DNS lookups are working
457 topbuild
459 The test has been started with '-i'
460 root The test has been invoked by the root user
462 user_varnish
464 The varnish user is present
465 user_vcache
467 The vcache user is present
468 group_varnish
470 The varnish group is present
471 cmd <command-line>
473 A command line that should execute with a zero exit status
474 ignore_unknown_macro
476 Do not fail the test if a string of the form ${...} is not rec‐
477 ognized as a macro.
478 persistent_storage
480 Varnish was built with the deprecated persistent storage.
481 coverage
483 Varnish was built with code coverage enabled.
484 asan Varnish was built with the address sanitizer.
486 msan Varnish was built with the memory sanitizer.
488 tsan Varnish was built with the thread sanitizer.
490 ubsan Varnish was built with the undefined behavior sanitizer.
492 sanitizer
494 Varnish was built with a sanitizer.
495 workspace_emulator
497 Varnish was built with its workspace emulator.
498 abstract_uds
500 Creation of an abstract unix domain socket succeeded
501 A feature name can be prefixed with an exclamation mark (!) to skip a
503 test if the feature is present.
504 Be careful with ignore_unknown_macro, because it may cause a test with
506 a misspelled macro to fail silently. You should only need it if you
507 must run a test with strings of the form "${...}".
508 filewrite
510 Write strings to file
511 filewrite [-a] /somefile "Hello" " " "Worldn"
512 The -a flag opens the file in append mode.
514 haproxy
516 Define and interact with haproxy instances.
517 To define a haproxy server, you'll use this syntax:
519 haproxy hNAME -conf-OK CONFIG
521 haproxy hNAME -conf-BAD ERROR CONFIG
522 haproxy hNAME [-D] [-W] [-arg STRING] [-conf[+vcl] STRING]
523 The first haproxy hNAME invocation will start the haproxy master
525 process in the background, waiting for the -start switch to actually
526 start the child.
527 Arguments:
529 hNAME Identify the HAProxy server with a string, it must starts with
531 'h'.
532 -conf-OK CONFIG
534 Run haproxy in '-c' mode to check config is OK
536 stdout/stderr should contain 'Configuration file is
537 valid' The exit code should be 0.
538 -conf-BAD ERROR CONFIG
540 Run haproxy in '-c' mode to check config is BAD.
542 "ERROR" should be part of the diagnostics on std‐
543 out/stderr. The exit code should be 1.
544 -D Run HAproxy in daemon mode. If not given '-d' mode used.
546 -W Enable HAproxy in Worker mode.
548 -S Enable HAproxy Master CLI in Worker mode
550 -arg STRING
552 Pass an argument to haproxy, for example "-h simple_list".
553 -cli STRING
555 Specify the spec to be run by the command line interface (CLI).
556 -mcli STRING
558 Specify the spec to be run by the command line interface (CLI)
559 of the Master process.
560 -conf STRING
562 Specify the configuration to be loaded by this HAProxy instance.
563 -conf+backend STRING
565 Specify the configuration to be loaded by this HAProxy instance,
567 all server instances will be automatically appended
568 -start Start this HAProxy instance.
570 -wait Stop this HAProxy instance.
572 -expectexit NUMBER
574 Expect haproxy to exit(3) with this value
575 haproxy CLI Specification
577 expect OP STRING
578 Regex match the CLI reception buffer with STRING if OP is ~ or,
579 on the contraty, if OP is !~ check that there is no regex match.
580 send STRING
582 Push STRING on the CLI connection. STRING will be terminated by
583 an end of line character (n).
584 logexpect
586 Reads the VSL and looks for records matching a given specification. It
587 will process records trying to match the first pattern, and when done,
588 will continue processing, trying to match the following pattern. If a
589 pattern isn't matched, the test will fail.
590 logexpect threads are declared this way:
592 logexpect lNAME -v <id> [-g <grouping>] [-d 0|1] [-q query] \
594 [vsl arguments] {
595 expect <skip> <vxid> <tag> <regex>
596 expect <skip> <vxid> <tag> <regex>
597 fail add <vxid> <tag> <regex>
598 fail clear
599 abort
600 ...
601 } [-start|-wait|-run]
602 And once declared, you can start them, or wait on them:
604 logexpect lNAME <-start|-wait>
606 With:
608 lNAME Name the logexpect thread, it must start with 'l'.
610 -v id Specify the varnish instance to use (most of the time, id=v1).
612 -g <session|request|vxid|raw
614 Decide how records are grouped, see -g in man varnishlog for
615 more information.
616 -d <0|1>
618 Start processing log records at the head of the log instead of
619 the tail.
620 -q query
622 Filter records using a query expression, see man vsl-query for
623 more information. Multiple -q options are not supported.
624 -m Also emit log records for misses (only for debugging)
626 -err Invert the meaning of success. Usually called once to expect the
628 logexpect to fail
629 -start Start the logexpect thread in the background.
631 -wait Wait for the logexpect thread to finish
633 -run Equivalent to "-start -wait".
635 VSL arguments (similar to the varnishlog options):
637 -C Use caseless regex
639 -i <taglist>
641 Include tags
642 -I <[taglist:]regex>
644 Include by regex
645 -T <seconds>
647 Transaction end timeout
648 expect specification:
650 skip: [uint|*|?]
652 Max number of record to skip
653 vxid: [uint|*|=]
655 vxid to match
656 tag: [tagname|*|=]
658 Tag to match against
659 regex: regular expression to match against (optional)
661 For skip, vxid and tag, '*' matches anything, '=' expects the value of
663 the previous matched record. The '?' marker is equivalent to zero, ex‐
664 pecting a match on the next record. The difference is that '?' can be
665 used when the order of individual consecutive logs is not determinis‐
666 tic. In other words, lines from a block of alternatives marked by '?'
667 can be matched in any order, but all need to match eventually.
668 fail specification:
670 add: Add to the fail list
672 Arguments are equivalent to expect, except for skip missing
673 clear: Clear the fail list
675 Any number of fail specifications can be active during execution of a
677 logexpect. All active fail specifications are matched against every log
678 line and, if any match, the logexpect fails immediately.
679 For a logexpect to end successfully, there must be no specs on the fail
681 list, so logexpects should always end with
682 expect <skip> <vxid> <tag> <termination-condition> fail clear
683 abort specification:
685 abort(3) varnishtest, intended to help debugging of the VSL client li‐
687 brary itself.
688 loop
690 loop NUMBER STRING
691 Process STRING as a specification, NUMBER times.
692 This works inside all specification strings
694 process
696 Run a process with stdin+stdout on a pseudo-terminal and stderr on a
697 pipe.
698 Output from the pseudo-terminal is copied verbatim to ${pNAME_out}, and
700 the -log/-dump/-hexdump flags will also put it in the vtc-log.
701 The pseudo-terminal is not in ECHO mode, but if the programs run set it
703 to ECHO mode ("stty sane") any input sent to the process will also ap‐
704 pear in this stream because of the ECHO.
705 Output from the stderr-pipe is copied verbatim to ${pNAME_err}, and is
707 always included in the vtc_log.
708 process pNAME SPEC [-allow-core] [-expect-exit N] [-expect-signal N]
710 [-dump] [-hexdump] [-log] [-run] [-close] [-kill SIGNAL]
711 [-start] [-stop] [-wait] [-write STRING] [-writeln STRING]
712 [-writehex HEXSTRING] [-need-bytes [+]NUMBER] [-screen-dump]
713 [-winsz LINES COLUMNSS] [-ansi-response] [-expect-cursor LINE
714 COLUMN] [-expect-text LINE COLUMN TEXT] [-match-text LINE
715 COLUMN REGEXP]
716 pNAME Name of the process. It must start with 'p'.
718 SPEC The command(s) to run in this process.
720 -hexdump
722 Log output with vtc_hexdump(). Must be before -start/-run.
723 -dump Log output with vtc_dump(). Must be before -start/-run.
725 -log Log output with VLU/vtc_log(). Must be before -start/-run.
727 -start Start the process.
729 -expect-exit N
731 Expect exit status N
732 -expect-signal N
734 Expect signal in exit status N
735 -allow-core
737 Core dump in exit status is OK
738 -wait Wait for the process to finish.
740 -run Shorthand for -start -wait.
742 In most cases, if you just want to start a process and wait for
744 it to finish, you can use the shell command instead. The fol‐
745 lowing commands are equivalent:
746 shell "do --something"
748 process p1 "do --something" -run
750 However, you may use the the process variant to conveniently
752 collect the standard input and output without dealing with shell
753 redirections yourself. The shell command can also expect an ex‐
754 pression from either output, consider using it if you only need
755 to match one.
756 -key KEYSYM
758 Send emulated key-press. KEYSYM can be one of (NPAGE, PPAGE,
759 HOME, END)
760 -kill SIGNAL
762 Send a signal to the process. The argument can be either the
763 string "TERM", "INT", or "KILL" for SIGTERM, SIGINT or SIGKILL
764 signals, respectively, or a hyphen (-) followed by the signal
765 number.
766 If you need to use other signal names, you can use the kill(1)
768 command directly:
769 shell "kill -USR1 ${pNAME_pid}"
771 Note that SIGHUP usage is discouraged in test cases.
773 -stop Shorthand for -kill TERM.
775 -close Alias for "-kill HUP"
777 -winsz LINES COLUMNS
779 Change the terminal window size to LIN lines and COL columns.
780 -write STRING
782 Write a string to the process' stdin.
783 -writeln STRING
785 Same as -write followed by a newline (\n).
786 -writehex HEXSTRING
788 Same as -write but interpreted as hexadecimal bytes.
789 -need-bytes [+]NUMBER
791 Wait until at least NUMBER bytes have been received in total.
792 If '+' is prefixed, NUMBER new bytes must be received.
793 -ansi-response
795 Respond to terminal respond-back sequences
796 -expect-cursor LINE COLUMN
798 Expect cursors location
799 -expect-text LINE COLUMNS TEXT
801 Wait for TEXT to appear at LIN,COL on the virtual screen. Lines
802 and columns are numbered 1...N LIN==0 means "on any line" COL==0
803 means "anywhere on the line"
804 -match-text LINE COLUMN REGEXP
806 Wait for the PAT regular expression to match the text at LIN,COL
807 on the virtual screen. Lines and columns are numbered 1...N
808 LIN==0 means "on any line" COL==0 means "anywhere on the line"
809 -screen-dump
811 Dump the virtual screen into vtc_log
812 setenv
814 Set or change an environment variable:
815 setenv FOO "bar baz"
817 The above will set the environment variable $FOO to the value provided.
819 There is also an -ifunset argument which will only set the value if the
820 the environment variable does not already exist:
821 setenv -ifunset FOO quux
823 shell
825 NOTE: This command is available everywhere commands are given.
826 Pass the string given as argument to a shell. If you have multiple com‐
828 mands to run, you can use curly brackets to describe a multi-lines
829 script, eg:
830 shell {
832 echo begin
833 cat /etc/fstab
834 echo end
835 }
836 By default a zero exit code is expected, otherwise the vtc will fail.
838 Notice that the commandstring is prefixed with "exec 2>&1;" to combine
840 stderr and stdout back to the test process.
841 Optional arguments:
843 -err Expect non-zero exit code.
845 -exit N
847 Expect exit code N instead of zero.
848 -expect STRING
850 Expect string to be found in stdout+err.
851 -match REGEXP
853 Expect regexp to match the stdout+err output.
854 stream
856 (note: this section is at the top-level for easier navigation, but it's
857 part of the client/server specification)
858 Streams map roughly to a request in HTTP/2, a request is sent on stream
860 N, the response too, then the stream is discarded. The main exception
861 is the first stream, 0, that serves as coordinator.
862 Stream syntax follow the client/server one:
864 stream ID [SPEC] [ACTION]
866 ID is the HTTP/2 stream number, while SPEC describes what will be done
868 in that stream.
869 Note that, when parsing a stream action, if the entity isn't operating
871 in HTTP/2 mode, these spec is ran before:
872 txpri/rxpri # client/server
874 stream 0 {
875 txsettings
876 rxsettings
877 txsettings -ack
878 rxsettings
879 expect settings.ack == true
880 } -run
881 And HTTP/2 mode is then activated before parsing the specification.
883 Actions
885 -start Run the specification in a thread, giving back control immedi‐
886 ately.
887 -wait Wait for the started thread to finish running the spec.
889 -run equivalent to calling -start then -wait.
891 Specification
893 The specification of a stream follows the exact same rules as one for a
894 client or a server.
895 txreq, txresp, txcont, txpush
897 These four commands are about sending headers. txreq and txresp will
898 send HEADER frames; txcont will send CONTINUATION frames; txpush PUSH
899 frames. The only difference between txreq and txresp are the default
900 headers set by each of them.
901 -noadd Do not add default headers. Useful to avoid duplicates when
903 sending default headers using -hdr, -idxHdr and -litIdxHdr.
904 -status INT (txresp)
906 Set the :status pseudo-header.
907 -url STRING (txreq, txpush)
909 Set the :path pseudo-header.
910 -method STRING (txreq, txpush)
912 Set the :method pseudo-header.
913 -req STRING (txreq, txpush)
915 Alias for -method.
916 -scheme STRING (txreq, txpush)
918 Set the :scheme pseudo-header.
919 -hdr STRING1 STRING2
921 Insert a header, STRING1 being the name, and STRING2 the value.
922 -idxHdr INT
924 Insert an indexed header, using INT as index.
925 -litIdxHdr inc|not|never INT huf|plain STRING
927 Insert an literal, indexed header. The first argument specify if
928 the header should be added to the table, shouldn't, or mustn't
929 be compressed if/when retransmitted.
930 INT is the idex of the header name to use.
932 The third argument informs about the Huffman encoding: yes (huf)
934 or no (plain).
935 The last term is the literal value of the header.
937 -litHdr inc|not|never huf|plain STRING1 huf|plain STRING2
939 Insert a literal header, with the same first argument as
940 -litIdxHdr.
941 The second and third terms tell what the name of the header is
943 and if it should be Huffman-encoded, while the last two do the
944 same regarding the value.
945 -body STRING (txreq, txresp)
947 Specify a body, effectively putting STRING into a DATA frame af‐
948 ter the HEADER frame is sent.
949 -bodyfrom FILE (txreq, txresp)
951 Same as -body but content is read from FILE.
952 -bodylen INT (txreq, txresp)
954 Do the same thing as -body but generate a string of INT length
955 for you.
956 -gzipbody STRING (txreq, txresp)
958 Gzip STRING and send it as body.
959 -gziplen NUMBER (txreq, txresp)
961 Combine -bodylen and -gzipbody: generate a string of length NUM‐
962 BER, gzip it and send as body.
963 -nostrend (txreq, txresp)
965 Don't set the END_STREAM flag automatically, making the peer ex‐
966 pect a body after the headers.
967 -nohdrend
969 Don't set the END_HEADERS flag automatically, making the peer
970 expect more HEADER frames.
971 -dep INT (txreq, txresp)
973 Tell the peer that this content depends on the stream with the
974 INT id.
975 -ex (txreq, txresp)
977 Make the dependency exclusive (-dep is still needed).
978 -weight (txreq, txresp)
980 Set the weight for the dependency.
981 -promised INT (txpush)
983 The id of the promised stream.
984 -pad STRING / -padlen INT (txreq, txresp, txpush)
986 Add string as padding to the frame, either the one you provided
987 with -pad, or one that is generated for you, of length INT is
988 -padlen case.
989 txdata
991 By default, data frames are empty. The receiving end will know the
992 whole body has been delivered thanks to the END_STREAM flag set in the
993 last DATA frame, and txdata automatically set it.
994 -data STRING
996 Data to be embedded into the frame.
997 -datalen INT
999 Generate and INT-bytes long string to be sent in the frame.
1000 -pad STRING / -padlen INT
1002 Add string as padding to the frame, either the one you provided
1003 with -pad, or one that is generated for you, of length INT is
1004 -padlen case.
1005 -nostrend
1007 Don't set the END_STREAM flag, allowing to send more data on
1008 this stream.
1009 rxreq, rxresp
1011 These are two convenience functions to receive headers and body of an
1012 incoming request or response. The only difference is that rxreq can
1013 only be by a server, and rxresp by a client.
1014 rxhdrs
1016 rxhdrs will expect one HEADER frame, then, depending on the arguments,
1017 zero or more CONTINUATION frame.
1018 -all Keep waiting for CONTINUATION frames until END_HEADERS flag is
1020 seen.
1021 -some INT
1023 Retrieve INT - 1 CONTINUATION frames after the HEADER frame.
1024 rxpush
1026 This works like rxhdrs, expecting a PUSH frame and then zero or more
1027 CONTINUATION frames.
1028 -all Keep waiting for CONTINUATION frames until END_HEADERS flag is
1030 seen.
1031 -some INT
1033 Retrieve INT - 1 CONTINUATION frames after the PUSH frame.
1034 rxdata
1036 Receiving data is done using the rxdata keywords and will retrieve one
1037 DATA frame, if you wish to receive more, you can use these two conve‐
1038 nience arguments:
1039 -all keep waiting for DATA frame until one sets the END_STREAM flag
1041 -some INT
1043 retrieve INT DATA frames.
1044 Receive a frame, any frame.
1046 sendhex
1048 Push bytes directly on the wire. sendhex takes exactly one argument: a
1049 string describing the bytes, in hex notation, with possible whitespaces
1050 between them. Here's an example:
1051 sendhex "00 00 08 00 0900 8d"
1053 rxgoaway
1055 Receive a GOAWAY frame.
1056 txgoaway
1058 Possible options include:
1059 -err STRING|INT
1061 set the error code to explain the termination. The second argu‐
1062 ment can be a integer or the string version of the error code as
1063 found in rfc7540#7.
1064 -laststream INT
1066 the id of the "highest-numbered stream identifier for which the
1067 sender of the GOAWAY frame might have taken some action on or
1068 might yet take action on".
1069 -debug specify the debug data, if any to append to the frame.
1071 gunzip
1073 Same as the gunzip command for HTTP/1.
1074 rxping
1076 Receive a PING frame.
1077 txping
1079 Send PING frame.
1080 -data STRING
1082 specify the payload of the frame, with STRING being an 8-char
1083 string.
1084 -ack set the ACK flag.
1086 rxprio
1088 Receive a PRIORITY frame.
1089 txprio
1091 Send a PRIORITY frame
1092 -stream INT
1094 indicate the id of the stream the sender stream depends on.
1095 -ex the dependency should be made exclusive (only this streams de‐
1097 pends on the parent stream).
1098 -weight INT
1100 an 8-bits integer is used to balance priority between streams
1101 depending on the same streams.
1102 rxrst
1104 Receive a RST_STREAM frame.
1105 txrst
1107 Send a RST_STREAM frame. By default, txrst will send a 0 error code
1108 (NO_ERROR).
1109 -err STRING|INT
1111 Sets the error code to be sent. The argument can be an integer
1112 or a string describing the error, such as NO_ERROR, or CANCEL
1113 (see rfc7540#11.4 for more strings).
1114 rxsettings
1116 Receive a SETTINGS frame.
1117 txsettings
1119 SETTINGS frames must be acknowledge, arguments are as follow (most of
1120 them are from rfc7540#6.5.2):
1121 -hdrtbl INT
1123 headers table size
1124 -push BOOL
1126 whether push frames are accepted or not
1127 -maxstreams INT
1129 maximum concurrent streams allowed
1130 -winsize INT
1132 sender's initial window size
1133 -framesize INT
1135 largest frame size authorized
1136 -hdrsize INT
1138 maximum size of the header list authorized
1139 -ack set the ack bit
1141 rxwinup
1143 Receive a WINDOW_UPDATE frame.
1144 txwinup
1146 Transmit a WINDOW_UPDATE frame, increasing the amount of credit of the
1147 connection (from stream 0) or of the stream (any other stream).
1148 -size INT
1150 give INT credits to the peer.
1151 write_body STRING
1153 Same as the write_body command for HTTP/1.
1154 expect
1156 expect in stream works as it does in client or server, except that the
1157 elements compared will be different.
1158 Most of these elements will be frame specific, meaning that the last
1160 frame received on that stream must of the correct type.
1161 Here the list of keywords you can look at.
1163 GOAWAY specific
1165 goaway.err
1166 The error code (as integer) of the GOAWAY frame.
1167 goaway.laststream
1169 Last-Stream-ID
1170 goaway.debug
1172 Debug data, if any.
1173 PING specific
1175 ping.data
1176 The 8-bytes string of the PING frame payload.
1177 ping.ack (PING)
1179 "true" if the ACK flag was set, "false" otherwise.
1180 PRIORITY specific
1182 prio.stream
1183 The stream ID announced.
1184 prio.exclusive
1186 "true" if the priority is exclusive, else "false".
1187 prio.weight
1189 The dependency weight.
1190 PUSH_PROMISE specific
1192 push.id
1193 The id of the promised stream.
1194 RESET_STREAM specific
1196 rst.err
1197 The error code (as integer) of the RESET_STREAM frame.
1198 SETTINGS specific
1200 settings.ack
1201 "true" if the ACK flag was set, else ""false.
1202 settings.push
1204 "true" if the push settings was set to yes, "false" if set to
1205 no, and <undef> if not present.
1206 settings.hdrtbl
1208 Value of HEADER_TABLE_SIZE if set, <undef> otherwise.
1209 settings.maxstreams
1211 Value of MAX_CONCURRENT_STREAMS if set, <undef> otherwise.
1212 settings.winsize
1214 Value of INITIAL_WINDOW_SIZE if set, <undef> otherwise.
1215 setting.framesize
1217 Value of MAX_FRAME_SIZE if set, <undef> otherwise.
1218 settings.hdrsize
1220 Value of MAX_HEADER_LIST_SIZE if set, <undef> otherwise.
1221 WINDOW_UPDATE specific
1223 winup.size
1224 The size of the upgrade given by the WINDOW_UPDATE frame.
1225 Generic frame
1227 frame.data
1228 Payload of the last frame
1229 frame.type
1231 Type of the frame, as integer.
1232 frame.size
1234 Size of the frame.
1235 frame.stream
1237 Stream of the frame (correspond to the one you are executing
1238 this from, obviously).
1239 frame.padding (for DATA, HEADERS, PUSH_PROMISE frames)
1241 Number of padded bytes.
1242 Request and response
1244 Note: it's possible to inspect a request or response while it is still
1245 being construct (in-between two frames for example).
1246 req.bodylen / resp.bodylen
1248 Length in bytes of the request/response so far.
1249 req.body / resp.body
1251 Body of the request/response so far.
1252 req.http.STRING / resp.http.STRING
1254 Value of the header STRING in the request/response.
1255 req.status / resp.status
1257 :status pseudo-header's value.
1258 req.url / resp.url
1260 :path pseudo-header's value.
1261 req.method / resp.method
1263 :method pseudo-header's value.
1264 req.authority / resp.authority
1266 :method pseudo-header's value.
1267 req.scheme / resp.scheme
1269 :method pseudo-header's value.
1270 Stream
1272 stream.window
1273 The current local window size of the stream, or, if on stream 0,
1274 of the connection.
1275 stream.peer_window
1277 The current peer window size of the stream, or, if on stream 0,
1278 of the connection.
1279 stream.weight
1281 Weight of the stream
1282 stream.dependency
1284 Id of the stream this one depends on.
1285 Index tables
1287 tbl.dec.size / tbl.enc.size
1288 Size (bytes) of the decoding/encoding table.
1289 tbl.dec.size / tbl.enc.maxsize
1291 Maximum size (bytes) of the decoding/encoding table.
1292 tbl.dec.length / tbl.enc.length
1294 Number of headers in decoding/encoding table.
1295 tbl.dec[INT].key / tbl.enc[INT].key
1297 Name of the header at index INT of the decoding/encoding table.
1298 tbl.dec[INT].value / tbl.enc[INT].value
1300 Value of the header at index INT of the decoding/encoding table.
1301 syslog
1303 Define and interact with syslog instances (for use with haproxy)
1304 To define a syslog server, you'll use this syntax:
1306 syslog SNAME
1308 Arguments:
1310 SNAME Identify the syslog server with a string which must start with
1312 'S'.
1313 -level STRING
1315 Set the default syslog priority level used by any subsequent
1316 "recv" command. Any syslog dgram with a different level will be
1317 skipped by "recv" command. This default level value may be su‐
1318 perseded by "recv" command if supplied as first argument: "recv
1319 <level>".
1320 -start Start the syslog server thread in the background.
1322 -repeat
1324 Instead of processing the specification only once, do it
1326 NUMBER times.
1327 -bind Bind the syslog socket to a local address.
1329 -wait Wait for that thread to terminate.
1331 -stop Stop the syslog server thread.
1333 tunnel
1335 The goal of a tunnel is to help control the data transfer between two
1336 parties, for example to trigger socket timeouts in the middle of proto‐
1337 col frames, without the need to change how both parties are imple‐
1338 mented.
1339 A tunnel accepts a connection and then connects on behalf of the source
1341 to the desired destination. Once both connections are established the
1342 tunnel will transfer bytes unchanged between the source and destina‐
1343 tion. Transfer can be interrupted, usually with the help of synchro‐
1344 nization methods like barriers. Once the transfer is paused, it is pos‐
1345 sible to let a specific amount of bytes move in either direction.
1346 Arguments
1348 -start Start the tunnel in background, processing the last given speci‐
1349 fication.
1350 -start+pause
1352 Start the tunnel, but already paused.
1353 -wait Block until the thread finishes.
1355 -listen STRING
1357 Dictate the listening socket for the server. STRING is of the
1358 form "IP PORT", or "HOST PORT".
1359 Listens by defaults to a local random port.
1361 -connect STRING
1363 Indicate the server to connect to. STRING is also of the form
1364 "IP PORT", or "HOST PORT".
1365 Connects by default to a varnish instance called v1.
1367 Specification
1369 The specification contains a list of tunnel commands that can be com‐
1370 bined with barriers and delays. For example:
1371 tunnel t1 {
1373 barrier b1 sync
1374 pause
1375 delay 1
1376 send 42
1377 barrier b2 sync
1378 resume
1379 } -start
1380 If one end of the tunnel is closed before the end of the specification
1382 the test case will fail. A specification that ends in a paused state
1383 will implicitely resume the tunnel.
1384 pause Wait for in-flight bytes to be transferred and pause the tunnel.
1386 The tunnel must be running.
1388 recv NUMBER
1390 Wait until NUMBER bytes are transferred from destination to
1391 source.
1392 The tunnel must be paused, it remains paused afterwards.
1394 resume Resume the transfer of bytes in both directions.
1396 The tunnel must be paused.
1398 send NUMBER
1400 Wait until NUMBER bytes are transferred from source to destina‐
1401 tion.
1402 The tunnel must be paused, it remains paused afterwards.
1404 varnish
1406 Define and interact with varnish instances.
1407 To define a Varnish server, you'll use this syntax:
1409 varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING]
1411 [-errvcl STRING STRING] [-jail STRING] [-proto PROXY]
1412 The first varnish vNAME invocation will start the varnishd master
1414 process in the background, waiting for the -start switch to actually
1415 start the child.
1416 Types used in the description below:
1418 PATTERN
1420 is a 'glob' style pattern (ie: fnmatch(3)) as used in shell
1421 filename expansion.
1422 Arguments:
1424 vNAME Identify the Varnish server with a string, it must starts with
1426 'v'.
1427 -arg STRING
1429 Pass an argument to varnishd, for example "-h simple_list".
1430 -vcl STRING
1432 Specify the VCL to load on this Varnish instance. You'll proba‐
1433 bly want to use multi-lines strings for this ({...}).
1434 -vcl+backend STRING
1436 Do the exact same thing as -vcl, but adds the definition block
1437 of known backends (ie. already defined).
1438 -errvcl STRING1 STRING2
1440 Load STRING2 as VCL, expecting it to fail, and Varnish to send
1441 an error string matching STRING1
1442 -jail STRING
1444 Look at man varnishd (-j) for more information.
1445 -proto PROXY
1447 Have Varnish use the proxy protocol. Note that PROXY here is the
1448 actual string.
1449 You can decide to start the Varnish instance and/or wait for several
1451 events:
1452 varnish vNAME [-start] [-wait] [-wait-running] [-wait-stopped]
1454 -start Start the child process.
1456 Once successfully started, the following macros are available
1458 for the default listen address: ${vNAME_addr}, ${vNAME_port} and
1459 ${vNAME_sock}. Additional macros are available, including the
1460 listen address name for each address vNAME listens to, like for
1461 example: ${vNAME_a0_addr}.
1462 -stop Stop the child process.
1464 -syntax
1466 Set the VCL syntax level for this command (default: 4.1)
1467 -wait Wait for that instance to terminate.
1469 -wait-running
1471 Wait for the Varnish child process to be started.
1472 -wait-stopped
1474 Wait for the Varnish child process to stop.
1475 -cleanup
1477 Once Varnish is stopped, clean everything after it. This is only
1478 used in very few tests and you should never need it.
1479 -expectexit NUMBER
1481 Expect varnishd to exit(3) with this value
1482 Once Varnish is started, you can talk to it (as you would through var‐
1484 nishadm) with these additional switches:
1485 varnish vNAME [-cli STRING] [-cliok STRING] [-clierr STRING]
1487 [-clijson STRING]
1488 -cli STRING|-cliok STRING|-clierr STATUS STRING|-cliexpect REGEXP
1490 STRING
1491 All four of these will send STRING to the CLI, the only differ‐
1492 ence is what they expect the result to be. -cli doesn't expect
1493 anything, -cliok expects 200, -clierr expects STATUS, and -cli‐
1494 expect expects the REGEXP to match the returned response.
1495 -clijson STRING
1497 Send STRING to the CLI, expect success (CLIS_OK/200) and check
1498 that the response is parsable JSON.
1499 It is also possible to interact with its shared memory (as you would
1501 through tools like varnishstat) with additional switches:
1502 -expect !PATTERN|PATTERN OP NUMBER|PATTERN OP PATTERN
1504 Look into the VSM and make sure the first VSC counter identified
1505 by PATTERN has a correct value. OP can be ==, >, >=, <, <=. For
1506 example:
1507 varnish v1 -expect SM?.s1.g_space > 1000000
1509 varnish v1 -expect cache_hit >= cache_hit_grace
1510 In the ! form the test fails if a counter matches PATTERN.
1512 The MAIN. namespace can be omitted from PATTERN.
1514 The test takes up to 5 seconds before timing out.
1516 -vsc PATTERN
1518 Dump VSC counters matching PATTERN.
1519 -vsl_catchup
1521 Wait until the logging thread has idled to make sure that all
1522 the generated log is flushed
1523 varnishtest
1525 Alternate name for 'vtest', see above.
1526 vtest
1528 This should be the first command in your vtc as it will identify the
1529 test case with a short yet descriptive sentence. It takes exactly one
1530 argument, a string, eg:
1531 vtest "Check that vtest is actually a valid command"
1533 It will also print that string in the log.
1535
1537 This document has been written by Guillaume Quintard.
1538
1540 • varnishtest(1)
1541 • vmod_vtc(3)
1543
1545 This document is licensed under the same licence as Varnish itself. See
1546 LICENCE for details.
1547 • Copyright (c) 2006-2016 Varnish Software AS
1549 VTC(7)