1VTC(7) VTC(7)
2
3
4
6 VTC - Varnish Test Case Syntax
7
9 This document describes the syntax used by Varnish Test Cases files
10 (.vtc). A vtc file describe a scenario with different scripted
11 HTTP-talking entities, and generally one or more Varnish instances to
12 test.
13
15 A vtc file will be read word after word, with very little tokenization,
16 meaning a syntax error won't be detected until the test actually reach
17 the relevant action in the test.
18
19 A parsing error will most of the time result in an assert being trig‐
20 gered. If this happens, please refer yourself to the related source
21 file and line number. However, this guide should help you avoid the
22 most common mistakes.
23
24 Words and strings
25 The parser splits words by detecting whitespace characters and a string
26 is a word, or a series of words on the same line enclosed by dou‐
27 ble-quotes ("..."), or, for multi-line strings, enclosed in curly
28 brackets ({...}).
29
30 Comments
31 The leading whitespaces of lines are ignored. Empty lines (or ones con‐
32 sisting only of whitespaces) are ignored too, as are the lines starting
33 with "#" that are comments.
34
35 Lines and commands
36 Test files take at most one command per line, with the first word of
37 the line being the command and the following ones being its arguments.
38 To continue over to a new line without breaking the argument string,
39 you can escape the newline character (\n) with a backslash (\).
40
42 When a string is processed, macro expansion is performed. Macros are in
43 the form ${<name>[,<args>...]}, they have a name followed by an op‐
44 tional comma- or space-separated list of arguments. Leading and trail‐
45 ing spaces are ignored.
46
47 The macros ${foo,bar,baz} and ${ foo bar baz } are equivalent. If an
48 argument contains a space or a comma, arguments can be quoted. For ex‐
49 ample the macro ${foo,"bar,baz"} gives one argument bar,baz to the
50 macro called foo.
51
52 Unless documented otherwise, all macros are simple macros that don't
53 take arguments.
54
55 Built-in macros
56 ${bad_backend}
57 A socket address that will reliably never accept connections.
58
59 ${bad_ip}
60 An unlikely IPv4 address.
61
62 ${date}
63 The current date and time formatted for HTTP.
64
65 ${listen_addr}
66 The default listen address various components use, by default a
67 random port on localhost.
68
69 ${localhost}
70 The first IP address that resolves to "localhost".
71
72 ${pwd} The working directory from which varnishtest was executed.
73
74 ${string,<action>[,<args>...]}
75 The string macro is the entry point for text generation, it
76 takes a specialized action with each its own set of arguments.
77
78 ${string,repeat,<uint>,<str>}
79 Repeat uint times the string str.
80
81 ${testdir}
82 The directory containing the VTC script of the ongoing test case
83 execution.
84
85 ${tmpdir}
86 The dedicated working directory for the ongoing test case execu‐
87 tion, which happens to also be the current working directory.
88 Useful when an absolute path to the working directory is needed.
89
90 ${topbuild}
91 Only present when the -i option is used, to work on Varnish it‐
92 self instead of a regular installation.
93
95 barrier
96 NOTE: This command is available everywhere commands are given.
97
98 Barriers allows you to synchronize different threads to make sure
99 events occur in the right order. It's even possible to use them in VCL.
100
101 First, it's necessary to declare the barrier:
102
103 barrier bNAME TYPE NUMBER [-cyclic]
104
105 With the arguments being:
106
107 bNAME this is the name of the barrier, used to identify it when you'll
108 create sync points. It must start with 'b'.
109
110 TYPE it can be "cond" (mutex) or "sock" (socket) and sets internal
111 behavior. If you don't need VCL synchronization, use cond.
112
113 NUMBER number of sync point needed to go through the barrier.
114
115 -cyclic
116 if present, the barrier will reset itself and be ready for an‐
117 other round once gotten through.
118
119 Then, to add a sync point:
120
121 barrier bNAME sync
122
123 This will block the parent thread until the number of sync points for
124 bNAME reaches the NUMBER given in the barrier declaration.
125
126 If you wish to synchronize the VCL, you need to declare a "sock" bar‐
127 rier. This will emit a macro definition named "bNAME_sock" that you
128 can use in VCL (after importing the vtc vmod):
129
130 vtc.barrier_sync("${bNAME_sock}");
131
132 This function returns 0 if everything went well and is the equivalent
133 of barrier bNAME sync at the VTC top-level.
134
135 client/server
136 Client and server threads are fake HTTP entities used to test your Var‐
137 nish and VCL. They take any number of arguments, and the one that are
138 not recognized, assuming they don't start with '-', are treated as
139 specifications, laying out the actions to undertake:
140
141 client cNAME [...]
142 server sNAME [...]
143
144 Clients and server are identified by a string that's the first argu‐
145 ment, clients' names start with 'c' and servers' names start with 's'.
146
147 As the client and server commands share a good deal of arguments and
148 specification actions, they are grouped in this single section, spe‐
149 cific items will be explicitly marked as such.
150
151 SECTION: client-server.macros Macros and automatic behaviour
152
153 To make things easier in the general case, clients will connect by de‐
154 fault to a Varnish server called v1. To connect to a different Varnish
155 server, use '-connect ${vNAME_sock}'.
156
157 The -vcl+backend switch of the varnish command will add all the de‐
158 clared servers as backends. Be careful though, servers will by default
159 listen to the 127.0.0.1 IP and will pick a random port, and publish 3
160 macros: sNAME_addr, sNAME_port and sNAME_sock, but only once they are
161 started. For 'varnish -vcl+backend' to create the vcl with the correct
162 values, the server must be started first.
163
164 SECTION: client-server.args Arguments
165
166 -start Start the thread in background, processing the last given speci‐
167 fication.
168
169 -wait Block until the thread finishes.
170
171 -run (client only)
172 Equivalent to "-start -wait".
173
174 -repeat NUMBER
175 Instead of processing the specification only once, do it NUMBER
176 times.
177
178 -keepalive
179 For repeat, do not open new connections but rather run all iter‐
180 ations in the same connection
181
182 -break (server only)
183 Stop the server.
184
185 -listen STRING (server only)
186 Dictate the listening socket for the server. STRING is of the
187 form "IP PORT", or "/PATH/TO/SOCKET" for a Unix domain socket.
188 In the latter case, the path must begin with '/', and the server
189 must be able to create it.
190
191 -connect STRING (client only)
192 Indicate the server to connect to. STRING is also of the form
193 "IP PORT", or "/PATH/TO/SOCKET". As with "server -listen", a
194 Unix domain socket is recognized when STRING begins with a '/'.
195
196 -dispatch (server only, s0 only)
197 Normally, to keep things simple, server threads only handle one
198 connection at a time, but the -dispatch switch allows to accept
199 any number of connection and handle them following the given
200 spec.
201
202 However, -dispatch is only allowed for the server name "s0".
203
204 -proxy1 STRING (client only)
205 Use the PROXY protocol version 1 for this connection. STRING is
206 of the form "CLIENTIP:PORT SERVERIP:PORT".
207
208 -proxy2 STRING (client only)
209 Use the PROXY protocol version 2 for this connection. STRING is
210 of the form "CLIENTIP:PORT SERVERIP:PORT".
211
212 SECTION: client-server.spec Specification
213
214 It's a string, either double-quoted "like this", but most of the time
215 enclosed in curly brackets, allowing multilining. Write a command per
216 line in it, empty line are ignored, and long line can be wrapped by us‐
217 ing a backslash. For example:
218
219 client c1 {
220 txreq -url /foo \
221 -hdr "bar: baz"
222
223 rxresp
224 } -run
225
226 accept (server only)
227 Close the current connection, if any, and accept a new one. Note
228 that this new connection is HTTP/1.x.
229
230 chunked STRING
231 Send STRING as chunked encoding.
232
233 chunkedlen NUMBER
234 Do as chunked except that the string will be generated for you,
235 with a length of NUMBER characters.
236
237 close (server only)
238 Close the connection. Note that if operating in HTTP/2 mode no
239 extra (GOAWAY) frame is sent, it's simply a TCP close.
240
241 expect STRING1 OP STRING2
242 Test if "STRING1 OP STRING2" is true, and if not, fails the
243 test. OP can be ==, <, <=, >, >= when STRING1 and STRING2 rep‐
244 resent numbers in which case it's an order operator. If STRING1
245 and STRING2 are meant as strings OP is a matching operator, ei‐
246 ther == (exact match) or ~ (regex match).
247
248 varnishtest will first try to resolve STRING1 and STRING2 by
249 looking if they have special meanings, in which case, the re‐
250 solved value is use for the test. Note that this value can be a
251 string representing a number, allowing for tests such as:
252
253 expect req.http.x-num > 2
254
255 Here's the list of recognized strings, most should be obvious as
256 they either match VCL logic, or the txreq/txresp options:
257
258 • remote.ip
259
260 • remote.port
261
262 • remote.path
263
264 • req.method
265
266 • req.url
267
268 • req.proto
269
270 • resp.proto
271
272 • resp.status
273
274 • resp.reason
275
276 • resp.chunklen
277
278 • req.bodylen
279
280 • req.body
281
282 • resp.bodylen
283
284 • resp.body
285
286 • req.http.NAME
287
288 • resp.http.NAME
289
290 expect_close
291 Reads from the connection, expecting nothing to read but an EOF.
292
293 fatal|non_fatal
294 Control whether a failure of this entity should stop the test.
295
296 gunzip Gunzip the body in place.
297
298 recv NUMBER
299 Read NUMBER bytes from the connection.
300
301 rxchunk
302 Receive an HTTP chunk.
303
304 rxpri (server only)
305 Receive a preface. If valid set the server to HTTP/2, abort oth‐
306 erwise.
307
308 rxreq (server only)
309 Receive and parse a request's headers and body.
310
311 rxreqbody (server only)
312 Receive a request's body.
313
314 rxreqhdrs (server only)
315 Receive and parse a request's headers (but not the body).
316
317 rxresp [-no_obj] (client only)
318 Receive and parse a response's headers and body. If -no_obj is
319 present, only get the headers.
320
321 rxrespbody (client only)
322 Receive (part of) a response's body.
323
324 -max : max length of this receive, 0 for all
325
326 rxresphdrs (client only)
327 Receive and parse a response's headers.
328
329 send STRING
330 Push STRING on the connection.
331
332 send_n NUMBER STRING
333 Write STRING on the socket NUMBER times.
334
335 send_urgent STRING
336 Send string as TCP OOB urgent data. You will never need this.
337
338 sendhex STRING
339 Send bytes as described by STRING. STRING should consist of hex
340 pairs possibly separated by whitespace or newlines. For example:
341 "0F EE a5 3df2".
342
343 settings -dectbl INT
344 Force internal HTTP/2 settings to certain values. Currently only
345 support setting the decoding table size.
346
347 shell Same as for the top-level shell.
348
349 stream HTTP/2 introduces the concept of streams, and these come with
350 their own specification, and as it's quite big, have been moved
351 to their own chapter.
352
353 SECTION: stream stream
354
355 (note: this section is at the top-level for easier navigation, but it's
356 part of the client/server specification)
357
358 Streams map roughly to a request in HTTP/2, a request is sent on stream
359 N, the response too, then the stream is discarded. The main exception
360 is the first stream, 0, that serves as coordinator.
361
362 Stream syntax follow the client/server one:
363
364 stream ID [SPEC] [ACTION]
365
366 ID is the HTTP/2 stream number, while SPEC describes what will be done
367 in that stream.
368
369 Note that, when parsing a stream action, if the entity isn't operating
370 in HTTP/2 mode, these spec is ran before:
371
372 txpri/rxpri # client/server
373 stream 0 {
374 txsettings
375 rxsettings
376 txsettings -ack
377 rxsettings
378 expect settings.ack == true
379 } -run
380
381 And HTTP/2 mode is then activated before parsing the specification.
382
383 SECTION: stream.actions Actions
384
385 -start Run the specification in a thread, giving back control immedi‐
386 ately.
387
388 -wait Wait for the started thread to finish running the spec.
389
390 -run equivalent to calling -start then -wait.
391
392 timeout NUMBER
393 Set the TCP timeout for this entity.
394
395 txpri (client only)
396 Send an HTTP/2 preface ("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") and
397 set client to HTTP/2.
398
399 txreq|txresp [...]
400 Send a minimal request or response, but overload it if neces‐
401 sary.
402
403 txreq is client-specific and txresp is server-specific.
404
405 The only thing different between a request and a response, apart
406 from who can send them is that the first line (request line vs
407 status line), so all the options are prety much the same.
408
409 -method STRING (txreq only)
410 What method to use (default: "GET").
411
412 -req STRING (txreq only)
413 Alias for -method.
414
415 -url STRING (txreq only)
416 What location to use (default "/").
417
418 -proto STRING
419 What protocol use in the status line. (default:
420 "HTTP/1.1").
421
422 -status NUMBER (txresp only)
423 What status code to return (default 200).
424
425 -reason STRING (txresp only)
426 What message to put in the status line (default: "OK").
427
428 These three switches can appear in any order but must come be‐
429 fore the following ones.
430
431 -nohost
432 Don't include a Host header in the request.
433
434 -nolen Don't include a Content-Length header.
435
436 -hdr STRING
437 Add STRING as a header, it must follow this format:
438 "name: value". It can be called multiple times.
439
440 -hdrlen STRING NUMBER
441 Add STRING as a header with NUMBER bytes of content.
442
443 You can then use the arguments related to the body:
444
445 -body STRING
446 Input STRING as body.
447
448 -bodyfrom FILE
449 Same as -body but content is read from FILE.
450
451 -bodylen NUMBER
452 Generate and input a body that is NUMBER bytes-long.
453
454 -gziplevel NUMBER
455 Set the gzip level (call it before any of the other gzip
456 switches).
457
458 -gzipresidual NUMBER
459 Add extra gzip bits. You should never need it.
460
461 -gzipbody STRING
462 Gzip STRING and send it as body.
463
464 -gziplen NUMBER
465 Combine -bodylen and -gzipbody: generate a string of
466 length NUMBER, gzip it and send as body.
467
468 write_body STRING
469 Write the body of a request or a response to a file. By using
470 the shell command, higher-level checks on the body can be per‐
471 formed (eg. XML, JSON, ...) provided that such checks can be
472 delegated to an external program.
473
474 delay
475 NOTE: This command is available everywhere commands are given.
476
477 Sleep for the number of seconds specified in the argument. The number
478 can include a fractional part, e.g. 1.5.
479
480 feature
481 Test that the required feature(s) for a test are available, and skip
482 the test otherwise; or change the interpretation of the test, as docu‐
483 mented below. feature takes any number of arguments from this list:
484
485 64bit The environment is 64 bits
486
487 ipv4 127.0.0.1 works
488
489 ipv6 [::1] works
490
491 dns DNS lookups are working
492
493 topbuild
494 The test has been started with '-i'
495
496 root The test has been invoked by the root user
497
498 user_varnish
499 The varnish user is present
500
501 user_vcache
502 The vcache user is present
503
504 group_varnish
505 The varnish group is present
506
507 cmd <command-line>
508 A command line that should execute with a zero exit status
509
510 ignore_unknown_macro
511 Do not fail the test if a string of the form ${...} is not rec‐
512 ognized as a macro.
513
514 persistent_storage
515 Varnish was built with the deprecated persistent storage.
516
517 coverage
518 Varnish was built with code coverage enabled.
519
520 asan Varnish was built with the address sanitizer.
521
522 msan Varnish was built with the memory sanitizer.
523
524 tsan Varnish was built with the thread sanitizer.
525
526 ubsan Varnish was built with the undefined behavior sanitizer.
527
528 sanitizer
529 Varnish was built with a sanitizer.
530
531 workspace_emulator
532 Varnish was built with its workspace emulator.
533
534 A feature name can be prefixed with an exclamation mark (!) to skip a
535 test if the feature is present.
536
537 Be careful with ignore_unknown_macro, because it may cause a test with
538 a misspelled macro to fail silently. You should only need it if you
539 must run a test with strings of the form "${...}".
540
541 filewrite
542 Write strings to file
543 filewrite [-a] /somefile "Hello" " " "Worldn"
544
545 The -a flag opens the file in append mode.
546
547 haproxy
548 Define and interact with haproxy instances.
549
550 To define a haproxy server, you'll use this syntax:
551
552 haproxy hNAME -conf-OK CONFIG
553 haproxy hNAME -conf-BAD ERROR CONFIG
554 haproxy hNAME [-D] [-W] [-arg STRING] [-conf[+vcl] STRING]
555
556 The first haproxy hNAME invocation will start the haproxy master
557 process in the background, waiting for the -start switch to actually
558 start the child.
559
560 Arguments:
561
562 hNAME Identify the HAProxy server with a string, it must starts with
563 'h'.
564
565 -conf-OK CONFIG
566
567 Run haproxy in '-c' mode to check config is OK
568 stdout/stderr should contain 'Configuration file is
569 valid' The exit code should be 0.
570
571 -conf-BAD ERROR CONFIG
572
573 Run haproxy in '-c' mode to check config is BAD.
574 "ERROR" should be part of the diagnostics on std‐
575 out/stderr. The exit code should be 1.
576
577 -D Run HAproxy in daemon mode. If not given '-d' mode used.
578
579 -W Enable HAproxy in Worker mode.
580
581 -S Enable HAproxy Master CLI in Worker mode
582
583 -arg STRING
584 Pass an argument to haproxy, for example "-h simple_list".
585
586 -cli STRING
587 Specify the spec to be run by the command line interface (CLI).
588
589 -mcli STRING
590 Specify the spec to be run by the command line interface (CLI)
591 of the Master process.
592
593 -conf STRING
594 Specify the configuration to be loaded by this HAProxy instance.
595
596 -conf+backend STRING
597
598 Specify the configuration to be loaded by this HAProxy instance,
599 all server instances will be automatically appended
600
601 -start Start this HAProxy instance.
602
603 -wait Stop this HAProxy instance.
604
605 -expectexit NUMBER
606 Expect haproxy to exit(3) with this value
607
608 logexpect
609 Reads the VSL and looks for records matching a given specification. It
610 will process records trying to match the first pattern, and when done,
611 will continue processing, trying to match the following pattern. If a
612 pattern isn't matched, the test will fail.
613
614 logexpect threads are declared this way:
615
616 logexpect lNAME -v <id> [-g <grouping>] [-d 0|1] [-q query] \
617 [vsl arguments] {
618 expect <skip> <vxid> <tag> <regex>
619 expect <skip> <vxid> <tag> <regex>
620 fail add <vxid> <tag> <regex>
621 fail clear
622 abort
623 ...
624 } [-start|-wait|-run]
625
626 And once declared, you can start them, or wait on them:
627
628 logexpect lNAME <-start|-wait>
629
630 With:
631
632 lNAME Name the logexpect thread, it must start with 'l'.
633
634 -v id Specify the varnish instance to use (most of the time, id=v1).
635
636 -g <session|request|vxid|raw
637 Decide how records are grouped, see -g in man varnishlog for
638 more information.
639
640 -d <0|1>
641 Start processing log records at the head of the log instead of
642 the tail.
643
644 -q query
645 Filter records using a query expression, see man vsl-query for
646 more information. Multiple -q options are not supported.
647
648 -m Also emit log records for misses (only for debugging)
649
650 -err Invert the meaning of success. Usually called once to expect the
651 logexpect to fail
652
653 -start Start the logexpect thread in the background.
654
655 -wait Wait for the logexpect thread to finish
656
657 -run Equivalent to "-start -wait".
658
659 VSL arguments (similar to the varnishlog options):
660
661 -C Use caseless regex
662
663 -i <taglist>
664 Include tags
665
666 -I <[taglist:]regex>
667 Include by regex
668
669 -T <seconds>
670 Transaction end timeout
671
672 expect specification:
673
674 skip: [uint|*|?]
675 Max number of record to skip
676
677 vxid: [uint|*|=]
678 vxid to match
679
680 tag: [tagname|*|=]
681 Tag to match against
682
683 regex: regular expression to match against (optional)
684
685 For skip, vxid and tag, '*' matches anything, '=' expects the value of
686 the previous matched record. The '?' marker is equivalent to zero, ex‐
687 pecting a match on the next record. The difference is that '?' can be
688 used when the order of individual consecutive logs is not determinis‐
689 tic. In other words, lines from a block of alternatives marked by '?'
690 can be matched in any order, but all need to match eventually.
691
692 fail specification:
693
694 add: Add to the fail list
695 Arguments are equivalent to expect, except for skip missing
696
697 clear: Clear the fail list
698
699 Any number of fail specifications can be active during execution of a
700 logexpect. All active fail specifications are matched against every log
701 line and, if any match, the logexpect fails immediately.
702
703 For a logexpect to end successfully, there must be no specs on the fail
704 list, so logexpects should always end with
705 expect <skip> <vxid> <tag> <termination-condition> fail clear
706
707 abort specification:
708
709 abort(3) varnishtest, intended to help debugging of the VSL client li‐
710 brary itself.
711
712 process
713 Run a process with stdin+stdout on a pseudo-terminal and stderr on a
714 pipe.
715
716 Output from the pseudo-terminal is copied verbatim to ${pNAME_out}, and
717 the -log/-dump/-hexdump flags will also put it in the vtc-log.
718
719 The pseudo-terminal is not in ECHO mode, but if the programs run set it
720 to ECHO mode ("stty sane") any input sent to the process will also ap‐
721 pear in this stream because of the ECHO.
722
723 Output from the stderr-pipe is copied verbatim to ${pNAME_err}, and is
724 always included in the vtc_log.
725
726 process pNAME SPEC [-log] [-dump] [-hexdump] [-expect-exit N]
727 [-start] [-run] [-write STRING] [-writeln STRING] [-kill
728 STRING] [-stop] [-wait] [-close]
729
730 pNAME Name of the process. It must start with 'p'.
731
732 SPEC The command(s) to run in this process.
733
734 -hexdump
735 Log output with vtc_hexdump(). Must be before -start/-run.
736
737 -dump Log output with vtc_dump(). Must be before -start/-run.
738
739 -log Log output with VLU/vtc_log(). Must be before -start/-run.
740
741 -start Start the process.
742
743 -expect-exit N
744 Expect exit status N
745
746 -wait Wait for the process to finish.
747
748 -run Shorthand for -start -wait.
749
750 In most cases, if you just want to start a process and wait for
751 it to finish, you can use the shell command instead. The fol‐
752 lowing commands are equivalent:
753
754 shell "do --something"
755
756 process p1 "do --something" -run
757
758 However, you may use the the process variant to conveniently
759 collect the standard input and output without dealing with shell
760 redirections yourself. The shell command can also expect an ex‐
761 pression from either output, consider using it if you only need
762 to match one.
763
764 -kill STRING
765 Send a signal to the process. The argument can be either the
766 string "TERM", "INT", or "KILL" for SIGTERM, SIGINT or SIGKILL
767 signals, respectively, or a hyphen (-) followed by the signal
768 number.
769
770 If you need to use other signal names, you can use the kill(1)
771 command directly:
772
773 shell "kill -USR1 ${pNAME_pid}"
774
775 Note that SIGHUP usage is discouraged in test cases.
776
777 -stop Shorthand for -kill TERM.
778
779 -winsz LIN COL
780 Change the terminal window size to LIN lines and COL columns.
781
782 -write STRING
783 Write a string to the process' stdin.
784
785 -writeln STRING
786 Same as -write followed by a newline (\n).
787
788 -writehex HEXSTRING
789 Same as -write but interpreted as hexadecimal bytes.
790
791 -need-bytes [+]NUMBER
792 Wait until at least NUMBER bytes have been received in total.
793 If '+' is prefixed, NUMBER new bytes must be received.
794
795 -expect-text LIN COL PAT
796 Wait for PAT to appear at LIN,COL on the virtual screen. Lines
797 and columns are numbered 1...N LIN==0 means "on any line" COL==0
798 means "anywhere on the line"
799
800 -match-text LIN COL PAT
801 Wait for the PAT regular expression to match the text at LIN,COL
802 on the virtual screen. Lines and columns are numbered 1...N
803 LIN==0 means "on any line" COL==0 means "anywhere on the line"
804
805 -close Alias for "-kill HUP"
806
807 -screen_dump
808 Dump the virtual screen into vtc_log
809
810 setenv
811 Set or change an environment variable:
812
813 setenv FOO "bar baz"
814
815 The above will set the environment variable $FOO to the value provided.
816 There is also an -ifunset argument which will only set the value if the
817 the environment variable does not already exist:
818
819 setenv -ifunset FOO quux
820
821 shell
822 NOTE: This command is available everywhere commands are given.
823
824 Pass the string given as argument to a shell. If you have multiple com‐
825 mands to run, you can use curly brackets to describe a multi-lines
826 script, eg:
827
828 shell {
829 echo begin
830 cat /etc/fstab
831 echo end
832 }
833
834 By default a zero exit code is expected, otherwise the vtc will fail.
835
836 Notice that the commandstring is prefixed with "exec 2>&1;" to combine
837 stderr and stdout back to the test process.
838
839 Optional arguments:
840
841 -err Expect non-zero exit code.
842
843 -exit N
844 Expect exit code N instead of zero.
845
846 -expect STRING
847 Expect string to be found in stdout+err.
848
849 -match REGEXP
850 Expect regexp to match the stdout+err output.
851
852 Specification
853 The specification of a stream follows the exact same rules as one for a
854 client or a server.
855
856 txreq, txresp, txcont, txpush
857 These four commands are about sending headers. txreq and txresp will
858 send HEADER frames; txcont will send CONTINUATION frames; txpush PUSH
859 frames. The only difference between txreq and txresp are the default
860 headers set by each of them.
861
862 -noadd Do not add default headers. Useful to avoid duplicates when
863 sending default headers using -hdr, -idxHdr and -litIdxHdr.
864
865 -status INT (txresp)
866 Set the :status pseudo-header.
867
868 -url STRING (txreq, txpush)
869 Set the :path pseudo-header.
870
871 -method STRING (txreq, txpush)
872 Set the :method pseudo-header.
873
874 -req STRING (txreq, txpush)
875 Alias for -method.
876
877 -scheme STRING (txreq, txpush)
878 Set the :scheme pseudo-header.
879
880 -hdr STRING1 STRING2
881 Insert a header, STRING1 being the name, and STRING2 the value.
882
883 -idxHdr INT
884 Insert an indexed header, using INT as index.
885
886 -litIdxHdr inc|not|never INT huf|plain STRING
887 Insert an literal, indexed header. The first argument specify if
888 the header should be added to the table, shouldn't, or mustn't
889 be compressed if/when retransmitted.
890
891 INT is the idex of the header name to use.
892
893 The third argument informs about the Huffman encoding: yes (huf)
894 or no (plain).
895
896 The last term is the literal value of the header.
897
898 -litHdr inc|not|never huf|plain STRING1 huf|plain STRING2
899 Insert a literal header, with the same first argument as
900 -litIdxHdr.
901
902 The second and third terms tell what the name of the header is
903 and if it should be Huffman-encoded, while the last two do the
904 same regarding the value.
905
906 -body STRING (txreq, txresp)
907 Specify a body, effectively putting STRING into a DATA frame af‐
908 ter the HEADER frame is sent.
909
910 -bodyfrom FILE (txreq, txresp)
911 Same as -body but content is read from FILE.
912
913 -bodylen INT (txreq, txresp)
914 Do the same thing as -body but generate a string of INT length
915 for you.
916
917 -gzipbody STRING (txreq, txresp)
918 Gzip STRING and send it as body.
919
920 -gziplen NUMBER (txreq, txresp)
921 Combine -bodylen and -gzipbody: generate a string of length NUM‐
922 BER, gzip it and send as body.
923
924 -nostrend (txreq, txresp)
925 Don't set the END_STREAM flag automatically, making the peer ex‐
926 pect a body after the headers.
927
928 -nohdrend
929 Don't set the END_HEADERS flag automatically, making the peer
930 expect more HEADER frames.
931
932 -dep INT (txreq, txresp)
933 Tell the peer that this content depends on the stream with the
934 INT id.
935
936 -ex (txreq, txresp)
937 Make the dependency exclusive (-dep is still needed).
938
939 -weight (txreq, txresp)
940 Set the weight for the dependency.
941
942 -promised INT (txpush)
943 The id of the promised stream.
944
945 -pad STRING / -padlen INT (txreq, txresp, txpush)
946 Add string as padding to the frame, either the one you provided
947 with -pad, or one that is generated for you, of length INT is
948 -padlen case.
949
950 txdata
951 By default, data frames are empty. The receiving end will know the
952 whole body has been delivered thanks to the END_STREAM flag set in the
953 last DATA frame, and txdata automatically set it.
954
955 -data STRING
956 Data to be embedded into the frame.
957
958 -datalen INT
959 Generate and INT-bytes long string to be sent in the frame.
960
961 -pad STRING / -padlen INT
962 Add string as padding to the frame, either the one you provided
963 with -pad, or one that is generated for you, of length INT is
964 -padlen case.
965
966 -nostrend
967 Don't set the END_STREAM flag, allowing to send more data on
968 this stream.
969
970 rxreq, rxresp
971 These are two convenience functions to receive headers and body of an
972 incoming request or response. The only difference is that rxreq can
973 only be by a server, and rxresp by a client.
974
975 rxhdrs
976 rxhdrs will expect one HEADER frame, then, depending on the arguments,
977 zero or more CONTINUATION frame.
978
979 -all Keep waiting for CONTINUATION frames until END_HEADERS flag is
980 seen.
981
982 -some INT
983 Retrieve INT - 1 CONTINUATION frames after the HEADER frame.
984
985 rxpush
986 This works like rxhdrs, expecting a PUSH frame and then zero or more
987 CONTINUATION frames.
988
989 -all Keep waiting for CONTINUATION frames until END_HEADERS flag is
990 seen.
991
992 -some INT
993 Retrieve INT - 1 CONTINUATION frames after the PUSH frame.
994
995 rxdata
996 Receiving data is done using the rxdata keywords and will retrieve one
997 DATA frame, if you wish to receive more, you can use these two conve‐
998 nience arguments:
999
1000 -all keep waiting for DATA frame until one sets the END_STREAM flag
1001
1002 -some INT
1003 retrieve INT DATA frames.
1004
1005 Receive a frame, any frame.
1006
1007 sendhex
1008 Push bytes directly on the wire. sendhex takes exactly one argument: a
1009 string describing the bytes, in hex notation, with possible whitespaces
1010 between them. Here's an example:
1011
1012 sendhex "00 00 08 00 0900 8d"
1013
1014 rxgoaway
1015 Receive a GOAWAY frame.
1016
1017 gunzip
1018 Same as the gunzip command for HTTP/1.
1019
1020 rxping
1021 Receive a PING frame.
1022
1023 txping
1024 Send PING frame.
1025
1026 -data STRING
1027 specify the payload of the frame, with STRING being an 8-char
1028 string.
1029
1030 -ack set the ACK flag.
1031
1032 rxprio
1033 Receive a PRIORITY frame.
1034
1035 txprio
1036 Send a PRIORITY frame
1037
1038 -stream INT
1039 indicate the id of the stream the sender stream depends on.
1040
1041 -ex the dependency should be made exclusive (only this streams de‐
1042 pends on the parent stream).
1043
1044 -weight INT
1045 an 8-bits integer is used to balance priority between streams
1046 depending on the same streams.
1047
1048 rxrst
1049 Receive a RST_STREAM frame.
1050
1051 txrst
1052 Send a RST_STREAM frame. By default, txrst will send a 0 error code
1053 (NO_ERROR).
1054
1055 -err STRING|INT
1056 Sets the error code to be sent. The argument can be an integer
1057 or a string describing the error, such as NO_ERROR, or CANCEL
1058 (see rfc7540#11.4 for more strings).
1059
1060 rxsettings
1061 Receive a SETTINGS frame.
1062
1063 txsettings
1064 SETTINGS frames must be acknowledge, arguments are as follow (most of
1065 them are from rfc7540#6.5.2):
1066
1067 -hdrtbl INT
1068 headers table size
1069
1070 -push BOOL
1071 whether push frames are accepted or not
1072
1073 -maxstreams INT
1074 maximum concurrent streams allowed
1075
1076 -winsize INT
1077 sender's initial window size
1078
1079 -framesize INT
1080 largest frame size authorized
1081
1082 -hdrsize INT
1083 maximum size of the header list authorized
1084
1085 -ack set the ack bit
1086
1087 rxwinup
1088 Receive a WINDOW_UPDATE frame.
1089
1090 txwinup
1091 Transmit a WINDOW_UPDATE frame, increasing the amount of credit of the
1092 connection (from stream 0) or of the stream (any other stream).
1093
1094 -size INT
1095 give INT credits to the peer.
1096
1097 write_body STRING
1098 Same as the write_body command for HTTP/1.
1099
1100 expect
1101 expect in stream works as it does in client or server, except that the
1102 elements compared will be different.
1103
1104 Most of these elements will be frame specific, meaning that the last
1105 frame received on that stream must of the correct type.
1106
1107 Here the list of keywords you can look at.
1108
1109 GOAWAY specific
1110 goaway.err
1111 The error code (as integer) of the GOAWAY frame.
1112
1113 goaway.laststream
1114 Last-Stream-ID
1115
1116 goaway.debug
1117 Debug data, if any.
1118
1119 PING specific
1120 ping.data
1121 The 8-bytes string of the PING frame payload.
1122
1123 ping.ack (PING)
1124 "true" if the ACK flag was set, "false" otherwise.
1125
1126 PRIORITY specific
1127 prio.stream
1128 The stream ID announced.
1129
1130 prio.exclusive
1131 "true" if the priority is exclusive, else "false".
1132
1133 prio.weight
1134 The dependency weight.
1135
1136 PUSH_PROMISE specific
1137 push.id
1138 The id of the promised stream.
1139
1140 RESET_STREAM specific
1141 rst.err
1142 The error code (as integer) of the RESET_STREAM frame.
1143
1144 SETTINGS specific
1145 settings.ack
1146 "true" if the ACK flag was set, else ""false.
1147
1148 settings.push
1149 "true" if the push settings was set to yes, "false" if set to
1150 no, and <undef> if not present.
1151
1152 settings.hdrtbl
1153 Value of HEADER_TABLE_SIZE if set, <undef> otherwise.
1154
1155 settings.maxstreams
1156 Value of MAX_CONCURRENT_STREAMS if set, <undef> otherwise.
1157
1158 settings.winsize
1159 Value of INITIAL_WINDOW_SIZE if set, <undef> otherwise.
1160
1161 setting.framesize
1162 Value of MAX_FRAME_SIZE if set, <undef> otherwise.
1163
1164 settings.hdrsize
1165 Value of MAX_HEADER_LIST_SIZE if set, <undef> otherwise.
1166
1167 WINDOW_UPDATE specific
1168 winup.size
1169 The size of the upgrade given by the WINDOW_UPDATE frame.
1170
1171 Generic frame
1172 frame.data
1173 Payload of the last frame
1174
1175 frame.type
1176 Type of the frame, as integer.
1177
1178 frame.size
1179 Size of the frame.
1180
1181 frame.stream
1182 Stream of the frame (correspond to the one you are executing
1183 this from, obviously).
1184
1185 frame.padding (for DATA, HEADERS, PUSH_PROMISE frames)
1186 Number of padded bytes.
1187
1188 Request and response
1189 Note: it's possible to inspect a request or response while it is still
1190 being construct (in-between two frames for example).
1191
1192 req.bodylen / resp.bodylen
1193 Length in bytes of the request/response so far.
1194
1195 req.body / resp.body
1196 Body of the request/response so far.
1197
1198 req.http.STRING / resp.http.STRING
1199 Value of the header STRING in the request/response.
1200
1201 req.status / resp.status
1202 :status pseudo-header's value.
1203
1204 req.url / resp.url
1205 :path pseudo-header's value.
1206
1207 req.method / resp.method
1208 :method pseudo-header's value.
1209
1210 req.authority / resp.authority
1211 :method pseudo-header's value.
1212
1213 req.scheme / resp.scheme
1214 :method pseudo-header's value.
1215
1216 Stream
1217 stream.window
1218 The current window size of the stream, or, if on stream 0, of
1219 the connection.
1220
1221 stream.weight
1222 Weight of the stream
1223
1224 stream.dependency
1225 Id of the stream this one depends on.
1226
1227 Index tables
1228 tbl.dec.size / tbl.enc.size
1229 Size (bytes) of the decoding/encoding table.
1230
1231 tbl.dec.size / tbl.enc.maxsize
1232 Maximum size (bytes) of the decoding/encoding table.
1233
1234 tbl.dec.length / tbl.enc.length
1235 Number of headers in decoding/encoding table.
1236
1237 tbl.dec[INT].key / tbl.enc[INT].key
1238 Name of the header at index INT of the decoding/encoding table.
1239
1240 tbl.dec[INT].value / tbl.enc[INT].value
1241 Value of the header at index INT of the decoding/encoding table.
1242
1243 syslog
1244 Define and interact with syslog instances (for use with haproxy)
1245
1246 To define a syslog server, you'll use this syntax:
1247
1248 syslog SNAME
1249
1250 Arguments:
1251
1252 SNAME Identify the syslog server with a string which must start with
1253 'S'.
1254
1255 -level STRING
1256 Set the default syslog priority level used by any subsequent
1257 "recv" command. Any syslog dgram with a different level will be
1258 skipped by "recv" command. This default level value may be su‐
1259 perseded by "recv" command if supplied as first argument: "recv
1260 <level>".
1261
1262 -start Start the syslog server thread in the background.
1263
1264 -repeat
1265
1266 Instead of processing the specification only once, do it
1267 NUMBER times.
1268
1269 -bind Bind the syslog socket to a local address.
1270
1271 -wait Wait for that thread to terminate.
1272
1273 -stop Stop the syslog server thread.
1274
1275 tunnel
1276 The goal of a tunnel is to help control the data transfer between two
1277 parties, for example to trigger socket timeouts in the middle of proto‐
1278 col frames, without the need to change how both parties are imple‐
1279 mented.
1280
1281 A tunnel accepts a connection and then connects on behalf of the source
1282 to the desired destination. Once both connections are established the
1283 tunnel will transfer bytes unchanged between the source and destina‐
1284 tion. Transfer can be interrupted, usually with the help of synchro‐
1285 nization methods like barriers. Once the transfer is paused, it is pos‐
1286 sible to let a specific amount of bytes move in either direction.
1287
1288 SECTION: tunnel.args Arguments
1289
1290 -start Start the tunnel in background, processing the last given speci‐
1291 fication.
1292
1293 -start+pause
1294 Start the tunnel, but already paused.
1295
1296 -wait Block until the thread finishes.
1297
1298 -listen STRING
1299 Dictate the listening socket for the server. STRING is of the
1300 form "IP PORT", or "HOST PORT".
1301
1302 Listens by defaults to a local random port.
1303
1304 -connect STRING
1305 Indicate the server to connect to. STRING is also of the form
1306 "IP PORT", or "HOST PORT".
1307
1308 Connects by default to a varnish instance called v1.
1309
1310 SECTION: tunnel.spec Specification
1311
1312 The specification contains a list of tunnel commands that can be com‐
1313 bined with barriers and delays. For example:
1314
1315 tunnel t1 {
1316 barrier b1 sync
1317 pause
1318 delay 1
1319 send 42
1320 barrier b2 sync
1321 resume
1322 } -start
1323
1324 If one end of the tunnel is closed before the end of the specification
1325 the test case will fail. A specification that ends in a paused state
1326 will implicitely resume the tunnel.
1327
1328 varnish
1329 Define and interact with varnish instances.
1330
1331 To define a Varnish server, you'll use this syntax:
1332
1333 varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING]
1334 [-errvcl STRING STRING] [-jail STRING] [-proto PROXY]
1335
1336 The first varnish vNAME invocation will start the varnishd master
1337 process in the background, waiting for the -start switch to actually
1338 start the child.
1339
1340 Types used in the description below:
1341
1342 PATTERN
1343 is a 'glob' style pattern (ie: fnmatch(3)) as used in shell
1344 filename expansion.
1345
1346 Arguments:
1347
1348 vNAME Identify the Varnish server with a string, it must starts with
1349 'v'.
1350
1351 -arg STRING
1352 Pass an argument to varnishd, for example "-h simple_list".
1353
1354 -vcl STRING
1355 Specify the VCL to load on this Varnish instance. You'll proba‐
1356 bly want to use multi-lines strings for this ({...}).
1357
1358 -vcl+backend STRING
1359 Do the exact same thing as -vcl, but adds the definition block
1360 of known backends (ie. already defined).
1361
1362 -errvcl STRING1 STRING2
1363 Load STRING2 as VCL, expecting it to fail, and Varnish to send
1364 an error string matching STRING1
1365
1366 -jail STRING
1367 Look at man varnishd (-j) for more information.
1368
1369 -proto PROXY
1370 Have Varnish use the proxy protocol. Note that PROXY here is the
1371 actual string.
1372
1373 You can decide to start the Varnish instance and/or wait for several
1374 events:
1375
1376 varnish vNAME [-start] [-wait] [-wait-running] [-wait-stopped]
1377
1378 -start Start the child process.
1379
1380 Once successfully started, the following macros are available
1381 for the default listen address: ${vNAME_addr}, ${vNAME_port} and
1382 ${vNAME_sock}. Additional macros are available, including the
1383 listen address name for each address vNAME listens to, like for
1384 example: ${vNAME_a0_addr}.
1385
1386 -stop Stop the child process.
1387
1388 -syntax
1389 Set the VCL syntax level for this command (default: 4.1)
1390
1391 -wait Wait for that instance to terminate.
1392
1393 -wait-running
1394 Wait for the Varnish child process to be started.
1395
1396 -wait-stopped
1397 Wait for the Varnish child process to stop.
1398
1399 -cleanup
1400 Once Varnish is stopped, clean everything after it. This is only
1401 used in very few tests and you should never need it.
1402
1403 -expectexit NUMBER
1404 Expect varnishd to exit(3) with this value
1405
1406 Once Varnish is started, you can talk to it (as you would through var‐
1407 nishadm) with these additional switches:
1408
1409 varnish vNAME [-cli STRING] [-cliok STRING] [-clierr STRING]
1410 [-clijson STRING]
1411
1412 -cli STRING|-cliok STRING|-clierr STATUS STRING|-cliexpect REGEXP
1413 STRING
1414 All four of these will send STRING to the CLI, the only differ‐
1415 ence is what they expect the result to be. -cli doesn't expect
1416 anything, -cliok expects 200, -clierr expects STATUS, and -cli‐
1417 expect expects the REGEXP to match the returned response.
1418
1419 -clijson STRING
1420 Send STRING to the CLI, expect success (CLIS_OK/200) and check
1421 that the response is parsable JSON.
1422
1423 It is also possible to interact with its shared memory (as you would
1424 through tools like varnishstat) with additional switches:
1425
1426 -expect !PATTERN|PATTERN OP NUMBER|PATTERN OP PATTERN
1427 Look into the VSM and make sure the first VSC counter identified
1428 by PATTERN has a correct value. OP can be ==, >, >=, <, <=. For
1429 example:
1430
1431 varnish v1 -expect SM?.s1.g_space > 1000000
1432 varnish v1 -expect cache_hit >= cache_hit_grace
1433
1434 In the ! form the test fails if a counter matches PATTERN.
1435
1436 The MAIN. namespace can be omitted from PATTERN.
1437
1438 The test takes up to 5 seconds before timing out.
1439
1440 -vsc PATTERN
1441 Dump VSC counters matching PATTERN.
1442
1443 -vsl_catchup
1444 Wait until the logging thread has idled to make sure that all
1445 the generated log is flushed
1446
1447 varnishtest
1448 Alternate name for 'vtest', see above.
1449
1450 vtest
1451 This should be the first command in your vtc as it will identify the
1452 test case with a short yet descriptive sentence. It takes exactly one
1453 argument, a string, eg:
1454
1455 vtest "Check that vtest is actually a valid command"
1456
1457 It will also print that string in the log.
1458
1460 This document has been written by Guillaume Quintard.
1461
1463 • varnishtest(1)
1464
1465 • vmod_vtc(3)
1466
1468 This document is licensed under the same licence as Varnish itself. See
1469 LICENCE for details.
1470
1471 • Copyright (c) 2006-2016 Varnish Software AS
1472
1473
1474
1475
1476 VTC(7)