1GABBI(1)                             Gabbi                            GABBI(1)
2
3
4

NAME

6       gabbi - Gabbi Documentation
7
8       Gabbi  tests  are  expressed  in YAML as a series of HTTP requests with
9       their expected response:
10
11          tests:
12             - name: retrieve root
13               GET: /
14               status: 200
15
16       This will trigger a GET request to / on the configured host.  The  test
17       will pass if the response's status code is 200.
18

TEST STRUCTURE

20       The  top-level tests category contains an ordered sequence of test dec‐
21       larations, each describing the expected response to a given request:
22
23   Metadata
24             ┌───────────────┬─────────────────────┬───────────────────┐
25             │Key            │ Description         │ Notes             │
26             ├───────────────┼─────────────────────┼───────────────────┤
27name           │ The  test's   name. │ required          
28             │               │ Must    be   unique │                   │
29             │               │ within a file.      │                   │
30             ├───────────────┼─────────────────────┼───────────────────┤
31desc           │ An arbitrary string │                   │
32             │               │ describing      the │                   │
33             │               │ test.               │                   │
34             ├───────────────┼─────────────────────┼───────────────────┤
35verbose        │ If  True   or   all │ defaults to False 
36             │               │ (synonymous),       │                   │
37             │               │ prints a  represen‐ │                   │
38             │               │ tation  of the cur‐ │                   │
39             │               │ rent  request   and │                   │
40             │               │ response to stdout, │                   │
41             │               │ including      both │                   │
42             │               │ headers  and  body. │                   │
43             │               │ If set  to  headers │                   │
44             │               │ or  body,  only the │                   │
45             │               │ corresponding  part │                   │
46             │               │ of  the request and │                   │
47             │               │ response  will   be │                   │
48             │               │ printed.   If   the │                   │
49             │               │ output  is  a  TTY, │                   │
50             │               │ colors    will   be │                   │
51             │               │ used. If  the  body │                   │
52             │               │ content-type     is │                   │
53             │               │ JSON  it  will   be │                   │
54             │               │ formatted       for │                   │
55             │               │ improved  readabil‐ │                   │
56             │               │ ity.  See  Verbose‐ │                   │
57             │               │ Http for details.   │                   │
58             ├───────────────┼─────────────────────┼───────────────────┤
59skip           │ A  string   message │ defaults to False 
60             │               │ which  if  set will │                   │
61             │               │ cause the  test  to │                   │
62             │               │ be skipped with the │                   │
63             │               │ provided message.   │                   │
64             └───────────────┴─────────────────────┴───────────────────┘
65
66
67xfail          │ Determines  whether │ defaults to False 
68             │               │ to expect this test │                   │
69             │               │ to fail. Note  that │                   │
70             │               │ the  test  will  be │                   │
71             │               │ run anyway.         │                   │
72             ├───────────────┼─────────────────────┼───────────────────┤
73use_prior_test │ Determines if  this │ defaults to True  
74             │               │ test will be run in │                   │
75             │               │ sequence    (after) │                   │
76             │               │ the  test  prior to │                   │
77             │               │ it in the  list  of │                   │
78             │               │ tests    within   a │                   │
79             │               │ file.  To  be  con‐ │                   │
80             │               │ crete, when this is │                   │
81             │               │ True  the  test  is │                   │
82             │               │ dependent   on  the │                   │
83             │               │ prior test  and  if │                   │
84             │               │ that  prior has not │                   │
85             │               │ yet run, it wil  be │                   │
86             │               │ run,  even  if only │                   │
87             │               │ the  current   test │                   │
88             │               │ has  been selected. │                   │
89             │               │ Set this  to  False │                   │
90             │               │ to  allow selecting │                   │
91             │               │ a   test    without │                   │
92             │               │ dependencies.       │                   │
93             ├───────────────┼─────────────────────┼───────────────────┤
94cert_validate  │ States  whether the │ defaults to True  
95             │               │ underlying     HTTP │                   │
96             │               │ client       should │                   │
97             │               │ attempt to validate │                   │
98             │               │ SSL   certificates. │                   │
99             │               │ In many test  envi‐ │                   │
100             │               │ ronment    certifi‐ │                   │
101             │               │ cates    will    be │                   │
102             │               │ self-signed      so │                   │
103             │               │ changing  this  may │                   │
104             │               │ be requried. It can │                   │
105             │               │ also   be   changed │                   │
106             │               │ when    loader   or │                   │
107             │               │ using gabbi-run.    │                   │
108             └───────────────┴─────────────────────┴───────────────────┘
109
110       NOTE:
111          When tests are generated dynamically, the TestCase name will include
112          the respective test's name, lowercased with spaces transformed to _.
113          In at least some test runners this will allow you to select and fil‐
114          ter on test name.
115
116   Request Parameters
117         ┌────────────────────┬───────────────────────┬─────────────────────┐
118         │Key                 │ Description           │ Notes               │
119         └────────────────────┴───────────────────────┴─────────────────────┘
120
121
122
123
124
125
126
127
128
129
130
131
132
133         │any       uppercase │ Any  such  key   is   │                     │
134         │string              │ considered  an HTTP   │                     │
135         │                    │ method,  with   the   │                     │
136         │                    │ corresponding value   │                     │
137         │                    │ expressing the URL.   │                     │
138         │                    │                       │                     │
139         │                    │ This is a  shortcut   │                     │
140         │                    │ combining    method   │                     │
141         │                    │ and url into a sin‐   │                     │
142         │                    │ gle statement:        │                     │
143         │                    │                       │                     │
144         │                    │           GET: /index │                     │
145         │                    │                       │                     │
146         │                    │        corresponds    │                     │
147         │                    │        to:            │                     │
148         │                    │                       │                     │
149         │                    │           method: GET │                     │
150         │                    │           url: /index │                     │
151         ├────────────────────┼───────────────────────┼─────────────────────┤
152method              │ The   HTTP    request │ defaults to GET     
153         │                    │ method.               │                     │
154         ├────────────────────┼───────────────────────┼─────────────────────┤
155url                 │ The  URL  to request. │ Either this or  the │
156         │                    │ This can either be  a │ shortcut  above  is │
157         │                    │ full    path    (e.g. │ required            
158         │                    │ "/index") or a  fully │                     │
159         │                    │ qualified  URL  (i.e. │                     │
160         │                    │ including  host   and │                     │
161         │                    │ scheme,    e.g.    "‐ │                     │
162         │                    │ http://exam          │                     │
163         │                    │ ple.org/index") — see │                     │
164         │                    │ host for details.     │                     │
165         ├────────────────────┼───────────────────────┼─────────────────────┤
166request_headers     │ A    dictionary    of │                     │
167         │                    │ key-value  pairs rep‐ │                     │
168         │                    │ resenting     request │                     │
169         │                    │ header names and val‐ │                     │
170         │                    │ ues.  These  will  be │                     │
171         │                    │ added   to  the  con‐ │                     │
172         │                    │ structed request.     │                     │
173         ├────────────────────┼───────────────────────┼─────────────────────┤
174query_parameters    │ A dictionary of query │                     │
175         │                    │ parameters  that will │                     │
176         │                    │ be added to  the  url │                     │
177         │                    │ as  query  string. If │                     │
178         │                    │ that URL already con‐ │                     │
179         │                    │ tains  a set of query │                     │
180         │                    │ parameters, those wil │                     │
181         │                    │ be    extended.   See │                     │
182         │                    │ example for a  demon‐ │                     │
183         │                    │ stration  of  how the │                     │
184         │                    │ data is structured.   │                     │
185         ├────────────────────┼───────────────────────┼─────────────────────┤
186data                │ A  representation  to │                     │
187         │                    │ pass as the body of a │                     │
188         │                    │ request.  Note   that │                     │
189         │                    │ content-type       in │                     │
190         │                    │ request_headers       │                     │
191         │                    │ should  also be set — │                     │
192         │                    │ see Data for details. │                     │
193         ├────────────────────┼───────────────────────┼─────────────────────┤
194redirects           │ If  True,   redirects │ defaults to False   
195         │                    │ will automatically be │                     │
196         │                    │ followed.             │                     │
197         └────────────────────┴───────────────────────┴─────────────────────┘
198
199ssl                 │ Determines    whether │ defaults to False   
200         │                    │ the  request uses SSL │                     │
201         │                    │ (i.e.  HTTPS).   Note │                     │
202         │                    │ that the url's scheme │                     │
203         │                    │ takes  precedence  if │                     │
204         │                    │ present  —  see  host │                     │
205         │                    │ for details.          │                     │
206         └────────────────────┴───────────────────────┴─────────────────────┘
207
208   Response Expectations
209            ┌────────────────────┬─────────────────────┬─────────────────┐
210            │Key                 │ Description         │ Notes           │
211            ├────────────────────┼─────────────────────┼─────────────────┤
212status              │ The        expected │ defaults to 200 
213            │                    │ response     status │                 │
214            │                    │ code.      Multiple │                 │
215            │                    │ acceptable response │                 │
216            │                    │ codes may  be  pro‐ │                 │
217            │                    │ vided, separated by │                 │
218            │                    │ || (e.g. 302 || 301 │                 │
219            │                    │ —   note,  however, │                 │
220            │                    │ that this indicates │                 │
221            │                    │ ambiguity, which is │                 │
222            │                    │ generally  undesir‐ │                 │
223            │                    │ able).              │                 │
224            ├────────────────────┼─────────────────────┼─────────────────┤
225response_headers    │ A   dictionary   of │                 │
226            │                    │ key-value     pairs │                 │
227            │                    │ representing        │                 │
228            │                    │ expected   response │                 │
229            │                    │ header   names  and │                 │
230            │                    │ values.    If     a │                 │
231            │                    │ header's  value  is │                 │
232            │                    │ wrapped  in  /.../, │                 │
233            │                    │ it  will be treated │                 │
234            │                    │ as    a     regular │                 │
235            │                    │ expression       to │                 │
236            │                    │ search for  in  the │                 │
237            │                    │ response header.    │                 │
238            ├────────────────────┼─────────────────────┼─────────────────┤
239response_forbid‐    │ A list  of  headers │                 │
240den_headers         │ which  must  not be │                 │
241            │                    │ present.            │                 │
242            ├────────────────────┼─────────────────────┼─────────────────┤
243response_strings    │ A  list  of  string │                 │
244            │                    │ fragments  expected │                 │
245            │                    │ to  be  present  in │                 │
246            │                    │ the response body.  │                 │
247            └────────────────────┴─────────────────────┴─────────────────┘
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265response_json_paths │ A   dictionary   of │                 │
266            │                    │ JSONPath      rules │                 │
267            │                    │ paired         with │                 │
268            │                    │ expected   matches. │                 │
269            │                    │ Using   this   rule │                 │
270            │                    │ requires  that  the │                 │
271            │                    │ content  being sent │                 │
272            │                    │ from the server  is │                 │
273            │                    │ JSON  (i.e.  a con‐ │                 │
274            │                    │ tent type of appli‐ │                 │
275            │                    │ cation/json or con‐ │                 │
276            │                    │ taining +json)      │                 │
277            │                    │                     │                 │
278            │                    │ If  the  value   is │                 │
279            │                    │ wrapped   in  /.../ │                 │
280            │                    │ the result  of  the │                 │
281            │                    │ JSONPath query will │                 │
282            │                    │ be searched for the │                 │
283            │                    │ value  as a regular │                 │
284            │                    │ expression.         │                 │
285            ├────────────────────┼─────────────────────┼─────────────────┤
286poll                │ A dictionary of two │                 │
287            │                    │ keys:               │                 │
288            │                    │                     │                 │
289            │                    │        · count:  An │                 │
290            │                    │          integer    │                 │
291            │                    │          stating    │                 │
292            │                    │          the number │                 │
293            │                    │          of   times │                 │
294            │                    │          to attempt │                 │
295            │                    │          this  test │                 │
296            │                    │          before     │                 │
297            │                    │          giving up. │                 │
298            │                    │                     │                 │
299            │                    │        · delay:   A │                 │
300            │                    │          floating   │                 │
301            │                    │          point num‐ │                 │
302            │                    │          ber     of │                 │
303            │                    │          seconds to │                 │
304            │                    │          delay      │                 │
305            │                    │          between    │                 │
306            │                    │          attempts.  │                 │
307            │                    │                     │                 │
308            │                    │        This   makes │                 │
309            │                    │        it  possible │                 │
310            │                    │        to poll  for │                 │
311            │                    │        a   resource │                 │
312            │                    │        created  via │                 │
313            │                    │        an asynchro‐ │                 │
314            │                    │        nous         │                 │
315            │                    │        request. Use │                 │
316            │                    │        with    cau‐ │                 │
317            │                    │        tion.        │                 │
318            └────────────────────┴─────────────────────┴─────────────────┘
319
320       Note that many of these items allow substitutions.
321
322       Default  values  for  a  file's tests may be provided via the top-level
323       defaults category. These  take  precedence  over  the  global  defaults
324       (explained below).
325
326       For examples see the gabbi tests, example and the gabbi-demo tutorial.
327

FIXTURES

329       The top-level fixtures category contains a sequence of named fixtures.
330

RESPONSE HANDLERS

332       response_*  keys are examples of Response Handlers. Custom handlers may
333       be created by test authors for specific use  cases.  See  handlers  for
334       more information.
335

SUBSTITUTION

337       There are a number of magical variables that can be used to make refer‐
338       ence to the state of a current test, the one just  prior  or  any  test
339       prior  to  the current one. The variables are replaced with real values
340       during test processing.
341
342   Global
343       · $ENVIRON['<environment variable>']: The name of an environment  vari‐
344         able.  Its  value  will  replace  the magical variable. If the string
345         value of the environment variable  is  "True"  or  "False"  then  the
346         resulting value will be the corresponding boolean, not a string.
347
348   Current Test
349       · $SCHEME: The current scheme/protocol (usually http or https).
350
351       · $NETLOC: The host and potentially port of the request.
352
353   Immediately Prior Test
354       · $COOKIE:  All  the cookies set by any Set-Cookie headers in the prior
355         response, including only the cookie key and value pairs and no  meta‐
356         data (e.g. expires or domain).
357
358       · $URL:  The URL defined in the prior request, after substitutions have
359         been  made.  For  backwards  compatibility  with   earlier   releases
360         $LAST_URL  may  also  be  used,  but if $HISTORY (see below) is being
361         used, $URL must be used.
362
363       · $LOCATION: The location header returned in the prior response.
364
365       · $HEADERS['<header>']:  The  value  of  any  header  from  the   prior
366         response.
367
368       · $RESPONSE['<json  path>']:  A JSONPath query into the prior response.
369         See jsonpath for more on formatting.
370
371   Any Previous Test
372       · $HISTORY['<test name>'].<magical variable expression>:  Any  variable
373         which refers to a prior test may be used in an expression that refers
374         to any earlier test in the same file by identifying the  target  test
375         by  its  name  in  a  $HISTORY dictionary. For example, to refer to a
376         value in a JSON object in the response of a test named post json:
377
378            $HISTORY['post json'].$RESPONSE['$.key']
379
380         This is a very powerful feature that could lead to test that are dif‐
381         ficult  for humans to read. Take care to optimize for the maintainers
382         that will come after you, not yourself.
383
384       NOTE:
385          Where a single-quote character, ', is shown in the  variables  above
386          you  may  also  use  a  double-quote  character, ", but in any given
387          expression the same character must be used at both ends.
388
389       All of these variables may be used in all of the following fields:
390
391       · url
392
393       · query_parameters
394
395       · data
396
397       · request_headers (in both the key and value)
398
399       · response_strings
400
401       · response_json_paths (in both the key and value, see json path substi‐
402         tution for more info)
403
404       · response_headers (in both the key and value)
405
406       · response_forbidden_headers
407
408       · count and delay fields of poll
409
410       With these variables it ought to be possible to traverse an API without
411       any explicit statements about the  URLs  being  used.  If  you  need  a
412       replacement  on a field that is not currently supported please raise an
413       issue or provide a patch.
414
415       As all of these features needed to be  tested  in  the  development  of
416       gabbi  itself,  the gabbi tests are a good source of examples on how to
417       use the functionality. See also example for a  collection  of  examples
418       and the gabbi-demo tutorial.
419

DATA

421       The data key has some special handing to allow for a bit more flexibil‐
422       ity when doing a POST or PUT:
423
424       · If the value is not a string (that is, it is a sequence or structure)
425         it  is  treated as a data structure that will be turned into a string
426         by the dumps method on the relevant content handler. For  example  if
427         the  content-type  of the body is application/json the data structure
428         will be turned into a JSON string.
429
430       · If the value is a string that begins with <@ then  the  rest  of  the
431         string is treated as a filepath to be loaded. The path is relative to
432         the test directory and may not traverse up into parent directories.
433
434       · If the value is an undecorated string, that's the value.
435
436       NOTE:
437          When reading from a file care should be taken to ensure that a  rea‐
438          sonable content-type is set for the data as this will control if any
439          encoding is done of the resulting string value. If it is text, json,
440          xml or javascript it will be encoded to UTF-8.
441
442       To  run  gabbi tests with a test harness they must be generated in some
443       fashion and then run. This is accomplished by a test loader.  Initially
444       gabbi only supported those test harnesses that supported the load_tests
445       protocol in UnitTest. It now possible to also build and run tests  with
446       pytest with some limitations described below.
447
448       NOTE:
449          It  is  also  possible to run gabbi tests from the command line. See
450          runner.
451
452       NOTE:
453          By default gabbi will load YAML files using the safe_load  function.
454          This  means  only basic YAML types are allowed in the file. For most
455          use cases this is fine. If you need custom types  (for  example,  to
456          match  NaN)  it  is  possible  to  set  the  safe_yaml  parameter of
457          build_tests() to False.  If custom types are used,  please  keep  in
458          mind  that this can limit the portability of the YAML files to other
459          contexts.
460
461       WARNING:
462          If test are being run with a runner that supports concurrency  (such
463          as  testrepository)  it is critical that the test runner is informed
464          of how to group the tests into their respective  suites.  The  usual
465          way  to  do this is to use a regular expression that groups based on
466          the name of the yaml files. For example, when  using  testrepository
467          the .testr.conf file needs an entry similar to the following:
468
469              group_regex=gabbi\.suitemaker\.(test_[^_]+_[^_]+)
470

UNITTEST STYLE LOADER

472       To  run the tests with a load_tests style loader a test file containing
473       a load_tests method is required. That will look a bit like:
474
475          """A sample test module."""
476
477          # For pathname munging
478          import os
479
480          # The module that build_tests comes from.
481          from gabbi import driver
482
483          # We need access to the WSGI application that hosts our service
484          from myapp import wsgiapp
485
486
487          # We're using fixtures in the YAML files, we need to know where to
488          # load them from.
489          from myapp.test import fixtures
490
491          # By convention the YAML files are put in a directory named
492          # "gabbits" that is in the same directory as the Python test file.
493          TESTS_DIR = 'gabbits'
494
495
496          def load_tests(loader, tests, pattern):
497              """Provide a TestSuite to the discovery process."""
498              test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
499              # Pass "require_ssl=True" as an argument to force all tests
500              # to use SSL in requests.
501              return driver.build_tests(test_dir, loader,
502                                        intercept=wsgiapp.app,
503                                        fixture_module=fixtures)
504
505
506       For  details  on  the  arguments  available  when  building  tests  see
507       build_tests().
508
509       Once  the  test  loader has been created, it needs to be run. There are
510       many options. Which is appropriate depends very much on  your  environ‐
511       ment.  Here  are some examples using unittest or testtools that require
512       minimal knowledge to get started.
513
514       By file:
515
516          python -m testtools.run -v test/test_loader.py
517
518       By module:
519
520          python -m testttols.run -v test.test_loader
521
522          python -m unittest -v test.test_loader
523
524       Using test discovery to locate all tests in a directory tree:
525
526          python -m testtools.run discover
527
528          python -m unittest discover test
529
530       See the source distribution and the tutorial  repo  for  more  advanced
531       options, including using testrepository and subunit.
532

PYTEST

534       Since pytest does not support the load_tests system, a different way of
535       generating tests is required. Two techniques are supported.
536
537       The original method (described below) used yield statements to generate
538       tests  which pytest would collect. This style of tests is deprecated as
539       of pytest>=3.0 so a new style using pytest fixtures has been developed.
540
541   pytest >= 3.0
542       In  the  newer  technique,  a  test  file  is  created  that  uses  the
543       pytest_generate_tests hook. Special care must be taken to always import
544       the test_pytest method which is the base  test  that  the  pytest  hook
545       parametrizes  to  generate  the tests from the YAML files.  Without the
546       method, the hook will not be called and no tests generated.
547
548       Here is a simple example file:
549
550          """A sample pytest module for pytest >= 3.0."""
551
552          # For pathname munging
553          import os
554
555          # The module that py_test_generator comes from.
556          from gabbi import driver
557
558          # We need test_pytest so that pytest test collection works properly.
559          # Without this, the pytest_generate_tests method below will not be
560          # called.
561          from gabbi.driver import test_pytest  # noqa
562
563          # We need access to the WSGI application that hosts our service
564          from myapp import wsgiapp
565
566          # We're using fixtures in the YAML files, we need to know where to
567          # load them from.
568          from myapp.test import fixtures
569
570          # By convention the YAML files are put in a directory named
571          # "gabbits" that is in the same directory as the Python test file.
572          TESTS_DIR = 'gabbits'
573
574
575          def pytest_generate_tests(metafunc):
576              test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
577              driver.py_test_generator(
578                  test_dir, intercept=wsgiapp.app,
579                  fixture_module=fixtures, metafunc=metafunc)
580
581
582       This can then be run with the usual pytest commands. For example:
583
584          py.test -svx pytest3.0-example.py
585
586   pytest < 3.0
587       When using the older technique, test file must be  created  that  calls
588       py_test_generator()  and  yields  the generated tests. That will look a
589       bit like this:
590
591          """A sample pytest module."""
592
593          # For pathname munging
594          import os
595
596          # The module that build_tests comes from.
597          from gabbi import driver
598
599          # We need access to the WSGI application that hosts our service
600          from myapp import wsgiapp
601
602          # We're using fixtures in the YAML files, we need to know where to
603          # load them from.
604          from myapp.test import fixtures
605
606          # By convention the YAML files are put in a directory named
607          # "gabbits" that is in the same directory as the Python test file.
608          TESTS_DIR = 'gabbits'
609
610
611          def test_gabbits():
612              test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
613              # Pass "require_ssl=True" as an argument to force all tests
614              # to use SSL in requests.
615              test_generator = driver.py_test_generator(
616                  test_dir, intercept=wsgiapp.app,
617                  fixture_module=fixtures)
618
619              for test in test_generator:
620                  yield test
621
622
623       This can then be run with the usual pytest commands. For example:
624
625          py.test -svx pytest-example.py
626
627       The older  technique  will  continue  to  work  with  all  versions  of
628       pytest<4.0  but  >=3.0  will  produce  warnings. If you want to use the
629       older technique but not see the warnings add  --disable-pytest-warnings
630       parameter to the invocation of py.test.
631
632       What  follows  is  a  commented  example of some tests in a single file
633       demonstrating many of the format features. See loader  for  the  Python
634       needed to integrate with a testing harness.
635
636
637          # Fixtures can be used to set any necessary configuration, such as a
638          # persistence layer, and establish sample data. They operate per
639          # file. They are context managers, each one wrapping the next in the
640          # sequence.
641
642          fixtures:
643              - ConfigFixture
644              - SampleDataFixture
645
646          # There is an included fixture named "SkipAllFixture" which can be
647          # used to declare that all the tests in the given file are to be
648          # skipped.
649
650          # Each test file can specify a set of defaults that will be used for
651          # every request. This is useful for always specifying a particular
652          # header or always requiring SSL. These values will be used on every
653          # test in the file unless overriden. Lists and dicts are merged one
654          # level deep, except for "data" which is copied verbatim whether it
655          # is a string, list or dict (it can be all three).
656
657          defaults:
658              ssl: True
659              request_headers:
660                  x-my-token: zoom
661
662          # The tests themselves are a list under a "tests" key. It's useful
663          # to use plenty of whitespace to help readability.
664
665          tests:
666
667          # Each request *must* have a name which is unique to the file. When it
668          # becomes a TestCase the name will be lowercased and spaces will
669          # become "_". Use that generated name when limiting test runs.
670
671              - name: a test for root
672                desc: Some explanatory text that could be used by other tooling
673
674          # The URL can either be relative to a host specified elsewhere or
675          # be a fully qualified "http" or "https" URL. *You* are responsible
676          # for url-encoding the URL.
677
678                url: /
679                method: GET
680
681          # If no status or method are provided they default to "200" and
682          # "GET".
683
684          # Instead of explicitly stating "url" and "method" you can join
685          # those two keys into one key representing the method. The method
686          # *must* be uppercase.
687
688              - name: another test for root
689                desc: Same test as above but with GET key
690                GET: /
691
692          # A single test can override settings in defaults (set above).
693
694              - name: root without ssl redirects
695                ssl: False
696                GET: /
697                status: 302
698
699          # When evaluating response headers it is possible to use a regular
700          # expression to not have to test the whole value. Regular expressions match
701          # anywhere in the output, not just at the beginning.
702
703                response_headers:
704                    location: /^https/
705
706          # By default redirects will not be followed. This can be changed.
707
708              - name: follow root without ssl redirect
709                ssl: False
710                redirects: True
711                GET: /
712                status: 200 # This is the response code after the redirect.
713
714          # URLs can express query parameters in two ways: either in the url
715          # value directly, or as query_parameters. If both are used then
716          # query_parameters are appended. In this example the resulting URL
717          # will be equivalient to
718          # /foo?section=news&article=1&article=2&date=yesterday
719          # but not necessarily in that order.
720
721              - name: create a url with parameters
722                GET: /foo?section=news
723                query_parameters:
724                    article:
725                        - 1
726                        - 2
727                    date: yesterday
728
729          # Request headers can be used to declare media-type choices and
730          # experiment with authorization handling (amongst other things).
731          # Response headers allow evaluating headers in the response. These
732          # two together form the core value of gabbi.
733
734              - name: test accept
735                GET: /resource
736                request_headers:
737                    accept: application/json
738                response_headers:
739                    content-type: /application/json/
740
741          # If a header must not be present in a response at all that can be
742          # expressed in a test as follows.
743
744              - name: test forbidden headers
745                GET: /resource
746                response_forbidden_headers:
747                    - x-special-header
748
749          # All of the above requests have defaulted to a "GET" method. When
750          # using "POST", "PUT" or "PATCH", the "data" key provides the
751          # request body.
752
753              - name: post some text
754                POST: /text_repo
755                request_headers:
756                    content-type: text/plain
757                data: "I'm storing this"
758                status: 201
759
760          # If the data is not a string, it will be transformed into JSON.
761          # You must supply an appropriate content-type request header.
762
763              - name: post some json
764                POST: /json_repo
765                request_headers:
766                    content-type: application/json
767                data:
768                    name: smith
769                    abode: castle
770                status: 201
771
772          # If the data is a string prepended with "<@" the value will be
773          # treated as the name of a file in the same directory as the YAML
774          # file. Again, you must supply an appropriate content-type. If the
775          # content-type is one of several "text-like" types, the content will
776          # be assumed to be UTF-8 encoded.
777
778              - name: post an image
779                POST: /image_repo
780                request_headers:
781                    content-type: image/png
782                data: <@kittens.png
783
784          # A single request can be marked to be skipped.
785
786              - name: patch an image
787                skip: patching images not yet implemented
788                PATCH: /image_repo/12d96fb8-e78c-11e4-8c03-685b35afa334
789
790          # Or a single request can be marked that it is expected to fail.
791
792              - name: check allow headers
793                desc: the framework doesn't do allow yet
794                xfail: True
795                PUT: /post_only_url
796                status: 405
797                response_headers:
798                    allow: POST
799
800          # The body of a response can be evaluated with response handlers.
801          # The most simple checks for a set of strings anywhere in the
802          # response. Note that the strings are members of a list.
803
804              - name: check for css file
805                GET: /blog/posts/12
806                response_strings:
807                    - normalize.css
808
809          # For JSON responses, JSONPath rules can be used.
810
811              - name: post some json get back json
812                POST: /json_repo
813                request_headers:
814                    content-type: application/json
815                data:
816                    name: smith
817                    abode: castle
818                status: 201
819                response_json_paths:
820                    $.name: smith
821                    $.abode: castle
822
823          # Requests run in sequence. One test can make reference to the test
824          # immediately prior using some special variables.
825          # "$LOCATION" contains the "location" header in the previous
826          # response.
827          # "$HEADERS" is a pseudo dictionary containing all the headers of
828          # the previous response.
829          # "$ENVIRON" is a pseudo dictionary providing access to the current
830          # environment.
831          # "$RESPONSE" provides access to the JSON in the prior response, via
832          # JSONPath. See http://jsonpath-rw.readthedocs.io/ for
833          # jsonpath-rw formatting.
834          # $SCHEME and $NETLOC provide access to the current protocol and
835          # location (host and port).
836
837              - name: get the thing we just posted
838                GET: $LOCATION
839                request_headers:
840                    x-magic-exchange: $HEADERS['x-magic-exchange']
841                    x-token: $ENVIRON['OS_TOKEN']
842                response_json_paths:
843                    $.name: $RESPONSE['$.name']
844                    $.abode: $RESPONSE['$.abode']
845                response_headers:
846                    content-location: /$SCHEME://$NETLOC/
847
848          # For APIs where resource creation is asynchronous it can be
849          # necessary to poll for the resulting resource. First we create the
850          # resource in one test. The next test uses the "poll" key to loop
851          # with a delay for a set number of times.
852
853              - name: create asynch
854                POST: /async_creator
855                request_headers:
856                    content-type: application/json
857                data:
858                    name: jones
859                    abode: bungalow
860                status: 202
861
862              - name: poll for created resource
863                GET: $LOCATION
864                poll:
865                    count: 10 # try up to ten times
866                    delay: .5 # wait .5 seconds between each try
867                response_json_paths:
868                    $.name: $RESPONSE['$.name']
869                    $.abode: $RESPONSE['$.abode']
870
871
872       Gabbi  supports  JSONPath  both for validating JSON response bodies and
873       within substitutions.
874
875       JSONPath expressions are provided by jsonpath_rw, with  jsonpath_rw_ext
876       custom extensions to address common requirements:
877
878       1. Sorting via sorted and [/property].
879
880       2. Filtering via [?property = value].
881
882       3. Returning the respective length via len.
883
884       (These apply both to arrays and key-value pairs.)
885
886       Here  is a JSONPath example demonstrating some of these features. Given
887       JSON data as follows:
888
889          {
890              "pets": [
891                  {"type": "cat", "sound": "meow"},
892                  {"type": "dog", "sound": "woof"}
893              ]
894          }
895
896       If the ordering of the list in pets is predictable and reliable  it  is
897       relatively straightforward to test values:
898
899          response_json_paths:
900              # length of list is two
901              $.pets.`len`: 2
902              # sound of second item in list is woof
903              $.pets[1].sound: woof
904
905       If the ordering is not predictable additional effort is required:
906
907          response_json_paths:
908              # sort by type
909              $.pets[/type][0].sound: meow
910              # sort by type, reversed
911              $.pets[\type][0].sound: woof
912              # all the sounds
913              $.pets[/type]..sound: ['meow', 'woof']
914              # filter by type = dog
915              $.pets[?type = "dog"].sound: woof
916
917       If  it is necessary to validate the entire JSON response use a JSONPath
918       of $:
919
920          response_json_paths:
921              $:
922                  pets:
923                      - type: cat
924                        sound: meow
925                      - type: dog
926                        sound: woof
927
928       This is not a technique that should be used frequently as it  can  lead
929       to  difficult to read tests and it also indicates that your gabbi tests
930       are being used to test your serializers and data models, not just  your
931       API interactions.
932
933       It  is  also possible to read raw JSON from disk for either all or some
934       of a JSON response:
935
936          response_json_paths:
937              $: @<data.json
938
939       or:
940
941          response_json_paths:
942              $.pets: <@pets.json
943              $.pets[0]: <@cat.json
944
945       Examples like this can be found in one of gabbi's own tests.
946
947       If it is desired to load YAML files  like  the  JSON  ones  above,  two
948       things must be done:
949
950       1. The YAMLDiskLoadingJSONHandler custom content handler must be passed
951          to the driver through the content_handlers argument. See  Extensions
952          on how to do this.
953
954       2. The  YAML  files to load must be placed in a subdirectory to prevent
955          the test runner from consuming them as test files to run:
956
957             response_json_paths:
958                 $: @<subdir/values.yaml
959
960       When reading from disk you can apply the same JSONPath by adding a  ':'
961       to  the  end  of  your file name. This allows you to store multiple API
962       responses into a single file to reduce file management when  construct‐
963       ing your tests.
964
965       Given JSON data as follows:
966
967          {
968              "values": [{
969                  "pets": [{
970                      "type": "cat",
971                      "sound": "meow"
972                  }, {
973                      "type": "dog",
974                      "sound": "woof"
975                  }]
976              }, {
977                  "people": [{
978                      "name": "chris",
979                      "id": 1
980                  }, {
981                      "name": "justin",
982                      "id": 2
983                  }]
984              }]
985          }
986
987       You can write your tests like the following:
988
989          response_json_paths:
990              $.pets: <@pets.json
991              $.pets[?type = "cat"].sound: <@values.json:$.values[0].pets[?type = "cat"].sound
992
993       Although placing more than one API response into a single JSON file may
994       seem convenient, keep in mind there is a tradeoff in  readability  that
995       should not be overlooked before implementing this technique.
996
997       Examples like this can be found in one of gabbi's yaml-from-disk tests.
998
999       There  are more JSONPath examples in example and in the jsonpath_rw and
1000       jsonpath_rw_ext documentation.
1001

SUBSTITUTION

1003       Substitutions can be made in both the left (query) and right (expected)
1004       hand  sides  of the json path expression. When subtitutions are used in
1005       the query, care must be taken to ensure proper quoting of the resulting
1006       value.   For   example   if   there   is   a  uuid  (with  hyphens)  at
1007       $RESPONSE['$.id'] then this expression may fail:
1008
1009          $.nested.structure.$RESPONSE['$.id'].name: foobar
1010
1011       as it will evaluate to something like:
1012
1013          $.nested.structure.ADC8AAFC-D564-40D1-9724-7680D3C010C2.name: foobar
1014
1015       which may be treated as an  arithemtic  expression  by  the  json  path
1016       parser. The test author should write:
1017
1018          $.nested.structure["$RESPONSE['$.id']"].name: foobar
1019
1020       to quote the result of the substitution.
1021
1022       The target host is the host on which the API to be tested can be found.
1023       Gabbi intends to preserve the flow and semantics of  HTTP  interactions
1024       as  much  as possible, and every HTTP request needs to be directed at a
1025       host of some form. Gabbi provides three ways to control this:
1026
1027       · Using wsgi-intercept to provide a fake socket and WSGI environment on
1028         an  arbitrary  host  and  port  attached  to  a WSGI application (see
1029         intercept examples).
1030
1031       · Using fully qualified url values in the YAML defined tests (see  full
1032         examples).
1033
1034       · Using  a  host  and (optionally) port defined at test build time (see
1035         live examples).
1036
1037       The intercept and live methods are mutually exclusive per test builder,
1038       but  either  kind of test can freely intermix fully qualified URLs into
1039       the sequence of tests in a YAML file.
1040
1041       For test driven development and local  tests  the  intercept  style  of
1042       testing  lowers test requirements (no web server required) and is fast.
1043       Interception is performed as part of fixtures processing  as  the  most
1044       deeply  nested fixture. This allows any configuration or database setup
1045       to be performed prior to the WSGI application being created.
1046
1047       For the implementation of the above see build_tests().
1048
1049       Each suite of tests is represented by  a  single  YAML  file,  and  may
1050       optionally  use  one or more fixtures to provide the necessary environ‐
1051       ment required by the tests in that file.
1052
1053       Fixtures are implemented as  nested  context  managers.  Subclasses  of
1054       GabbiFixture  must implement start_fixture and stop_fixture methods for
1055       creating and destroying, respectively, any  resources  managed  by  the
1056       fixture.   While  the  subclass  may choose to implement __init__ it is
1057       important that no exceptions are thrown in that method,  otherwise  the
1058       stack  of  context  managers will fail in unexpected ways. Instead ini‐
1059       tialization of real resources should happen in start_fixture.
1060
1061       At this time there is no mechanism for the individual tests to have any
1062       direct  awareness of the fixtures. The fixtures exist, conceptually, on
1063       the server side of the API being tested.
1064
1065       Fixtures may do whatever is required by the testing  environment,  how‐
1066       ever there are two common scenarios:
1067
1068       · Establishing  (and then resetting when a test suite has finished) any
1069         baseline configuration settings and persistence systems required  for
1070         the tests.
1071
1072       · Creating sample data for use by the tests.
1073
1074       If a fixture raises unittest.case.SkipTest during start_fixture all the
1075       tests in the current file will be skipped.  This makes it  possible  to
1076       skip  the  tests  if  some optional configuration (such as a particular
1077       type of database) is not available.
1078
1079       If an exception is raised while a fixture is  being  used,  information
1080       about the exception will be stored on the fixture so that the stop_fix‐
1081       ture method can decide if the exception should change how  the  fixture
1082       should  clean  up.  The exception information can be found on exc_type,
1083       exc_value and traceback method attributes.
1084
1085       If an exception is raised when a fixture is started (in  start_fixture)
1086       the  first  test  in the suite using the fixture will be marked with an
1087       error using the traceback from the exception and all the tests  in  the
1088       suite  will be skipped. This ensures that fixture failure is adequately
1089       captured and reported by test runners.
1090
1091       In some contexts (for example CI environments with a  large  number  of
1092       tests  being  run  in  a broadly concurrent environment where output is
1093       logged to a single file) it can be important to capture and consolidate
1094       stray  output  that is produced during the tests and display it associ‐
1095       ated with an individual test. This can help debugging and avoids  unus‐
1096       able output that is the result of multiple streams being interleaved.
1097
1098       Inner fixtures have been added to support this. These are fixtures more
1099       in line with the tradtional unittest concept of fixtures:  a  class  on
1100       which setUp and cleanUp is automatically called.
1101
1102       build_tests()  accepts  a  named parameter arguments of inner_fixtures.
1103       The value of that argument may be an ordered list  of  fixtures.Fixture
1104       classes that will be called when each individual test is set up.
1105
1106       An example fixture that could be useful is the FakeLogger.
1107
1108       NOTE:
1109          At  this time inner_fixtures are not supported when using the pytest
1110          loader.
1111
1112       Content handlers are responsible for preparing request data and  evalu‐
1113       ating  response  data  based  on  the  content-type  of the request and
1114       response. A content handler operates as follows:
1115
1116       · Structured YAML data provided via the data attribute is converted  to
1117         a string or bytes sequence and used as request body.
1118
1119       · The response body (a string or sequence of bytes) is transformed into
1120         a  content-type  dependent  structure  and  stored  in  an   internal
1121         attribute named response_data that is:
1122
1123         · used when evaluating the response body
1124
1125         · used in $RESPONSE[] substitutions
1126
1127       By  default,  gabbi provides content handlers for JSON. In that content
1128       handler the data test key is converted from structured YAML into a JSON
1129       string.  Response  bodies  are converted from a JSON string into a data
1130       structure   in   response_data   that   is   used    when    evaluating
1131       response_json_paths   entries   in   a  test  or  doing  JSONPath-based
1132       $RESPONSE[] substitutions.
1133
1134       A YAMLDiskLoadingJSONHandler has been added to extend the JSON handler.
1135       It  works  the  same way as the JSON handler except for when evaluating
1136       the response_json_paths handle, data that is  read  from  disk  can  be
1137       either  in  JSON  or YAML format. The YAMLDiskLoadingJSONHandler is not
1138       enabled by default and must be added as shown in the Extensions section
1139       in order to be used in the tests.
1140
1141       Further  content  handlers can be added as extensions. Test authors may
1142       need these extensions for their own suites, or enterprising  developers
1143       may wish to create and distribute extensions for others to use.
1144
1145       NOTE:
1146          One  extension that is likely to be useful is a content handler that
1147          turns data into url-encoded form data suitable for POST and turns an
1148          HTML response into a DOM object.
1149

EXTENSIONS

1151       Content  handlers  are  an evolution of the response handler concept in
1152       earlier versions gabbi. To preserve backwards compatibility with exist‐
1153       ing  response  handlers, old style response handlers are still allowed,
1154       but  new  handlers  should  implement  the  content  handler  interface
1155       (described below).
1156
1157       Registering additional custom handlers is done by passing a subclass of
1158       ContentHandler to build_tests():
1159
1160          driver.build_tests(test_dir, loader, host=None,
1161                             intercept=simple_wsgi.SimpleWsgi,
1162                             content_handlers=[MyContentHandler])
1163
1164       If pytest is being used:
1165
1166          driver.py_test_generator(test_dir, intercept=simple_wsgi.SimpleWsgi,
1167                                   content_handlers=[MyContenHandler])
1168
1169       Gabbi provides an additional custom handler named  YAMLDiskLoadingJSON‐
1170       Handler.   This  can  be used for loading JSON and YAML files from disk
1171       when evaluating the response_json_paths handle.
1172
1173       WARNING:
1174          YAMLDiskLoadingJSONHandler  shares  the  same  content-type  as  the
1175          default  JSONHandler.  When  there are multiple handlers listed that
1176          accept the same content-type, the one that is earliest in  the  list
1177          will be used.
1178
1179       With  gabbi-run,  custom handlers can be loaded via the --response-han‐
1180       dler option -- see load_response_handlers() for details.
1181
1182       NOTE:
1183          The use of the --response-handler argument is done to preserve back‐
1184          wards  compatibility  and  avoid excessive arguments.  Both types of
1185          handler may be passed to the argument.
1186
1187   Implementation Details
1188       Creating a content  handler  requires  subclassing  ContentHandler  and
1189       implementing  several  methods.  These methods are described below, but
1190       inspecting JSONHandler will be  instructive  in  highlighting  required
1191       arguments and techniques.
1192
1193       To  provide  a  response_<something> response-body evaluator a subclass
1194       must define:
1195
1196       · test_key_suffix: This, along with the prefix response_, forms the key
1197         used in the test structure. It is a class level string.
1198
1199       · test_key_value: The key's default value, either an empty list ([]) or
1200         empty dict ({}). It is a class level value.
1201
1202       · action: An instance method which tests the  expected  values  against
1203         the HTTP response - it is invoked for each entry, with the parameters
1204         depending on the default value.  The  arguments  to  action  are  (in
1205         order):
1206
1207         · self: The current instance.
1208
1209         · test: The currently active HTTPTestCase
1210
1211         · item:  The current entry if test_key_value is a list, otherwise the
1212           key half of the key/value pair at this entry.
1213
1214         · value: None if test_key_value is a list, otherwise the  value  half
1215           of the key/value pair at this entry.
1216
1217       To  translate  request  or response bodies to or from structured data a
1218       subclass must define an accepts method. This should return True if this
1219       class is willing to translate the provided content-type. During request
1220       processing it is given the value of the content-type header  that  will
1221       be  sent  in  the  request.  During response processing it is given the
1222       value of the content-type header of the response. This makes it  possi‐
1223       ble  to  handle  different request and response bodies in the same han‐
1224       dler,  if  desired.  For  example  a  handler  might  accept   applica‐
1225       tion/x-www-form-urlencoded and text/html.
1226
1227       If accepts is defined two additional static methods should be defined:
1228
1229       · dumps: Turn structured Python data from the data key in a test into a
1230         string or byte stream. The optional test param allows you  to  access
1231         the  current  test  case which may help with manipulations for custom
1232         content handlers, e.g. multipart/form-data needs to add a boundary to
1233         the  Content-Type header in order to mark the appropriate sections of
1234         the body.
1235
1236       · loads: Turn a string or byte stream in a response into a Python  data
1237         structure. Gabbi will put this data on the response_data attribute on
1238         the test, where it can be used in the evaluations described above (in
1239         the   action method) or in $RESPONSE handling.  An example usage here
1240         would be to turn HTML into a DOM.
1241
1242       · load_data_file: Load data from disk into  a  Python  data  structure.
1243         Gabbi  will  call  this  method when response_<something> contains an
1244         item where the right hand side value starts with <@. The  test  param
1245         allows   you   to  access  the  current  test  case  and  provides  a
1246         load_data_file method which should be used because  it  verifies  the
1247         data  is  loaded within the test diectory and returns the file source
1248         as a string. The load_data_file method was introduced to  re-use  the
1249         JSONHandler  in order to support loading YAML files from disk through
1250         the implementation of an additional custom handler, see YAMLDiskLoad‐
1251         ingJSONHandler for details.
1252
1253       Finally  if  a  replacer class method is defined, then when a $RESPONSE
1254       substitution is encountered, replacer will be passed the  response_data
1255       of the prior test and the argument within the $RESPONSE.
1256
1257       Please see the JSONHandler source for additional detail.
1258
1259       If  there is a running web service that needs to be tested and creating
1260       a test loader with build_tests() is either inconvenient or overkill  it
1261       is  possible to run YAML test files directly from the command line with
1262       the console-script gabbi-run. It accepts YAML on stdin or  as  multiple
1263       file  arguments,  and generates and runs tests and outputs a summary of
1264       the results.
1265
1266       The provided YAML may not use custom fixtures but  otherwise  uses  the
1267       default  format.  host  information is either expressed directly in the
1268       YAML file or provided on the command line:
1269
1270          gabbi-run [host[:port]] < /my/test.yaml
1271
1272       or:
1273
1274          gabbi-run http://host:port < /my/test.yaml
1275
1276       To test with one or more files the  following  command  syntax  may  be
1277       used:
1278
1279          gabbi-run http://host:port -- /my/test.yaml /my/other.yaml
1280
1281       NOTE:
1282          The  filename arguments must come after a -- and all other arguments
1283          (host, port, prefix, failfast) must come before the --.
1284
1285       NOTE:
1286          If files are provided, test output will use names including the name
1287          of  the  file. If any single file includes an error, the name of the
1288          file will be included in a summary of failed files at the end of the
1289          test report.
1290
1291       To facilitate using the same tests against the same application mounted
1292       in different locations in a WSGI server, a prefix may be provided as  a
1293       second argument:
1294
1295          gabbi-run host[:port] [prefix] < /my/test.yaml
1296
1297       or in the target URL:
1298
1299          gabbi-run http://host:port/prefix < /my/test.yaml
1300
1301       The  value of prefix will be prepended to the path portion of URLs that
1302       are not fully qualified.
1303
1304       Anywhere host is used, if it is a raw IPV6 address it should be wrapped
1305       in [ and ].
1306
1307       If  https  is  used  in the target, then the tests in the provided YAML
1308       will default to ssl: True.
1309
1310       Use -k or --insecure to not validate  certificates  when  making  https
1311       connections.
1312
1313       If  a  -x  or  --failfast argument is provided then gabbi-run will exit
1314       after the first test failure.
1315
1316       Use -v or --verbose with a value of all, headers or  body  to  turn  on
1317       verbosity for all tests being run.
1318
1319       Use -q or --quiet to silence test runner output.
1320
1321       These  are  informal release notes for gabbi since version 1.0.0, high‐
1322       lighting major features and changes. For more  detail  see  the  commit
1323       logs on GitHub.
1324

1.49.0

1326       · Add  support  for not validating certificates in https requests. Con‐
1327         trolled by  the  cert_validate  attribute  in  individual  tests  and
1328         build_tests() and the -k or --insecure argument to gabbi-run.
1329

1.48.0

1331       · Support pytest 5.0.0 in Python >=3.5. For earlier versions of Python,
1332         pytest<5.0.0 will be used; the pytest project is dropping support for
1333         older versions of Python.
1334

1.47.0

1336       · Use pytest<5.0.0 until gabbi has solutions for the changes in 5.0.0.
1337

1.46.0

1339       · A  -q argument is added to gabbi-run to suppress output from the test
1340         runner.
1341

1.45.0

1343       · Adjust loading of YAML to be ready for new release of PyYAML.
1344

1.44.0

1346       · Provide the YAMLDiskLoadingJSONHandler class that allows test  result
1347         data for response_json_path checks to be loaded from YAML-on-disk.
1348

1.43.0

1350       · Use    jsonpath    to   select   a   portion   of   data-on-disk   in
1351         response_json_path checks.
1352
1353       · Restrict PyYAML to <4.0.
1354

1.42.0

1356       · Allow listing of tests with no host configured. When host is an empty
1357         string,  tests  can be listed (for discovery), but will be skipped on
1358         run.
1359

1.41.0

1361       · JSON $RESPONSE substitutions in the data field may be  complex  types
1362         (lists and dicts), not solely strings.
1363

1.40.0

1365       · When  the  HTTP  response begins with a bad status line, have BadSta‐
1366         tusLine be raised from urllib3.
1367

1.39.0

1369       · Allow substitutions in the key portion of request and response  head‐
1370         ers, not just the value.
1371

1.38.0

1373       · Remove support for Python 3.3.
1374
1375       · Make handling of fixture-level skips in pytest actually work.
1376

1.37.0

1378       · Add safe_yaml parameter to build_tests().
1379

1.36.0

1381       · use_prior_test is added to test metadata.
1382
1383       · Extensive  cleanups  in regular expression handling when constructing
1384         tests from YAML.
1385

1.35.0

1387       jsonpath handling gets two improvements:
1388
1389       · The value side of a response_json_paths entry can be  loaded  from  a
1390         file using the <@file.json syntax also used in data.
1391
1392       · The  key  side  of a response_json_paths entry can use substitutions.
1393         This was already true for the value side.
1394

1.34.0

1396       Substitutions in $RESPONSE handling now preserve numeric types  instead
1397       of  casting  to  a  string.  This  is useful when servers are expecting
1398       strong types and tests want to send response data back to the server.
1399

1.33.0

1401       count     and     delay     test     keys     allow      substitutions.
1402       gabbi.driver.build_tests() accepts a verbose parameter to set test ver‐
1403       bosity for an entire session.
1404

1.32.0

1406       Better failure reporting when using gabbi-run with multiple files. Test
1407       names  are based on the files and a summary of failed files is provided
1408       at the end of the report.
1409

1.31.0

1411       Effectively capture a failure in a fixture and  report  the  traceback.
1412       Without  this some test runners swallow the error and discovering prob‐
1413       lems when developing fixtures can be quite challenging.
1414

1.30.0

1416       Thanks to Samuel Fekete, tests can use the $HISTORY dictionary to refer
1417       to any prior test in the same file, not just the one immediately prior,
1418       when doing substitutions.
1419

1.29.0

1421       Filenames used to read data into tests using the <@ syntax may now  use
1422       pathnames relative to the YAML file. See data.
1423
1424       gabbi-run  gains a --verbose parameter to force all tests run in a ses‐
1425       sion to run with verbose set.
1426
1427       When using pytest to load tests, a new  mechanism  is  available  which
1428       avoids warnings produced in when using a version of pytest greater than
1429       3.0.
1430

1.28.0

1432       When verbosely displaying request and response bodies  that  are  JSON,
1433       pretty print for improved readability.
1434

1.27.0

1436       Allow  gabbi-run to accept multiple filenames as command line arguments
1437       instead of reading tests from stdin.
1438

1.26.0

1440       Switch from response handlers to handlers to allow more  flexible  pro‐
1441       cessing of both response _and_ request bodies.
1442
1443       Add inner fixtures for per test fixtures, useful for output capturing.
1444

1.25.0

1446       Allow  the  test_loader_name arg to gabbi.driver.build_tests() to over‐
1447       ride the prefix of the pretty printed name of generated tests.
1448

1.24.0

1450       String values in JSONPath matches  may  be  wrapped  in  /.../`  to  be
1451       treated as regular expressions.
1452

1.23.0

1454       Better  documentation  of how to run gabbi in a concurrent environment.
1455       Improved handling of pytest fixtures and test counts.
1456

1.22.0

1458       Add url to gabbi.driver.build_tests() to use instead of host, port  and
1459       prefix.
1460

1.21.0

1462       Add require_ssl to gabbi.driver.build_tests() to force use of SSL.
1463

1.20.0

1465       Add $COOKIE substitution.
1466

1.19.1

1468       Correctly support IPV6 hosts.
1469

1.19.0

1471       Add $LAST_URL substitution.
1472

1.17.0

1474       Introduce support for loading and running tests with pytest.
1475

1.16.0

1477       Use urllib3 instead of httplib2 for driving HTTP requests.
1478

1.13.0

1480       Add sorting and filtering to jsonpath handling.
1481

1.11.0

1483       Add the response_forbidden_headers to response expectations.
1484

1.7.0

1486       Instead of:
1487
1488          tests:
1489          - name: a simple get
1490            url: /some/path
1491            method: get
1492
1493       1.7.0 also makes it possible to:
1494
1495          tests:
1496          - name: a simple get
1497            GET: /some/path
1498
1499       Any upper case key is treated as a method.
1500

1.4.0 AND 1.5.0

1502       Enhanced flexibility and colorization when setting tests to be verbose.
1503

1.3.0

1505       Adds the query_parameters key to request parameters.
1506

1.2.0

1508       The  start of improvements and extensions to jsonpath handling. In this
1509       case the addition of the len function.
1510

1.1.0

1512       Vastly improved output and behavior in gabbi-run.
1513

1.0.0

1515       Version 1 was the first release with a commitment to a  stable  format.
1516       Since then new fields have been added but have not been taken away.
1517
1518       The  following  people  have contributed code to gabbi. Thanks to them.
1519       Thanks also to all the people who have made gabbi better  by  reporting
1520       issues and their successes and failures with using gabbi.
1521
1522       · Chris Dent
1523
1524       · FND
1525
1526       · Mehdi Abaakouk
1527
1528       · Tom Viner
1529
1530       · Jason Myers
1531
1532       · Josh Leeb-du Toit
1533
1534       · Duc Truong
1535
1536       · Zane Bitter
1537
1538       · Ryan Spencer
1539
1540       · Kim Raymoure
1541
1542       · Travis Truman
1543
1544       · Samuel Fekete
1545
1546       · Michael McCune
1547
1548       · Imran Hayder
1549
1550       · Julien Danjou
1551
1552       · Trevor McCasland
1553
1554       · Danek Duvall
1555
1556       · Marc Abramowitz
1557
1558       NOTE:
1559          This  section  provides  a collection of questions with answers that
1560          don't otherwise fit in the rest of the documentation.  If  something
1561          is missing, please create an issue.
1562
1563          As this document grows it will gain a more refined structure.
1564

GENERAL

1566   Is gabbi only for testing Python-based APIs?
1567       No, you can use gabbi-run to test an HTTP service built in any program‐
1568       ming language.
1569
1570   How do I run just one test?
1571       Each YAML file contains a sequence of tests, each test within each file
1572       has  a name. That name is translated to the name of the test by replac‐
1573       ing spaces with an _.
1574
1575       When running tests that are generated dynamically, filtering  based  on
1576       the  test  name prior to the test being collected will not work in some
1577       test runners.  Test runners that use a --load-list functionality can be
1578       convinced to filter after discovery.
1579
1580       pytest does this directly with the -k keyword flag.
1581
1582       When  using  testrepository with tox as used in gabbi's own tests it is
1583       possible to pass a filter in the tox command:
1584
1585          tox -epy27 -- get_the_widget
1586
1587       When using testtools.run and similar test runners it's a bit more  com‐
1588       plicated.  It  is  necessary  to provide the full name of the test as a
1589       list to --load-list:
1590
1591          python -m testtools.run --load-list \
1592              <(echo package.tests.test_api.yamlfile_get_the_widge.test_request)
1593
1594   How do I run just one test, without running prior tests in a sequence?
1595       By default, when you select a single test to run, all  tests  prior  to
1596       that  one  in  a  file  will  be run as well: the file is treated as as
1597       sequence of dependent tests. If you do not want this you can adjust the
1598       use_prior_test test metadata in one of three ways:
1599
1600       · Set it in the YAML file for the one test you are concerned with.
1601
1602       · Set the defaults for all tests in that file.
1603
1604       · set use_prior_test to false when calling build_tests()
1605
1606       Be  aware  that  doing  this breaks a fundamental assumption that gabbi
1607       makes about how tests work. Any substitutions will fail.
1608

TESTING STYLE

1610   Can I have variables in my YAML file?
1611       Gabbi provides the $ENVIRON substitution which can operate a  bit  like
1612       variables  that are set elsewhere and then used in the tests defined by
1613       the YAML.
1614
1615       If you find it necessary to have variables within a  single  YAML  file
1616       you take advantage of YAML alias nodes list this:
1617
1618          vars:
1619            - &uuid_1 5613AABF-BAED-4BBA-887A-252B2D3543F8
1620
1621          tests:
1622          - name: send a uuid to a post
1623            POST: /resource
1624            request_headers:
1625              content-type: application/json
1626            data:
1627              uuid: *uuid_1
1628
1629       You  can alias all sorts of nodes, not just single items. Be aware that
1630       the replacement of an alias  node  happens  while  the  YAML  is  being
1631       loaded, before gabbi does any processing.
1632
1633   How many tests should be put in one YAML file?
1634       For  the  sake  of  readability it is best to keep each YAML file rela‐
1635       tively short. Since each YAML file represents a sequence  of  requests,
1636       it  usually  makes sense to create a new file when a test is not depen‐
1637       dent on any before it.
1638
1639       It's tempting to put all the tests for any resource or URL in the  same
1640       file, but this eventually leads to files that are too long and are thus
1641       difficult to read.
1642

CASE MODULE

1644       A single HTTP request represented as a subclass of testtools.TestCase
1645
1646       The test case encapsulates the request headers and  body  and  expected
1647       response headers and body. When the test is run an HTTP request is made
1648       using urllib3. Assertions are made against the response.
1649
1650       class gabbi.case.HTTPTestCase(*args, **kwargs)
1651              Bases: testtools.testcase.TestCase
1652
1653              Encapsulate a single HTTP request as a TestCase.
1654
1655              If the test is a member of a sequence of requests,  ensure  that
1656              prior tests are run.
1657
1658              To  keep  the  test harness happy we need to make sure the setUp
1659              and tearDown are only run once.
1660
1661              assert_in_or_print_output(expected, iterable)
1662                     Assert the iterable contains expected or print some  out‐
1663                     put.
1664
1665                     If   the   output  is  long,  it  is  limited  by  either
1666                     GABBI_MAX_CHARS_OUTPUT  in   the   environment   or   the
1667                     MAX_CHARS_OUTPUT constant.
1668
1669              base_test  =  {'cert_validate':  True,  'data':  '', 'desc': '',
1670              'method': 'GET', 'name': '', 'poll': {}, 'query_parameters': {},
1671              'redirects':  False,  'request_headers':  {}, 'skip': '', 'ssl':
1672              False, 'status': '200', 'url': '', 'use_prior_test': True, 'ver‐
1673              bose': False, 'xfail': False}
1674
1675              get_content_handler(content_type)
1676                     Determine the content handler for this media type.
1677
1678              load_data_file(filename)
1679                     Read a file from the current test directory.
1680
1681              replace_template(message, escape_regex=False)
1682                     Replace magic strings in message.
1683
1684              run(result=None)
1685                     Store the current result handler on this test.
1686
1687              setUp()
1688                     Hook  method for setting up the test fixture before exer‐
1689                     cising it.
1690
1691              tearDown()
1692                     Hook method for deconstructing  the  test  fixture  after
1693                     testing it.
1694
1695              test_request()
1696                     Run this request if it has not yet run.
1697
1698                     If there is a prior test in the sequence, run it first.
1699
1700       gabbi.case.potentialFailure(func)
1701              Decorate  a  test  method that is expected to fail if 'xfail' is
1702              true.
1703

DRIVER MODULE

SUITEMAKER MODULE

FIXTURE MODULE

1707       Manage fixtures for gabbi at the test suite level.
1708
1709       class gabbi.fixture.GabbiFixture
1710              Bases: object
1711
1712              A context manager that operates as a fixture.
1713
1714              Subclasses must implement start_fixture and  stop_fixture,  each
1715              of  which  contain  the logic for stopping and starting whatever
1716              the fixture is. What a fixture is is left as an exercise for the
1717              implementor.
1718
1719              These  context  managers will be nested so any actual work needs
1720              to happen in start_fixture and stop_fixture and not in __init__.
1721              Otherwise exception handling will not work properly.
1722
1723              start_fixture()
1724                     Implement  the  actual  workings  of starting the fixture
1725                     here.
1726
1727              stop_fixture()
1728                     Implement the actual workings  of  stopping  the  fixture
1729                     here.
1730
1731       exception gabbi.fixture.GabbiFixtureError
1732              Bases: Exception
1733
1734              Generic exception for GabbiFixture.
1735
1736       class gabbi.fixture.SkipAllFixture
1737              Bases: gabbi.fixture.GabbiFixture
1738
1739              A fixture that skips all the tests in the current suite.
1740
1741              start_fixture()
1742                     Implement  the  actual  workings  of starting the fixture
1743                     here.
1744
1745       gabbi.fixture.nest(fixtures)
1746              Nest a series of fixtures.
1747
1748              This is duplicated from nested in the  stdlib,  which  has  been
1749              deprecated  because  of issues with how exceptions are difficult
1750              to handle during __init__. Gabbi needs to nest an unknown number
1751              of fixtures dynamically, so the with syntax that replaces nested
1752              will not work.
1753

HANDLERS MODULE

1755       Package for response and content handlers that process the  body  of  a
1756       response in various ways.
1757
1758   handlers.base Module
1759       Base classes for response and content handlers.
1760
1761       class gabbi.handlers.base.ContentHandler
1762              Bases: gabbi.handlers.base.ResponseHandler
1763
1764              A subclass of ResponseHandlers that adds content handling.
1765
1766              static accepts(content_type)
1767                     Return True if this handler can handler this type.
1768
1769              static dumps(data, pretty=False, test=None)
1770                     Return structured data as a string.
1771
1772                     If pretty is true, prettify.
1773
1774              static load_data_file(test, file_path)
1775                     Return  the  string  content of the file specified by the
1776                     file_path.
1777
1778              static loads(data)
1779                     Create structured (Python) data from a stream.
1780
1781              classmethod replacer(response_data, path)
1782                     Return the string that is replacing RESPONSE.
1783
1784       class gabbi.handlers.base.ResponseHandler
1785              Bases: object
1786
1787              Add functionality for making assertions about an HTTP response.
1788
1789              A subclass may implement two methods: action and preprocess.
1790
1791              preprocess takes  one  argument,  the  TestCase.  It  is  called
1792              exactly once for each test before looping across the assertions.
1793              It is used, rarely, to copy the test.output into a  useful  form
1794              (such as a parsed DOM).
1795
1796              action takes two or three arguments. If test_key_value is a list
1797              action is called with the test case and a single list  item.  If
1798              test_key_value  is  a  dict  then action is called with the test
1799              case and a key and value pair.
1800
1801              action(test, item, value=None)
1802                     Test an individual entry for this response handler.
1803
1804                     If the entry is a key value pair the key is in  item  and
1805                     the  value  in value. Otherwise the entry is considered a
1806                     single item from a list.
1807
1808              preprocess(test)
1809                     Do any pre-single-test preprocessing.
1810
1811              test_key_suffix = ''
1812
1813              test_key_value = []
1814
1815   handlers.core Module
1816       Core response handlers.
1817
1818       class gabbi.handlers.core.ForbiddenHeadersResponseHandler
1819              Bases: gabbi.handlers.base.ResponseHandler
1820
1821              Test that listed headers are not in the response.
1822
1823              action(test, forbidden, value=None)
1824                     Test an individual entry for this response handler.
1825
1826                     If the entry is a key value pair the key is in  item  and
1827                     the  value  in value. Otherwise the entry is considered a
1828                     single item from a list.
1829
1830              test_key_suffix = 'forbidden_headers'
1831
1832              test_key_value = []
1833
1834       class gabbi.handlers.core.HeadersResponseHandler
1835              Bases: gabbi.handlers.base.ResponseHandler
1836
1837              Compare expected headers with actual headers.
1838
1839              If a header value is wrapped in / it is treated as a raw regular
1840              expression.
1841
1842              Headers values are always treated as strings.
1843
1844              action(test, header, value=None)
1845                     Test an individual entry for this response handler.
1846
1847                     If  the  entry is a key value pair the key is in item and
1848                     the value in value. Otherwise the entry is  considered  a
1849                     single item from a list.
1850
1851              test_key_suffix = 'headers'
1852
1853              test_key_value = {}
1854
1855       class gabbi.handlers.core.StringResponseHandler
1856              Bases: gabbi.handlers.base.ResponseHandler
1857
1858              Test for matching strings in the the response body.
1859
1860              action(test, expected, value=None)
1861                     Test an individual entry for this response handler.
1862
1863                     If  the  entry is a key value pair the key is in item and
1864                     the value in value. Otherwise the entry is  considered  a
1865                     single item from a list.
1866
1867              test_key_suffix = 'strings'
1868
1869              test_key_value = []
1870
1871   handlers.jsonhandler Module
1872       JSON-related content handling.
1873
1874       class gabbi.handlers.jsonhandler.JSONHandler
1875              Bases: gabbi.handlers.base.ContentHandler
1876
1877              A ContentHandler for JSON
1878
1879              · Structured  test  data  is  turned into JSON when request con‐
1880                tent-type is JSON.
1881
1882              · Response bodies that are JSON strings  are  made  into  Python
1883                data  on  the  test  response_data attribute when the response
1884                content-type is JSON.
1885
1886              · A response_json_paths response handler is added.
1887
1888              · JSONPaths in $RESPONSE substitutions are supported.
1889
1890              static accepts(content_type)
1891                     Return True if this handler can handler this type.
1892
1893              action(test, path, value=None)
1894                     Test json_paths against json data.
1895
1896              static dumps(data, pretty=False, test=None)
1897                     Return structured data as a string.
1898
1899                     If pretty is true, prettify.
1900
1901              static extract_json_path_value(data, path)
1902                     Extract the value at JSON Path path from the data.
1903
1904                     The input data is a  Python  datastructure,  not  a  JSON
1905                     string.
1906
1907              static load_data_file(test, file_path)
1908                     Return  the  string  content of the file specified by the
1909                     file_path.
1910
1911              static loads(data)
1912                     Create structured (Python) data from a stream.
1913
1914              classmethod replacer(response_data, match)
1915                     Return the string that is replacing RESPONSE.
1916
1917              test_key_suffix = 'json_paths'
1918
1919              test_key_value = {}
1920
1921   handlers.yaml_disk_loading_jsonhandler Module
1922       JSON-related content handling with YAML data disk loading.
1923
1924       class gabbi.handlers.yaml_disk_loading_jsonhandler.YAMLDiskLoadingJSON‐
1925       Handler
1926              Bases: gabbi.handlers.jsonhandler.JSONHandler
1927
1928              A ContentHandler for JSON responses that loads YAML from disk
1929
1930              · Structured  test  data  is  turned into JSON when request con‐
1931                tent-type is JSON.
1932
1933              · Response bodies that are JSON strings  are  made  into  Python
1934                data  on  the  test  response_data attribute when the response
1935                content-type is JSON.
1936
1937              · A response_json_paths response handler  is  added.  Data  read
1938                from   disk  during  this  handle  will  be  loaded  with  the
1939                yaml.safe_load method to  support  both  JSON  and  YAML  data
1940                sources from disk.
1941
1942              · JSONPaths in $RESPONSE substitutions are supported.
1943
1944              static load_data_file(test, file_path)
1945                     Return  the  string  content of the file specified by the
1946                     file_path.
1947

SUITE MODULE

1949       A TestSuite for containing gabbi tests.
1950
1951       This suite has two features: the contained tests are ordered and  there
1952       are suite-level fixtures that operate as context managers.
1953
1954       class gabbi.suite.GabbiSuite(tests=())
1955              Bases: unittest.suite.TestSuite
1956
1957              A TestSuite with fixtures.
1958
1959              The  suite wraps the tests with a set of nested context managers
1960              that operate as fixtures.
1961
1962              If a fixture raises unittest.case.SkipTest during setup, all the
1963              tests in this suite will be skipped.
1964
1965              run(result, debug=False)
1966                     Override TestSuite run to start suite-level fixtures.
1967
1968                     To  avoid  exception  confusion,  use a null Fixture when
1969                     there are no fixtures.
1970
1971              start(result, tests)
1972                     Start fixtures when using pytest.
1973
1974              stop() Stop fixtures when using pytest.
1975
1976       gabbi.suite.noop(*args)
1977              A noop method used to disable collected tests.
1978

RUNNER MODULE

REPORTER MODULE

1981       TestRunner and TestResult for gabbi-run.
1982
1983       class gabbi.reporter.ConciseTestResult(stream, descriptions, verbosity)
1984              Bases: unittest.runner.TextTestResult
1985
1986              A TextTestResult with simple but useful output.
1987
1988              If the output is a tty or GABBI_FORCE_COLOR is set in the  envi‐
1989              ronment, output will be colorized.
1990
1991              addError(test, err)
1992                     Called  when  an  error has occurred. 'err' is a tuple of
1993                     values as returned by sys.exc_info().
1994
1995              addExpectedFailure(test, err)
1996                     Called when an expected failure/error occurred.
1997
1998              addFailure(test, err)
1999                     Called when an error has occurred. 'err' is  a  tuple  of
2000                     values as returned by sys.exc_info().
2001
2002              addSkip(test, reason)
2003                     Called when a test is skipped.
2004
2005              addSuccess(test)
2006                     Called when a test has completed successfully
2007
2008              addUnexpectedSuccess(test)
2009                     Called when a test was expected to fail, but succeed.
2010
2011              getDescription(test)
2012
2013              printErrorList(flavor, errors)
2014
2015              startTest(test)
2016                     Called when the given test is about to be run
2017
2018       class  gabbi.reporter.ConciseTestRunner(stream=None, descriptions=True,
2019       verbosity=1,  failfast=False,  buffer=False,  resultclass=None,   warn‐
2020       ings=None, *, tb_locals=False)
2021              Bases: unittest.runner.TextTestRunner
2022
2023              A  TextTestRunner  that  uses  ConciseTestResult  for  reporting
2024              results.
2025
2026              resultclass
2027                     alias of ConciseTestResult
2028
2029       class gabbi.reporter.PyTestResult(stream=None, descriptions=None,  ver‐
2030       bosity=None)
2031              Bases: unittest.result.TestResult
2032
2033              Wrap a test result to allow it to work with pytest.
2034
2035              The main behaviors here are:
2036
2037              · to turn what had been exceptions back into exceptions
2038
2039              · use pytest's skip and xfail methods
2040
2041              addError(test, err)
2042                     Called  when  an  error has occurred. 'err' is a tuple of
2043                     values as returned by sys.exc_info().
2044
2045              addExpectedFailure(test, err)
2046                     Called when an expected failure/error occurred.
2047
2048              addFailure(test, err)
2049                     Called when an error has occurred. 'err' is  a  tuple  of
2050                     values as returned by sys.exc_info().
2051
2052              addSkip(test, reason)
2053                     Called when a test is skipped.
2054

UTILS MODULE

2056       Utility functions grab bag.
2057
2058       gabbi.utils.create_url(base_url, host, port=None, prefix='', ssl=False)
2059              Given pieces of a path-based url, return a fully qualified url.
2060
2061       gabbi.utils.decode_response_content(header_dict, content)
2062              Decode content to a proper string.
2063
2064       gabbi.utils.extract_content_type(header_dict,         default='applica‐
2065       tion/binary')
2066              Extract parsed content-type from headers.
2067
2068       gabbi.utils.get_colorizer(stream)
2069              Return a function to colorize a string.
2070
2071              Only if stream is a tty .
2072
2073       gabbi.utils.host_info_from_target(target, prefix=None)
2074              Turn url or host:port and target into test destination.
2075
2076       gabbi.utils.load_yaml(handle=None, yaml_file=None, safe=True)
2077              Read and parse any YAML file or filehandle.
2078
2079              Let exceptions flow where they may.
2080
2081              If no file or handle is provided, read from STDIN.
2082
2083       gabbi.utils.not_binary(content_type)
2084              Decide if something is content we'd like to treat as a string.
2085
2086       gabbi.utils.parse_content_type(content_type, default_charset='utf-8')
2087              Parse content type value for media type and charset.
2088

EXCEPTION MODULE

2090       Gabbi specific exceptions.
2091
2092       exception gabbi.exception.GabbiFormatError
2093              Bases: ValueError
2094
2095              An exception to encapsulate poorly formed test data.
2096
2097       exception gabbi.exception.GabbiSyntaxWarning
2098              Bases: SyntaxWarning
2099
2100              A warning about syntax that is not desirable.
2101

HTTPCLIENT MODULE

JSON_PARSER MODULE

2104       Keep one single global jsonpath parser.
2105
2106       gabbi.json_parser.parse(path)
2107              Parse a JSONPath expression use the global parser.
2108
2109       Gabbi is a tool for running HTTP tests where requests and responses are
2110       expressed as declarations in YAML files:
2111
2112          tests:
2113          - name: retrieve items
2114            GET: /items
2115
2116       See  the  rest  of these docs for more details on the many features and
2117       formats  for  setting  request  headers  and  bodies   and   evaluating
2118       responses.
2119
2120       Tests  can  be run from the command line with gabbi-run or programmati‐
2121       cally  using  either  py.test  or  unittest-style  test  runners.   See
2122       installation instructions below.
2123
2124       The  name  is  derived  from  "gabby": excessively talkative. In a test
2125       environment having visibility of what a test is  actually  doing  is  a
2126       good  thing. This is especially true when the goal of a test is to test
2127       the HTTP, not the testing infrastructure. Gabbi tries to put  the  HTTP
2128       interaction in the foreground of testing.
2129
2130       If you want to get straight to creating tests look at example, the test
2131       files in the source distribution and format.  A  gabbi-demo  repository
2132       provides a tutorial of using gabbi to build an API, via the commit his‐
2133       tory of the repo.
2134

PURPOSE

2136       Gabbi works to bridge the gap between human readable  YAML  files  (see
2137       format for details) that represent HTTP requests and expected responses
2138       and the rather complex world of automated testing.
2139
2140       Each YAML file represents an ordered list of HTTP requests  along  with
2141       the  expected  responses.  This  allows  a  single  file to represent a
2142       process in the API being tested. For example:
2143
2144       · Create a resource.
2145
2146       · Retrieve a resource.
2147
2148       · Delete a resource.
2149
2150       · Retrieve a resource again to confirm it is gone.
2151
2152       At the same time it is still possible to ask  gabbi  to  run  just  one
2153       request.  If  it  is in a sequence of tests, those tests prior to it in
2154       the YAML file will be run (in order). In any single  process  any  test
2155       will  only  be run once. Concurrency is handled such that one file runs
2156       in one process.
2157
2158       These features mean that it is possible to create tests that are useful
2159       for  both humans (as tools for learning, improving and developing APIs)
2160       and automated CI systems.
2161
2162       Significant flexibility and power is available in the format to make it
2163       relatively   straightforward  to  test  existing  complex  APIs.   This
2164       extended functionality includes the use of JSONPath to  query  response
2165       bodies  and  templating  of test data to allow access to the prior HTTP
2166       response in the current request. For APIs which do not use  JSON  addi‐
2167       tional handlers can be created.
2168
2169       Care  should be taken when using this functionality when you are creat‐
2170       ing a new API. If your API is so complex that  it  needs  complex  test
2171       files then you may wish to take that as a sign that your API itself too
2172       complex. One goal of gabbi is to encourage transparent and comprehensi‐
2173       ble APIs.
2174
2175       Though  gabbi  is  written in Python and under the covers uses unittest
2176       data structures and processes, there is no requirement that the host be
2177       a  Python-based  service. Anything talking HTTP can be tested. A runner
2178       makes it possible to simply create YAML files and point them at a  run‐
2179       ning server.
2180

INSTALLATION

2182       As a Python package, gabbi is typically installed via pip:
2183
2184          pip install gabbi
2185
2186       (both Python 2 and Python 3 are supported)
2187
2188       You might want to create a virtual environment; an isolated context for
2189       Python packages, keeping gabbi cleany separated from the rest  of  your
2190       system.
2191
2192       Python 3 comes with a built-in tool to create virtual environments:
2193
2194          python3 -m venv venv
2195          . venv/bin/activate
2196
2197          pip install gabbi
2198
2199       Alternatively, with Python 2 we can use virtualenv:
2200
2201          pip install virtualenv
2202          virtualenv venv
2203          . venv/bin/activate
2204
2205          pip install gabbi
2206
2207       This  way we can later use deactivate and safely remove the venv direc‐
2208       tory, thus erasing any trace of gabbi from the system.
2209

AUTHOR

2211       Chris Dent
2212
2213
2214
2215
2216                                 Feb 11, 2020                         GABBI(1)
Impressum