1GABBI(1)                             Gabbi                            GABBI(1)
2
3
4

NAME

6       gabbi - Gabbi Documentation
7
8       Gabbi  tests  are  expressed  in YAML as a series of HTTP requests with
9       their expected response:
10
11          tests:
12             - name: retrieve root
13               GET: /
14               status: 200
15
16       This will trigger a GET request to / on the configured host.  The  test
17       will pass if the response's status code is 200.
18

TEST STRUCTURE

20       The  top-level tests category contains an ordered sequence of test dec‐
21       larations, each describing the expected response to a given request:
22
23   Metadata
24             ┌───────────────┬─────────────────────┬───────────────────┐
25             │Key            │ Description         │ Notes             │
26             ├───────────────┼─────────────────────┼───────────────────┤
27name           │ The  test's   name. │ required          
28             │               │ Must    be   unique │                   │
29             │               │ within a file.      │                   │
30             ├───────────────┼─────────────────────┼───────────────────┤
31desc           │ An arbitrary string │                   │
32             │               │ describing      the │                   │
33             │               │ test.               │                   │
34             ├───────────────┼─────────────────────┼───────────────────┤
35verbose        │ If  True   or   all │ defaults to False 
36             │               │ (synonymous),       │                   │
37             │               │ prints a  represen‐ │                   │
38             │               │ tation  of the cur‐ │                   │
39             │               │ rent  request   and │                   │
40             │               │ response to stdout, │                   │
41             │               │ including      both │                   │
42             │               │ headers  and  body. │                   │
43             │               │ If set  to  headers │                   │
44             │               │ or  body,  only the │                   │
45             │               │ corresponding  part │                   │
46             │               │ of  the request and │                   │
47             │               │ response  will   be │                   │
48             │               │ printed.   If   the │                   │
49             │               │ output  is  a  TTY, │                   │
50             │               │ colors    will   be │                   │
51             │               │ used. If  the  body │                   │
52             │               │ content-type     is │                   │
53             │               │ JSON  it  will   be │                   │
54             │               │ formatted  for  im‐ │                   │
55             │               │ proved readability. │                   │
56             │               │ See VerboseHttp for │                   │
57             │               │ details.            │                   │
58             ├───────────────┼─────────────────────┼───────────────────┤
59skip           │ A  string   message │ defaults to False 
60             │               │ which  if  set will │                   │
61             │               │ cause the  test  to │                   │
62             │               │ be skipped with the │                   │
63             │               │ provided message.   │                   │
64             └───────────────┴─────────────────────┴───────────────────┘
65
66
67xfail          │ Determines  whether │ defaults to False 
68             │               │ to expect this test │                   │
69             │               │ to fail. Note  that │                   │
70             │               │ the  test  will  be │                   │
71             │               │ run anyway.         │                   │
72             ├───────────────┼─────────────────────┼───────────────────┤
73use_prior_test │ Determines if  this │ defaults to True  
74             │               │ test will be run in │                   │
75             │               │ sequence    (after) │                   │
76             │               │ the  test  prior to │                   │
77             │               │ it in the  list  of │                   │
78             │               │ tests    within   a │                   │
79             │               │ file.  To  be  con‐ │                   │
80             │               │ crete, when this is │                   │
81             │               │ True  the  test  is │                   │
82             │               │ dependent   on  the │                   │
83             │               │ prior test  and  if │                   │
84             │               │ that  prior has not │                   │
85             │               │ yet run, it wil  be │                   │
86             │               │ run,  even  if only │                   │
87             │               │ the  current   test │                   │
88             │               │ has  been selected. │                   │
89             │               │ Set this  to  False │                   │
90             │               │ to  allow selecting │                   │
91             │               │ a test without  de‐ │                   │
92             │               │ pendencies.         │                   │
93             ├───────────────┼─────────────────────┼───────────────────┤
94cert_validate  │ States  whether the │ defaults to True  
95             │               │ underlying     HTTP │                   │
96             │               │ client  should  at‐ │                   │
97             │               │ tempt  to  validate │                   │
98             │               │ SSL   certificates. │                   │
99             │               │ In many test  envi‐ │                   │
100             │               │ ronment    certifi‐ │                   │
101             │               │ cates    will    be │                   │
102             │               │ self-signed      so │                   │
103             │               │ changing  this  may │                   │
104             │               │ be requried. It can │                   │
105             │               │ also   be   changed │                   │
106             │               │ when  loader or us‐ │                   │
107             │               │ ing gabbi-run.      │                   │
108             └───────────────┴─────────────────────┴───────────────────┘
109
110       NOTE:
111          When tests are generated dynamically, the TestCase name will include
112          the respective test's name, lowercased with spaces transformed to _.
113          In at least some test runners this will allow you to select and fil‐
114          ter on test name.
115
116   Request Parameters
117         ┌────────────────────┬───────────────────────┬─────────────────────┐
118         │Key                 │ Description           │ Notes               │
119         └────────────────────┴───────────────────────┴─────────────────────┘
120
121
122
123
124
125
126
127
128
129
130
131
132
133         │any       uppercase │ Any  such  key   is   │                     │
134         │string              │ considered  an HTTP   │                     │
135         │                    │ method,  with   the   │                     │
136         │                    │ corresponding value   │                     │
137         │                    │ expressing the URL.   │                     │
138         │                    │                       │                     │
139         │                    │ This is a  shortcut   │                     │
140         │                    │ combining    method   │                     │
141         │                    │ and url into a sin‐   │                     │
142         │                    │ gle statement:        │                     │
143         │                    │                       │                     │
144         │                    │           GET: /index │                     │
145         │                    │                       │                     │
146         │                    │        corresponds    │                     │
147         │                    │        to:            │                     │
148         │                    │                       │                     │
149         │                    │           method: GET │                     │
150         │                    │           url: /index │                     │
151         ├────────────────────┼───────────────────────┼─────────────────────┤
152method              │ The   HTTP    request │ defaults to GET     
153         │                    │ method.               │                     │
154         ├────────────────────┼───────────────────────┼─────────────────────┤
155url                 │ The  URL  to request. │ Either this or  the │
156         │                    │ This can either be  a │ shortcut  above  is │
157         │                    │ full path (e.g. "/in‐ │ required            
158         │                    │ dex")  or   a   fully │                     │
159         │                    │ qualified  URL  (i.e. │                     │
160         │                    │ including  host   and │                     │
161         │                    │ scheme,    e.g.    "‐ │                     │
162         │                    │ http://exam          │                     │
163         │                    │ ple.org/index") — see │                     │
164         │                    │ host for details.     │                     │
165         ├────────────────────┼───────────────────────┼─────────────────────┤
166request_headers     │ A    dictionary    of │                     │
167         │                    │ key-value  pairs rep‐ │                     │
168         │                    │ resenting     request │                     │
169         │                    │ header names and val‐ │                     │
170         │                    │ ues.  These  will  be │                     │
171         │                    │ added   to  the  con‐ │                     │
172         │                    │ structed request.     │                     │
173         ├────────────────────┼───────────────────────┼─────────────────────┤
174query_parameters    │ A dictionary of query │                     │
175         │                    │ parameters  that will │                     │
176         │                    │ be added to  the  url │                     │
177         │                    │ as  query  string. If │                     │
178         │                    │ that URL already con‐ │                     │
179         │                    │ tains  a set of query │                     │
180         │                    │ parameters, those wil │                     │
181         │                    │ be  extended. See ex‐ │                     │
182         │                    │ ample  for  a  demon‐ │                     │
183         │                    │ stration  of  how the │                     │
184         │                    │ data is structured.   │                     │
185         ├────────────────────┼───────────────────────┼─────────────────────┤
186data                │ A  representation  to │                     │
187         │                    │ pass as the body of a │                     │
188         │                    │ request.  Note   that │                     │
189         │                    │ content-type  in  re‐ │                     │
190         │                    │ quest_headers  should │                     │
191         │                    │ also  be  set  —  see │                     │
192         │                    │ Data for details.     │                     │
193         ├────────────────────┼───────────────────────┼─────────────────────┤
194redirects           │ If  True,   redirects │ defaults to False   
195         │                    │ will automatically be │                     │
196         │                    │ followed.             │                     │
197         └────────────────────┴───────────────────────┴─────────────────────┘
198
199ssl                 │ Determines    whether │ defaults to False   
200         │                    │ the  request uses SSL │                     │
201         │                    │ (i.e.  HTTPS).   Note │                     │
202         │                    │ that the url's scheme │                     │
203         │                    │ takes  precedence  if │                     │
204         │                    │ present  —  see  host │                     │
205         │                    │ for details.          │                     │
206         └────────────────────┴───────────────────────┴─────────────────────┘
207
208   Response Expectations
209            ┌────────────────────┬─────────────────────┬─────────────────┐
210            │Key                 │ Description         │ Notes           │
211            ├────────────────────┼─────────────────────┼─────────────────┤
212status              │ The  expected   re‐ │ defaults to 200 
213            │                    │ sponse status code. │                 │
214            │                    │ Multiple acceptable │                 │
215            │                    │ response  codes may │                 │
216            │                    │ be provided,  sepa‐ │                 │
217            │                    │ rated  by  || (e.g. │                 │
218            │                    │ 302 || 301 —  note, │                 │
219            │                    │ however,  that this │                 │
220            │                    │ indicates   ambigu‐ │                 │
221            │                    │ ity,  which is gen‐ │                 │
222            │                    │ erally     undesir‐ │                 │
223            │                    │ able).              │                 │
224            ├────────────────────┼─────────────────────┼─────────────────┤
225response_headers    │ A   dictionary   of │                 │
226            │                    │ key-value     pairs │                 │
227            │                    │ representing    ex‐ │                 │
228            │                    │ pected     response │                 │
229            │                    │ header   names  and │                 │
230            │                    │ values.    If     a │                 │
231            │                    │ header's  value  is │                 │
232            │                    │ wrapped  in  /.../, │                 │
233            │                    │ it  will be treated │                 │
234            │                    │ as  a  regular  ex‐ │                 │
235            │                    │ pression  to search │                 │
236            │                    │ for in the response │                 │
237            │                    │ header.             │                 │
238            ├────────────────────┼─────────────────────┼─────────────────┤
239response_forbid‐    │ A list  of  headers │                 │
240den_headers         │ which  must  not be │                 │
241            │                    │ present.            │                 │
242            ├────────────────────┼─────────────────────┼─────────────────┤
243response_strings    │ A  list  of  string │                 │
244            │                    │ fragments  expected │                 │
245            │                    │ to  be  present  in │                 │
246            │                    │ the response body.  │                 │
247            └────────────────────┴─────────────────────┴─────────────────┘
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265response_json_paths │ A   dictionary   of │                 │
266            │                    │ JSONPath      rules │                 │
267            │                    │ paired   with   ex‐ │                 │
268            │                    │ pected matches. Us‐ │                 │
269            │                    │ ing  this  rule re‐ │                 │
270            │                    │ quires   that   the │                 │
271            │                    │ content  being sent │                 │
272            │                    │ from the server  is │                 │
273            │                    │ JSON  (i.e.  a con‐ │                 │
274            │                    │ tent type of appli‐ │                 │
275            │                    │ cation/json or con‐ │                 │
276            │                    │ taining +json)      │                 │
277            │                    │                     │                 │
278            │                    │ If  the  value   is │                 │
279            │                    │ wrapped   in  /.../ │                 │
280            │                    │ the result  of  the │                 │
281            │                    │ JSONPath query will │                 │
282            │                    │ be searched for the │                 │
283            │                    │ value  as a regular │                 │
284            │                    │ expression.         │                 │
285            ├────────────────────┼─────────────────────┼─────────────────┤
286poll                │ A dictionary of two │                 │
287            │                    │ keys:               │                 │
288            │                    │                     │                 │
289            │                    │        • count:  An │                 │
290            │                    │          integer    │                 │
291            │                    │          stating    │                 │
292            │                    │          the number │                 │
293            │                    │          of   times │                 │
294            │                    │          to attempt │                 │
295            │                    │          this  test │                 │
296            │                    │          before     │                 │
297            │                    │          giving up. │                 │
298            │                    │                     │                 │
299            │                    │        • delay:   A │                 │
300            │                    │          floating   │                 │
301            │                    │          point num‐ │                 │
302            │                    │          ber     of │                 │
303            │                    │          seconds to │                 │
304            │                    │          delay  be‐ │                 │
305            │                    │          tween  at‐ │                 │
306            │                    │          tempts.    │                 │
307            │                    │                     │                 │
308            │                    │        This   makes │                 │
309            │                    │        it  possible │                 │
310            │                    │        to poll  for │                 │
311            │                    │        a   resource │                 │
312            │                    │        created  via │                 │
313            │                    │        an asynchro‐ │                 │
314            │                    │        nous     re‐ │                 │
315            │                    │        quest.   Use │                 │
316            │                    │        with    cau‐ │                 │
317            │                    │        tion.        │                 │
318            └────────────────────┴─────────────────────┴─────────────────┘
319
320       Note that many of these items allow substitutions.
321
322       Default values for a file's tests may be provided via the top-level de‐
323       faults category. These take precedence over the  global  defaults  (ex‐
324       plained below).
325
326       For examples see the gabbi tests, example and the gabbi-demo tutorial.
327

FIXTURES

329       The top-level fixtures category contains a sequence of named fixtures.
330

RESPONSE HANDLERS

332       response_*  keys are examples of Response Handlers. Custom handlers may
333       be created by test authors for specific use  cases.  See  handlers  for
334       more information.
335

SUBSTITUTION

337       There are a number of magical variables that can be used to make refer‐
338       ence to the state of a current test, the one just  prior  or  any  test
339       prior  to  the current one. The variables are replaced with real values
340       during test processing.
341
342   Global
343$ENVIRON['<environment variable>']: The name of an environment  vari‐
344         able.  Its  value  will  replace  the magical variable. If the string
345         value of the environment variable is "True" or "False" then  the  re‐
346         sulting value will be the corresponding boolean, not a string.
347
348   Current Test
349$SCHEME: The current scheme/protocol (usually http or https).
350
351$NETLOC: The host and potentially port of the request.
352
353   Immediately Prior Test
354$COOKIE:  All  the cookies set by any Set-Cookie headers in the prior
355         response, including only the cookie key and value pairs and no  meta‐
356         data (e.g. expires or domain).
357
358$URL:  The URL defined in the prior request, after substitutions have
359         been  made.  For  backwards  compatibility  with   earlier   releases
360         $LAST_URL  may  also  be  used,  but if $HISTORY (see below) is being
361         used, $URL must be used.
362
363$LOCATION: The location header returned in the prior response.
364
365$HEADERS['<header>']: The value of any  header  from  the  prior  re‐
366         sponse.
367
368$RESPONSE['<json  path>']:  A JSONPath query into the prior response.
369         See jsonpath for more on formatting.
370
371   Any Previous Test
372$HISTORY['<test name>'].<magical variable expression>:  Any  variable
373         which refers to a prior test may be used in an expression that refers
374         to any earlier test in the same file by identifying the  target  test
375         by  its  name  in  a  $HISTORY dictionary. For example, to refer to a
376         value in a JSON object in the response of a test named post json:
377
378            $HISTORY['post json'].$RESPONSE['$.key']
379
380         This is a very powerful feature that could lead to test that are dif‐
381         ficult  for humans to read. Take care to optimize for the maintainers
382         that will come after you, not yourself.
383
384       NOTE:
385          Where a single-quote character, ', is shown in the  variables  above
386          you  may  also use a double-quote character, ", but in any given ex‐
387          pression the same character must be used at both ends.
388
389       All of these variables may be used in all of the following fields:
390
391skip
392
393url
394
395query_parameters
396
397data
398
399request_headers (in both the key and value)
400
401response_strings
402
403response_json_paths (in both the key and value, see json path substi‐
404         tution for more info)
405
406response_headers (in both the key and value)
407
408response_forbidden_headers
409
410count and delay fields of poll
411
412       With these variables it ought to be possible to traverse an API without
413       any explicit statements about the URLs being used. If you  need  a  re‐
414       placement  on  a  field that is not currently supported please raise an
415       issue or provide a patch.
416
417       As all of these features needed to be  tested  in  the  development  of
418       gabbi  itself,  the gabbi tests are a good source of examples on how to
419       use the functionality. See also example for a  collection  of  examples
420       and the gabbi-demo tutorial.
421

DATA

423       The data key has some special handing to allow for a bit more flexibil‐
424       ity when doing a POST or PUT:
425
426       • If the value is not a string (that is, it is a sequence or structure)
427         it  is  treated as a data structure that will be turned into a string
428         by the dumps method on the relevant content handler. For  example  if
429         the  content-type  of the body is application/json the data structure
430         will be turned into a JSON string.
431
432       • If the value is a string that begins with <@ then  the  rest  of  the
433         string is treated as a filepath to be loaded. The path is relative to
434         the test directory and may not traverse up into parent directories.
435
436       • If the value is an undecorated string, that's the value.
437
438       NOTE:
439          When reading from a file care should be taken to ensure that a  rea‐
440          sonable content-type is set for the data as this will control if any
441          encoding is done of the resulting string value. If it is text, json,
442          xml or javascript it will be encoded to UTF-8.
443
444       To  run  gabbi tests with a test harness they must be generated in some
445       fashion and then run. This is accomplished by a test loader.  Initially
446       gabbi only supported those test harnesses that supported the load_tests
447       protocol in UnitTest. It now possible to also build and run tests  with
448       pytest with some limitations described below.
449
450       NOTE:
451          It  is  also  possible to run gabbi tests from the command line. See
452          runner.
453
454       NOTE:
455          By default gabbi will load YAML files using the safe_load  function.
456          This  means  only basic YAML types are allowed in the file. For most
457          use cases this is fine. If you need custom types  (for  example,  to
458          match  NaN)  it  is  possible  to  set  the  safe_yaml  parameter of
459          build_tests() to False.  If custom types are used,  please  keep  in
460          mind  that this can limit the portability of the YAML files to other
461          contexts.
462
463       WARNING:
464          If test are being run with a runner that supports concurrency  (such
465          as  testrepository)  it is critical that the test runner is informed
466          of how to group the tests into their respective  suites.  The  usual
467          way  to  do this is to use a regular expression that groups based on
468          the name of the yaml files. For example, when  using  testrepository
469          the .testr.conf file needs an entry similar to the following:
470
471              group_regex=gabbi\.suitemaker\.(test_[^_]+_[^_]+)
472

UNITTEST STYLE LOADER

474       To  run the tests with a load_tests style loader a test file containing
475       a load_tests method is required. That will look a bit like:
476
477          """A sample test module."""
478
479          # For pathname munging
480          import os
481
482          # The module that build_tests comes from.
483          from gabbi import driver
484
485          # We need access to the WSGI application that hosts our service
486          from myapp import wsgiapp
487
488
489          # We're using fixtures in the YAML files, we need to know where to
490          # load them from.
491          from myapp.test import fixtures
492
493          # By convention the YAML files are put in a directory named
494          # "gabbits" that is in the same directory as the Python test file.
495          TESTS_DIR = 'gabbits'
496
497
498          def load_tests(loader, tests, pattern):
499              """Provide a TestSuite to the discovery process."""
500              test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
501              # Pass "require_ssl=True" as an argument to force all tests
502              # to use SSL in requests.
503              return driver.build_tests(test_dir, loader,
504                                        intercept=wsgiapp.app,
505                                        fixture_module=fixtures)
506
507
508       For  details  on  the  arguments  available  when  building  tests  see
509       build_tests().
510
511       Once  the  test  loader has been created, it needs to be run. There are
512       many options. Which is appropriate depends very much on  your  environ‐
513       ment.  Here  are some examples using unittest or testtools that require
514       minimal knowledge to get started.
515
516       By file:
517
518          python -m testtools.run -v test/test_loader.py
519
520       By module:
521
522          python -m testttols.run -v test.test_loader
523
524          python -m unittest -v test.test_loader
525
526       Using test discovery to locate all tests in a directory tree:
527
528          python -m testtools.run discover
529
530          python -m unittest discover test
531
532       See the source distribution and the tutorial repo for more advanced op‐
533       tions, including using testrepository and subunit.
534

PYTEST

536       Since pytest does not support the load_tests system, a different way of
537       generating tests is required. Two techniques are supported.
538
539       The original method (described below) used yield statements to generate
540       tests  which pytest would collect. This style of tests is deprecated as
541       of pytest>=3.0 so a new style using pytest fixtures has been developed.
542
543   pytest >= 3.0
544       In  the  newer  technique,  a  test  file  is  created  that  uses  the
545       pytest_generate_tests hook. Special care must be taken to always import
546       the test_pytest method which is the base  test  that  the  pytest  hook
547       parametrizes  to  generate  the tests from the YAML files.  Without the
548       method, the hook will not be called and no tests generated.
549
550       Here is a simple example file:
551
552          """A sample pytest module for pytest >= 3.0."""
553
554          # For pathname munging
555          import os
556
557          # The module that py_test_generator comes from.
558          from gabbi import driver
559
560          # We need test_pytest so that pytest test collection works properly.
561          # Without this, the pytest_generate_tests method below will not be
562          # called.
563          from gabbi.driver import test_pytest  # noqa
564
565          # We need access to the WSGI application that hosts our service
566          from myapp import wsgiapp
567
568          # We're using fixtures in the YAML files, we need to know where to
569          # load them from.
570          from myapp.test import fixtures
571
572          # By convention the YAML files are put in a directory named
573          # "gabbits" that is in the same directory as the Python test file.
574          TESTS_DIR = 'gabbits'
575
576
577          def pytest_generate_tests(metafunc):
578              test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
579              driver.py_test_generator(
580                  test_dir, intercept=wsgiapp.app,
581                  fixture_module=fixtures, metafunc=metafunc)
582
583
584       This can then be run with the usual pytest commands. For example:
585
586          py.test -svx pytest3.0-example.py
587
588   pytest < 3.0
589       When using the older technique, test file must be  created  that  calls
590       py_test_generator()  and  yields  the generated tests. That will look a
591       bit like this:
592
593          """A sample pytest module."""
594
595          # For pathname munging
596          import os
597
598          # The module that build_tests comes from.
599          from gabbi import driver
600
601          # We need access to the WSGI application that hosts our service
602          from myapp import wsgiapp
603
604          # We're using fixtures in the YAML files, we need to know where to
605          # load them from.
606          from myapp.test import fixtures
607
608          # By convention the YAML files are put in a directory named
609          # "gabbits" that is in the same directory as the Python test file.
610          TESTS_DIR = 'gabbits'
611
612
613          def test_gabbits():
614              test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
615              # Pass "require_ssl=True" as an argument to force all tests
616              # to use SSL in requests.
617              test_generator = driver.py_test_generator(
618                  test_dir, intercept=wsgiapp.app,
619                  fixture_module=fixtures)
620
621              for test in test_generator:
622                  yield test
623
624
625       This can then be run with the usual pytest commands. For example:
626
627          py.test -svx pytest-example.py
628
629       The older  technique  will  continue  to  work  with  all  versions  of
630       pytest<4.0  but  >=3.0  will  produce  warnings. If you want to use the
631       older technique but not see the warnings add  --disable-pytest-warnings
632       parameter to the invocation of py.test.
633
634       What  follows  is  a  commented  example of some tests in a single file
635       demonstrating many of the format features. See loader  for  the  Python
636       needed to integrate with a testing harness.
637
638
639          # Fixtures can be used to set any necessary configuration, such as a
640          # persistence layer, and establish sample data. They operate per
641          # file. They are context managers, each one wrapping the next in the
642          # sequence.
643
644          fixtures:
645              - ConfigFixture
646              - SampleDataFixture
647
648          # There is an included fixture named "SkipAllFixture" which can be
649          # used to declare that all the tests in the given file are to be
650          # skipped.
651
652          # Each test file can specify a set of defaults that will be used for
653          # every request. This is useful for always specifying a particular
654          # header or always requiring SSL. These values will be used on every
655          # test in the file unless overriden. Lists and dicts are merged one
656          # level deep, except for "data" which is copied verbatim whether it
657          # is a string, list or dict (it can be all three).
658
659          defaults:
660              ssl: True
661              request_headers:
662                  x-my-token: zoom
663
664          # The tests themselves are a list under a "tests" key. It's useful
665          # to use plenty of whitespace to help readability.
666
667          tests:
668
669          # Each request *must* have a name which is unique to the file. When it
670          # becomes a TestCase the name will be lowercased and spaces will
671          # become "_". Use that generated name when limiting test runs.
672
673              - name: a test for root
674                desc: Some explanatory text that could be used by other tooling
675
676          # The URL can either be relative to a host specified elsewhere or
677          # be a fully qualified "http" or "https" URL. *You* are responsible
678          # for url-encoding the URL.
679
680                url: /
681                method: GET
682
683          # If no status or method are provided they default to "200" and
684          # "GET".
685
686          # Instead of explicitly stating "url" and "method" you can join
687          # those two keys into one key representing the method. The method
688          # *must* be uppercase.
689
690              - name: another test for root
691                desc: Same test as above but with GET key
692                GET: /
693
694          # A single test can override settings in defaults (set above).
695
696              - name: root without ssl redirects
697                ssl: False
698                GET: /
699                status: 302
700
701          # When evaluating response headers it is possible to use a regular
702          # expression to not have to test the whole value. Regular expressions match
703          # anywhere in the output, not just at the beginning.
704
705                response_headers:
706                    location: /^https/
707
708          # By default redirects will not be followed. This can be changed.
709
710              - name: follow root without ssl redirect
711                ssl: False
712                redirects: True
713                GET: /
714                status: 200 # This is the response code after the redirect.
715
716          # URLs can express query parameters in two ways: either in the url
717          # value directly, or as query_parameters. If both are used then
718          # query_parameters are appended. In this example the resulting URL
719          # will be equivalient to
720          # /foo?section=news&article=1&article=2&date=yesterday
721          # but not necessarily in that order.
722
723              - name: create a url with parameters
724                GET: /foo?section=news
725                query_parameters:
726                    article:
727                        - 1
728                        - 2
729                    date: yesterday
730
731          # Request headers can be used to declare media-type choices and
732          # experiment with authorization handling (amongst other things).
733          # Response headers allow evaluating headers in the response. These
734          # two together form the core value of gabbi.
735
736              - name: test accept
737                GET: /resource
738                request_headers:
739                    accept: application/json
740                response_headers:
741                    content-type: /application/json/
742
743          # If a header must not be present in a response at all that can be
744          # expressed in a test as follows.
745
746              - name: test forbidden headers
747                GET: /resource
748                response_forbidden_headers:
749                    - x-special-header
750
751          # All of the above requests have defaulted to a "GET" method. When
752          # using "POST", "PUT" or "PATCH", the "data" key provides the
753          # request body.
754
755              - name: post some text
756                POST: /text_repo
757                request_headers:
758                    content-type: text/plain
759                data: "I'm storing this"
760                status: 201
761
762          # If the data is not a string, it will be transformed into JSON.
763          # You must supply an appropriate content-type request header.
764
765              - name: post some json
766                POST: /json_repo
767                request_headers:
768                    content-type: application/json
769                data:
770                    name: smith
771                    abode: castle
772                status: 201
773
774          # If the data is a string prepended with "<@" the value will be
775          # treated as the name of a file in the same directory as the YAML
776          # file. Again, you must supply an appropriate content-type. If the
777          # content-type is one of several "text-like" types, the content will
778          # be assumed to be UTF-8 encoded.
779
780              - name: post an image
781                POST: /image_repo
782                request_headers:
783                    content-type: image/png
784                data: <@kittens.png
785
786          # A single request can be marked to be skipped.
787
788              - name: patch an image
789                skip: patching images not yet implemented
790                PATCH: /image_repo/12d96fb8-e78c-11e4-8c03-685b35afa334
791
792          # Or a single request can be marked that it is expected to fail.
793
794              - name: check allow headers
795                desc: the framework doesn't do allow yet
796                xfail: True
797                PUT: /post_only_url
798                status: 405
799                response_headers:
800                    allow: POST
801
802          # The body of a response can be evaluated with response handlers.
803          # The most simple checks for a set of strings anywhere in the
804          # response. Note that the strings are members of a list.
805
806              - name: check for css file
807                GET: /blog/posts/12
808                response_strings:
809                    - normalize.css
810
811          # For JSON responses, JSONPath rules can be used.
812
813              - name: post some json get back json
814                POST: /json_repo
815                request_headers:
816                    content-type: application/json
817                data:
818                    name: smith
819                    abode: castle
820                status: 201
821                response_json_paths:
822                    $.name: smith
823                    $.abode: castle
824
825          # Requests run in sequence. One test can make reference to the test
826          # immediately prior using some special variables.
827          # "$LOCATION" contains the "location" header in the previous
828          # response.
829          # "$HEADERS" is a pseudo dictionary containing all the headers of
830          # the previous response.
831          # "$ENVIRON" is a pseudo dictionary providing access to the current
832          # environment.
833          # "$RESPONSE" provides access to the JSON in the prior response, via
834          # JSONPath. See http://jsonpath-rw.readthedocs.io/ for
835          # jsonpath-rw formatting.
836          # $SCHEME and $NETLOC provide access to the current protocol and
837          # location (host and port).
838
839              - name: get the thing we just posted
840                GET: $LOCATION
841                request_headers:
842                    x-magic-exchange: $HEADERS['x-magic-exchange']
843                    x-token: $ENVIRON['OS_TOKEN']
844                response_json_paths:
845                    $.name: $RESPONSE['$.name']
846                    $.abode: $RESPONSE['$.abode']
847                response_headers:
848                    content-location: /$SCHEME://$NETLOC/
849
850          # For APIs where resource creation is asynchronous it can be
851          # necessary to poll for the resulting resource. First we create the
852          # resource in one test. The next test uses the "poll" key to loop
853          # with a delay for a set number of times.
854
855              - name: create asynch
856                POST: /async_creator
857                request_headers:
858                    content-type: application/json
859                data:
860                    name: jones
861                    abode: bungalow
862                status: 202
863
864              - name: poll for created resource
865                GET: $LOCATION
866                poll:
867                    count: 10 # try up to ten times
868                    delay: .5 # wait .5 seconds between each try
869                response_json_paths:
870                    $.name: $RESPONSE['$.name']
871                    $.abode: $RESPONSE['$.abode']
872
873
874       Gabbi  supports  JSONPath  both for validating JSON response bodies and
875       within substitutions.
876
877       JSONPath expressions are provided by jsonpath_rw, with  jsonpath_rw_ext
878       custom extensions to address common requirements:
879
880       1. Sorting via sorted and [/property].
881
882       2. Filtering via [?property = value].
883
884       3. Returning the respective length via len.
885
886       (These apply both to arrays and key-value pairs.)
887
888       Here  is a JSONPath example demonstrating some of these features. Given
889       JSON data as follows:
890
891          {
892              "pets": [
893                  {"type": "cat", "sound": "meow"},
894                  {"type": "dog", "sound": "woof"}
895              ]
896          }
897
898       If the ordering of the list in pets is predictable and reliable  it  is
899       relatively straightforward to test values:
900
901          response_json_paths:
902              # length of list is two
903              $.pets.`len`: 2
904              # sound of second item in list is woof
905              $.pets[1].sound: woof
906
907       If the ordering is not predictable additional effort is required:
908
909          response_json_paths:
910              # sort by type
911              $.pets[/type][0].sound: meow
912              # sort by type, reversed
913              $.pets[\type][0].sound: woof
914              # all the sounds
915              $.pets[/type]..sound: ['meow', 'woof']
916              # filter by type = dog
917              $.pets[?type = "dog"].sound: woof
918
919       If  it is necessary to validate the entire JSON response use a JSONPath
920       of $:
921
922          response_json_paths:
923              $:
924                  pets:
925                      - type: cat
926                        sound: meow
927                      - type: dog
928                        sound: woof
929
930       This is not a technique that should be used frequently as it  can  lead
931       to  difficult to read tests and it also indicates that your gabbi tests
932       are being used to test your serializers and data models, not just  your
933       API interactions.
934
935       It  is  also possible to read raw JSON from disk for either all or some
936       of a JSON response:
937
938          response_json_paths:
939              $: @<data.json
940
941       or:
942
943          response_json_paths:
944              $.pets: <@pets.json
945              $.pets[0]: <@cat.json
946
947       Examples like this can be found in one of gabbi's own tests.
948
949       If it is desired to load YAML files  like  the  JSON  ones  above,  two
950       things must be done:
951
952       1. The YAMLDiskLoadingJSONHandler custom content handler must be passed
953          to the driver through the content_handlers argument. See  Extensions
954          on how to do this.
955
956       2. The  YAML  files to load must be placed in a subdirectory to prevent
957          the test runner from consuming them as test files to run:
958
959             response_json_paths:
960                 $: @<subdir/values.yaml
961
962       When reading from disk you can apply the same JSONPath by adding a  ':'
963       to the end of your file name. This allows you to store multiple API re‐
964       sponses into a single file to reduce file management when  constructing
965       your tests.
966
967       Given JSON data as follows:
968
969          {
970              "values": [{
971                  "pets": [{
972                      "type": "cat",
973                      "sound": "meow"
974                  }, {
975                      "type": "dog",
976                      "sound": "woof"
977                  }]
978              }, {
979                  "people": [{
980                      "name": "chris",
981                      "id": 1
982                  }, {
983                      "name": "justin",
984                      "id": 2
985                  }]
986              }]
987          }
988
989       You can write your tests like the following:
990
991          response_json_paths:
992              $.pets: <@pets.json
993              $.pets[?type = "cat"].sound: <@values.json:$.values[0].pets[?type = "cat"].sound
994
995       Although placing more than one API response into a single JSON file may
996       seem convenient, keep in mind there is a tradeoff in  readability  that
997       should not be overlooked before implementing this technique.
998
999       Examples like this can be found in one of gabbi's yaml-from-disk tests.
1000
1001       There  are more JSONPath examples in example and in the jsonpath_rw and
1002       jsonpath_rw_ext documentation.
1003

SUBSTITUTION

1005       Substitutions can be made in both the left (query) and right (expected)
1006       hand  sides  of the json path expression. When subtitutions are used in
1007       the query, care must be taken to ensure proper quoting of the resulting
1008       value.  For  example  if  there  is  a  uuid  (with  hyphens)  at  $RE‐
1009       SPONSE['$.id'] then this expression may fail:
1010
1011          $.nested.structure.$RESPONSE['$.id'].name: foobar
1012
1013       as it will evaluate to something like:
1014
1015          $.nested.structure.ADC8AAFC-D564-40D1-9724-7680D3C010C2.name: foobar
1016
1017       which may be treated as an  arithemtic  expression  by  the  json  path
1018       parser. The test author should write:
1019
1020          $.nested.structure["$RESPONSE['$.id']"].name: foobar
1021
1022       to quote the result of the substitution.
1023
1024       The target host is the host on which the API to be tested can be found.
1025       Gabbi intends to preserve the flow and semantics of  HTTP  interactions
1026       as  much  as possible, and every HTTP request needs to be directed at a
1027       host of some form. Gabbi provides three ways to control this:
1028
1029       • Using wsgi-intercept to provide a fake socket and WSGI environment on
1030         an  arbitrary  host  and  port  attached  to  a WSGI application (see
1031         intercept examples).
1032
1033       • Using fully qualified url values in the YAML defined tests (see  full
1034         examples).
1035
1036       • Using  a  host  and (optionally) port defined at test build time (see
1037         live examples).
1038
1039       The intercept and live methods are mutually exclusive per test builder,
1040       but  either  kind of test can freely intermix fully qualified URLs into
1041       the sequence of tests in a YAML file.
1042
1043       For test driven development and local  tests  the  intercept  style  of
1044       testing  lowers test requirements (no web server required) and is fast.
1045       Interception is performed as part of fixtures processing  as  the  most
1046       deeply  nested fixture. This allows any configuration or database setup
1047       to be performed prior to the WSGI application being created.
1048
1049       For the implementation of the above see build_tests().
1050
1051       Each suite of tests is represented by a single YAML file, and  may  op‐
1052       tionally  use one or more fixtures to provide the necessary environment
1053       required by the tests in that file.
1054
1055       Fixtures are implemented as  nested  context  managers.  Subclasses  of
1056       GabbiFixture  must implement start_fixture and stop_fixture methods for
1057       creating and destroying, respectively, any  resources  managed  by  the
1058       fixture.  While the subclass may choose to implement __init__ it is im‐
1059       portant that no exceptions are thrown in  that  method,  otherwise  the
1060       stack  of  context  managers will fail in unexpected ways. Instead ini‐
1061       tialization of real resources should happen in start_fixture.
1062
1063       At this time there is no mechanism for the individual tests to have any
1064       direct  awareness of the fixtures. The fixtures exist, conceptually, on
1065       the server side of the API being tested.
1066
1067       Fixtures may do whatever is required by the testing  environment,  how‐
1068       ever there are two common scenarios:
1069
1070       • Establishing  (and then resetting when a test suite has finished) any
1071         baseline configuration settings and persistence systems required  for
1072         the tests.
1073
1074       • Creating sample data for use by the tests.
1075
1076       If a fixture raises unittest.case.SkipTest during start_fixture all the
1077       tests in the current file will be skipped.  This makes it  possible  to
1078       skip  the  tests  if  some optional configuration (such as a particular
1079       type of database) is not available.
1080
1081       If an exception is raised while a fixture is  being  used,  information
1082       about the exception will be stored on the fixture so that the stop_fix‐
1083       ture method can decide if the exception should change how  the  fixture
1084       should  clean  up.  The exception information can be found on exc_type,
1085       exc_value and traceback method attributes.
1086
1087       If an exception is raised when a fixture is started (in  start_fixture)
1088       the  first  test  in the suite using the fixture will be marked with an
1089       error using the traceback from the exception and all the tests  in  the
1090       suite  will be skipped. This ensures that fixture failure is adequately
1091       captured and reported by test runners.
1092
1093       In some contexts (for example CI environments with a  large  number  of
1094       tests  being  run  in  a broadly concurrent environment where output is
1095       logged to a single file) it can be important to capture and consolidate
1096       stray  output  that is produced during the tests and display it associ‐
1097       ated with an individual test. This can help debugging and avoids  unus‐
1098       able output that is the result of multiple streams being interleaved.
1099
1100       Inner fixtures have been added to support this. These are fixtures more
1101       in line with the tradtional unittest concept of fixtures:  a  class  on
1102       which setUp and cleanUp is automatically called.
1103
1104       build_tests()  accepts  a  named parameter arguments of inner_fixtures.
1105       The value of that argument may be an ordered list  of  fixtures.Fixture
1106       classes that will be called when each individual test is set up.
1107
1108       An example fixture that could be useful is the FakeLogger.
1109
1110       NOTE:
1111          At  this time inner_fixtures are not supported when using the pytest
1112          loader.
1113
1114       Content handlers are responsible for preparing request data and  evalu‐
1115       ating  response  data  based on the content-type of the request and re‐
1116       sponse. A content handler operates as follows:
1117
1118       • Structured YAML data provided via the data attribute is converted  to
1119         a string or bytes sequence and used as request body.
1120
1121       • The response body (a string or sequence of bytes) is transformed into
1122         a content-type dependent structure and stored in an  internal  attri‐
1123         bute named response_data that is:
1124
1125         • used when evaluating the response body
1126
1127         • used in $RESPONSE[] substitutions
1128
1129       By  default,  gabbi provides content handlers for JSON. In that content
1130       handler the data test key is converted from structured YAML into a JSON
1131       string.  Response  bodies  are converted from a JSON string into a data
1132       structure  in  response_data  that  is   used   when   evaluating   re‐
1133       sponse_json_paths entries in a test or doing JSONPath-based $RESPONSE[]
1134       substitutions.
1135
1136       A YAMLDiskLoadingJSONHandler has been added to extend the JSON handler.
1137       It  works  the  same way as the JSON handler except for when evaluating
1138       the response_json_paths handle, data that is read from disk can be  ei‐
1139       ther  in JSON or YAML format. The YAMLDiskLoadingJSONHandler is not en‐
1140       abled by default and must be added as shown in the  Extensions  section
1141       in order to be used in the tests.
1142
1143       Further  content  handlers can be added as extensions. Test authors may
1144       need these extensions for their own suites, or enterprising  developers
1145       may wish to create and distribute extensions for others to use.
1146
1147       NOTE:
1148          One  extension that is likely to be useful is a content handler that
1149          turns data into url-encoded form data suitable for POST and turns an
1150          HTML response into a DOM object.
1151

EXTENSIONS

1153       Content  handlers  are  an evolution of the response handler concept in
1154       earlier versions gabbi. To preserve backwards compatibility with exist‐
1155       ing  response  handlers, old style response handlers are still allowed,
1156       but new handlers should implement the content  handler  interface  (de‐
1157       scribed below).
1158
1159       Registering additional custom handlers is done by passing a subclass of
1160       ContentHandler to build_tests():
1161
1162          driver.build_tests(test_dir, loader, host=None,
1163                             intercept=simple_wsgi.SimpleWsgi,
1164                             content_handlers=[MyContentHandler])
1165
1166       If pytest is being used:
1167
1168          driver.py_test_generator(test_dir, intercept=simple_wsgi.SimpleWsgi,
1169                                   content_handlers=[MyContenHandler])
1170
1171       Gabbi provides an additional custom handler named  YAMLDiskLoadingJSON‐
1172       Handler.   This  can  be used for loading JSON and YAML files from disk
1173       when evaluating the response_json_paths handle.
1174
1175       WARNING:
1176          YAMLDiskLoadingJSONHandler shares the same content-type as  the  de‐
1177          fault  JSONHandler. When there are multiple handlers listed that ac‐
1178          cept the same content-type, the one that is  earliest  in  the  list
1179          will be used.
1180
1181       With  gabbi-run,  custom handlers can be loaded via the --response-han‐
1182       dler option -- see load_response_handlers() for details.
1183
1184       NOTE:
1185          The use of the --response-handler argument is done to preserve back‐
1186          wards  compatibility  and  avoid excessive arguments.  Both types of
1187          handler may be passed to the argument.
1188
1189   Implementation Details
1190       Creating a content handler requires subclassing ContentHandler and  im‐
1191       plementing several methods.  These methods are described below, but in‐
1192       specting JSONHandler will be instructive in highlighting required argu‐
1193       ments and techniques.
1194
1195       To  provide  a  response_<something> response-body evaluator a subclass
1196       must define:
1197
1198test_key_suffix: This, along with the prefix response_, forms the key
1199         used in the test structure. It is a class level string.
1200
1201test_key_value: The key's default value, either an empty list ([]) or
1202         empty dict ({}). It is a class level value.
1203
1204action: An instance method which tests the  expected  values  against
1205         the HTTP response - it is invoked for each entry, with the parameters
1206         depending on the default value. The arguments to action are  (in  or‐
1207         der):
1208
1209self: The current instance.
1210
1211test: The currently active HTTPTestCase
1212
1213item:  The current entry if test_key_value is a list, otherwise the
1214           key half of the key/value pair at this entry.
1215
1216value: None if test_key_value is a list, otherwise the  value  half
1217           of the key/value pair at this entry.
1218
1219       To  translate  request  or response bodies to or from structured data a
1220       subclass must define an accepts method. This should return True if this
1221       class is willing to translate the provided content-type. During request
1222       processing it is given the value of the content-type header  that  will
1223       be  sent  in  the  request.  During response processing it is given the
1224       value of the content-type header of the response. This makes it  possi‐
1225       ble  to  handle  different request and response bodies in the same han‐
1226       dler,  if  desired.  For  example  a  handler  might  accept   applica‐
1227       tion/x-www-form-urlencoded and text/html.
1228
1229       If accepts is defined two additional static methods should be defined:
1230
1231dumps: Turn structured Python data from the data key in a test into a
1232         string or byte stream. The optional test param allows you  to  access
1233         the  current  test  case which may help with manipulations for custom
1234         content handlers, e.g. multipart/form-data needs to add a boundary to
1235         the  Content-Type header in order to mark the appropriate sections of
1236         the body.
1237
1238loads: Turn a string or byte stream in a response into a Python  data
1239         structure. Gabbi will put this data on the response_data attribute on
1240         the test, where it can be used in the evaluations described above (in
1241         the   action method) or in $RESPONSE handling.  An example usage here
1242         would be to turn HTML into a DOM.
1243
1244load_data_file: Load data from disk into  a  Python  data  structure.
1245         Gabbi  will  call  this  method when response_<something> contains an
1246         item where the right hand side value starts with <@. The  test  param
1247         allows   you   to  access  the  current  test  case  and  provides  a
1248         load_data_file method which should be used because  it  verifies  the
1249         data  is  loaded within the test diectory and returns the file source
1250         as a string. The load_data_file method was introduced to  re-use  the
1251         JSONHandler  in order to support loading YAML files from disk through
1252         the implementation of an additional custom handler, see YAMLDiskLoad‐
1253         ingJSONHandler for details.
1254
1255       Finally  if  a  replacer class method is defined, then when a $RESPONSE
1256       substitution is encountered, replacer will be passed the  response_data
1257       of the prior test and the argument within the $RESPONSE.
1258
1259       Please see the JSONHandler source for additional detail.
1260
1261       If  there is a running web service that needs to be tested and creating
1262       a test loader with build_tests() is either inconvenient or overkill  it
1263       is  possible to run YAML test files directly from the command line with
1264       the console-script gabbi-run. It accepts YAML on stdin or  as  multiple
1265       file  arguments,  and generates and runs tests and outputs a summary of
1266       the results.
1267
1268       The provided YAML may not use custom fixtures but  otherwise  uses  the
1269       default  format.  host  information is either expressed directly in the
1270       YAML file or provided on the command line:
1271
1272          gabbi-run [host[:port]] < /my/test.yaml
1273
1274       or:
1275
1276          gabbi-run http://host:port < /my/test.yaml
1277
1278       To test with one or more files the  following  command  syntax  may  be
1279       used:
1280
1281          gabbi-run http://host:port -- /my/test.yaml /my/other.yaml
1282
1283       NOTE:
1284          The  filename arguments must come after a -- and all other arguments
1285          (host, port, prefix, failfast) must come before the --.
1286
1287       NOTE:
1288          If files are provided, test output will use names including the name
1289          of  the  file. If any single file includes an error, the name of the
1290          file will be included in a summary of failed files at the end of the
1291          test report.
1292
1293       To facilitate using the same tests against the same application mounted
1294       in different locations in a WSGI server, a prefix may be provided as  a
1295       second argument:
1296
1297          gabbi-run host[:port] [prefix] < /my/test.yaml
1298
1299       or in the target URL:
1300
1301          gabbi-run http://host:port/prefix < /my/test.yaml
1302
1303       The  value of prefix will be prepended to the path portion of URLs that
1304       are not fully qualified.
1305
1306       Anywhere host is used, if it is a raw IPV6 address it should be wrapped
1307       in [ and ].
1308
1309       If  https  is  used  in the target, then the tests in the provided YAML
1310       will default to ssl: True.
1311
1312       Use -k or --insecure to not validate  certificates  when  making  https
1313       connections.
1314
1315       If a -x or --failfast argument is provided then gabbi-run will exit af‐
1316       ter the first test failure.
1317
1318       Use -v or --verbose with a value of all, headers or  body  to  turn  on
1319       verbosity for all tests being run.
1320
1321       Use -q or --quiet to silence test runner output.
1322
1323       These  are  informal release notes for gabbi since version 1.0.0, high‐
1324       lighting major features and changes. For more  detail  see  the  commit
1325       logs on GitHub.
1326

2.2.0

1328       • Allow substitutions in skip for more flexible skipping, notablly with
1329         $ENVIRON.
1330
1331       • Better test name output when running with unittest and verbose.
1332

2.1.0

1334       • Remove support for Python 3.5. Ensure testing support for  3.8,  3.9,
1335         and pypy3.
1336
1337       • Adapt to new behavior in urllib.parse.
1338
1339       • Correct fixture start when using pytest.
1340

2.0.4

1342       • If  no content-type is provided with a response and verbose is on for
1343         responses, display the response body as if it were text.
1344

2.0.3

1346       • Properly declare that gabbi 2.x is Python 3 only.
1347

2.0.1

1349       • Correct management of response handler default fields.
1350

2.0.0

1352       • Drop support for Python 2. If you need Python 2 support, use an older
1353         version.
1354
1355       • Stop  using testtools and fixtures. These two modules present several
1356         difficulties and their maintenance situation suggests those difficul‐
1357         ties  will  not be resolved. Since Python 2 support is being removed,
1358         the need for the modules can be removed as well without losing  func‐
1359         tionality.  "Inner  fixtures" that use the fixtures.Fixture interface
1360         should continue to work.
1361

1.49.0

1363       • Add support for not validating certificates in https  requests.  Con‐
1364         trolled  by  the  cert_validate  attribute  in  individual  tests and
1365         build_tests() and the -k or --insecure argument to gabbi-run.
1366

1.48.0

1368       • Support pytest 5.0.0 in Python >=3.5. For earlier versions of Python,
1369         pytest<5.0.0 will be used; the pytest project is dropping support for
1370         older versions of Python.
1371

1.47.0

1373       • Use pytest<5.0.0 until gabbi has solutions for the changes in 5.0.0.
1374

1.46.0

1376       • A -q argument is added to gabbi-run to suppress output from the  test
1377         runner.
1378

1.45.0

1380       • Adjust loading of YAML to be ready for new release of PyYAML.
1381

1.44.0

1383       • Provide  the YAMLDiskLoadingJSONHandler class that allows test result
1384         data for response_json_path checks to be loaded from YAML-on-disk.
1385

1.43.0

1387       • Use  jsonpath  to  select  a   portion   of   data-on-disk   in   re‐
1388         sponse_json_path checks.
1389
1390       • Restrict PyYAML to <4.0.
1391

1.42.0

1393       • Allow listing of tests with no host configured. When host is an empty
1394         string, tests can be listed (for discovery), but will be  skipped  on
1395         run.
1396

1.41.0

1398       • JSON  $RESPONSE  substitutions in the data field may be complex types
1399         (lists and dicts), not solely strings.
1400

1.40.0

1402       • When the HTTP response begins with a bad status  line,  have  BadSta‐
1403         tusLine be raised from urllib3.
1404

1.39.0

1406       • Allow  substitutions in the key portion of request and response head‐
1407         ers, not just the value.
1408

1.38.0

1410       • Remove support for Python 3.3.
1411
1412       • Make handling of fixture-level skips in pytest actually work.
1413

1.37.0

1415       • Add safe_yaml parameter to build_tests().
1416

1.36.0

1418use_prior_test is added to test metadata.
1419
1420       • Extensive cleanups in regular expression handling  when  constructing
1421         tests from YAML.
1422

1.35.0

1424       jsonpath handling gets two improvements:
1425
1426       • The  value  side  of a response_json_paths entry can be loaded from a
1427         file using the <@file.json syntax also used in data.
1428
1429       • The key side of a response_json_paths entry  can  use  substitutions.
1430         This was already true for the value side.
1431

1.34.0

1433       Substitutions  in $RESPONSE handling now preserve numeric types instead
1434       of casting to a string. This  is  useful  when  servers  are  expecting
1435       strong types and tests want to send response data back to the server.
1436

1.33.0

1438       count      and     delay     test     keys     allow     substitutions.
1439       gabbi.driver.build_tests() accepts a verbose parameter to set test ver‐
1440       bosity for an entire session.
1441

1.32.0

1443       Better failure reporting when using gabbi-run with multiple files. Test
1444       names are based on the files and a summary of failed files is  provided
1445       at the end of the report.
1446

1.31.0

1448       Effectively  capture  a  failure in a fixture and report the traceback.
1449       Without this some test runners swallow the error and discovering  prob‐
1450       lems when developing fixtures can be quite challenging.
1451

1.30.0

1453       Thanks to Samuel Fekete, tests can use the $HISTORY dictionary to refer
1454       to any prior test in the same file, not just the one immediately prior,
1455       when doing substitutions.
1456

1.29.0

1458       Filenames  used to read data into tests using the <@ syntax may now use
1459       pathnames relative to the YAML file. See data.
1460
1461       gabbi-run gains a --verbose parameter to force all tests run in a  ses‐
1462       sion to run with verbose set.
1463
1464       When  using  pytest  to  load tests, a new mechanism is available which
1465       avoids warnings produced in when using a version of pytest greater than
1466       3.0.
1467

1.28.0

1469       When  verbosely  displaying  request and response bodies that are JSON,
1470       pretty print for improved readability.
1471

1.27.0

1473       Allow gabbi-run to accept multiple filenames as command line  arguments
1474       instead of reading tests from stdin.
1475

1.26.0

1477       Switch  from  response handlers to handlers to allow more flexible pro‐
1478       cessing of both response _and_ request bodies.
1479
1480       Add inner fixtures for per test fixtures, useful for output capturing.
1481

1.25.0

1483       Allow the test_loader_name arg to gabbi.driver.build_tests()  to  over‐
1484       ride the prefix of the pretty printed name of generated tests.
1485

1.24.0

1487       String  values  in  JSONPath  matches  may  be  wrapped in /.../` to be
1488       treated as regular expressions.
1489

1.23.0

1491       Better documentation of how to run gabbi in a  concurrent  environment.
1492       Improved handling of pytest fixtures and test counts.
1493

1.22.0

1495       Add  url to gabbi.driver.build_tests() to use instead of host, port and
1496       prefix.
1497

1.21.0

1499       Add require_ssl to gabbi.driver.build_tests() to force use of SSL.
1500

1.20.0

1502       Add $COOKIE substitution.
1503

1.19.1

1505       Correctly support IPV6 hosts.
1506

1.19.0

1508       Add $LAST_URL substitution.
1509

1.17.0

1511       Introduce support for loading and running tests with pytest.
1512

1.16.0

1514       Use urllib3 instead of httplib2 for driving HTTP requests.
1515

1.13.0

1517       Add sorting and filtering to jsonpath handling.
1518

1.11.0

1520       Add the response_forbidden_headers to response expectations.
1521

1.7.0

1523       Instead of:
1524
1525          tests:
1526          - name: a simple get
1527            url: /some/path
1528            method: get
1529
1530       1.7.0 also makes it possible to:
1531
1532          tests:
1533          - name: a simple get
1534            GET: /some/path
1535
1536       Any upper case key is treated as a method.
1537

1.4.0 AND 1.5.0

1539       Enhanced flexibility and colorization when setting tests to be verbose.
1540

1.3.0

1542       Adds the query_parameters key to request parameters.
1543

1.2.0

1545       The start of improvements and extensions to jsonpath handling. In  this
1546       case the addition of the len function.
1547

1.1.0

1549       Vastly improved output and behavior in gabbi-run.
1550

1.0.0

1552       Version  1  was the first release with a commitment to a stable format.
1553       Since then new fields have been added but have not been taken away.
1554
1555       The following people have contributed code to gabbi.  Thanks  to  them.
1556       Thanks  also  to all the people who have made gabbi better by reporting
1557       issues and their successes and failures with using gabbi.
1558
1559       • Chris Dent
1560
1561       • FND
1562
1563       • Mehdi Abaakouk
1564
1565       • Tom Viner
1566
1567       • Jason Myers
1568
1569       • Josh Leeb-du Toit
1570
1571       • Duc Truong
1572
1573       • Zane Bitter
1574
1575       • Ryan Spencer
1576
1577       • Kim Raymoure
1578
1579       • Travis Truman
1580
1581       • Samuel Fekete
1582
1583       • Michael McCune
1584
1585       • Imran Hayder
1586
1587       • Julien Danjou
1588
1589       • Trevor McCasland
1590
1591       • Danek Duvall
1592
1593       • Marc Abramowitz
1594
1595       NOTE:
1596          This section provides a collection of questions  with  answers  that
1597          don't  otherwise  fit in the rest of the documentation. If something
1598          is missing, please create an issue.
1599
1600          As this document grows it will gain a more refined structure.
1601

GENERAL

1603   Is gabbi only for testing Python-based APIs?
1604       No, you can use gabbi-run to test an HTTP service built in any program‐
1605       ming language.
1606
1607   How do I run just one test?
1608       Each YAML file contains a sequence of tests, each test within each file
1609       has a name. That name is translated to the name of the test by  replac‐
1610       ing spaces with an _.
1611
1612       When  running  tests that are generated dynamically, filtering based on
1613       the test name prior to the test being collected will not work  in  some
1614       test runners.  Test runners that use a --load-list functionality can be
1615       convinced to filter after discovery.
1616
1617       pytest does this directly with the -k keyword flag.
1618
1619       When using testrepository with tox as used in gabbi's own tests  it  is
1620       possible to pass a filter in the tox command:
1621
1622          tox -epy27 -- get_the_widget
1623
1624       When  using testtools.run and similar test runners it's a bit more com‐
1625       plicated. It is necessary to provide the full name of  the  test  as  a
1626       list to --load-list:
1627
1628          python -m testtools.run --load-list \
1629              <(echo package.tests.test_api.yamlfile_get_the_widge.test_request)
1630
1631   How do I run just one test, without running prior tests in a sequence?
1632       By  default,  when  you select a single test to run, all tests prior to
1633       that one in a file will be run as well: the file is treated as  as  se‐
1634       quence  of  dependent tests. If you do not want this you can adjust the
1635       use_prior_test test metadata in one of three ways:
1636
1637       • Set it in the YAML file for the one test you are concerned with.
1638
1639       • Set the defaults for all tests in that file.
1640
1641       • set use_prior_test to false when calling build_tests()
1642
1643       Be aware that doing this breaks a  fundamental  assumption  that  gabbi
1644       makes about how tests work. Any substitutions will fail.
1645

TESTING STYLE

1647   Can I have variables in my YAML file?
1648       Gabbi  provides  the $ENVIRON substitution which can operate a bit like
1649       variables that are set elsewhere and then used in the tests defined  by
1650       the YAML.
1651
1652       If  you  find  it necessary to have variables within a single YAML file
1653       you take advantage of YAML alias nodes list this:
1654
1655          vars:
1656            - &uuid_1 5613AABF-BAED-4BBA-887A-252B2D3543F8
1657
1658          tests:
1659          - name: send a uuid to a post
1660            POST: /resource
1661            request_headers:
1662              content-type: application/json
1663            data:
1664              uuid: *uuid_1
1665
1666       You can alias all sorts of nodes, not just single items. Be aware  that
1667       the  replacement  of  an  alias  node  happens  while the YAML is being
1668       loaded, before gabbi does any processing.
1669
1670   How many tests should be put in one YAML file?
1671       For the sake of readability it is best to keep  each  YAML  file  rela‐
1672       tively  short.  Since each YAML file represents a sequence of requests,
1673       it usually makes sense to create a new file when a test is  not  depen‐
1674       dent on any before it.
1675
1676       It's  tempting to put all the tests for any resource or URL in the same
1677       file, but this eventually leads to files that are too long and are thus
1678       difficult to read.
1679

CASE MODULE

1681       A single HTTP request represented as a subclass of unittest.TestCase
1682
1683       The  test  case  encapsulates the request headers and body and expected
1684       response headers and body. When the test is run an HTTP request is made
1685       using urllib3. Assertions are made against the response.
1686
1687       class gabbi.case.HTTPTestCase(methodName='runTest')
1688              Bases: unittest.case.TestCase
1689
1690              Encapsulate a single HTTP request as a TestCase.
1691
1692              If  the  test is a member of a sequence of requests, ensure that
1693              prior tests are run.
1694
1695              To keep the test harness happy we need to make  sure  the  setUp
1696              and tearDown are only run once.
1697
1698              assert_in_or_print_output(expected, iterable)
1699                     Assert  the iterable contains expected or print some out‐
1700                     put.
1701
1702                     If  the  output  is  long,  it  is  limited   by   either
1703                     GABBI_MAX_CHARS_OUTPUT   in   the   environment   or  the
1704                     MAX_CHARS_OUTPUT constant.
1705
1706              base_test = {'cert_validate':  True,  'data':  '',  'desc':  '',
1707              'method': 'GET', 'name': '', 'poll': {}, 'query_parameters': {},
1708              'redirects': False, 'request_headers': {},  'skip':  '',  'ssl':
1709              False, 'status': '200', 'url': '', 'use_prior_test': True, 'ver‐
1710              bose': False, 'xfail': False}
1711
1712              get_content_handler(content_type)
1713                     Determine the content handler for this media type.
1714
1715              load_data_file(filename)
1716                     Read a file from the current test directory.
1717
1718              replace_template(message, escape_regex=False)
1719                     Replace magic strings in message.
1720
1721              run(result=None)
1722                     Store the current result handler on this test.
1723
1724              setUp()
1725                     Hook method for setting up the test fixture before  exer‐
1726                     cising it.
1727
1728              shortDescription()
1729                     Returns a one-line description of the test, or None if no
1730                     description has been provided.
1731
1732                     The default implementation of  this  method  returns  the
1733                     first line of the specified test method's docstring.
1734
1735              tearDown()
1736                     Hook  method  for  deconstructing  the test fixture after
1737                     testing it.
1738
1739              test_request()
1740                     Run this request if it has not yet run.
1741
1742                     If there is a prior test in the sequence, run it first.
1743
1744       gabbi.case.potentialFailure(func)
1745              Decorate a test method that is expected to fail  if  'xfail'  is
1746              true.
1747

DRIVER MODULE

SUITEMAKER MODULE

FIXTURE MODULE

1751       Manage fixtures for gabbi at the test suite level.
1752
1753       class gabbi.fixture.GabbiFixture
1754              Bases: object
1755
1756              A context manager that operates as a fixture.
1757
1758              Subclasses  must  implement start_fixture and stop_fixture, each
1759              of which contain the logic for stopping  and  starting  whatever
1760              the fixture is. What a fixture is is left as an exercise for the
1761              implementor.
1762
1763              These context managers will be nested so any actual  work  needs
1764              to happen in start_fixture and stop_fixture and not in __init__.
1765              Otherwise exception handling will not work properly.
1766
1767              start_fixture()
1768                     Implement the actual workings  of  starting  the  fixture
1769                     here.
1770
1771              stop_fixture()
1772                     Implement  the  actual  workings  of stopping the fixture
1773                     here.
1774
1775       exception gabbi.fixture.GabbiFixtureError
1776              Bases: Exception
1777
1778              Generic exception for GabbiFixture.
1779
1780       class gabbi.fixture.SkipAllFixture
1781              Bases: gabbi.fixture.GabbiFixture
1782
1783              A fixture that skips all the tests in the current suite.
1784
1785              start_fixture()
1786                     Implement the actual workings  of  starting  the  fixture
1787                     here.
1788
1789       gabbi.fixture.nest(fixtures)
1790              Nest a series of fixtures.
1791
1792              This  is  duplicated  from  nested in the stdlib, which has been
1793              deprecated because of issues with how exceptions  are  difficult
1794              to handle during __init__. Gabbi needs to nest an unknown number
1795              of fixtures dynamically, so the with syntax that replaces nested
1796              will not work.
1797

HANDLERS MODULE

1799       Package  for  response  and content handlers that process the body of a
1800       response in various ways.
1801
1802   handlers.base Module
1803       Base classes for response and content handlers.
1804
1805       class gabbi.handlers.base.ContentHandler
1806              Bases: gabbi.handlers.base.ResponseHandler
1807
1808              A subclass of ResponseHandlers that adds content handling.
1809
1810              static accepts(content_type)
1811                     Return True if this handler can handler this type.
1812
1813              static dumps(data, pretty=False, test=None)
1814                     Return structured data as a string.
1815
1816                     If pretty is true, prettify.
1817
1818              static load_data_file(test, file_path)
1819                     Return the string content of the file  specified  by  the
1820                     file_path.
1821
1822              static loads(data)
1823                     Create structured (Python) data from a stream.
1824
1825              classmethod replacer(response_data, path)
1826                     Return the string that is replacing RESPONSE.
1827
1828       class gabbi.handlers.base.ResponseHandler
1829              Bases: object
1830
1831              Add functionality for making assertions about an HTTP response.
1832
1833              A subclass may implement two methods: action and preprocess.
1834
1835              preprocess  takes  one  argument, the TestCase. It is called ex‐
1836              actly once for each test before looping across  the  assertions.
1837              It  is  used, rarely, to copy the test.output into a useful form
1838              (such as a parsed DOM).
1839
1840              action takes two or three arguments. If test_key_value is a list
1841              action  is  called with the test case and a single list item. If
1842              test_key_value is a dict then action is  called  with  the  test
1843              case and a key and value pair.
1844
1845              action(test, item, value=None)
1846                     Test an individual entry for this response handler.
1847
1848                     If  the  entry is a key value pair the key is in item and
1849                     the value in value. Otherwise the entry is  considered  a
1850                     single item from a list.
1851
1852              preprocess(test)
1853                     Do any pre-single-test preprocessing.
1854
1855              test_key_suffix = ''
1856
1857              test_key_value = []
1858
1859   handlers.core Module
1860       Core response handlers.
1861
1862       class gabbi.handlers.core.ForbiddenHeadersResponseHandler
1863              Bases: gabbi.handlers.base.ResponseHandler
1864
1865              Test that listed headers are not in the response.
1866
1867              action(test, forbidden, value=None)
1868                     Test an individual entry for this response handler.
1869
1870                     If  the  entry is a key value pair the key is in item and
1871                     the value in value. Otherwise the entry is  considered  a
1872                     single item from a list.
1873
1874              test_key_suffix = 'forbidden_headers'
1875
1876              test_key_value = []
1877
1878       class gabbi.handlers.core.HeadersResponseHandler
1879              Bases: gabbi.handlers.base.ResponseHandler
1880
1881              Compare expected headers with actual headers.
1882
1883              If a header value is wrapped in / it is treated as a raw regular
1884              expression.
1885
1886              Headers values are always treated as strings.
1887
1888              action(test, header, value=None)
1889                     Test an individual entry for this response handler.
1890
1891                     If the entry is a key value pair the key is in  item  and
1892                     the  value  in value. Otherwise the entry is considered a
1893                     single item from a list.
1894
1895              test_key_suffix = 'headers'
1896
1897              test_key_value = {}
1898
1899       class gabbi.handlers.core.StringResponseHandler
1900              Bases: gabbi.handlers.base.ResponseHandler
1901
1902              Test for matching strings in the the response body.
1903
1904              action(test, expected, value=None)
1905                     Test an individual entry for this response handler.
1906
1907                     If the entry is a key value pair the key is in  item  and
1908                     the  value  in value. Otherwise the entry is considered a
1909                     single item from a list.
1910
1911              test_key_suffix = 'strings'
1912
1913              test_key_value = []
1914
1915   handlers.jsonhandler Module
1916       JSON-related content handling.
1917
1918       class gabbi.handlers.jsonhandler.JSONHandler
1919              Bases: gabbi.handlers.base.ContentHandler
1920
1921              A ContentHandler for JSON
1922
1923              • Structured test data is turned into  JSON  when  request  con‐
1924                tent-type is JSON.
1925
1926              • Response  bodies  that  are  JSON strings are made into Python
1927                data on the test response_data  attribute  when  the  response
1928                content-type is JSON.
1929
1930              • A response_json_paths response handler is added.
1931
1932              • JSONPaths in $RESPONSE substitutions are supported.
1933
1934              static accepts(content_type)
1935                     Return True if this handler can handler this type.
1936
1937              action(test, path, value=None)
1938                     Test json_paths against json data.
1939
1940              static dumps(data, pretty=False, test=None)
1941                     Return structured data as a string.
1942
1943                     If pretty is true, prettify.
1944
1945              static extract_json_path_value(data, path)
1946                     Extract the value at JSON Path path from the data.
1947
1948                     The  input  data  is  a  Python datastructure, not a JSON
1949                     string.
1950
1951              static load_data_file(test, file_path)
1952                     Return the string content of the file  specified  by  the
1953                     file_path.
1954
1955              static loads(data)
1956                     Create structured (Python) data from a stream.
1957
1958              classmethod replacer(response_data, match)
1959                     Return the string that is replacing RESPONSE.
1960
1961              test_key_suffix = 'json_paths'
1962
1963              test_key_value = {}
1964
1965   handlers.yaml_disk_loading_jsonhandler Module
1966       JSON-related content handling with YAML data disk loading.
1967
1968       class gabbi.handlers.yaml_disk_loading_jsonhandler.YAMLDiskLoadingJSON‐
1969       Handler
1970              Bases: gabbi.handlers.jsonhandler.JSONHandler
1971
1972              A ContentHandler for JSON responses that loads YAML from disk
1973
1974              • Structured test data is turned into  JSON  when  request  con‐
1975                tent-type is JSON.
1976
1977              • Response  bodies  that  are  JSON strings are made into Python
1978                data on the test response_data  attribute  when  the  response
1979                content-type is JSON.
1980
1981              • A  response_json_paths  response  handler  is added. Data read
1982                from  disk  during  this  handle  will  be  loaded  with   the
1983                yaml.safe_load  method  to  support  both  JSON  and YAML data
1984                sources from disk.
1985
1986              • JSONPaths in $RESPONSE substitutions are supported.
1987
1988              static load_data_file(test, file_path)
1989                     Return the string content of the file  specified  by  the
1990                     file_path.
1991

SUITE MODULE

1993       A TestSuite for containing gabbi tests.
1994
1995       This  suite has two features: the contained tests are ordered and there
1996       are suite-level fixtures that operate as context managers.
1997
1998       class gabbi.suite.GabbiSuite(tests=())
1999              Bases: unittest.suite.TestSuite
2000
2001              A TestSuite with fixtures.
2002
2003              The suite wraps the tests with a set of nested context  managers
2004              that operate as fixtures.
2005
2006              If a fixture raises unittest.case.SkipTest during setup, all the
2007              tests in this suite will be skipped.
2008
2009              run(result, debug=False)
2010                     Override TestSuite run to start suite-level fixtures.
2011
2012                     To avoid exception confusion, use  a  null  Fixture  when
2013                     there are no fixtures.
2014
2015              start(result, tests=None)
2016                     Start fixtures when using pytest.
2017
2018              stop() Stop fixtures when using pytest.
2019
2020       gabbi.suite.noop(*args)
2021              A noop method used to disable collected tests.
2022

RUNNER MODULE

REPORTER MODULE

2025       TestRunner and TestResult for gabbi-run.
2026
2027       class gabbi.reporter.ConciseTestResult(stream, descriptions, verbosity)
2028              Bases: unittest.runner.TextTestResult
2029
2030              A TextTestResult with simple but useful output.
2031
2032              If  the output is a tty or GABBI_FORCE_COLOR is set in the envi‐
2033              ronment, output will be colorized.
2034
2035              addError(test, err)
2036                     Called when an error has occurred. 'err' is  a  tuple  of
2037                     values as returned by sys.exc_info().
2038
2039              addExpectedFailure(test, err)
2040                     Called when an expected failure/error occurred.
2041
2042              addFailure(test, err)
2043                     Called  when  an  error has occurred. 'err' is a tuple of
2044                     values as returned by sys.exc_info().
2045
2046              addSkip(test, reason)
2047                     Called when a test is skipped.
2048
2049              addSuccess(test)
2050                     Called when a test has completed successfully
2051
2052              addUnexpectedSuccess(test)
2053                     Called when a test was expected to fail, but succeed.
2054
2055              getDescription(test)
2056
2057              printErrorList(flavor, errors)
2058
2059              startTest(test)
2060                     Called when the given test is about to be run
2061
2062       class gabbi.reporter.ConciseTestRunner(stream=None,  descriptions=True,
2063       verbosity=1,   failfast=False,  buffer=False,  resultclass=None,  warn‐
2064       ings=None, *, tb_locals=False)
2065              Bases: unittest.runner.TextTestRunner
2066
2067              A TextTestRunner that uses ConciseTestResult for  reporting  re‐
2068              sults.
2069
2070              resultclass
2071                     alias of gabbi.reporter.ConciseTestResult
2072
2073       class  gabbi.reporter.PyTestResult(stream=None, descriptions=None, ver‐
2074       bosity=None)
2075              Bases: unittest.result.TestResult
2076
2077              Wrap a test result to allow it to work with pytest.
2078
2079              The main behaviors here are:
2080
2081              • to turn what had been exceptions back into exceptions
2082
2083              • use pytest's skip and xfail methods
2084
2085              addError(test, err)
2086                     Called when an error has occurred. 'err' is  a  tuple  of
2087                     values as returned by sys.exc_info().
2088
2089              addExpectedFailure(test, err)
2090                     Called when an expected failure/error occurred.
2091
2092              addFailure(test, err)
2093                     Called  when  an  error has occurred. 'err' is a tuple of
2094                     values as returned by sys.exc_info().
2095
2096              addSkip(test, reason)
2097                     Called when a test is skipped.
2098

UTILS MODULE

2100       Utility functions grab bag.
2101
2102       gabbi.utils.create_url(base_url, host, port=None, prefix='', ssl=False)
2103              Given pieces of a path-based url, return a fully qualified url.
2104
2105       gabbi.utils.decode_response_content(header_dict, content)
2106              Decode content to a proper string.
2107
2108       gabbi.utils.extract_content_type(header_dict,  default='application/bi‐
2109       nary')
2110              Extract parsed content-type from headers.
2111
2112       gabbi.utils.get_colorizer(stream)
2113              Return a function to colorize a string.
2114
2115              Only if stream is a tty .
2116
2117       gabbi.utils.host_info_from_target(target, prefix=None)
2118              Turn url or host:port and target into test destination.
2119
2120       gabbi.utils.load_yaml(handle=None, yaml_file=None, safe=True)
2121              Read and parse any YAML file or filehandle.
2122
2123              Let exceptions flow where they may.
2124
2125              If no file or handle is provided, read from STDIN.
2126
2127       gabbi.utils.not_binary(content_type)
2128              Decide if something is content we'd like to treat as a string.
2129
2130       gabbi.utils.parse_content_type(content_type, default_charset='utf-8')
2131              Parse content type value for media type and charset.
2132

EXCEPTION MODULE

2134       Gabbi specific exceptions.
2135
2136       exception gabbi.exception.GabbiFormatError
2137              Bases: ValueError
2138
2139              An exception to encapsulate poorly formed test data.
2140
2141       exception gabbi.exception.GabbiSyntaxWarning
2142              Bases: SyntaxWarning
2143
2144              A warning about syntax that is not desirable.
2145

HTTPCLIENT MODULE

JSON_PARSER MODULE

2148       Keep one single global jsonpath parser.
2149
2150       gabbi.json_parser.parse(path)
2151              Parse a JSONPath expression use the global parser.
2152
2153       Gabbi is a tool for running HTTP tests where requests and responses are
2154       expressed as declarations in YAML files:
2155
2156          tests:
2157          - name: retrieve items
2158            GET: /items
2159
2160       See the rest of these docs for more details on the  many  features  and
2161       formats  for  setting  request  headers  and  bodies and evaluating re‐
2162       sponses.
2163
2164       Tests can be run from the command line with gabbi-run  or  programmati‐
2165       cally   using  either  py.test  or  unittest-style  test  runners.  See
2166       installation instructions below.
2167
2168       The name is derived from "gabby": excessively talkative. In a test  en‐
2169       vironment  having visibility of what a test is actually doing is a good
2170       thing. This is especially true when the goal of a test is to  test  the
2171       HTTP,  not  the testing infrastructure. Gabbi tries to put the HTTP in‐
2172       teraction in the foreground of testing.
2173
2174       If you want to get straight to creating tests look at example, the test
2175       files  in  the  source distribution and format. A gabbi-demo repository
2176       provides a tutorial of using gabbi to build an API, via the commit his‐
2177       tory of the repo.
2178

PURPOSE

2180       Gabbi  works  to  bridge the gap between human readable YAML files (see
2181       format for details) that represent HTTP requests and expected responses
2182       and the rather complex world of automated testing.
2183
2184       Each  YAML  file represents an ordered list of HTTP requests along with
2185       the expected responses. This  allows  a  single  file  to  represent  a
2186       process in the API being tested. For example:
2187
2188       • Create a resource.
2189
2190       • Retrieve a resource.
2191
2192       • Delete a resource.
2193
2194       • Retrieve a resource again to confirm it is gone.
2195
2196       At  the same time it is still possible to ask gabbi to run just one re‐
2197       quest. If it is in a sequence of tests, those tests prior to it in  the
2198       YAML  file  will be run (in order). In any single process any test will
2199       only be run once. Concurrency is handled such that one file runs in one
2200       process.
2201
2202       These features mean that it is possible to create tests that are useful
2203       for both humans (as tools for learning, improving and developing  APIs)
2204       and automated CI systems.
2205
2206       Significant flexibility and power is available in the format to make it
2207       relatively straightforward to test existing  complex  APIs.   This  ex‐
2208       tended  functionality  includes  the  use of JSONPath to query response
2209       bodies and templating of test data to allow access to  the  prior  HTTP
2210       response  in  the current request. For APIs which do not use JSON addi‐
2211       tional handlers can be created.
2212
2213       Care should be taken when using this functionality when you are  creat‐
2214       ing  a  new  API.  If your API is so complex that it needs complex test
2215       files then you may wish to take that as a sign that your API itself too
2216       complex. One goal of gabbi is to encourage transparent and comprehensi‐
2217       ble APIs.
2218
2219       Though gabbi is written in Python and under the  covers  uses  unittest
2220       data structures and processes, there is no requirement that the host be
2221       a Python-based service. Anything talking HTTP can be tested.  A  runner
2222       makes  it possible to simply create YAML files and point them at a run‐
2223       ning server.
2224

INSTALLATION

2226       As a Python package, gabbi is typically installed via pip:
2227
2228          pip install gabbi
2229
2230       You might want to create a virtual environment; an isolated context for
2231       Python  packages,  keeping gabbi cleany separated from the rest of your
2232       system.
2233
2234       Python 3 comes with a built-in tool to create virtual environments:
2235
2236          python3 -m venv venv
2237          . venv/bin/activate
2238
2239          pip install gabbi
2240
2241       This way we can later use deactivate and safely remove the venv  direc‐
2242       tory, thus erasing any trace of gabbi from the system.
2243

AUTHOR

2245       Chris Dent
2246
2247
2248
2249
2250                                 Jul 23, 2021                         GABBI(1)
Impressum