1GABBI(1) Gabbi GABBI(1)
2
6 gabbi - Gabbi Documentation
7 Gabbi tests are expressed in YAML as a series of HTTP requests with
9 their expected response:
10 tests:
12 - name: retrieve root
13 GET: /
14 status: 200
15 This will trigger a GET request to / on the configured host. The test
17 will pass if the response's status code is 200.
18
20 The top-level tests category contains an ordered sequence of test dec‐
21 larations, each describing the expected response to a given request:
22 Metadata
24 ┌───────────────┬─────────────────────┬───────────────────┐
25 │Key │ Description │ Notes │
26 ├───────────────┼─────────────────────┼───────────────────┤
27 │name │ The test's name. │ required │
28 │ │ Must be unique │ │
29 │ │ within a file. │ │
30 ├───────────────┼─────────────────────┼───────────────────┤
31 │desc │ An arbitrary string │ │
32 │ │ describing the │ │
33 │ │ test. │ │
34 ├───────────────┼─────────────────────┼───────────────────┤
35 │verbose │ If True or all │ defaults to False │
36 │ │ (synonymous), │ │
37 │ │ prints a represen‐ │ │
38 │ │ tation of the cur‐ │ │
39 │ │ rent request and │ │
40 │ │ response to stdout, │ │
41 │ │ including both │ │
42 │ │ headers and body. │ │
43 │ │ If set to headers │ │
44 │ │ or body, only the │ │
45 │ │ corresponding part │ │
46 │ │ of the request and │ │
47 │ │ response will be │ │
48 │ │ printed. If the │ │
49 │ │ output is a TTY, │ │
50 │ │ colors will be │ │
51 │ │ used. If the body │ │
52 │ │ content-type is │ │
53 │ │ JSON it will be │ │
54 │ │ formatted for │ │
55 │ │ improved readabil‐ │ │
56 │ │ ity. See Verbose‐ │ │
57 │ │ Http for details. │ │
58 ├───────────────┼─────────────────────┼───────────────────┤
59 │skip │ A string message │ defaults to False │
60 │ │ which if set will │ │
61 │ │ cause the test to │ │
62 │ │ be skipped with the │ │
63 │ │ provided message. │ │
64 └───────────────┴─────────────────────┴───────────────────┘
65 │xfail │ Determines whether │ defaults to False │
68 │ │ to expect this test │ │
69 │ │ to fail. Note that │ │
70 │ │ the test will be │ │
71 │ │ run anyway. │ │
72 ├───────────────┼─────────────────────┼───────────────────┤
73 │use_prior_test │ Determines if this │ defaults to True │
74 │ │ test will be run in │ │
75 │ │ sequence (after) │ │
76 │ │ the test prior to │ │
77 │ │ it in the list of │ │
78 │ │ tests within a │ │
79 │ │ file. To be con‐ │ │
80 │ │ crete, when this is │ │
81 │ │ True the test is │ │
82 │ │ dependent on the │ │
83 │ │ prior test and if │ │
84 │ │ that prior has not │ │
85 │ │ yet run, it wil be │ │
86 │ │ run, even if only │ │
87 │ │ the current test │ │
88 │ │ has been selected. │ │
89 │ │ Set this to False │ │
90 │ │ to allow selecting │ │
91 │ │ a test without │ │
92 │ │ dependencies. │ │
93 └───────────────┴─────────────────────┴───────────────────┘
94 NOTE:
96 When tests are generated dynamically, the TestCase name will include
97 the respective test's name, lowercased with spaces transformed to _.
98 In at least some test runners this will allow you to select and fil‐
99 ter on test name.
100 Request Parameters
102 ┌────────────────────┬───────────────────────┬─────────────────────┐
103 │Key │ Description │ Notes │
104 ├────────────────────┼───────────────────────┼─────────────────────┤
105 │any uppercase │ Any such key is │ │
106 │string │ considered an HTTP │ │
107 │ │ method, with the │ │
108 │ │ corresponding value │ │
109 │ │ expressing the URL. │ │
110 │ │ │ │
111 │ │ This is a shortcut │ │
112 │ │ combining method │ │
113 │ │ and url into a sin‐ │ │
114 │ │ gle statement: │ │
115 │ │ │ │
116 │ │ GET: /index │ │
117 │ │ │ │
118 │ │ corresponds │ │
119 │ │ to: │ │
120 │ │ │ │
121 │ │ method: GET │ │
122 │ │ url: /index │ │
123 ├────────────────────┼───────────────────────┼─────────────────────┤
124 │method │ The HTTP request │ defaults to GET │
125 │ │ method. │ │
126 └────────────────────┴───────────────────────┴─────────────────────┘
127 │url │ The URL to request. │ Either this or the │
134 │ │ This can either be a │ shortcut above is │
135 │ │ full path (e.g. │ required │
136 │ │ "/index") or a fully │ │
137 │ │ qualified URL (i.e. │ │
138 │ │ including host and │ │
139 │ │ scheme, e.g. "‐ │ │
140 │ │ http://exam‐ │ │
141 │ │ ple.org/index") — see │ │
142 │ │ host for details. │ │
143 ├────────────────────┼───────────────────────┼─────────────────────┤
144 │request_headers │ A dictionary of │ │
145 │ │ key-value pairs rep‐ │ │
146 │ │ resenting request │ │
147 │ │ header names and val‐ │ │
148 │ │ ues. These will be │ │
149 │ │ added to the con‐ │ │
150 │ │ structed request. │ │
151 ├────────────────────┼───────────────────────┼─────────────────────┤
152 │query_parameters │ A dictionary of query │ │
153 │ │ parameters that will │ │
154 │ │ be added to the url │ │
155 │ │ as query string. If │ │
156 │ │ that URL already con‐ │ │
157 │ │ tains a set of query │ │
158 │ │ parameters, those wil │ │
159 │ │ be extended. See │ │
160 │ │ example for a demon‐ │ │
161 │ │ stration of how the │ │
162 │ │ data is structured. │ │
163 ├────────────────────┼───────────────────────┼─────────────────────┤
164 │data │ A representation to │ │
165 │ │ pass as the body of a │ │
166 │ │ request. Note that │ │
167 │ │ content-type in │ │
168 │ │ request_headers │ │
169 │ │ should also be set — │ │
170 │ │ see Data for details. │ │
171 ├────────────────────┼───────────────────────┼─────────────────────┤
172 │redirects │ If True, redirects │ defaults to False │
173 │ │ will automatically be │ │
174 │ │ followed. │ │
175 ├────────────────────┼───────────────────────┼─────────────────────┤
176 │ssl │ Determines whether │ defaults to False │
177 │ │ the request uses SSL │ │
178 │ │ (i.e. HTTPS). Note │ │
179 │ │ that the url's scheme │ │
180 │ │ takes precedence if │ │
181 │ │ present — see host │ │
182 │ │ for details. │ │
183 └────────────────────┴───────────────────────┴─────────────────────┘
184 Response Expectations
186 ┌────────────────────┬─────────────────────┬─────────────────┐
187 │Key │ Description │ Notes │
188 └────────────────────┴─────────────────────┴─────────────────┘
189 │status │ The expected │ defaults to 200 │
200 │ │ response status │ │
201 │ │ code. Multiple │ │
202 │ │ acceptable response │ │
203 │ │ codes may be pro‐ │ │
204 │ │ vided, separated by │ │
205 │ │ || (e.g. 302 || 301 │ │
206 │ │ — note, however, │ │
207 │ │ that this indicates │ │
208 │ │ ambiguity, which is │ │
209 │ │ generally undesir‐ │ │
210 │ │ able). │ │
211 ├────────────────────┼─────────────────────┼─────────────────┤
212 │response_headers │ A dictionary of │ │
213 │ │ key-value pairs │ │
214 │ │ representing │ │
215 │ │ expected response │ │
216 │ │ header names and │ │
217 │ │ values. If a │ │
218 │ │ header's value is │ │
219 │ │ wrapped in /.../, │ │
220 │ │ it will be treated │ │
221 │ │ as a regular │ │
222 │ │ expression to │ │
223 │ │ search for in the │ │
224 │ │ response header. │ │
225 ├────────────────────┼─────────────────────┼─────────────────┤
226 │response_forbid‐ │ A list of headers │ │
227 │den_headers │ which must not be │ │
228 │ │ present. │ │
229 ├────────────────────┼─────────────────────┼─────────────────┤
230 │response_strings │ A list of string │ │
231 │ │ fragments expected │ │
232 │ │ to be present in │ │
233 │ │ the response body. │ │
234 ├────────────────────┼─────────────────────┼─────────────────┤
235 │response_json_paths │ A dictionary of │ │
236 │ │ JSONPath rules │ │
237 │ │ paired with │ │
238 │ │ expected matches. │ │
239 │ │ Using this rule │ │
240 │ │ requires that the │ │
241 │ │ content being sent │ │
242 │ │ from the server is │ │
243 │ │ JSON (i.e. a con‐ │ │
244 │ │ tent type of appli‐ │ │
245 │ │ cation/json or con‐ │ │
246 │ │ taining +json) │ │
247 │ │ │ │
248 │ │ If the value is │ │
249 │ │ wrapped in /.../ │ │
250 │ │ the result of the │ │
251 │ │ JSONPath query will │ │
252 │ │ be searched for the │ │
253 │ │ value as a regular │ │
254 │ │ expression. │ │
255 └────────────────────┴─────────────────────┴─────────────────┘
256 │poll │ A dictionary of two │ │
266 │ │ keys: │ │
267 │ │ │ │
268 │ │ · count: An │ │
269 │ │ integer │ │
270 │ │ stating │ │
271 │ │ the number │ │
272 │ │ of times │ │
273 │ │ to attempt │ │
274 │ │ this test │ │
275 │ │ before │ │
276 │ │ giving up. │ │
277 │ │ │ │
278 │ │ · delay: A │ │
279 │ │ floating │ │
280 │ │ point num‐ │ │
281 │ │ ber of │ │
282 │ │ seconds to │ │
283 │ │ delay │ │
284 │ │ between │ │
285 │ │ attempts. │ │
286 │ │ │ │
287 │ │ This makes │ │
288 │ │ it possible │ │
289 │ │ to poll for │ │
290 │ │ a resource │ │
291 │ │ created via │ │
292 │ │ an asynchro‐ │ │
293 │ │ nous │ │
294 │ │ request. Use │ │
295 │ │ with cau‐ │ │
296 │ │ tion. │ │
297 └────────────────────┴─────────────────────┴─────────────────┘
298 Note that many of these items allow substitutions.
300 Default values for a file's tests may be provided via the top-level
302 defaults category. These take precedence over the global defaults
303 (explained below).
304 For examples see the gabbi tests, example and the gabbi-demo tutorial.
306
308 The top-level fixtures category contains a sequence of named fixtures.
309
311 response_* keys are examples of Response Handlers. Custom handlers may
312 be created by test authors for specific use cases. See handlers for
313 more information.
314
316 There are a number of magical variables that can be used to make refer‐
317 ence to the state of a current test, the one just prior or any test
318 prior to the current one. The variables are replaced with real values
319 during test processing.
320 Global
322 · $ENVIRON['<environment variable>']: The name of an environment vari‐
323 able. Its value will replace the magical variable. If the string
324 value of the environment variable is "True" or "False" then the
325 resulting value will be the corresponding boolean, not a string.
326 Current Test
328 · $SCHEME: The current scheme/protocol (usually http or https).
329 · $NETLOC: The host and potentially port of the request.
331 Immediately Prior Test
333 · $COOKIE: All the cookies set by any Set-Cookie headers in the prior
334 response, including only the cookie key and value pairs and no meta‐
335 data (e.g. expires or domain).
336 · $URL: The URL defined in the prior request, after substitutions have
338 been made. For backwards compatibility with earlier releases
339 $LAST_URL may also be used, but if $HISTORY (see below) is being
340 used, $URL must be used.
341 · $LOCATION: The location header returned in the prior response.
343 · $HEADERS['<header>']: The value of any header from the prior
345 response.
346 · $RESPONSE['<json path>']: A JSONPath query into the prior response.
348 See jsonpath for more on formatting.
349 Any Previous Test
351 · $HISTORY['<test name>'].<magical variable expression>: Any variable
352 which refers to a prior test may be used in an expression that refers
353 to any earlier test in the same file by identifying the target test
354 by its name in a $HISTORY dictionary. For example, to refer to a
355 value in a JSON object in the response of a test named post json:
356 $HISTORY['post json'].$RESPONSE['$.key']
358 This is a very powerful feature that could lead to test that are dif‐
360 ficult for humans to read. Take care to optimize for the maintainers
361 that will come after you, not yourself.
362 NOTE:
364 Where a single-quote character, ', is shown in the variables above
365 you may also use a double-quote character, ", but in any given
366 expression the same character must be used at both ends.
367 All of these variables may be used in all of the following fields:
369 · url
371 · query_parameters
373 · data
375 · request_headers (in both the key and value)
377 · response_strings
379 · response_json_paths (in both the key and value, see json path substi‐
381 tution for more info)
382 · response_headers (in both the key and value)
384 · response_forbidden_headers
386 · count and delay fields of poll
388 With these variables it ought to be possible to traverse an API without
390 any explicit statements about the URLs being used. If you need a
391 replacement on a field that is not currently supported please raise an
392 issue or provide a patch.
393 As all of these features needed to be tested in the development of
395 gabbi itself, the gabbi tests are a good source of examples on how to
396 use the functionality. See also example for a collection of examples
397 and the gabbi-demo tutorial.
398
400 The data key has some special handing to allow for a bit more flexibil‐
401 ity when doing a POST or PUT:
402 · If the value is not a string (that is, it is a sequence or structure)
404 it is treated as a data structure that will be turned into a string
405 by the dumps method on the relevant content handler. For example if
406 the content-type of the body is application/json the data structure
407 will be turned into a JSON string.
408 · If the value is a string that begins with <@ then the rest of the
410 string is treated as a filepath to be loaded. The path is relative to
411 the test directory and may not traverse up into parent directories.
412 · If the value is an undecorated string, that's the value.
414 NOTE:
416 When reading from a file care should be taken to ensure that a rea‐
417 sonable content-type is set for the data as this will control if any
418 encoding is done of the resulting string value. If it is text, json,
419 xml or javascript it will be encoded to UTF-8.
420 To run gabbi tests with a test harness they must be generated in some
422 fashion and then run. This is accomplished by a test loader. Initially
423 gabbi only supported those test harnesses that supported the load_tests
424 protocol in UnitTest. It now possible to also build and run tests with
425 pytest with some limitations described below.
426 NOTE:
428 It is also possible to run gabbi tests from the command line. See
429 runner.
430 NOTE:
432 By default gabbi will load YAML files using the safe_load function.
433 This means only basic YAML types are allowed in the file. For most
434 use cases this is fine. If you need custom types (for example, to
435 match NaN) it is possible to set the safe_yaml parameter of
436 build_tests() to False. If custom types are used, please keep in
437 mind that this can limit the portability of the YAML files to other
438 contexts.
439 WARNING:
441 If test are being run with a runner that supports concurrency (such
442 as testrepository) it is critical that the test runner is informed
443 of how to group the tests into their respective suites. The usual
444 way to do this is to use a regular expression that groups based on
445 the name of the yaml files. For example, when using testrepository
446 the .testr.conf file needs an entry similar to the following:
447 group_regex=gabbi\.suitemaker\.(test_[^_]+_[^_]+)
449
451 To run the tests with a load_tests style loader a test file containing
452 a load_tests method is required. That will look a bit like:
453 """A sample test module."""
455 # For pathname munging
457 import os
458 # The module that build_tests comes from.
460 from gabbi import driver
461 # We need access to the WSGI application that hosts our service
463 from myapp import wsgiapp
464 # We're using fixtures in the YAML files, we need to know where to
467 # load them from.
468 from myapp.test import fixtures
469 # By convention the YAML files are put in a directory named
471 # "gabbits" that is in the same directory as the Python test file.
472 TESTS_DIR = 'gabbits'
473 def load_tests(loader, tests, pattern):
476 """Provide a TestSuite to the discovery process."""
477 test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
478 # Pass "require_ssl=True" as an argument to force all tests
479 # to use SSL in requests.
480 return driver.build_tests(test_dir, loader,
481 intercept=wsgiapp.app,
482 fixture_module=fixtures)
483 For details on the arguments available when building tests see
486 build_tests().
487 Once the test loader has been created, it needs to be run. There are
489 many options. Which is appropriate depends very much on your environ‐
490 ment. Here are some examples using unittest or testtools that require
491 minimal knowledge to get started.
492 By file:
494 python -m testtools.run -v test/test_loader.py
496 By module:
498 python -m testttols.run -v test.test_loader
500 python -m unittest -v test.test_loader
502 Using test discovery to locate all tests in a directory tree:
504 python -m testtools.run discover
506 python -m unittest discover test
508 See the source distribution and the tutorial repo for more advanced
510 options, including using testrepository and subunit.
511
513 Since pytest does not support the load_tests system, a different way of
514 generating tests is required. Two techniques are supported.
515 The original method (described below) used yield statements to generate
517 tests which pytest would collect. This style of tests is deprecated as
518 of pytest>=3.0 so a new style using pytest fixtures has been developed.
519 pytest >= 3.0
521 In the newer technique, a test file is created that uses the
522 pytest_generate_tests hook. Special care must be taken to always import
523 the test_pytest method which is the base test that the pytest hook
524 parametrizes to generate the tests from the YAML files. Without the
525 method, the hook will not be called and no tests generated.
526 Here is a simple example file:
528 """A sample pytest module for pytest >= 3.0."""
530 # For pathname munging
532 import os
533 # The module that py_test_generator comes from.
535 from gabbi import driver
536 # We need test_pytest so that pytest test collection works properly.
538 # Without this, the pytest_generate_tests method below will not be
539 # called.
540 from gabbi.driver import test_pytest # noqa
541 # We need access to the WSGI application that hosts our service
543 from myapp import wsgiapp
544 # We're using fixtures in the YAML files, we need to know where to
546 # load them from.
547 from myapp.test import fixtures
548 # By convention the YAML files are put in a directory named
550 # "gabbits" that is in the same directory as the Python test file.
551 TESTS_DIR = 'gabbits'
552 def pytest_generate_tests(metafunc):
555 test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
556 driver.py_test_generator(
557 test_dir, intercept=wsgiapp.app,
558 fixture_module=fixtures, metafunc=metafunc)
559 This can then be run with the usual pytest commands. For example:
562 py.test -svx pytest3.0-example.py
564 pytest < 3.0
566 When using the older technique, test file must be created that calls
567 py_test_generator() and yields the generated tests. That will look a
568 bit like this:
569 """A sample pytest module."""
571 # For pathname munging
573 import os
574 # The module that build_tests comes from.
576 from gabbi import driver
577 # We need access to the WSGI application that hosts our service
579 from myapp import wsgiapp
580 # We're using fixtures in the YAML files, we need to know where to
582 # load them from.
583 from myapp.test import fixtures
584 # By convention the YAML files are put in a directory named
586 # "gabbits" that is in the same directory as the Python test file.
587 TESTS_DIR = 'gabbits'
588 def test_gabbits():
591 test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
592 # Pass "require_ssl=True" as an argument to force all tests
593 # to use SSL in requests.
594 test_generator = driver.py_test_generator(
595 test_dir, intercept=wsgiapp.app,
596 fixture_module=fixtures)
597 for test in test_generator:
599 yield test
600 This can then be run with the usual pytest commands. For example:
603 py.test -svx pytest-example.py
605 The older technique will continue to work with all versions of
607 pytest<4.0 but >=3.0 will produce warnings. If you want to use the
608 older technique but not see the warnings add --disable-pytest-warnings
609 parameter to the invocation of py.test.
610 What follows is a commented example of some tests in a single file
612 demonstrating many of the format features. See loader for the Python
613 needed to integrate with a testing harness.
614 # Fixtures can be used to set any necessary configuration, such as a
617 # persistence layer, and establish sample data. They operate per
618 # file. They are context managers, each one wrapping the next in the
619 # sequence.
620 fixtures:
622 - ConfigFixture
623 - SampleDataFixture
624 # There is an included fixture named "SkipAllFixture" which can be
626 # used to declare that all the tests in the given file are to be
627 # skipped.
628 # Each test file can specify a set of defaults that will be used for
630 # every request. This is useful for always specifying a particular
631 # header or always requiring SSL. These values will be used on every
632 # test in the file unless overriden. Lists and dicts are merged one
633 # level deep, except for "data" which is copied verbatim whether it
634 # is a string, list or dict (it can be all three).
635 defaults:
637 ssl: True
638 request_headers:
639 x-my-token: zoom
640 # The tests themselves are a list under a "tests" key. It's useful
642 # to use plenty of whitespace to help readability.
643 tests:
645 # Each request *must* have a name which is unique to the file. When it
647 # becomes a TestCase the name will be lowercased and spaces will
648 # become "_". Use that generated name when limiting test runs.
649 - name: a test for root
651 desc: Some explanatory text that could be used by other tooling
652 # The URL can either be relative to a host specified elsewhere or
654 # be a fully qualified "http" or "https" URL. *You* are responsible
655 # for url-encoding the URL.
656 url: /
658 method: GET
659 # If no status or method are provided they default to "200" and
661 # "GET".
662 # Instead of explicitly stating "url" and "method" you can join
664 # those two keys into one key representing the method. The method
665 # *must* be uppercase.
666 - name: another test for root
668 desc: Same test as above but with GET key
669 GET: /
670 # A single test can override settings in defaults (set above).
672 - name: root without ssl redirects
674 ssl: False
675 GET: /
676 status: 302
677 # When evaluating response headers it is possible to use a regular
679 # expression to not have to test the whole value. Regular expressions match
680 # anywhere in the output, not just at the beginning.
681 response_headers:
683 location: /^https/
684 # By default redirects will not be followed. This can be changed.
686 - name: follow root without ssl redirect
688 ssl: False
689 redirects: True
690 GET: /
691 status: 200 # This is the response code after the redirect.
692 # URLs can express query parameters in two ways: either in the url
694 # value directly, or as query_parameters. If both are used then
695 # query_parameters are appended. In this example the resulting URL
696 # will be equivalient to
697 # /foo?section=news&article=1&article=2&date=yesterday
698 # but not necessarily in that order.
699 - name: create a url with parameters
701 GET: /foo?section=news
702 query_parameters:
703 article:
704 - 1
705 - 2
706 date: yesterday
707 # Request headers can be used to declare media-type choices and
709 # experiment with authorization handling (amongst other things).
710 # Response headers allow evaluating headers in the response. These
711 # two together form the core value of gabbi.
712 - name: test accept
714 GET: /resource
715 request_headers:
716 accept: application/json
717 response_headers:
718 content-type: /application/json/
719 # If a header must not be present in a response at all that can be
721 # expressed in a test as follows.
722 - name: test forbidden headers
724 GET: /resource
725 response_forbidden_headers:
726 - x-special-header
727 # All of the above requests have defaulted to a "GET" method. When
729 # using "POST", "PUT" or "PATCH", the "data" key provides the
730 # request body.
731 - name: post some text
733 POST: /text_repo
734 request_headers:
735 content-type: text/plain
736 data: "I'm storing this"
737 status: 201
738 # If the data is not a string, it will be transformed into JSON.
740 # You must supply an appropriate content-type request header.
741 - name: post some json
743 POST: /json_repo
744 request_headers:
745 content-type: application/json
746 data:
747 name: smith
748 abode: castle
749 status: 201
750 # If the data is a string prepended with "<@" the value will be
752 # treated as the name of a file in the same directory as the YAML
753 # file. Again, you must supply an appropriate content-type. If the
754 # content-type is one of several "text-like" types, the content will
755 # be assumed to be UTF-8 encoded.
756 - name: post an image
758 POST: /image_repo
759 request_headers:
760 content-type: image/png
761 data: <@kittens.png
762 # A single request can be marked to be skipped.
764 - name: patch an image
766 skip: patching images not yet implemented
767 PATCH: /image_repo/12d96fb8-e78c-11e4-8c03-685b35afa334
768 # Or a single request can be marked that it is expected to fail.
770 - name: check allow headers
772 desc: the framework doesn't do allow yet
773 xfail: True
774 PUT: /post_only_url
775 status: 405
776 response_headers:
777 allow: POST
778 # The body of a response can be evaluated with response handlers.
780 # The most simple checks for a set of strings anywhere in the
781 # response. Note that the strings are members of a list.
782 - name: check for css file
784 GET: /blog/posts/12
785 response_strings:
786 - normalize.css
787 # For JSON responses, JSONPath rules can be used.
789 - name: post some json get back json
791 POST: /json_repo
792 request_headers:
793 content-type: application/json
794 data:
795 name: smith
796 abode: castle
797 status: 201
798 response_json_paths:
799 $.name: smith
800 $.abode: castle
801 # Requests run in sequence. One test can make reference to the test
803 # immediately prior using some special variables.
804 # "$LOCATION" contains the "location" header in the previous
805 # response.
806 # "$HEADERS" is a pseudo dictionary containing all the headers of
807 # the previous response.
808 # "$ENVIRON" is a pseudo dictionary providing access to the current
809 # environment.
810 # "$RESPONSE" provides access to the JSON in the prior response, via
811 # JSONPath. See http://jsonpath-rw.readthedocs.io/ for
812 # jsonpath-rw formatting.
813 # $SCHEME and $NETLOC provide access to the current protocol and
814 # location (host and port).
815 - name: get the thing we just posted
817 GET: $LOCATION
818 request_headers:
819 x-magic-exchange: $HEADERS['x-magic-exchange']
820 x-token: $ENVIRON['OS_TOKEN']
821 response_json_paths:
822 $.name: $RESPONSE['$.name']
823 $.abode: $RESPONSE['$.abode']
824 response_headers:
825 content-location: /$SCHEME://$NETLOC/
826 # For APIs where resource creation is asynchronous it can be
828 # necessary to poll for the resulting resource. First we create the
829 # resource in one test. The next test uses the "poll" key to loop
830 # with a delay for a set number of times.
831 - name: create asynch
833 POST: /async_creator
834 request_headers:
835 content-type: application/json
836 data:
837 name: jones
838 abode: bungalow
839 status: 202
840 - name: poll for created resource
842 GET: $LOCATION
843 poll:
844 count: 10 # try up to ten times
845 delay: .5 # wait .5 seconds between each try
846 response_json_paths:
847 $.name: $RESPONSE['$.name']
848 $.abode: $RESPONSE['$.abode']
849 Gabbi supports JSONPath both for validating JSON response bodies and
852 within substitutions.
853 JSONPath expressions are provided by jsonpath_rw, with jsonpath_rw_ext
855 custom extensions to address common requirements:
856 1. Sorting via sorted and [/property].
858 2. Filtering via [?property = value].
860 3. Returning the respective length via len.
862 (These apply both to arrays and key-value pairs.)
864 Here is a JSONPath example demonstrating some of these features. Given
866 JSON data as follows:
867 {
869 "pets": [
870 {"type": "cat", "sound": "meow"},
871 {"type": "dog", "sound": "woof"}
872 ]
873 }
874 If the ordering of the list in pets is predictable and reliable it is
876 relatively straightforward to test values:
877 response_json_paths:
879 # length of list is two
880 $.pets.`len`: 2
881 # sound of second item in list is woof
882 $.pets[1].sound: woof
883 If the ordering is not predictable additional effort is required:
885 response_json_paths:
887 # sort by type
888 $.pets[/type][0].sound: meow
889 # sort by type, reversed
890 $.pets[\type][0].sound: woof
891 # all the sounds
892 $.pets[/type]..sound: ['meow', 'woof']
893 # filter by type = dog
894 $.pets[?type = "dog"].sound: woof
895 If it is necessary to validate the entire JSON response use a JSONPath
897 of $:
898 response_json_paths:
900 $:
901 pets:
902 - type: cat
903 sound: meow
904 - type: dog
905 sound: woof
906 This is not a technique that should be used frequently as it can lead
908 to difficult to read tests and it also indicates that your gabbi tests
909 are being used to test your serializers and data models, not just your
910 API interactions.
911 It is also possible to read raw JSON from disk for either all or some
913 of a JSON response:
914 response_json_paths:
916 $: @<data.json
917 or:
919 response_json_paths:
921 $.pets: <@pets.json
922 $.pets[0]: <@cat.json
923 Examples like this can be found in one of gabbi's own tests.
925 If it is desired to load YAML files like the JSON ones above, two
927 things must be done:
928 1. The YAMLDiskLoadingJSONHandler custom content handler must be passed
930 to the driver through the content_handlers argument. See Extensions
931 on how to do this.
932 2. The YAML files to load must be placed in a subdirectory to prevent
934 the test runner from consuming them as test files to run:
935 response_json_paths:
937 $: @<subdir/values.yaml
938 When reading from disk you can apply the same JSONPath by adding a ':'
940 to the end of your file name. This allows you to store multiple API
941 responses into a single file to reduce file management when construct‐
942 ing your tests.
943 Given JSON data as follows:
945 {
947 "values": [{
948 "pets": [{
949 "type": "cat",
950 "sound": "meow"
951 }, {
952 "type": "dog",
953 "sound": "woof"
954 }]
955 }, {
956 "people": [{
957 "name": "chris",
958 "id": 1
959 }, {
960 "name": "justin",
961 "id": 2
962 }]
963 }]
964 }
965 You can write your tests like the following:
967 response_json_paths:
969 $.pets: <@pets.json
970 $.pets[?type = "cat"].sound: <@values.json:$.values[0].pets[?type = "cat"].sound
971 Although placing more than one API response into a single JSON file may
973 seem convenient, keep in mind there is a tradeoff in readability that
974 should not be overlooked before implementing this technique.
975 Examples like this can be found in one of gabbi's yaml-from-disk tests.
977 There are more JSONPath examples in example and in the jsonpath_rw and
979 jsonpath_rw_ext documentation.
980
982 Substitutions can be made in both the left (query) and right (expected)
983 hand sides of the json path expression. When subtitutions are used in
984 the query, care must be taken to ensure proper quoting of the resulting
985 value. For example if there is a uuid (with hyphens) at
986 $RESPONSE['$.id'] then this expression may fail:
987 $.nested.structure.$RESPONSE['$.id'].name: foobar
989 as it will evaluate to something like:
991 $.nested.structure.ADC8AAFC-D564-40D1-9724-7680D3C010C2.name: foobar
993 which may be treated as an arithemtic expression by the json path
995 parser. The test author should write:
996 $.nested.structure["$RESPONSE['$.id']"].name: foobar
998 to quote the result of the substitution.
1000 The target host is the host on which the API to be tested can be found.
1002 Gabbi intends to preserve the flow and semantics of HTTP interactions
1003 as much as possible, and every HTTP request needs to be directed at a
1004 host of some form. Gabbi provides three ways to control this:
1005 · Using wsgi-intercept to provide a fake socket and WSGI environment on
1007 an arbitrary host and port attached to a WSGI application (see
1008 intercept examples).
1009 · Using fully qualified url values in the YAML defined tests (see full
1011 examples).
1012 · Using a host and (optionally) port defined at test build time (see
1014 live examples).
1015 The intercept and live methods are mutually exclusive per test builder,
1017 but either kind of test can freely intermix fully qualified URLs into
1018 the sequence of tests in a YAML file.
1019 For test driven development and local tests the intercept style of
1021 testing lowers test requirements (no web server required) and is fast.
1022 Interception is performed as part of fixtures processing as the most
1023 deeply nested fixture. This allows any configuration or database setup
1024 to be performed prior to the WSGI application being created.
1025 For the implementation of the above see build_tests().
1027 Each suite of tests is represented by a single YAML file, and may
1029 optionally use one or more fixtures to provide the necessary environ‐
1030 ment required by the tests in that file.
1031 Fixtures are implemented as nested context managers. Subclasses of
1033 GabbiFixture must implement start_fixture and stop_fixture methods for
1034 creating and destroying, respectively, any resources managed by the
1035 fixture. While the subclass may choose to implement __init__ it is
1036 important that no exceptions are thrown in that method, otherwise the
1037 stack of context managers will fail in unexpected ways. Instead ini‐
1038 tialization of real resources should happen in start_fixture.
1039 At this time there is no mechanism for the individual tests to have any
1041 direct awareness of the fixtures. The fixtures exist, conceptually, on
1042 the server side of the API being tested.
1043 Fixtures may do whatever is required by the testing environment, how‐
1045 ever there are two common scenarios:
1046 · Establishing (and then resetting when a test suite has finished) any
1048 baseline configuration settings and persistence systems required for
1049 the tests.
1050 · Creating sample data for use by the tests.
1052 If a fixture raises unittest.case.SkipTest during start_fixture all the
1054 tests in the current file will be skipped. This makes it possible to
1055 skip the tests if some optional configuration (such as a particular
1056 type of database) is not available.
1057 If an exception is raised while a fixture is being used, information
1059 about the exception will be stored on the fixture so that the stop_fix‐
1060 ture method can decide if the exception should change how the fixture
1061 should clean up. The exception information can be found on exc_type,
1062 exc_value and traceback method attributes.
1063 If an exception is raised when a fixture is started (in start_fixture)
1065 the first test in the suite using the fixture will be marked with an
1066 error using the traceback from the exception and all the tests in the
1067 suite will be skipped. This ensures that fixture failure is adequately
1068 captured and reported by test runners.
1069 In some contexts (for example CI environments with a large number of
1071 tests being run in a broadly concurrent environment where output is
1072 logged to a single file) it can be important to capture and consolidate
1073 stray output that is produced during the tests and display it associ‐
1074 ated with an individual test. This can help debugging and avoids unus‐
1075 able output that is the result of multiple streams being interleaved.
1076 Inner fixtures have been added to support this. These are fixtures more
1078 in line with the tradtional unittest concept of fixtures: a class on
1079 which setUp and cleanUp is automatically called.
1080 build_tests() accepts a named parameter arguments of inner_fixtures.
1082 The value of that argument may be an ordered list of fixtures.Fixture
1083 classes that will be called when each individual test is set up.
1084 An example fixture that could be useful is the FakeLogger.
1086 NOTE:
1088 At this time inner_fixtures are not supported when using the pytest
1089 loader.
1090 Content handlers are responsible for preparing request data and evalu‐
1092 ating response data based on the content-type of the request and
1093 response. A content handler operates as follows:
1094 · Structured YAML data provided via the data attribute is converted to
1096 a string or bytes sequence and used as request body.
1097 · The response body (a string or sequence of bytes) is transformed into
1099 a content-type dependent structure and stored in an internal
1100 attribute named response_data that is:
1101 · used when evaluating the response body
1103 · used in $RESPONSE[] substitutions
1105 By default, gabbi provides content handlers for JSON. In that content
1107 handler the data test key is converted from structured YAML into a JSON
1108 string. Response bodies are converted from a JSON string into a data
1109 structure in response_data that is used when evaluating
1110 response_json_paths entries in a test or doing JSONPath-based
1111 $RESPONSE[] substitutions.
1112 A YAMLDiskLoadingJSONHandler has been added to extend the JSON handler.
1114 It works the same way as the JSON handler except for when evaluating
1115 the response_json_paths handle, data that is read from disk can be
1116 either in JSON or YAML format. The YAMLDiskLoadingJSONHandler is not
1117 enabled by default and must be added as shown in the Extensions section
1118 in order to be used in the tests.
1119 Further content handlers can be added as extensions. Test authors may
1121 need these extensions for their own suites, or enterprising developers
1122 may wish to create and distribute extensions for others to use.
1123 NOTE:
1125 One extension that is likely to be useful is a content handler that
1126 turns data into url-encoded form data suitable for POST and turns an
1127 HTML response into a DOM object.
1128
1130 Content handlers are an evolution of the response handler concept in
1131 earlier versions gabbi. To preserve backwards compatibility with exist‐
1132 ing response handlers, old style response handlers are still allowed,
1133 but new handlers should implement the content handler interface
1134 (described below).
1135 Registering additional custom handlers is done by passing a subclass of
1137 ContentHandler to build_tests():
1138 driver.build_tests(test_dir, loader, host=None,
1140 intercept=simple_wsgi.SimpleWsgi,
1141 content_handlers=[MyContentHandler])
1142 If pytest is being used:
1144 driver.py_test_generator(test_dir, intercept=simple_wsgi.SimpleWsgi,
1146 content_handlers=[MyContenHandler])
1147 Gabbi provides an additional custom handler named YAMLDiskLoadingJSON‐
1149 Handler. This can be used for loading JSON and YAML files from disk
1150 when evaluating the response_json_paths handle.
1151 WARNING:
1153 YAMLDiskLoadingJSONHandler shares the same content-type as the
1154 default JSONHandler. When there are multiple handlers listed that
1155 accept the same content-type, the one that is earliest in the list
1156 will be used.
1157 With gabbi-run, custom handlers can be loaded via the --response-han‐
1159 dler option -- see load_response_handlers() for details.
1160 NOTE:
1162 The use of the --response-handler argument is done to preserve back‐
1163 wards compatibility and avoid excessive arguments. Both types of
1164 handler may be passed to the argument.
1165 Implementation Details
1167 Creating a content handler requires subclassing ContentHandler and
1168 implementing several methods. These methods are described below, but
1169 inspecting JSONHandler will be instructive in highlighting required
1170 arguments and techniques.
1171 To provide a response_<something> response-body evaluator a subclass
1173 must define:
1174 · test_key_suffix: This, along with the prefix response_, forms the key
1176 used in the test structure. It is a class level string.
1177 · test_key_value: The key's default value, either an empty list ([]) or
1179 empty dict ({}). It is a class level value.
1180 · action: An instance method which tests the expected values against
1182 the HTTP response - it is invoked for each entry, with the parameters
1183 depending on the default value. The arguments to action are (in
1184 order):
1185 · self: The current instance.
1187 · test: The currently active HTTPTestCase
1189 · item: The current entry if test_key_value is a list, otherwise the
1191 key half of the key/value pair at this entry.
1192 · value: None if test_key_value is a list, otherwise the value half
1194 of the key/value pair at this entry.
1195 To translate request or response bodies to or from structured data a
1197 subclass must define an accepts method. This should return True if this
1198 class is willing to translate the provided content-type. During request
1199 processing it is given the value of the content-type header that will
1200 be sent in the request. During response processing it is given the
1201 value of the content-type header of the response. This makes it possi‐
1202 ble to handle different request and response bodies in the same han‐
1203 dler, if desired. For example a handler might accept applica‐
1204 tion/x-www-form-urlencoded and text/html.
1205 If accepts is defined two additional static methods should be defined:
1207 · dumps: Turn structured Python data from the data key in a test into a
1209 string or byte stream. The optional test param allows you to access
1210 the current test case which may help with manipulations for custom
1211 content handlers, e.g. multipart/form-data needs to add a boundary to
1212 the Content-Type header in order to mark the appropriate sections of
1213 the body.
1214 · loads: Turn a string or byte stream in a response into a Python data
1216 structure. Gabbi will put this data on the response_data attribute on
1217 the test, where it can be used in the evaluations described above (in
1218 the action method) or in $RESPONSE handling. An example usage here
1219 would be to turn HTML into a DOM.
1220 · load_data_file: Load data from disk into a Python data structure.
1222 Gabbi will call this method when response_<something> contains an
1223 item where the right hand side value starts with <@. The test param
1224 allows you to access the current test case and provides a
1225 load_data_file method which should be used because it verifies the
1226 data is loaded within the test diectory and returns the file source
1227 as a string. The load_data_file method was introduced to re-use the
1228 JSONHandler in order to support loading YAML files from disk through
1229 the implementation of an additional custom handler, see YAMLDiskLoad‐
1230 ingJSONHandler for details.
1231 Finally if a replacer class method is defined, then when a $RESPONSE
1233 substitution is encountered, replacer will be passed the response_data
1234 of the prior test and the argument within the $RESPONSE.
1235 Please see the JSONHandler source for additional detail.
1237 If there is a running web service that needs to be tested and creating
1239 a test loader with build_tests() is either inconvenient or overkill it
1240 is possible to run YAML test files directly from the command line with
1241 the console-script gabbi-run. It accepts YAML on stdin or as multiple
1242 file arguments, and generates and runs tests and outputs a summary of
1243 the results.
1244 The provided YAML may not use custom fixtures but otherwise uses the
1246 default format. host information is either expressed directly in the
1247 YAML file or provided on the command line:
1248 gabbi-run [host[:port]] < /my/test.yaml
1250 or:
1252 gabbi-run http://host:port < /my/test.yaml
1254 To test with one or more files the following command syntax may be
1256 used:
1257 gabbi-run http://host:port -- /my/test.yaml /my/other.yaml
1259 NOTE:
1261 The filename arguments must come after a -- and all other arguments
1262 (host, port, prefix, failfast) must come before the --.
1263 NOTE:
1265 If files are provided, test output will use names including the name
1266 of the file. If any single file includes an error, the name of the
1267 file will be included in a summary of failed files at the end of the
1268 test report.
1269 To facilitate using the same tests against the same application mounted
1271 in different locations in a WSGI server, a prefix may be provided as a
1272 second argument:
1273 gabbi-run host[:port] [prefix] < /my/test.yaml
1275 or in the target URL:
1277 gabbi-run http://host:port/prefix < /my/test.yaml
1279 The value of prefix will be prepended to the path portion of URLs that
1281 are not fully qualified.
1282 Anywhere host is used, if it is a raw IPV6 address it should be wrapped
1284 in [ and ].
1285 If https is used in the target, then the tests in the provided YAML
1287 will default to ssl: True.
1288 If a -x or --failfast argument is provided then gabbi-run will exit
1290 after the first test failure.
1291 Use -v or --verbose with a value of all, headers or body to turn on
1293 verbosity for all tests being run.
1294 These are informal release notes for gabbi since version 1.0.0, high‐
1296 lighting major features and changes. For more detail see the commit
1297 logs on GitHub.
1298
1300 · Provide the YAMLDiskLoadingJSONHandler class that allows test result
1301 data for response_json_path checks to be loaded from YAML-on-disk.
1302
1304 · Use JSONPath to select a portion of data-on-disk in
1305 response_json_path checks.
1306 · Restrict PyYAML to <4.0.
1308
1310 · Allow listing of tests with no host configured. When host is an empty
1311 string, tests can be listed (for discovery), but will be skipped on
1312 run.
1313
1315 · JSON $RESPONSE substitutions in the data field may be complex types
1316 (lists and dicts), not solely strings.
1317
1319 · When the HTTP response begins with a bad status line, have BadSta‐
1320 tusLine be raised from urllib3.
1321
1323 · Allow substitutions in the key portion of request and response head‐
1324 ers, not just the value.
1325
1327 · Remove support for Python 3.3.
1328 · Make handling of fixture-level skips in pytest actually work.
1330
1332 · Add safe_yaml parameter to build_tests().
1333
1335 · use_prior_test is added to test metadata.
1336 · Extensive cleanups in regular expression handling when constructing
1338 tests from YAML.
1339
1341 jsonpath handling gets two improvements:
1342 · The value side of a response_json_paths entry can be loaded from a
1344 file using the <@file.json syntax also used in data.
1345 · The key side of a response_json_paths entry can use substitutions.
1347 This was already true for the value side.
1348
1350 Substitutions in $RESPONSE handling now preserve numeric types instead
1351 of casting to a string. This is useful when servers are expecting
1352 strong types and tests want to send response data back to the server.
1353
1355 count and delay test keys allow substitutions.
1356 gabbi.driver.build_tests() accepts a verbose parameter to set test ver‐
1357 bosity for an entire session.
1358
1360 Better failure reporting when using gabbi-run with multiple files. Test
1361 names are based on the files and a summary of failed files is provided
1362 at the end of the report.
1363
1365 Effectively capture a failure in a fixture and report the traceback.
1366 Without this some test runners swallow the error and discovering prob‐
1367 lems when developing fixtures can be quite challenging.
1368
1370 Thanks to Samuel Fekete, tests can use the $HISTORY dictionary to refer
1371 to any prior test in the same file, not just the one immediately prior,
1372 when doing substitutions.
1373
1375 Filenames used to read data into tests using the <@ syntax may now use
1376 pathnames relative to the YAML file. See data.
1377 gabbi-run gains a --verbose parameter to force all tests run in a ses‐
1379 sion to run with verbose set.
1380 When using pytest to load tests, a new mechanism is available which
1382 avoids warnings produced in when using a version of pytest greater than
1383 3.0.
1384
1386 When verbosely displaying request and response bodies that are JSON,
1387 pretty print for improved readability.
1388
1390 Allow gabbi-run to accept multiple filenames as command line arguments
1391 instead of reading tests from stdin.
1392
1394 Switch from response handlers to handlers to allow more flexible pro‐
1395 cessing of both response _and_ request bodies.
1396 Add inner fixtures for per test fixtures, useful for output capturing.
1398
1400 Allow the test_loader_name arg to gabbi.driver.build_tests() to over‐
1401 ride the prefix of the pretty printed name of generated tests.
1402
1404 String values in JSONPath matches may be wrapped in /.../` to be
1405 treated as regular expressions.
1406
1408 Better documentation of how to run gabbi in a concurrent environment.
1409 Improved handling of pytest fixtures and test counts.
1410
1412 Add url to gabbi.driver.build_tests() to use instead of host, port and
1413 prefix.
1414
1416 Add require_ssl to gabbi.driver.build_tests() to force use of SSL.
1417
1419 Add $COOKIE substitution.
1420
1422 Correctly support IPV6 hosts.
1423
1425 Add $LAST_URL substitution.
1426
1428 Introduce support for loading and running tests with pytest.
1429
1431 Use urllib3 instead of httplib2 for driving HTTP requests.
1432
1434 Add sorting and filtering to jsonpath handling.
1435
1437 Add the response_forbidden_headers to response expectations.
1438
1440 Instead of:
1441 tests:
1443 - name: a simple get
1444 url: /some/path
1445 method: get
1446 1.7.0 also makes it possible to:
1448 tests:
1450 - name: a simple get
1451 GET: /some/path
1452 Any upper case key is treated as a method.
1454
1456 Enhanced flexibility and colorization when setting tests to be verbose.
1457
1459 Adds the query_parameters key to request parameters.
1460
1462 The start of improvements and extensions to jsonpath handling. In this
1463 case the addition of the len function.
1464
1466 Vastly improved output and behavior in gabbi-run.
1467
1469 Version 1 was the first release with a commitment to a stable format.
1470 Since then new fields have been added but have not been taken away.
1471 The following people have contributed code to gabbi. Thanks to them.
1473 Thanks also to all the people who have made gabbi better by reporting
1474 issues and their successes and failures with using gabbi.
1475 · Chris Dent
1477 · FND
1479 · Mehdi Abaakouk
1481 · Tom Viner
1483 · Jason Myers
1485 · Josh Leeb-du Toit
1487 · Duc Truong
1489 · Zane Bitter
1491 · Ryan Spencer
1493 · Kim Raymoure
1495 · Travis Truman
1497 · Samuel Fekete
1499 · Michael McCune
1501 · Imran Hayder
1503 · Julien Danjou
1505 · Trevor McCasland
1507 · Danek Duvall
1509 · Marc Abramowitz
1511 NOTE:
1513 This section provides a collection of questions with answers that
1514 don't otherwise fit in the rest of the documentation. If something
1515 is missing, please create an issue.
1516 As this document grows it will gain a more refined structure.
1518
1520 Is gabbi only for testing Python-based APIs?
1521 No, you can use gabbi-run to test an HTTP service built in any program‐
1522 ming language.
1523 How do I run just one test?
1525 Each YAML file contains a sequence of tests, each test within each file
1526 has a name. That name is translated to the name of the test by replac‐
1527 ing spaces with an _.
1528 When running tests that are generated dynamically, filtering based on
1530 the test name prior to the test being collected will not work in some
1531 test runners. Test runners that use a --load-list functionality can be
1532 convinced to filter after discovery.
1533 pytest does this directly with the -k keyword flag.
1535 When using testrepository with tox as used in gabbi's own tests it is
1537 possible to pass a filter in the tox command:
1538 tox -epy27 -- get_the_widget
1540 When using testtools.run and similar test runners it's a bit more com‐
1542 plicated. It is necessary to provide the full name of the test as a
1543 list to --load-list:
1544 python -m testtools.run --load-list \
1546 <(echo package.tests.test_api.yamlfile_get_the_widge.test_request)
1547 How do I run just one test, without running prior tests in a sequence?
1549 By default, when you select a single test to run, all tests prior to
1550 that one in a file will be run as well: the file is treated as as
1551 sequence of dependent tests. If you do not want this you can adjust the
1552 use_prior_test test metadata in one of three ways:
1553 · Set it in the YAML file for the one test you are concerned with.
1555 · Set the defaults for all tests in that file.
1557 · set use_prior_test to false when calling build_tests()
1559 Be aware that doing this breaks a fundamental assumption that gabbi
1561 makes about how tests work. Any substitutions will fail.
1562
1564 Can I have variables in my YAML file?
1565 Gabbi provides the $ENVIRON substitution which can operate a bit like
1566 variables that are set elsewhere and then used in the tests defined by
1567 the YAML.
1568 If you find it necessary to have variables within a single YAML file
1570 you take advantage of YAML alias nodes list this:
1571 vars:
1573 - &uuid_1 5613AABF-BAED-4BBA-887A-252B2D3543F8
1574 tests:
1576 - name: send a uuid to a post
1577 POST: /resource
1578 request_headers:
1579 content-type: application/json
1580 data:
1581 uuid: *uuid_1
1582 You can alias all sorts of nodes, not just single items. Be aware that
1584 the replacement of an alias node happens while the YAML is being
1585 loaded, before gabbi does any processing.
1586 How many tests should be put in one YAML file?
1588 For the sake of readability it is best to keep each YAML file rela‐
1589 tively short. Since each YAML file represents a sequence of requests,
1590 it usually makes sense to create a new file when a test is not depen‐
1591 dent on any before it.
1592 It's tempting to put all the tests for any resource or URL in the same
1594 file, but this eventually leads to files that are too long and are thus
1595 difficult to read.
1596
1598 A single HTTP request represented as a subclass of testtools.TestCase
1599 The test case encapsulates the request headers and body and expected
1601 response headers and body. When the test is run an HTTP request is made
1602 using urllib3. Assertions are made against the response.
1603 class gabbi.case.HTTPTestCase(*args, **kwargs)
1605 Bases: testtools.testcase.TestCase
1606 Encapsulate a single HTTP request as a TestCase.
1608 If the test is a member of a sequence of requests, ensure that
1610 prior tests are run.
1611 To keep the test harness happy we need to make sure the setUp
1613 and tearDown are only run once.
1614 assert_in_or_print_output(expected, iterable)
1616 Assert the iterable contains expected or print some out‐
1617 put.
1618 If the output is long, it is limited by either
1620 GABBI_MAX_CHARS_OUTPUT in the environment or the
1621 MAX_CHARS_OUTPUT constant.
1622 base_test = {'data': '', 'desc': '', 'method': 'GET', 'name':
1624 '', 'poll': {}, 'query_parameters': {}, 'redirects': False,
1625 'request_headers': {}, 'skip': '', 'ssl': False, 'status':
1626 '200', 'url': '', 'use_prior_test': True, 'verbose': False,
1627 'xfail': False}
1628 get_content_handler(content_type)
1630 Determine the content handler for this media type.
1631 load_data_file(filename)
1633 Read a file from the current test directory.
1634 replace_template(message, escape_regex=False)
1636 Replace magic strings in message.
1637 run(result=None)
1639 Store the current result handler on this test.
1640 setUp()
1642 Hook method for setting up the test fixture before exer‐
1643 cising it.
1644 tearDown()
1646 Hook method for deconstructing the test fixture after
1647 testing it.
1648 test_request()
1650 Run this request if it has not yet run.
1651 If there is a prior test in the sequence, run it first.
1653 gabbi.case.potentialFailure(func)
1655 Decorate a test method that is expected to fail if 'xfail' is
1656 true.
1657
1659 Generate HTTP tests from YAML files
1660 Each HTTP request is its own TestCase and can be requested to be run in
1662 isolation from other tests. If it is a member of a sequence of
1663 requests, prior requests will be run.
1664 A sequence is represented by an ordered list in a single YAML file.
1666 Each sequence becomes a TestSuite.
1668 An entire directory of YAML files is a TestSuite of TestSuites.
1670 gabbi.driver.build_tests(path, loader, host=None, port=8001, inter‐
1672 cept=None, test_loader_name=None, fixture_module=None, response_han‐
1673 dlers=None, content_handlers=None, prefix='', require_ssl=False,
1674 url=None, inner_fixtures=None, verbose=False, use_prior_test=True,
1675 safe_yaml=True)
1676 Read YAML files from a directory to create tests.
1677 Each YAML file represents a list of HTTP requests.
1679 Parameters
1681 · path -- The directory where yaml files are located.
1683 · loader -- The TestLoader.
1685 · host -- The host to test against. Do not use with
1687 intercept.
1688 · port -- The port to test against. Used with host.
1690 · intercept -- WSGI app factory for wsgi-intercept.
1692 · test_loader_name -- Base name for test classes. Use
1694 this to align the naming of the tests with other tests
1695 in a system.
1696 · fixture_module -- Python module containing fixture
1698 classes.
1699 · response_handers -- ResponseHandler classes.
1701 · content_handlers (List of ContentHandler classes.) --
1703 ContentHandler classes.
1704 · prefix -- A URL prefix for all URLs that are not fully
1706 qualified.
1707 · url -- A full URL to test against. Replaces host, port
1709 and prefix.
1710 · require_ssl -- If True, make all tests default to using
1712 SSL.
1713 · inner_fixtures (List of fixtures.Fixture classes.) -- A
1715 list of Fixtures to use with each individual test
1716 request.
1717 · verbose -- If True or 'all', make tests verbose by
1719 default 'headers' and 'body' are also accepted.
1720 · use_prior_test -- If True, uses prior test to create
1722 ordered sequence of tests
1723 · safe_yaml -- If True, recognizes only standard YAML
1725 tags and not Python object
1726 Return type
1728 TestSuite containing multiple TestSuites (one for each
1729 YAML file)
1730 gabbi.driver.py_test_generator(test_dir, host=None, port=8001, inter‐
1732 cept=None, prefix=None, test_loader_name=None, fixture_module=None,
1733 response_handlers=None, content_handlers=None, require_ssl=False,
1734 url=None, metafunc=None, use_prior_test=True, inner_fixtures=None,
1735 safe_yaml=True)
1736 Generate tests cases for py.test
1737 This uses build_tests to create TestCases and then yields them
1739 in a way that pytest can handle.
1740 gabbi.driver.test_pytest(test, result)
1742 gabbi.driver.test_suite_from_yaml(loader, test_base_name, test_yaml,
1744 test_directory, host, port, fixture_module, intercept, prefix='')
1745 Legacy wrapper retained for backwards compatibility.
1746
1748 The code that creates a suite of tests.
1749 The key piece of code is test_suite_from_dict(). It produces a
1751 gabbi.suite.GabbiSuite containing one or more gabbi.case.HTTPTestCase.
1752 class gabbi.suitemaker.TestBuilder
1754 Bases: type
1755 Metaclass to munge a dynamically created test.
1757 required_attributes = {'has_run': False}
1759 class gabbi.suitemaker.TestMaker(test_base_name, test_defaults,
1761 test_directory, fixture_classes, loader, host, port, intercept, prefix,
1762 response_handlers, content_handlers, test_loader_name=None, inner_fix‐
1763 tures=None)
1764 Bases: object
1765 A class for encapsulating test invariants.
1767 All of the tests in a single gabbi file have invariants which
1769 are provided when creating each HTTPTestCase. It is not useful
1770 to pass these around when making each test case. So they are
1771 wrapped in this class which then has make_one_test called multi‐
1772 ple times to generate all the tests in the suite.
1773 make_one_test(test_dict, prior_test)
1775 Create one single HTTPTestCase.
1776 The returned HTTPTestCase is added to the TestSuite cur‐
1778 rently being built (one per YAML file).
1779 gabbi.suitemaker.test_suite_from_dict(loader, test_base_name,
1781 suite_dict, test_directory, host, port, fixture_module, intercept, pre‐
1782 fix='', handlers=None, test_loader_name=None, inner_fixtures=None)
1783 Generate a GabbiSuite from a dict represent a list of tests.
1784 The dict takes the form:
1786 Parameters
1788 · fixtures -- An optional list of fixture classes that
1790 this suite can use.
1791 · defaults -- An optional dictionary of default values to
1793 be used in each test.
1794 · tests -- A list of individual tests, themselves each
1796 being a dictionary. See gabbi.case.BASE_TEST.
1797 gabbi.suitemaker.test_update(orig_dict, new_dict)
1799 Modify test in place to update with new data.
1800
1802 Manage fixtures for gabbi at the test suite level.
1803 class gabbi.fixture.GabbiFixture
1805 Bases: object
1806 A context manager that operates as a fixture.
1808 Subclasses must implement start_fixture and stop_fixture, each
1810 of which contain the logic for stopping and starting whatever
1811 the fixture is. What a fixture is is left as an exercise for the
1812 implementor.
1813 These context managers will be nested so any actual work needs
1815 to happen in start_fixture and stop_fixture and not in __init__.
1816 Otherwise exception handling will not work properly.
1817 start_fixture()
1819 Implement the actual workings of starting the fixture
1820 here.
1821 stop_fixture()
1823 Implement the actual workings of stopping the fixture
1824 here.
1825 exception gabbi.fixture.GabbiFixtureError
1827 Bases: Exception
1828 Generic exception for GabbiFixture.
1830 class gabbi.fixture.SkipAllFixture
1832 Bases: gabbi.fixture.GabbiFixture
1833 A fixture that skips all the tests in the current suite.
1835 start_fixture()
1837 Implement the actual workings of starting the fixture
1838 here.
1839 gabbi.fixture.nest(fixtures)
1841 Nest a series of fixtures.
1842 This is duplicated from nested in the stdlib, which has been
1844 deprecated because of issues with how exceptions are difficult
1845 to handle during __init__. Gabbi needs to nest an unknown number
1846 of fixtures dynamically, so the with syntax that replaces nested
1847 will not work.
1848
1850 Package for response and content handlers that process the body of a
1851 response in various ways.
1852 handlers.base Module
1854 Base classes for response and content handlers.
1855 class gabbi.handlers.base.ContentHandler
1857 Bases: gabbi.handlers.base.ResponseHandler
1858 A subclass of ResponseHandlers that adds content handling.
1860 static accepts(content_type)
1862 Return True if this handler can handler this type.
1863 static dumps(data, pretty=False, test=None)
1865 Return structured data as a string.
1866 If pretty is true, prettify.
1868 static load_data_file(test, file_path)
1870 Return the string content of the file specified by the
1871 file_path.
1872 static loads(data)
1874 Create structured (Python) data from a stream.
1875 classmethod replacer(response_data, path)
1877 Return the string that is replacing RESPONSE.
1878 class gabbi.handlers.base.ResponseHandler
1880 Bases: object
1881 Add functionality for making assertions about an HTTP response.
1883 A subclass may implement two methods: action and preprocess.
1885 preprocess takes one argument, the TestCase. It is called
1887 exactly once for each test before looping across the assertions.
1888 It is used, rarely, to copy the test.output into a useful form
1889 (such as a parsed DOM).
1890 action takes two or three arguments. If test_key_value is a list
1892 action is called with the test case and a single list item. If
1893 test_key_value is a dict then action is called with the test
1894 case and a key and value pair.
1895 action(test, item, value=None)
1897 Test an individual entry for this response handler.
1898 If the entry is a key value pair the key is in item and
1900 the value in value. Otherwise the entry is considered a
1901 single item from a list.
1902 preprocess(test)
1904 Do any pre-single-test preprocessing.
1905 test_key_suffix = ''
1907 test_key_value = []
1909 handlers.core Module
1911 Core response handlers.
1912 class gabbi.handlers.core.ForbiddenHeadersResponseHandler
1914 Bases: gabbi.handlers.base.ResponseHandler
1915 Test that listed headers are not in the response.
1917 action(test, forbidden, value=None)
1919 Test an individual entry for this response handler.
1920 If the entry is a key value pair the key is in item and
1922 the value in value. Otherwise the entry is considered a
1923 single item from a list.
1924 test_key_suffix = 'forbidden_headers'
1926 test_key_value = []
1928 class gabbi.handlers.core.HeadersResponseHandler
1930 Bases: gabbi.handlers.base.ResponseHandler
1931 Compare expected headers with actual headers.
1933 If a header value is wrapped in / it is treated as a raw regular
1935 expression.
1936 Headers values are always treated as strings.
1938 action(test, header, value=None)
1940 Test an individual entry for this response handler.
1941 If the entry is a key value pair the key is in item and
1943 the value in value. Otherwise the entry is considered a
1944 single item from a list.
1945 test_key_suffix = 'headers'
1947 test_key_value = {}
1949 class gabbi.handlers.core.StringResponseHandler
1951 Bases: gabbi.handlers.base.ResponseHandler
1952 Test for matching strings in the the response body.
1954 action(test, expected, value=None)
1956 Test an individual entry for this response handler.
1957 If the entry is a key value pair the key is in item and
1959 the value in value. Otherwise the entry is considered a
1960 single item from a list.
1961 test_key_suffix = 'strings'
1963 test_key_value = []
1965 handlers.jsonhandler Module
1967 JSON-related content handling.
1968 class gabbi.handlers.jsonhandler.JSONHandler
1970 Bases: gabbi.handlers.base.ContentHandler
1971 A ContentHandler for JSON
1973 · Structured test data is turned into JSON when request con‐
1975 tent-type is JSON.
1976 · Response bodies that are JSON strings are made into Python
1978 data on the test response_data attribute when the response
1979 content-type is JSON.
1980 · A response_json_paths response handler is added.
1982 · JSONPaths in $RESPONSE substitutions are supported.
1984 static accepts(content_type)
1986 Return True if this handler can handler this type.
1987 action(test, path, value=None)
1989 Test json_paths against json data.
1990 static dumps(data, pretty=False, test=None)
1992 Return structured data as a string.
1993 If pretty is true, prettify.
1995 static extract_json_path_value(data, path)
1997 Extract the value at JSON Path path from the data.
1998 The input data is a Python datastructure, not a JSON
2000 string.
2001 static load_data_file(test, file_path)
2003 Return the string content of the file specified by the
2004 file_path.
2005 static loads(data)
2007 Create structured (Python) data from a stream.
2008 classmethod replacer(response_data, match)
2010 Return the string that is replacing RESPONSE.
2011 test_key_suffix = 'json_paths'
2013 test_key_value = {}
2015 handlers.yaml_disk_loading_jsonhandler Module
2017 JSON-related content handling with YAML data disk loading.
2018 class gabbi.handlers.yaml_disk_loading_jsonhandler.YAMLDiskLoadingJSON‐
2020 Handler
2021 Bases: gabbi.handlers.jsonhandler.JSONHandler
2022 A ContentHandler for JSON responses that loads YAML from disk
2024 · Structured test data is turned into JSON when request con‐
2026 tent-type is JSON.
2027 · Response bodies that are JSON strings are made into Python
2029 data on the test response_data attribute when the response
2030 content-type is JSON.
2031 · A response_json_paths response handler is added. Data read
2033 from disk during this handle will be loaded with the
2034 yaml.safe_load method to support both JSON and YAML data
2035 sources from disk.
2036 · JSONPaths in $RESPONSE substitutions are supported.
2038 static load_data_file(test, file_path)
2040 Return the string content of the file specified by the
2041 file_path.
2042
2044 A TestSuite for containing gabbi tests.
2045 This suite has two features: the contained tests are ordered and there
2047 are suite-level fixtures that operate as context managers.
2048 class gabbi.suite.GabbiSuite(tests=())
2050 Bases: unittest.suite.TestSuite
2051 A TestSuite with fixtures.
2053 The suite wraps the tests with a set of nested context managers
2055 that operate as fixtures.
2056 If a fixture raises unittest.case.SkipTest during setup, all the
2058 tests in this suite will be skipped.
2059 run(result, debug=False)
2061 Override TestSuite run to start suite-level fixtures.
2062 To avoid exception confusion, use a null Fixture when
2064 there are no fixtures.
2065 start(result, tests)
2067 Start fixtures when using pytest.
2068 stop() Stop fixtures when using pytest.
2070 gabbi.suite.noop(*args)
2072 A noop method used to disable collected tests.
2073
2075 Implementation of a command-line runner for gabbi files (AKA suites).
2076 gabbi.runner.extract_file_paths(argv)
2078 Extract file paths from the command-line.
2079 File path arguments follow a -- end-of-options delimiter, if
2081 any.
2082 gabbi.runner.initialize_handlers(response_handlers)
2084 gabbi.runner.load_response_handlers(import_path)
2086 Load and return custom response handlers from the import path.
2087 The import path references either a specific response handler
2089 class ("package.module:class") or a module that contains one or
2090 more response handler classes ("package.module").
2091 For the latter, the module is expected to contain a
2093 gabbi_response_handlers object, which is either a list of
2094 response handler classes or a function returning such a list.
2095 gabbi.runner.run()
2097 Run simple tests from STDIN.
2098 This command provides a way to run a set of tests encoded in
2100 YAML that is provided on STDIN. No fixtures are supported, so
2101 this is primarily designed for use with real running services.
2102 Host and port information may be provided in three different
2104 ways:
2105 · In the URL value of the tests.
2107 · In a host or host:port argument on the command line.
2109 · In a URL on the command line.
2111 An example run might looks like this:
2113 gabbi-run example.com:9999 < mytest.yaml
2115 or:
2117 gabbi-run http://example.com:999 < mytest.yaml
2119 It is also possible to provide a URL prefix which can be useful
2121 if the target application might be mounted in different loca‐
2122 tions. An example:
2123 gabbi-run example.com:9999 /mountpoint < mytest.yaml
2125 or:
2127 gabbi-run http://example.com:9999/mountpoint < mytest.yaml
2129 Use -x or --failfast to abort after the first error or failure:
2131 gabbi-run -x example.com:9999 /mountpoint < mytest.yaml
2133 Use -v or --verbose with a value of all, headers or body to turn
2135 on verbosity for all tests being run.
2136 Multiple files may be named as arguments, separated from other
2138 arguments by a --. Each file will be run as a separate test
2139 suite:
2140 gabbi-run http://example.com -- /path/to/x.yaml /path/to/y.yaml
2142 Output is formatted as unittest summary information.
2144 gabbi.runner.run_suite(handle, handler_objects, host, port, prefix,
2146 force_ssl=False, failfast=False, data_dir='.', verbosity=False,
2147 name='input', safe_yaml=True)
2148 Run the tests from the YAML in handle.
2149
2151 TestRunner and TestResult for gabbi-run.
2152 class gabbi.reporter.ConciseTestResult(stream, descriptions, verbosity)
2154 Bases: unittest.runner.TextTestResult
2155 A TextTestResult with simple but useful output.
2157 If the output is a tty or GABBI_FORCE_COLOR is set in the envi‐
2159 ronment, output will be colorized.
2160 addError(test, err)
2162 Called when an error has occurred. 'err' is a tuple of
2163 values as returned by sys.exc_info().
2164 addExpectedFailure(test, err)
2166 Called when an expected failure/error occurred.
2167 addFailure(test, err)
2169 Called when an error has occurred. 'err' is a tuple of
2170 values as returned by sys.exc_info().
2171 addSkip(test, reason)
2173 Called when a test is skipped.
2174 addSuccess(test)
2176 Called when a test has completed successfully
2177 addUnexpectedSuccess(test)
2179 Called when a test was expected to fail, but succeed.
2180 getDescription(test)
2182 printErrorList(flavor, errors)
2184 startTest(test)
2186 Called when the given test is about to be run
2187 class gabbi.reporter.ConciseTestRunner(stream=None, descriptions=True,
2189 verbosity=1, failfast=False, buffer=False, resultclass=None, warn‐
2190 ings=None, *, tb_locals=False)
2191 Bases: unittest.runner.TextTestRunner
2192 A TextTestRunner that uses ConciseTestResult for reporting
2194 results.
2195 resultclass
2197 alias of ConciseTestResult
2198 class gabbi.reporter.PyTestResult(stream=None, descriptions=None, ver‐
2200 bosity=None)
2201 Bases: unittest.result.TestResult
2202 Wrap a test result to allow it to work with pytest.
2204 The main behaviors here are:
2206 · to turn what had been exceptions back into exceptions
2208 · use pytest's skip and xfail methods
2210 addError(test, err)
2212 Called when an error has occurred. 'err' is a tuple of
2213 values as returned by sys.exc_info().
2214 addExpectedFailure(test, err)
2216 Called when an expected failure/error occurred.
2217 addFailure(test, err)
2219 Called when an error has occurred. 'err' is a tuple of
2220 values as returned by sys.exc_info().
2221 addSkip(test, reason)
2223 Called when a test is skipped.
2224
2226 Utility functions grab bag.
2227 gabbi.utils.create_url(base_url, host, port=None, prefix='', ssl=False)
2229 Given pieces of a path-based url, return a fully qualified url.
2230 gabbi.utils.decode_response_content(header_dict, content)
2232 Decode content to a proper string.
2233 gabbi.utils.extract_content_type(header_dict, default='applica‐
2235 tion/binary')
2236 Extract parsed content-type from headers.
2237 gabbi.utils.get_colorizer(stream)
2239 Return a function to colorize a string.
2240 Only if stream is a tty .
2242 gabbi.utils.host_info_from_target(target, prefix=None)
2244 Turn url or host:port and target into test destination.
2245 gabbi.utils.load_yaml(handle=None, yaml_file=None, safe=True)
2247 Read and parse any YAML file or filehandle.
2248 Let exceptions flow where they may.
2250 If no file or handle is provided, read from STDIN.
2252 gabbi.utils.not_binary(content_type)
2254 Decide if something is content we'd like to treat as a string.
2255 gabbi.utils.parse_content_type(content_type, default_charset='utf-8')
2257 Parse content type value for media type and charset.
2258
2260 Gabbi specific exceptions.
2261 exception gabbi.exception.GabbiFormatError
2263 Bases: ValueError
2264 An exception to encapsulate poorly formed test data.
2266 exception gabbi.exception.GabbiSyntaxWarning
2268 Bases: SyntaxWarning
2269 A warning about syntax that is not desirable.
2271
2273 class gabbi.httpclient.Http(num_pools=10, headers=None, **connec‐
2274 tion_pool_kw)
2275 Bases: urllib3.poolmanager.PoolManager
2276 A subclass of the urllib3.PoolManager to munge the data.
2278 This transforms the response to look more like what httplib2
2280 provided when it was used as the HTTP client.
2281 request(absolute_uri, method, body, headers, redirect)
2283 Make a request using urlopen() with the appropriate
2284 encoding of fields based on the method used.
2285 This is a convenience method that requires the least
2287 amount of manual effort. It can be used in most situa‐
2288 tions, while still having the option to drop down to more
2289 specific methods when necessary, such as
2290 request_encode_url(), request_encode_body(), or even the
2291 lowest level urlopen().
2292 class gabbi.httpclient.VerboseHttp(**kwargs)
2294 Bases: gabbi.httpclient.Http
2295 A subclass of Http that verbosely reports on activity.
2297 If the output is a tty or GABBI_FORCE_COLOR is set in the envi‐
2299 ronment, then output will be colorized according to COLORMAP.
2300 Output can include request and response headers, request and
2302 response body content (if of a printable content type), or both.
2303 The color of the output has reasonable defaults. These may be
2305 overridden by setting the following environment variables
2306 · GABBI_CAPTION_COLOR
2308 · GABBI_HEADER_COLOR
2310 · GABBI_REQUEST_COLOR
2312 · GABBI_STATUS_COLOR
2314 to any of: BLACK RED GREEN YELLOW BLUE MAGENTA CYAN WHITE
2316 COLORMAP = {'caption': 'BLUE', 'header': 'YELLOW', 'request':
2318 'CYAN', 'status': 'CYAN'}
2319 HEADER_BLACKLIST = ['status', 'reason']
2321 REQUEST_PREFIX = '>'
2323 RESPONSE_PREFIX = '<'
2325 request(absolute_uri, method, body, headers, redirect)
2327 Display request parameters before requesting.
2328 gabbi.httpclient.get_http(verbose=False, caption='')
2330 Return an Http class for making requests.
2331
2333 Keep one single global jsonpath parser.
2334 gabbi.json_parser.parse(path)
2336 Parse a JSONPath expression use the global parser.
2337 Gabbi is a tool for running HTTP tests where requests and responses are
2339 expressed as declarations in a collection of YAML files. The simplest
2340 test looks like this:
2341 tests:
2343 - name: A test
2344 GET: /api/resources/id
2345 See the rest of these docs for more details on the many features and
2347 formats for setting request headers and bodies and evaluating
2348 responses.
2349 The name is derived from "gabby": excessively talkative. In a test
2351 environment having visibility of what a test is actually doing is a
2352 good thing. This is especially true when the goal of a test is to test
2353 the HTTP, not the testing infrastructure. Gabbi tries to put the HTTP
2354 interaction in the foreground of testing.
2355 Tests can be run using unittest style test runners or py.test or from
2357 the command line with a gabbi-run script.
2358 If you want to get straight to creating tests look at example, the test
2360 files in the source distribution and format. A gabbi-demo repository
2361 provides a tutorial of using gabbi to build an API, via the commit his‐
2362 tory of the repo.
2363
2365 Gabbi works to bridge the gap between human readable YAML files (see
2366 format for details) that represent HTTP requests and expected responses
2367 and the rather complex world of automated testing.
2368 Each YAML file represents an ordered list of HTTP requests along with
2370 the expected responses. This allows a single file to represent a
2371 process in the API being tested. For example:
2372 · Create a resource.
2374 · Retrieve a resource.
2376 · Delete a resource.
2378 · Retrieve a resource again to confirm it is gone.
2380 At the same time it is still possible to ask gabbi to run just one
2382 request. If it is in a sequence of tests, those tests prior to it in
2383 the YAML file will be run (in order). In any single process any test
2384 will only be run once. Concurrency is handled such that one file runs
2385 in one process.
2386 These features mean that it is possible to create tests that are useful
2388 for both humans (as tools for learning, improving and developing APIs)
2389 and automated CI systems.
2390 Significant flexibility and power is available in the format to make it
2392 relatively straightforward to test existing complex APIs. This
2393 extended functionality includes the use of JSONPath to query response
2394 bodies and templating of test data to allow access to the prior HTTP
2395 response in the current request. For APIs which do not use JSON addi‐
2396 tional handlers can be created.
2397 Care should be taken when using this functionality when you are creat‐
2399 ing a new API. If your API is so complex that it needs complex test
2400 files then you may wish to take that as a sign that your API itself too
2401 complex. One goal of gabbi is to encourage transparent and comprehensi‐
2402 ble APIs.
2403 Though gabbi is written in Python and under the covers uses unittest
2405 data structures and processes, there is no requirement that the host be
2406 a Python-based service. Anything talking HTTP can be tested. A runner
2407 makes it possible to simply create YAML files and point them at a run‐
2408 ning server.
2409
2411 Chris Dent
2412 Mar 01, 2019 GABBI(1)