1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the
41 future.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 · Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 · If you delete an object, it is no longer visible.
53 · If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ''
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ''
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Lets see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
115 def test_decode_inverts_encode(s):
116 assert decode(encode(s)) == s
117 (For this example we'll just let pytest discover and run the test.
119 We'll cover other ways you could have run it later).
120 The text function returns what Hypothesis calls a search strategy. An
122 object with methods that describe how to generate and simplify certain
123 kinds of values. The @given decorator then takes our test function and
124 turns it into a parametrized one which, when called, will run the test
125 function over a wide range of matching data from that strategy.
126 Anyway, this test immediately finds a bug in the code:
128 Falsifying example: test_decode_inverts_encode(s='')
130 UnboundLocalError: local variable 'character' referenced before assignment
132 Hypothesis correctly points out that this code is simply wrong if
134 called on an empty string.
135 If we fix that by just adding the following code to the beginning of
137 the function then Hypothesis tells us the code is correct (by doing
138 nothing as you'd expect a passing test to).
139 if not input_string:
141 return []
142 If we wanted to make sure this example was always checked we could add
144 it in explicitly by using the @example decorator:
145 from hypothesis import given, example
147 from hypothesis.strategies import text
148 @given(text())
150 @example('')
151 def test_decode_inverts_encode(s):
152 assert decode(encode(s)) == s
153 This can be useful to show other developers (or your future self) what
155 kinds of data are valid inputs, or to ensure that particular edge cases
156 such as "" are tested every time. It's also great for regression tests
157 because although Hypothesis will remember failing examples, we don't
158 recommend distributing that database.
159 It's also worth noting that both example and given support keyword
161 arguments as well as positional. The following would have worked just
162 as well:
163 @given(s=text())
165 @example(s='')
166 def test_decode_inverts_encode(s):
167 assert decode(encode(s)) == s
168 Suppose we had a more interesting bug and forgot to reset the count
170 each time. Say we missed a line in our encode method:
171 def encode(input_string):
173 count = 1
174 prev = ''
175 lst = []
176 for character in input_string:
177 if character != prev:
178 if prev:
179 entry = (prev, count)
180 lst.append(entry)
181 # count = 1 # Missing reset operation
182 prev = character
183 else:
184 count += 1
185 entry = (character, count)
186 lst.append(entry)
187 return lst
188 Hypothesis quickly informs us of the following example:
190 Falsifying example: test_decode_inverts_encode(s='001')
192 Note that the example provided is really quite simple. Hypothesis
194 doesn't just find any counter-example to your tests, it knows how to
195 simplify the examples it finds to produce small easy to understand
196 ones. In this case, two identical values are enough to set the count to
197 a number different from one, followed by another distinct value which
198 should have reset the count but in this case didn't.
199 The examples Hypothesis provides are valid Python code you can run. Any
201 arguments that you explicitly provide when calling the function are not
202 generated by Hypothesis, and if you explicitly provide all the argu‐
203 ments Hypothesis will just call the underlying function once rather
204 than running it multiple times.
205 Installing
207 Hypothesis is available on PyPI as "hypothesis". You can install it
208 with:
209 pip install hypothesis
211 You can install the dependencies for optional extensions with e.g. pip
213 install hypothesis[pandas,django].
214 If you want to install directly from the source code (e.g. because you
216 want to make changes and install the changed version), check out the
217 instructions in CONTRIBUTING.rst.
218 Running tests
220 In our example above we just let pytest discover and run our tests, but
221 we could also have run it explicitly ourselves:
222 if __name__ == '__main__':
224 test_decode_inverts_encode()
225 We could also have done this as a python:unittest.TestCase:
227 import unittest
229 class TestEncoding(unittest.TestCase):
231 @given(text())
232 def test_decode_inverts_encode(self, s):
233 self.assertEqual(decode(encode(s)), s)
234 if __name__ == '__main__':
236 unittest.main()
237 A detail: This works because Hypothesis ignores any arguments it hasn't
239 been told to provide (positional arguments start from the right), so
240 the self argument to the test is simply ignored and works as normal.
241 This also means that Hypothesis will play nicely with other ways of
242 parameterizing tests. e.g it works fine if you use pytest fixtures for
243 some arguments and Hypothesis for others.
244 Writing tests
246 A test in Hypothesis consists of two parts: A function that looks like
247 a normal test in your test framework of choice but with some additional
248 arguments, and a @given decorator that specifies how to provide those
249 arguments.
250 Here are some other examples of how you could use that:
252 from hypothesis import given
254 import hypothesis.strategies as st
255 @given(st.integers(), st.integers())
257 def test_ints_are_commutative(x, y):
258 assert x + y == y + x
259 @given(x=st.integers(), y=st.integers())
261 def test_ints_cancel(x, y):
262 assert (x + y) - y == x
263 @given(st.lists(st.integers()))
265 def test_reversing_twice_gives_same_list(xs):
266 # This will generate lists of arbitrary length (usually between 0 and
267 # 100 elements) whose elements are integers.
268 ys = list(xs)
269 ys.reverse()
270 ys.reverse()
271 assert xs == ys
272 @given(st.tuples(st.booleans(), st.text()))
274 def test_look_tuples_work_too(t):
275 # A tuple is generated as the one you provided, with the corresponding
276 # types in those positions.
277 assert len(t) == 2
278 assert isinstance(t[0], bool)
279 assert isinstance(t[1], str)
280 Note that as we saw in the above example you can pass arguments to
282 @given either as positional or as keywords.
283 Where to start
285 You should now know enough of the basics to write some tests for your
286 code using Hypothesis. The best way to learn is by doing, so go have a
287 try.
288 If you're stuck for ideas for how to use this sort of test for your
290 code, here are some good starting points:
291 1. Try just calling functions with appropriate arbitrary data and see
293 if they crash. You may be surprised how often this works. e.g. note
294 that the first bug we found in the encoding example didn't even get
295 as far as our assertion: It crashed because it couldn't handle the
296 data we gave it, not because it did the wrong thing.
297 2. Look for duplication in your tests. Are there any cases where you're
299 testing the same thing with multiple different examples? Can you
300 generalise that to a single test using Hypothesis?
301 3. This piece is designed for an F# implementation, but is still very
303 good advice which you may find helps give you good ideas for using
304 Hypothesis.
305 If you have any trouble getting started, don't feel shy about asking
307 for help.
308
310 This is an account of slightly less common Hypothesis features that you
311 don't need to get started but will nevertheless make your life easier.
312 Additional test output
314 Normally the output of a failing test will look something like:
315 Falsifying example: test_a_thing(x=1, y="foo")
317 With the repr of each keyword argument being printed.
319 Sometimes this isn't enough, either because you have values with a repr
321 that isn't very descriptive or because you need to see the output of
322 some intermediate steps of your test. That's where the note function
323 comes in:
324 hypothesis.note(value)
326 Report this value in the final execution.
327 >>> from hypothesis import given, note, strategies as st
329 >>> @given(st.lists(st.integers()), st.randoms())
330 ... def test_shuffle_is_noop(ls, r):
331 ... ls2 = list(ls)
332 ... r.shuffle(ls2)
333 ... note("Shuffle: %r" % (ls2))
334 ... assert ls == ls2
335 ...
336 >>> try:
337 ... test_shuffle_is_noop()
338 ... except AssertionError:
339 ... print('ls != ls2')
340 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
341 Shuffle: [1, 0]
342 ls != ls2
343 The note is printed in the final run of the test in order to include
345 any additional information you might need in your test.
346 Test Statistics
348 If you are using pytest you can see a number of statistics about the
349 executed tests by passing the command line argument --hypothe‐
350 sis-show-statistics. This will include some general statistics about
351 the test:
352 For example if you ran the following with --hypothesis-show-statistics:
354 from hypothesis import given, strategies as st
356 @given(st.integers())
358 def test_integers(i):
359 pass
360 You would see:
362 test_integers:
364 - 100 passing examples, 0 failing examples, 0 invalid examples
366 - Typical runtimes: ~ 1ms
367 - Fraction of time spent in data generation: ~ 12%
368 - Stopped because settings.max_examples=100
369 The final "Stopped because" line is particularly important to note: It
371 tells you the setting value that determined when the test should stop
372 trying new examples. This can be useful for understanding the behaviour
373 of your tests. Ideally you'd always want this to be max_examples.
374 In some cases (such as filtered and recursive strategies) you will see
376 events mentioned which describe some aspect of the data generation:
377 from hypothesis import given, strategies as st
379 @given(st.integers().filter(lambda x: x % 2 == 0))
381 def test_even_integers(i):
382 pass
383 You would see something like:
385 test_even_integers:
387 - 100 passing examples, 0 failing examples, 36 invalid examples
389 - Typical runtimes: 0-1 ms
390 - Fraction of time spent in data generation: ~ 16%
391 - Stopped because settings.max_examples=100
392 - Events:
393 * 80.88%, Retried draw from integers().filter(lambda x: <unknown>) to satisfy filter
394 * 26.47%, Aborted test because unable to satisfy integers().filter(lambda x: <unknown>)
395 You can also mark custom events in a test using the event function:
397 hypothesis.event(value)
399 Record an event that occurred this test. Statistics on number of
400 test runs with each event will be reported at the end if you run
401 Hypothesis in statistics reporting mode.
402 Events should be strings or convertible to them.
404 from hypothesis import given, event, strategies as st
406 @given(st.integers().filter(lambda x: x % 2 == 0))
408 def test_even_integers(i):
409 event("i mod 3 = %d" % (i % 3,))
410 You will then see output like:
412 test_even_integers:
414 - 100 passing examples, 0 failing examples, 38 invalid examples
416 - Typical runtimes: 0-1 ms
417 - Fraction of time spent in data generation: ~ 16%
418 - Stopped because settings.max_examples=100
419 - Events:
420 * 80.43%, Retried draw from integers().filter(lambda x: <unknown>) to satisfy filter
421 * 31.88%, i mod 3 = 0
422 * 27.54%, Aborted test because unable to satisfy integers().filter(lambda x: <unknown>)
423 * 21.74%, i mod 3 = 1
424 * 18.84%, i mod 3 = 2
425 Arguments to event can be any hashable type, but two events will be
427 considered the same if they are the same when converted to a string
428 with python:str.
429 Making assumptions
431 Sometimes Hypothesis doesn't give you exactly the right sort of data
432 you want - it's mostly of the right shape, but some examples won't work
433 and you don't want to care about them. You can just ignore these by
434 aborting the test early, but this runs the risk of accidentally testing
435 a lot less than you think you are. Also it would be nice to spend less
436 time on bad examples - if you're running 100 examples per test (the
437 default) and it turns out 70 of those examples don't match your needs,
438 that's a lot of wasted time.
439 hypothesis.assume(condition)
441 Calling assume is like an assert that marks the example as bad,
442 rather than failing the test.
443 This allows you to specify properties that you assume will be
445 true, and let Hypothesis try to avoid similar examples in
446 future.
447 For example suppose you had the following test:
449 @given(floats())
451 def test_negation_is_self_inverse(x):
452 assert x == -(-x)
453 Running this gives us:
455 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
457 AssertionError
458 This is annoying. We know about NaN and don't really care about it, but
460 as soon as Hypothesis finds a NaN example it will get distracted by
461 that and tell us about it. Also the test will fail and we want it to
462 pass.
463 So lets block off this particular example:
465 from math import isnan
467 @given(floats())
469 def test_negation_is_self_inverse_for_non_nan(x):
470 assume(not isnan(x))
471 assert x == -(-x)
472 And this passes without a problem.
474 In order to avoid the easy trap where you assume a lot more than you
476 intended, Hypothesis will fail a test when it can't find enough exam‐
477 ples passing the assumption.
478 If we'd written:
480 @given(floats())
482 def test_negation_is_self_inverse_for_non_nan(x):
483 assume(False)
484 assert x == -(-x)
485 Then on running we'd have got the exception:
487 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
489 How good is assume?
491 Hypothesis has an adaptive exploration strategy to try to avoid things
492 which falsify assumptions, which should generally result in it still
493 being able to find examples in hard to find situations.
494 Suppose we had the following:
496 @given(lists(integers()))
498 def test_sum_is_positive(xs):
499 assert sum(xs) > 0
500 Unsurprisingly this fails and gives the falsifying example [].
502 Adding assume(xs) to this removes the trivial empty example and gives
504 us [0].
505 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
507 of positive integers is positive.
508 The reason that this should be surprising is not that it doesn't find a
510 counter-example, but that it finds enough examples at all.
511 In order to make sure something interesting is happening, suppose we
513 wanted to try this for long lists. e.g. suppose we added an
514 assume(len(xs) > 10) to it. This should basically never find an exam‐
515 ple: a naive strategy would find fewer than one in a thousand examples,
516 because if each element of the list is negative with probability
517 one-half, you'd have to have ten of these go the right way by chance.
518 In the default configuration Hypothesis gives up long before it's tried
519 1000 examples (by default it tries 200).
520 Here's what happens if we try to run this:
522 @given(lists(integers()))
524 def test_sum_is_positive(xs):
525 assume(len(xs) > 10)
526 assume(all(x > 0 for x in xs))
527 print(xs)
528 assert sum(xs) > 0
529 In: test_sum_is_positive()
531 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
532 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
533 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2...
534 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
535 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...
536 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
537 As you can see, Hypothesis doesn't find many examples here, but it
539 finds some - enough to keep it happy.
540 In general if you can shape your strategies better to your tests you
542 should - for example integers(1, 1000) is a lot better than assume(1 <=
543 x <= 1000), but assume will take you a long way if you can't.
544 Defining strategies
546 The type of object that is used to explore the examples given to your
547 test function is called a SearchStrategy. These are created using the
548 functions exposed in the hypothesis.strategies module.
549 Many of these strategies expose a variety of arguments you can use to
551 customize generation. For example for integers you can specify min and
552 max values of integers you want. If you want to see exactly what a
553 strategy produces you can ask for an example:
554 >>> integers(min_value=0, max_value=10).example()
556 1
557 Many strategies are built out of other strategies. For example, if you
559 want to define a tuple you need to say what goes in each element:
560 >>> from hypothesis.strategies import tuples
562 >>> tuples(integers(), integers()).example()
563 (-24597, 12566)
564 Further details are available in a separate document.
566 The gory details of given parameters
568 hypothesis.given(*given_arguments, **given_kwargs)
569 A decorator for turning a test function that accepts arguments
570 into a randomized test.
571 This is the main entry point to Hypothesis.
573 The @given decorator may be used to specify which arguments of a func‐
575 tion should be parametrized over. You can use either positional or key‐
576 word arguments, but not a mixture of both.
577 For example all of the following are valid uses:
579 @given(integers(), integers())
581 def a(x, y):
582 pass
583 @given(integers())
585 def b(x, y):
586 pass
587 @given(y=integers())
589 def c(x, y):
590 pass
591 @given(x=integers())
593 def d(x, y):
594 pass
595 @given(x=integers(), y=integers())
597 def e(x, **kwargs):
598 pass
599 @given(x=integers(), y=integers())
601 def f(x, *args, **kwargs):
602 pass
603 class SomeTest(TestCase):
606 @given(integers())
607 def test_a_thing(self, x):
608 pass
609 The following are not:
611 @given(integers(), integers(), integers())
613 def g(x, y):
614 pass
615 @given(integers())
617 def h(x, *args):
618 pass
619 @given(integers(), x=integers())
621 def i(x, y):
622 pass
623 @given()
625 def j(x, y):
626 pass
627 The rules for determining what are valid uses of given are as follows:
629 1. You may pass any keyword argument to given.
631 2. Positional arguments to given are equivalent to the rightmost named
633 arguments for the test function.
634 3. Positional arguments may not be used if the underlying test function
636 has varargs, arbitrary keywords, or keyword-only arguments.
637 4. Functions tested with given may not have any defaults.
639 The reason for the "rightmost named arguments" behaviour is so that
641 using @given with instance methods works: self will be passed to the
642 function as normal and not be parametrized over.
643 The function returned by given has all the same arguments as the origi‐
645 nal test, minus those that are filled in by @given. Check the notes on
646 framework compatibility to see how this affects other testing libraries
647 you may be using.
648 Custom function execution
650 Hypothesis provides you with a hook that lets you control how it runs
651 examples.
652 This lets you do things like set up and tear down around each example,
654 run examples in a subprocess, transform coroutine tests into normal
655 tests, etc. For example, TransactionTestCase in the Django extra runs
656 each example in a separate database transaction.
657 The way this works is by introducing the concept of an executor. An
659 executor is essentially a function that takes a block of code and run
660 it. The default executor is:
661 def default_executor(function):
663 return function()
664 You define executors by defining a method execute_example on a class.
666 Any test methods on that class with @given used on them will use
667 self.execute_example as an executor with which to run tests. For exam‐
668 ple, the following executor runs all its code twice:
669 from unittest import TestCase
671 class TestTryReallyHard(TestCase):
673 @given(integers())
674 def test_something(self, i):
675 perform_some_unreliable_operation(i)
676 def execute_example(self, f):
678 f()
679 return f()
680 Note: The functions you use in map, etc. will run inside the executor.
682 i.e. they will not be called until you invoke the function passed to
683 execute_example.
684 An executor must be able to handle being passed a function which
686 returns None, otherwise it won't be able to run normal test cases. So
687 for example the following executor is invalid:
688 from unittest import TestCase
690 class TestRunTwice(TestCase):
692 def execute_example(self, f):
693 return f()()
694 and should be rewritten as:
696 from unittest import TestCase
698 class TestRunTwice(TestCase):
700 def execute_example(self, f):
701 result = f()
702 if callable(result):
703 result = result()
704 return result
705 An alternative hook is provided for use by test runner extensions such
707 as pytest-trio, which cannot use the execute_example method. This is
708 not recommended for end-users - it is better to write a complete test
709 function directly, perhaps by using a decorator to perform the same
710 transformation before applying @given.
711 @given(x=integers())
713 @pytest.mark.trio
714 async def test(x):
715 ...
716 # Illustrative code, inside the pytest-trio plugin
717 test.hypothesis.inner_test = lambda x: trio.run(test, x)
718 For authors of test runners however, assigning to the inner_test
720 attribute of the hypothesis attribute of the test will replace the
721 interior test.
722 NOTE:
724 The new inner_test must accept and pass through all the *args and
725 **kwargs expected by the original test.
726 If the end user has also specified a custom executor using the exe‐
728 cute_example method, it - and all other execution-time logic - will be
729 applied to the new inner test assigned by the test runner.
730 Making random code deterministic
732 While Hypothesis' example generation can be used for nondeterministic
733 tests, debugging anything nondeterministic is usually a very frustrat‐
734 ing excercise. To make things worse, our example shrinking relies on
735 the same input causing the same failure each time - though we show the
736 un-shrunk failure and a decent error message if it doesn't.
737 By default, Hypothesis will handle the global random and numpy.random
739 random number generators for you, and you can register others:
740 hypothesis.register_random(r)
742 Register the given Random instance for management by Hypothesis.
743 You can pass random.Random instances (or other objects with
745 seed, getstate, and setstate methods) to register_random(r) to
746 have their states seeded and restored in the same way as the
747 global PRNGs from the random and numpy.random modules.
748 All global PRNGs, from e.g. simulation or scheduling frameworks,
750 should be registered to prevent flaky tests. Hypothesis will
751 ensure that the PRNG state is consistent for all test runs, or
752 reproducibly varied if you choose to use the random_module()
753 strategy.
754 Using Hypothesis to find values
756 You can use Hypothesis's data exploration features to find values sat‐
757 isfying some predicate. This is generally useful for exploring custom
758 strategies defined with @composite, or experimenting with conditions
759 for filtering data.
760 hypothesis.find(specifier, condition, settings=None, random=None, data‐
762 base_key=None)
763 Returns the minimal example from the given strategy specifier
764 that matches the predicate function condition.
765 >>> from hypothesis import find
767 >>> from hypothesis.strategies import sets, lists, integers
768 >>> find(lists(integers()), lambda x: sum(x) >= 10)
769 [10]
770 >>> find(lists(integers()), lambda x: sum(x) >= 10 and len(x) >= 3)
771 [0, 0, 10]
772 >>> find(sets(integers()), lambda x: sum(x) >= 10 and len(x) >= 3)
773 {0, 1, 9}
774 The first argument to find() describes data in the usual way for an
776 argument to given(), and supports all the same data types. The second
777 is a predicate it must satisfy.
778 Of course not all conditions are satisfiable. If you ask Hypothesis for
780 an example to a condition that is always false it will raise an error:
781 >>> find(integers(), lambda x: False)
783 Traceback (most recent call last):
784 ...
785 hypothesis.errors.NoSuchExample: No examples of condition lambda x: <unknown>
786 (The lambda x: unknown is because Hypothesis can't retrieve the source
788 code of lambdas from the interactive python console. It gives a better
789 error message most of the time which contains the actual condition)
790 Inferred Strategies
792 In some cases, Hypothesis can work out what to do when you omit argu‐
793 ments. This is based on introspection, not magic, and therefore has
794 well-defined limits.
795 builds() will check the signature of the target (using get‐
797 fullargspec()). If there are required arguments with type annotations
798 and no strategy was passed to builds(), from_type() is used to fill
799 them in. You can also pass the special value hypothesis.infer as a
800 keyword argument, to force this inference for arguments with a default
801 value.
802 >>> def func(a: int, b: str):
804 ... return [a, b]
805 >>> builds(func).example()
806 [-6993, '']
807 hypothesis.infer
809 @given does not perform any implicit inference for required arguments,
811 as this would break compatibility with pytest fixtures. infer can be
812 used as a keyword argument to explicitly fill in an argument from its
813 type annotation.
814 @given(a=infer)
816 def test(a: int): pass
817 # is equivalent to
818 @given(a=integers())
819 def test(a): pass
820 Limitations
822 PEP 3107 type annotations are not supported on Python 2, and Hypothesis
823 does not inspect PEP 484 type comments at runtime. While from_type()
824 will work as usual, inference in builds() and @given will only work if
825 you manually create the __annotations__ attribute (e.g. by using @anno‐
826 tations(...) and @returns(...) decorators). The python:typing module
827 is fully supported on Python 2 if you have the backport installed.
828 The python:typing module is provisional and has a number of internal
830 changes between Python 3.5.0 and 3.6.1, including at minor versions.
831 These are all supported on a best-effort basis, but you may encounter
832 problems with an old version of the module. Please report them to us,
833 and consider updating to a newer version of Python as a workaround.
834 Type Annotations in Hypothesis
836 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
837 patible tool, the type checker should automatically pick up our type
838 hints.
839 NOTE:
841 Hypothesis' type hints may make breaking changes between minor
842 releases.
843 Upstream tools and conventions about type hints remain in flux - for
845 example the python:typing module itself is provisional, and Mypy has
846 not yet reached version 1.0 - and we plan to support the latest ver‐
847 sion of this ecosystem, as well as older versions where practical.
848 We may also find more precise ways to describe the type of various
850 interfaces, or change their type and runtime behaviour togther in a
851 way which is otherwise backwards-compatible. We often omit type
852 hints for deprecated features or arguments, as an additional form of
853 warning.
854 There are known issues inferring the type of examples generated by
856 deferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
857 ies(). We will fix these, and require correspondingly newer versions
858 of Mypy for type hinting, as the ecosystem improves.
859 Writing downstream type hints
861 Projects that provide Hypothesis strategies and use type hints may wish
862 to annotate their strategies too. This is a supported use-case, again
863 on a best-effort provisional basis. For example:
864 def foo_strategy() -> SearchStrategy[Foo]: ...
866 class hypothesis.strategies.SearchStrategy
868 SearchStrategy is the type of all strategy objects. It is a generic
870 type, and covariant in the type of the examples it creates. For exam‐
871 ple:
872 · integers() is of type SearchStrategy[int].
874 · lists(integers()) is of type SearchStrategy[List[int]].
876 · SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
878 a subtype of Animal (as seems likely).
879 WARNING:
881 SearchStrategy should only be used in type hints. Please do not
882 inherit from, compare to, or otherwise use it in any way outside of
883 type hints. The only supported way to construct objects of this
884 type is to use the functions provided by the hypothesis.strategies
885 module!
886 The Hypothesis pytest Plugin
888 Hypothesis includes a tiny plugin to improve integration with pytest,
889 which is activated by default (but does not affect other test runners).
890 It aims to improve the integration between Hypothesis and Pytest by
891 providing extra information and convenient access to config options.
892 · pytest --hypothesis-show-statistics can be used to display test and
894 data generation statistics.
895 · pytest --hypothesis-profile=<profile name> can be used to load a set‐
897 tings profile. pytest --hypothesis-verbosity=<level name> can be
898 used to override the current verbosity level.
899 · pytest --hypothesis-seed=<an int> can be used to reproduce a failure
901 with a particular seed.
902 Finally, all tests that are defined with Hypothesis automatically have
904 @pytest.mark.hypothesis applied to them. See here for information on
905 working with markers.
906 NOTE:
908 Pytest will load the plugin automatically if Hypothesis is
909 installed. You don't need to do anything at all to use it.
910
912 Hypothesis tries to have good defaults for its behaviour, but sometimes
913 that's not enough and you need to tweak it.
914 The mechanism for doing this is the settings object. You can set up a
916 @given based test to use this using a settings decorator:
917 @given invocation is as follows:
919 from hypothesis import given, settings
921 @given(integers())
923 @settings(max_examples=500)
924 def test_this_thoroughly(x):
925 pass
926 This uses a settings object which causes the test to receive a much
928 larger set of examples than normal.
929 This may be applied either before or after the given and the results
931 are the same. The following is exactly equivalent:
932 from hypothesis import given, settings
934 @settings(max_examples=500)
936 @given(integers())
937 def test_this_thoroughly(x):
938 pass
939 Available settings
941 class hypothesis.settings(parent=None, **kwargs)
942 A settings object controls a variety of parameters that are used
943 in falsification. These may control both the falsification
944 strategy and the details of the data that is generated.
945 Default values are picked up from the settings.default object
947 and changes made there will be picked up in newly created set‐
948 tings.
949 buffer_size
951 The size of the underlying data used to generate exam‐
952 ples. If you need to generate really large examples you
953 may want to increase this, but it will make your tests
954 slower.
955 default value: (dynamically calculated)
957 database
959 An instance of hypothesis.database.ExampleDatabase that
960 will be used to save examples to and load previous exam‐
961 ples from. May be None in which case no storage will be
962 used, :memory: for an in-memory database, or any path for
963 a directory-based example database.
964 default value: (dynamically calculated)
966 deadline
968 If set, a duration (as timedelta, or integer or float
969 number of milliseconds) that each individual example
970 (i.e. each time your test function is called, not the
971 whole decorated test) within a test is not allowed to
972 exceed. Tests which take longer than that may be con‐
973 verted into errors (but will not necessarily be if close
974 to the deadline, to allow some variability in test run
975 time).
976 Set this to None to disable this behaviour entirely.
978 default value: timedelta(milliseconds=200.0)
980 derandomize
982 If this is True then hypothesis will run in deterministic
983 mode where each falsification uses a random number gener‐
984 ator that is seeded based on the hypothesis to falsify,
985 which will be consistent across multiple runs. This has
986 the advantage that it will eliminate any randomness from
987 your tests, which may be preferable for some situations.
988 It does have the disadvantage of making your tests less
989 likely to find novel breakages.
990 default value: False
992 max_examples
994 Once this many satisfying examples have been considered
995 without finding any counter-example, falsification will
996 terminate.
997 The default value is chosen to suit a workflow where the
999 test will be part of a suite that is regularly executed
1000 locally or on a CI server, balancing total running time
1001 against the chance of missing a bug.
1002 If you are writing one-off tests, running tens of thou‐
1004 sands of examples is quite reasonable as Hypothesis may
1005 miss uncommon bugs with default settings. For very com‐
1006 plex code, we have observed Hypothesis finding novel bugs
1007 after several million examples while testing SymPy.
1008 default value: 100
1010 phases Control which phases should be run. See the full documen‐
1012 tation for more details
1013 default value: (<Phase.explicit: 0>, <Phase.reuse: 1>,
1015 <Phase.generate: 2>, <Phase.shrink: 3>)
1016 print_blob
1018 Determines whether to print blobs after tests that can be
1019 used to reproduce failures.
1020 See the documentation on @reproduce_failure for more
1022 details of this behaviour.
1023 default value: PrintSettings.INFER
1025 report_multiple_bugs
1027 Because Hypothesis runs the test many times, it can some‐
1028 times find multiple bugs in a single run. Reporting all
1029 of them at once is usually very useful, but replacing the
1030 exceptions can occasionally clash with debuggers. If
1031 disabled, only the exception with the smallest minimal
1032 example is raised.
1033 default value: True
1035 stateful_step_count
1037 Number of steps to run a stateful program for before giv‐
1038 ing up on it breaking.
1039 default value: 50
1041 suppress_health_check
1043 A list of health checks to disable.
1044 default value: ()
1046 timeout
1048 The timeout setting has been deprecated and no longer
1049 does anything.
1050 default value: not_set
1052 The timeout setting can safely be removed with no effect.
1054 verbosity
1056 Control the verbosity level of Hypothesis messages
1057 default value: Verbosity.normal
1059 Controlling What Runs
1061 Hypothesis divides tests into four logically distinct phases:
1062 1. Running explicit examples provided with the @example decorator.
1064 2. Rerunning a selection of previously failing examples to reproduce a
1066 previously seen error
1067 3. Generating new examples.
1069 4. Attempting to shrink an example found in phases 2 or 3 to a more
1071 manageable one (explicit examples cannot be shrunk).
1072 The phases setting provides you with fine grained control over which of
1074 these run, with each phase corresponding to a value on the Phase enum:
1075 class hypothesis.Phase
1077 1. Phase.explicit controls whether explicit examples are run.
1079 2. Phase.reuse controls whether previous examples will be reused.
1081 3. Phase.generate controls whether new examples will be generated.
1083 4. Phase.shrink controls whether examples will be shrunk.
1085 The phases argument accepts a collection with any subset of these. e.g.
1087 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1088 ples and shrink them, but will not run explicit examples or reuse pre‐
1089 vious failures, while settings(phases=[Phase.explicit]) will only run
1090 the explicit examples.
1091 Seeing intermediate result
1093 To see what's going on while Hypothesis runs your tests, you can turn
1094 up the verbosity setting. This works with both find() and @given.
1095 >>> from hypothesis import find, settings, Verbosity
1097 >>> from hypothesis.strategies import lists, integers
1098 >>> find(lists(integers()), any, settings=settings(verbosity=Verbosity.verbose))
1099 Tried non-satisfying example []
1100 Found satisfying example [-1198601713, -67, 116, -29578]
1101 Shrunk example to [-1198601713]
1102 Shrunk example to [-1198601600]
1103 Shrunk example to [-1191228800]
1104 Shrunk example to [-8421504]
1105 Shrunk example to [-32896]
1106 Shrunk example to [-128]
1107 Shrunk example to [64]
1108 Shrunk example to [32]
1109 Shrunk example to [16]
1110 Shrunk example to [8]
1111 Shrunk example to [4]
1112 Shrunk example to [3]
1113 Shrunk example to [2]
1114 Shrunk example to [1]
1115 [1]
1116 The four levels are quiet, normal, verbose and debug. normal is the
1118 default, while in quiet mode Hypothesis will not print anything out,
1119 not even the final falsifying example. debug is basically verbose but a
1120 bit more so. You probably don't want it.
1121 If you are using pytest, you may also need to disable output capturing
1123 for passing tests.
1124 Building settings objects
1126 Settings can be created by calling settings with any of the available
1127 settings values. Any absent ones will be set to defaults:
1128 >>> from hypothesis import settings
1130 >>> settings().max_examples
1131 100
1132 >>> settings(max_examples=10).max_examples
1133 10
1134 You can also pass a 'parent' settings object as the first argument, and
1136 any settings you do not specify as keyword arguments will be copied
1137 from the parent settings:
1138 >>> parent = settings(max_examples=10)
1140 >>> child = settings(parent, deadline=None)
1141 >>> parent.max_examples == child.max_examples == 10
1142 True
1143 >>> parent.deadline
1144 200
1145 >>> child.deadline is None
1146 True
1147 Default settings
1149 At any given point in your program there is a current default settings,
1150 available as settings.default. As well as being a settings object in
1151 its own right, all newly created settings objects which are not explic‐
1152 itly based off another settings are based off the default, so will
1153 inherit any values that are not explicitly set from it.
1154 You can change the defaults by using profiles.
1156 settings Profiles
1158 Depending on your environment you may want different default settings.
1159 For example: during development you may want to lower the number of
1160 examples to speed up the tests. However, in a CI environment you may
1161 want more examples so you are more likely to find bugs.
1162 Hypothesis allows you to define different settings profiles. These pro‐
1164 files can be loaded at any time.
1165 class hypothesis.settings(parent=None, **kwargs)
1167 A settings object controls a variety of parameters that are used
1168 in falsification. These may control both the falsification
1169 strategy and the details of the data that is generated.
1170 Default values are picked up from the settings.default object
1172 and changes made there will be picked up in newly created set‐
1173 tings.
1174 static register_profile(name, parent=None, **kwargs)
1176 Registers a collection of values to be used as a settings
1177 profile.
1178 Settings profiles can be loaded by name - for example,
1180 you might create a 'fast' profile which runs fewer exam‐
1181 ples, keep the 'default' profile, and create a 'ci' pro‐
1182 file that increases the number of examples and uses a
1183 different database to store failures.
1184 The arguments to this method are exactly as for settings:
1186 optional parent settings, and keyword arguments for each
1187 setting that will be set differently to parent (or set‐
1188 tings.default, if parent is None).
1189 static get_profile(name)
1191 Return the profile with the given name.
1192 static load_profile(name)
1194 Loads in the settings defined in the profile provided.
1195 If the profile does not exist, InvalidArgument will be
1197 raised. Any setting not defined in the profile will be
1198 the library defined default for that setting.
1199 Loading a profile changes the default settings but will not change the
1201 behavior of tests that explicitly change the settings.
1202 >>> from hypothesis import settings
1204 >>> settings.register_profile("ci", max_examples=1000)
1205 >>> settings().max_examples
1206 100
1207 >>> settings.load_profile("ci")
1208 >>> settings().max_examples
1209 1000
1210 Instead of loading the profile and overriding the defaults you can
1212 retrieve profiles for specific tests.
1213 >>> settings.get_profile("ci").max_examples
1215 1000
1216 Optionally, you may define the environment variable to load a profile
1218 for you. This is the suggested pattern for running your tests on CI.
1219 The code below should run in a conftest.py or any setup/initialization
1220 section of your test suite. If this variable is not defined the
1221 Hypothesis defined defaults will be loaded.
1222 >>> import os
1224 >>> from hypothesis import settings, Verbosity
1225 >>> settings.register_profile("ci", max_examples=1000)
1226 >>> settings.register_profile("dev", max_examples=10)
1227 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1228 >>> settings.load_profile(os.getenv(u'HYPOTHESIS_PROFILE', 'default'))
1229 If you are using the hypothesis pytest plugin and your profiles are
1231 registered by your conftest you can load one with the command line
1232 option --hypothesis-profile.
1233 $ pytest tests --hypothesis-profile <profile-name>
1235
1237 Most things should be easy to generate and everything should be possi‐
1238 ble.
1239 To support this principle Hypothesis provides strategies for most
1241 built-in types with arguments to constrain or adjust the output, as
1242 well as higher-order strategies that can be composed to generate more
1243 complex types.
1244 This document is a guide to what strategies are available for generat‐
1246 ing data and how to build them. Strategies have a variety of other
1247 important internal features, such as how they simplify, but the data
1248 they can generate is the only public part of their API.
1249 Functions for building strategies are all available in the hypothe‐
1251 sis.strategies module. The salient functions from it are as follows:
1252 class hypothesis.strategies.DataObject(data)
1254 This type only exists so that you can write type hints for tests
1255 using the data() strategy. Do not use it directly!
1256 hypothesis.strategies.binary(min_size=0, max_size=None)
1258 Generates the appropriate binary type (str in python 2, bytes in
1259 python 3).
1260 min_size and max_size have the usual interpretations.
1262 Examples from this strategy shrink towards smaller strings and
1264 lower byte values.
1265 hypothesis.strategies.booleans()
1267 Returns a strategy which generates instances of python:bool.
1268 Examples from this strategy will shrink towards False (i.e.
1270 shrinking will try to replace True with False where possible).
1271 hypothesis.strategies.builds(*callable_and_args, **kwargs)
1273 Generates values by drawing from args and kwargs and passing
1274 them to the callable (provided as the first positional argument)
1275 in the appropriate argument position.
1276 e.g. builds(target, integers(), flag=booleans()) would draw an
1278 integer i and a boolean b and call target(i, flag=b).
1279 If the callable has type annotations, they will be used to infer
1281 a strategy for required arguments that were not passed to
1282 builds. You can also tell builds to infer a strategy for an
1283 optional argument by passing the special value hypothesis.infer
1284 as a keyword argument to builds, instead of a strategy for that
1285 argument to the callable.
1286 If the callable is a class defined with attrs, missing required
1288 arguments will be inferred from the attribute on a best-effort
1289 basis, e.g. by checking attrs standard validators. Dataclasses
1290 are handled natively by the inference from type hints.
1291 Examples from this strategy shrink by shrinking the argument
1293 values to the callable.
1294 hypothesis.strategies.characters(whitelist_categories=None, black‐
1296 list_categories=None, blacklist_characters=None, min_codepoint=None,
1297 max_codepoint=None, whitelist_characters=None)
1298 Generates unicode text type (unicode on python 2, str on python
1299 3) characters following specified filtering rules.
1300 · When no filtering rules are specifed, any character can be
1302 produced.
1303 · If min_codepoint or max_codepoint is specifed, then only char‐
1305 acters having a codepoint in that range will be produced.
1306 · If whitelist_categories is specified, then only characters
1308 from those Unicode categories will be produced. This is a fur‐
1309 ther restriction, characters must also satisfy min_codepoint
1310 and max_codepoint.
1311 · If blacklist_categories is specified, then any character from
1313 those categories will not be produced. Any overlap between
1314 whitelist_categories and blacklist_categories will raise an
1315 exception, as each character can only belong to a single
1316 class.
1317 · If whitelist_characters is specified, then any additional
1319 characters in that list will also be produced.
1320 · If blacklist_characters is specified, then any characters in
1322 that list will be not be produced. Any overlap between
1323 whitelist_characters and blacklist_characters will raise an
1324 exception.
1325 The _codepoint arguments must be integers between zero and
1327 python:sys.maxunicode. The _characters arguments must be col‐
1328 lections of length-one unicode strings, such as a unicode
1329 string.
1330 The _categories arguments must be used to specify either the
1332 one-letter Unicode major category or the two-letter Unicode
1333 general category. For example, ('Nd', 'Lu') signifies "Number,
1334 decimal digit" and "Letter, uppercase". A single letter ('major
1335 category') can be given to match all corresponding categories,
1336 for example 'P' for characters in any punctuation category.
1337 Examples from this strategy shrink towards the codepoint for
1339 '0', or the first allowable codepoint after it if '0' is
1340 excluded.
1341 hypothesis.strategies.complex_numbers(min_magnitude=0, max_magni‐
1343 tude=None, allow_infinity=None, allow_nan=None)
1344 Returns a strategy that generates complex numbers.
1345 This strategy draws complex numbers with constrained magnitudes.
1347 The min_magnitude and max_magnitude parameters should be
1348 non-negative Real numbers; values of None correspond to zero and
1349 infinite values respectively.
1350 If min_magnitude is positive or max_magnitude is finite, it is
1352 an error to enable allow_nan. If max_magnitude is finite, it is
1353 an error to enable allow_infinity.
1354 The magnitude contraints are respected up to a relative error of
1356 (around) floating-point epsilon, due to implementation via the
1357 system sqrt function.
1358 Examples from this strategy shrink by shrinking their real and
1360 imaginary parts, as floats().
1361 If you need to generate complex numbers with particular real and
1363 imaginary parts or relationships between parts, consider using
1364 builds(complex, ...) or @composite respectively.
1365 hypothesis.strategies.composite(f)
1367 Defines a strategy that is built out of potentially arbitrarily
1368 many other strategies.
1369 This is intended to be used as a decorator. See the full docu‐
1371 mentation for more details about how to use this function.
1372 Examples from this strategy shrink by shrinking the output of
1374 each draw call.
1375 hypothesis.strategies.data()
1377 This isn't really a normal strategy, but instead gives you an
1378 object which can be used to draw data interactively from other
1379 strategies.
1380 It can only be used within @given, not find(). This is because
1382 the lifetime of the object cannot outlast the test body.
1383 See the rest of the documentation for more complete information.
1385 Examples from this strategy do not shrink (because there is only
1387 one), but the result of calls to each draw() call shrink as they
1388 normally would.
1389 hypothesis.strategies.dates(min_value=datetime.date(1, 1, 1),
1391 max_value=datetime.date(9999, 12, 31))
1392 A strategy for dates between min_value and max_value.
1393 Examples from this strategy shrink towards January 1st 2000.
1395 hypothesis.strategies.datetimes(min_value=datetime.datetime(1, 1, 1, 0,
1397 0), max_value=datetime.datetime(9999, 12, 31, 23, 59, 59, 999999),
1398 timezones=none())
1399 A strategy for generating datetimes, which may be time‐
1400 zone-aware.
1401 This strategy works by drawing a naive datetime between
1403 min_value and max_value, which must both be naive (have no time‐
1404 zone).
1405 timezones must be a strategy that generates tzinfo objects (or
1407 None, which is valid for naive datetimes). A value drawn from
1408 this strategy will be added to a naive datetime, and the result‐
1409 ing tz-aware datetime returned.
1410 NOTE:
1412 tz-aware datetimes from this strategy may be ambiguous or
1413 non-existent due to daylight savings, leap seconds, timezone
1414 and calendar adjustments, etc. This is intentional, as mal‐
1415 formed timestamps are a common source of bugs.
1416 hypothesis.extra.pytz.timezones() requires the pytz package, but
1418 provides all timezones in the Olsen database. If you want to
1419 allow naive datetimes, combine strategies like none() | time‐
1420 zones().
1421 hypothesis.extra.dateutil.timezones() requires the
1423 python-dateutil package, and similarly provides all timezones
1424 there.
1425 Alternatively, you can create a list of the timezones you wish
1427 to allow (e.g. from the standard library, datetutil, or pytz)
1428 and use sampled_from(). Ensure that simple values such as None
1429 or UTC are at the beginning of the list for proper minimisation.
1430 Examples from this strategy shrink towards midnight on January
1432 1st 2000.
1433 hypothesis.strategies.decimals(min_value=None, max_value=None,
1435 allow_nan=None, allow_infinity=None, places=None)
1436 Generates instances of python:decimal.Decimal, which may be:
1437 · A finite rational number, between min_value and max_value.
1439 · Not a Number, if allow_nan is True. None means "allow NaN,
1441 unless min_value and max_value are not None".
1442 · Positive or negative infinity, if max_value and min_value
1444 respectively are None, and allow_infinity is not False. None
1445 means "allow infinity, unless excluded by the min and max val‐
1446 ues".
1447 Note that where floats have one NaN value, Decimals have four:
1449 signed, and either quiet or signalling. See the decimal module
1450 docs for more information on special values.
1451 If places is not None, all finite values drawn from the strategy
1453 will have that number of digits after the decimal place.
1454 Examples from this strategy do not have a well defined shrink
1456 order but try to maximize human readability when shrinking.
1457 hypothesis.strategies.deferred(definition)
1459 A deferred strategy allows you to write a strategy that refer‐
1460 ences other strategies that have not yet been defined. This
1461 allows for the easy definition of recursive and mutually recur‐
1462 sive strategies.
1463 The definition argument should be a zero-argument function that
1465 returns a strategy. It will be evaluated the first time the
1466 strategy is used to produce an example.
1467 Example usage:
1469 >>> import hypothesis.strategies as st
1471 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1472 >>> x.example()
1473 (((False, (True, True)), (False, True)), (True, True))
1474 >>> x.example()
1475 True
1476 Mutual recursion also works fine:
1478 >>> a = st.deferred(lambda: st.booleans() | b)
1480 >>> b = st.deferred(lambda: st.tuples(a, a))
1481 >>> a.example()
1482 True
1483 >>> b.example()
1484 (False, (False, ((False, True), False)))
1485 Examples from this strategy shrink as they normally would from
1487 the strategy returned by the definition.
1488 hypothesis.strategies.dictionaries(keys, values, dict_class=<class
1490 'dict'>, min_size=0, max_size=None)
1491 Generates dictionaries of type dict_class with keys drawn from
1492 the keys argument and values drawn from the values argument.
1493 The size parameters have the same interpretation as for lists.
1495 Examples from this strategy shrink by trying to remove keys from
1497 the generated dictionary, and by shrinking each generated key
1498 and value.
1499 hypothesis.strategies.emails()
1501 A strategy for generating email addresses as unicode strings.
1502 The address format is specified in RFC 5322#section-3.4.1. Val‐
1503 ues shrink towards shorter local-parts and host domains.
1504 This strategy is useful for generating "user data" for tests, as
1506 mishandling of email addresses is a common source of bugs.
1507 hypothesis.strategies.fixed_dictionaries(mapping)
1509 Generates a dictionary of the same type as mapping with a fixed
1510 set of keys mapping to strategies. mapping must be a dict sub‐
1511 class.
1512 Generated values have all keys present in mapping, with the cor‐
1514 responding values drawn from mapping[key]. If mapping is an
1515 instance of OrderedDict the keys will also be in the same order,
1516 otherwise the order is arbitrary.
1517 Examples from this strategy shrink by shrinking each individual
1519 value in the generated dictionary.
1520 hypothesis.strategies.floats(min_value=None, max_value=None,
1522 allow_nan=None, allow_infinity=None, width=64, exclude_min=False,
1523 exclude_max=False)
1524 Returns a strategy which generates floats.
1525 · If min_value is not None, all values will be >= min_value.
1527 · If max_value is not None, all values will be <= max_value.
1529 · If min_value or max_value is not None, it is an error to
1531 enable allow_nan.
1532 · If both min_value and max_value are not None, it is an error
1534 to enable allow_infinity.
1535 Where not explicitly ruled out by the bounds, all of infinity,
1537 -infinity and NaN are possible values generated by this strat‐
1538 egy.
1539 The width argument specifies the maximum number of bits of pre‐
1541 cision required to represent the generated float. Valid values
1542 are 16, 32, or 64. Passing width=32 will still use the builtin
1543 64-bit float class, but always for values which can be exactly
1544 represented as a 32-bit float. Half-precision floats (width=16)
1545 are only supported on Python 3.6, or if Numpy is installed.
1546 The exclude_min and exclude_max argument can be used to generate
1548 numbers from open or half-open intervals, by excluding the
1549 respective endpoints. Attempting to exclude an endpoint which
1550 is None will raise an error; use allow_infinity=False to gener‐
1551 ate finite floats. You can however use e.g.
1552 min_value=float("-inf"), exclude_min=True to exclude only one
1553 infinite endpoint.
1554 Examples from this strategy have a complicated and hard to
1556 explain shrinking behaviour, but it tries to improve "human
1557 readability". Finite numbers will be preferred to infinity and
1558 infinity will be preferred to NaN.
1559 hypothesis.strategies.fractions(min_value=None, max_value=None,
1561 max_denominator=None)
1562 Returns a strategy which generates Fractions.
1563 If min_value is not None then all generated values are no less
1565 than min_value. If max_value is not None then all generated
1566 values are no greater than max_value. min_value and max_value
1567 may be anything accepted by the Fraction constructor.
1568 If max_denominator is not None then the denominator of any gen‐
1570 erated values is no greater than max_denominator. Note that
1571 max_denominator must be None or a positive integer.
1572 Examples from this strategy shrink towards smaller denominators,
1574 then closer to zero.
1575 hypothesis.strategies.from_regex(regex, fullmatch=False)
1577 Generates strings that contain a match for the given regex (i.e.
1578 ones for which python:re.search() will return a non-None
1579 result).
1580 regex may be a pattern or compiled regex. Both byte-strings and
1582 unicode strings are supported, and will generate examples of the
1583 same type.
1584 You can use regex flags such as python:re.IGNORECASE or
1586 python:re.DOTALL to control generation. Flags can be passed
1587 either in compiled regex or inside the pattern with a (?iLmsux)
1588 group.
1589 Some regular expressions are only partly supported - the under‐
1591 lying strategy checks local matching and relies on filtering to
1592 resolve context-dependent expressions. Using too many of these
1593 constructs may cause health-check errors as too many examples
1594 are filtered out. This mainly includes (positive or negative)
1595 lookahead and lookbehind groups.
1596 If you want the generated string to match the whole regex you
1598 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1599 gle character string, while "." will return any string, and
1600 r"\A.$" will return a single character optionally followed by a
1601 "\n". Alternatively, passing fullmatch=True will ensure that
1602 the whole string is a match, as if you had used the \A and \Z
1603 markers.
1604 Examples from this strategy shrink towards shorter strings and
1606 lower character values, with exact behaviour that may depend on
1607 the pattern.
1608 hypothesis.strategies.from_type(thing)
1610 Looks up the appropriate search strategy for the given type.
1611 from_type is used internally to fill in missing arguments to
1613 builds() and can be used interactively to explore what strate‐
1614 gies are available or to debug type resolution.
1615 You can use register_type_strategy() to handle your custom
1617 types, or to globally redefine certain strategies - for example
1618 excluding NaN from floats, or use timezone-aware instead of
1619 naive time and datetime strategies.
1620 The resolution logic may be changed in a future version, but
1622 currently tries these four options:
1623 1. If thing is in the default lookup mapping or user-registered
1625 lookup, return the corresponding strategy. The default
1626 lookup covers all types with Hypothesis strategies, including
1627 extras where possible.
1628 2. If thing is from the python:typing module, return the corre‐
1630 sponding strategy (special logic).
1631 3. If thing has one or more subtypes in the merged lookup,
1633 return the union of the strategies for those types that are
1634 not subtypes of other elements in the lookup.
1635 4. Finally, if thing has type annotations for all required argu‐
1637 ments, it is resolved via builds().
1638 There is a valuable recipe for leveraging from_type() to gener‐
1640 ate "everything except" values from a specified type. I.e.
1641 def everything_except(excluded_types):
1643 return (
1644 from_type(type).flatmap(from_type)
1645 .filter(lambda x: not isinstance(x, excluded_types))
1646 )
1647 For example, everything_except(int) returns a strategy that can
1649 generate anything that from_type() can ever generate, except for
1650 instances of :class:python:int, and excluding instances of types
1651 added via :func:~hypothesis.strategies.register_type_strategy.
1652 This is useful when writing tests which check that invalid input
1654 is rejected in a certain way.
1655 hypothesis.strategies.frozensets(elements, min_size=0, max_size=None)
1657 This is identical to the sets function but instead returns
1658 frozensets.
1659 hypothesis.strategies.functions(like=<function <lambda>>,
1661 returns=none())
1662 A strategy for functions, which can be used in callbacks.
1663 The generated functions will mimic the interface of like, which
1665 must be a callable (including a class, method, or function).
1666 The return value for the function is drawn from the returns
1667 argument, which must be a strategy.
1668 Note that the generated functions do not validate their argu‐
1670 ments, and may return a different value if called again with the
1671 same arguments.
1672 Generated functions can only be called within the scope of the
1674 @given which created them. This strategy does not support
1675 .example().
1676 hypothesis.strategies.integers(min_value=None, max_value=None)
1678 Returns a strategy which generates integers; in Python 2 these
1679 may be ints or longs.
1680 If min_value is not None then all values will be >= min_value.
1682 If max_value is not None then all values will be <= max_value
1683 Examples from this strategy will shrink towards zero, and nega‐
1685 tive values will also shrink towards positive (i.e. -n may be
1686 replaced by +n).
1687 hypothesis.strategies.iterables(elements, min_size=0, max_size=None,
1689 unique_by=None, unique=False)
1690 This has the same behaviour as lists, but returns iterables
1691 instead.
1692 Some iterables cannot be indexed (e.g. sets) and some do not
1694 have a fixed length (e.g. generators). This strategy produces
1695 iterators, which cannot be indexed and do not have a fixed
1696 length. This ensures that you do not accidentally depend on
1697 sequence behaviour.
1698 hypothesis.strategies.just(value)
1700 Return a strategy which only generates value.
1701 Note: value is not copied. Be wary of using mutable values.
1703 If value is the result of a callable, you can use
1705 builds(callable) instead of just(callable()) to get a fresh
1706 value each time.
1707 Examples from this strategy do not shrink (because there is only
1709 one).
1710 hypothesis.strategies.lists(elements, min_size=0, max_size=None,
1712 unique_by=None, unique=False)
1713 Returns a list containing values drawn from elements with length
1714 in the interval [min_size, max_size] (no bounds in that direc‐
1715 tion if these are None). If max_size is 0 then elements may be
1716 None and only the empty list will be drawn.
1717 If unique is True (or something that evaluates to True), we com‐
1719 pare direct object equality, as if unique_by was lambda x: x.
1720 This comparison only works for hashable types.
1721 If unique_by is not None it must be a callable or tuple of
1723 callables returning a hashable type when given a value drawn
1724 from elements. The resulting list will satisfy the condition
1725 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1726 If unique_by is a tuple of callables the uniqueness will be
1728 respective to each callable.
1729 For example, the following will produce two columns of integers
1731 with both columns being unique respectively.
1732 >>> twoints = st.tuples(st.integers(), st.integers())
1734 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1735 Examples from this strategy shrink by trying to remove elements
1737 from the list, and by shrinking each individual element of the
1738 list.
1739 hypothesis.strategies.none()
1741 Return a strategy which only generates None.
1742 Examples from this strategy do not shrink (because there is only
1744 one).
1745 hypothesis.strategies.nothing()
1747 This strategy never successfully draws a value and will always
1748 reject on an attempt to draw.
1749 Examples from this strategy do not shrink (because there are
1751 none).
1752 hypothesis.strategies.one_of(*args)
1754 Return a strategy which generates values from any of the argu‐
1755 ment strategies.
1756 This may be called with one iterable argument instead of multi‐
1758 ple strategy arguments, in which case one_of(x) and one_of(*x)
1759 are equivalent.
1760 Examples from this strategy will generally shrink to ones that
1762 come from strategies earlier in the list, then shrink according
1763 to behaviour of the strategy that produced them. In order to get
1764 good shrinking behaviour, try to put simpler strategies first.
1765 e.g. one_of(none(), text()) is better than one_of(text(),
1766 none()).
1767 This is especially important when using recursive strategies.
1769 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1770 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1771 st.none()) will shrink very badly indeed.
1772 hypothesis.strategies.permutations(values)
1774 Return a strategy which returns permutations of the ordered col‐
1775 lection values.
1776 Examples from this strategy shrink by trying to become closer to
1778 the original order of values.
1779 hypothesis.strategies.random_module()
1781 The Hypothesis engine handles PRNG state for the stdlib and
1782 Numpy random modules internally, always seeding them to zero and
1783 restoring the previous state after the test.
1784 If having a fixed seed would unacceptably weaken your tests, and
1786 you cannot use a random.Random instance provided by randoms(),
1787 this strategy calls python:random.seed() with an arbitrary inte‐
1788 ger and passes you an opaque object whose repr displays the seed
1789 value for debugging. If numpy.random is available, that state
1790 is also managed.
1791 Examples from these strategy shrink to seeds closer to zero.
1793 hypothesis.strategies.randoms()
1795 Generates instances of random.Random, tweaked to show the seed
1796 value in the repr for reproducibility.
1797 Examples from this strategy shrink to seeds closer to zero.
1799 hypothesis.strategies.recursive(base, extend, max_leaves=100)
1801 base: A strategy to start from.
1802 extend: A function which takes a strategy and returns a new
1804 strategy.
1805 max_leaves: The maximum number of elements to be drawn from base
1807 on a given run.
1808 This returns a strategy S such that S = extend(base | S). That
1810 is, values may be drawn from base, or from any strategy reach‐
1811 able by mixing applications of | and extend.
1812 An example may clarify: recursive(booleans(), lists) would
1814 return a strategy that may return arbitrarily nested and mixed
1815 lists of booleans. So e.g. False, [True], [False, []], and
1816 [[[[True]]]] are all valid values to be drawn from that strat‐
1817 egy.
1818 Examples from this strategy shrink by trying to reduce the
1820 amount of recursion and by shrinking according to the shrinking
1821 behaviour of base and the result of extend.
1822 hypothesis.strategies.register_type_strategy(custom_type, strategy)
1824 Add an entry to the global type-to-strategy lookup.
1825 This lookup is used in builds() and @given.
1827 builds() will be used automatically for classes with type anno‐
1829 tations on __init__ , so you only need to register a strategy if
1830 one or more arguments need to be more tightly defined than their
1831 type-based default, or if you want to supply a strategy for an
1832 argument with a default value.
1833 strategy may be a search strategy, or a function that takes a
1835 type and returns a strategy (useful for generic types).
1836 hypothesis.strategies.runner(default=not_set)
1838 A strategy for getting "the current test runner", whatever that
1839 may be. The exact meaning depends on the entry point, but it
1840 will usually be the associated 'self' value for it.
1841 If there is no current test runner and a default is provided,
1843 return that default. If no default is provided, raises Invali‐
1844 dArgument.
1845 Examples from this strategy do not shrink (because there is only
1847 one).
1848 hypothesis.strategies.sampled_from(elements)
1850 Returns a strategy which generates any value present in ele‐
1851 ments.
1852 Note that as with just(), values will not be copied and thus you
1854 should be careful of using mutable data.
1855 sampled_from supports ordered collections, as well as Enum
1857 objects. Flag objects may also generate any combination of
1858 their members.
1859 Examples from this strategy shrink by replacing them with values
1861 earlier in the list. So e.g. sampled_from((10, 1)) will shrink
1862 by trying to replace 1 values with 10, and sampled_from((1, 10))
1863 will shrink by trying to replace 10 values with 1.
1864 hypothesis.strategies.sets(elements, min_size=0, max_size=None)
1866 This has the same behaviour as lists, but returns sets instead.
1867 Note that Hypothesis cannot tell if values are drawn from ele‐
1869 ments are hashable until running the test, so you can define a
1870 strategy for sets of an unhashable type but it will fail at test
1871 time.
1872 Examples from this strategy shrink by trying to remove elements
1874 from the set, and by shrinking each individual element of the
1875 set.
1876 hypothesis.strategies.shared(base, key=None)
1878 Returns a strategy that draws a single shared value per run,
1879 drawn from base. Any two shared instances with the same key will
1880 share the same value, otherwise the identity of this strategy
1881 will be used. That is:
1882 >>> s = integers() # or any other strategy
1884 >>> x = shared(s)
1885 >>> y = shared(s)
1886 In the above x and y may draw different (or potentially the
1888 same) values. In the following they will always draw the same:
1889 >>> x = shared(s, key="hi")
1891 >>> y = shared(s, key="hi")
1892 Examples from this strategy shrink as per their base strategy.
1894 hypothesis.strategies.slices(size)
1896 Generates slices that will select indices up to the supplied
1897 size
1898 Generated slices will have start and stop indices that range
1900 from 0 to size - 1 and will step in the appropriate direction.
1901 Slices should only produce an empty selection if the start and
1902 end are the same.
1903 Examples from this strategy shrink toward 0 and smaller values
1905 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
1907 gories=('Cs', )), min_size=0, max_size=None)
1908 Generates values of a unicode text type (unicode on python 2,
1909 str on python 3) with values drawn from alphabet, which should
1910 be an iterable of length one strings or a strategy generating
1911 such strings.
1912 The default alphabet strategy can generate the full unicode
1914 range but excludes surrogate characters because they are invalid
1915 in the UTF-8 encoding. You can use characters() without argu‐
1916 ments to find surrogate-related bugs such as bpo-34454.
1917 min_size and max_size have the usual interpretations. Note that
1919 Python measures string length by counting codepoints: U+00C5 Å
1920 is a single character, while U+0041 U+030A Å is two - the A,
1921 and a combining ring above.
1922 Examples from this strategy shrink towards shorter strings, and
1924 with the characters in the text shrinking as per the alphabet
1925 strategy. This strategy does not normalize() examples, so gen‐
1926 erated strings may be in any or none of the 'normal forms'.
1927 hypothesis.strategies.timedeltas(min_value=date‐
1929 time.timedelta(days=-999999999), max_value=date‐
1930 time.timedelta(days=999999999, seconds=86399, microseconds=999999))
1931 A strategy for timedeltas between min_value and max_value.
1932 Examples from this strategy shrink towards zero.
1934 hypothesis.strategies.times(min_value=datetime.time(0, 0),
1936 max_value=datetime.time(23, 59, 59, 999999), timezones=none())
1937 A strategy for times between min_value and max_value.
1938 The timezones argument is handled as for datetimes().
1940 Examples from this strategy shrink towards midnight, with the
1942 timezone component shrinking as for the strategy that provided
1943 it.
1944 hypothesis.strategies.tuples(*args)
1946 Return a strategy which generates a tuple of the same length as
1947 args by generating the value at index i from args[i].
1948 e.g. tuples(integers(), integers()) would generate a tuple of
1950 length two with both values an integer.
1951 Examples from this strategy shrink by shrinking their component
1953 parts.
1954 hypothesis.strategies.uuids(version=None)
1956 Returns a strategy that generates UUIDs.
1957 If the optional version argument is given, value is passed
1959 through to UUID and only UUIDs of that version will be gener‐
1960 ated.
1961 All returned values from this will be unique, so e.g. if you do
1963 lists(uuids()) the resulting list will never contain duplicates.
1964 Examples from this strategy don't have any meaningful shrink
1966 order.
1967 Shrinking
1969 When using strategies it is worth thinking about how the data shrinks.
1970 Shrinking is the process by which Hypothesis tries to produce human
1971 readable examples when it finds a failure - it takes a complex example
1972 and turns it into a simpler one.
1973 Each strategy defines an order in which it shrinks - you won't usually
1975 need to care about this much, but it can be worth being aware of as it
1976 can affect what the best way to write your own strategies is.
1977 The exact shrinking behaviour is not a guaranteed part of the API, but
1979 it doesn't change that often and when it does it's usually because we
1980 think the new way produces nicer examples.
1981 Possibly the most important one to be aware of is one_of(), which has a
1983 preference for values produced by strategies earlier in its argument
1984 list. Most of the others should largely "do the right thing" without
1985 you having to think about it.
1986 Adapting strategies
1988 Often it is the case that a strategy doesn't produce exactly what you
1989 want it to and you need to adapt it. Sometimes you can do this in the
1990 test, but this hurts reuse because you then have to repeat the adaption
1991 in every test.
1992 Hypothesis gives you ways to build strategies from other strategies
1994 given functions for transforming the data.
1995 Mapping
1997 map is probably the easiest and most useful of these to use. If you
1998 have a strategy s and a function f, then an example s.map(f).example()
1999 is f(s.example()), i.e. we draw an example from s and then apply f to
2000 it.
2001 e.g.:
2003 >>> lists(integers()).map(sorted).example()
2005 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2006 Note that many things that you might use mapping for can also be done
2008 with builds().
2009 Filtering
2011 filter lets you reject some examples. s.filter(f).example() is some
2012 example of s such that f(example) is truthy.
2013 >>> integers().filter(lambda x: x > 11).example()
2015 26126
2016 >>> integers().filter(lambda x: x > 11).example()
2017 23324
2018 It's important to note that filter isn't magic and if your condition is
2020 too hard to satisfy then this can fail:
2021 >>> integers().filter(lambda x: False).example()
2023 Traceback (most recent call last):
2024 ...
2025 hypothesis.errors.NoExamples: Could not find any valid examples in 20 tries
2026 In general you should try to use filter only to avoid corner cases that
2028 you don't want rather than attempting to cut out a large chunk of the
2029 search space.
2030 A technique that often works well here is to use map to first transform
2032 the data and then use filter to remove things that didn't work out. So
2033 for example if you wanted pairs of integers (x,y) such that x < y you
2034 could do the following:
2035 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2037 [-8543729478746591815, 3760495307320535691]
2038 Chaining strategies together
2040 Finally there is flatmap. flatmap draws an example, then turns that
2041 example into a strategy, then draws an example from that strategy.
2042 It may not be obvious why you want this at first, but it turns out to
2044 be quite useful because it lets you generate different types of data
2045 with relationships to each other.
2046 For example suppose we wanted to generate a list of lists of the same
2048 length:
2049 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2051 ... lambda n: lists(lists(integers(), min_size=n, max_size=n)))
2052 >>> find(rectangle_lists, lambda x: True)
2053 []
2054 >>> find(rectangle_lists, lambda x: len(x) >= 10)
2055 [[], [], [], [], [], [], [], [], [], []]
2056 >>> find(rectangle_lists, lambda t: len(t) >= 3 and len(t[0]) >= 3)
2057 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2058 >>> find(rectangle_lists, lambda t: sum(len(s) for s in t) >= 10)
2059 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2060 In this example we first choose a length for our tuples, then we build
2062 a strategy which generates lists containing lists precisely of that
2063 length. The finds show what simple examples for this look like.
2064 Most of the time you probably don't want flatmap, but unlike filter and
2066 map which are just conveniences for things you could just do in your
2067 tests, flatmap allows genuinely new data generation that you wouldn't
2068 otherwise be able to easily do.
2069 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2071 don't know Haskell, ignore everything in these parentheses. You do not
2072 need to understand anything about monads to use this, or anything else
2073 in Hypothesis).
2074 Recursive data
2076 Sometimes the data you want to generate has a recursive definition.
2077 e.g. if you wanted to generate JSON data, valid JSON is:
2078 1. Any float, any boolean, any unicode string.
2080 2. Any list of valid JSON data
2082 3. Any dictionary mapping unicode strings to valid JSON data.
2084 The problem is that you cannot call a strategy recursively and expect
2086 it to not just blow up and eat all your memory. The other problem here
2087 is that not all unicode strings display consistently on different
2088 machines, so we'll restrict them in our doctest.
2089 The way Hypothesis handles this is with the recursive() function which
2091 you pass in a base case and a function that, given a strategy for your
2092 data type, returns a new strategy for it. So for example:
2093 >>> from string import printable; from pprint import pprint
2095 >>> json = recursive(none() | booleans() | floats() | text(printable),
2096 ... lambda children: lists(children, 1) | dictionaries(text(printable), children, min_size=1))
2097 >>> pprint(json.example())
2098 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2099 >>> pprint(json.example())
2100 {'de(l': None,
2101 'nK': {'(Rt)': None,
2102 '+hoZh1YU]gy8': True,
2103 '8z]EIFA06^l`i^': 'LFE{Q',
2104 '9,': 'l{cA=/'}}
2105 That is, we start with our leaf data and then we augment it by allowing
2107 lists and dictionaries of anything we can generate as JSON data.
2108 The size control of this works by limiting the maximum number of values
2110 that can be drawn from the base strategy. So for example if we wanted
2111 to only generate really small JSON we could do this as:
2112 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2114 >>> small_lists.example()
2115 True
2116 >>> small_lists.example()
2117 [False]
2118 Composite strategies
2120 The @composite decorator lets you combine other strategies in more or
2121 less arbitrary ways. It's probably the main thing you'll want to use
2122 for complicated custom strategies.
2123 The composite decorator works by converting a function that returns one
2125 example into a function that returns a strategy that produces such
2126 examples - which you can pass to @given, modify with .map or .filter,
2127 and generally use like any other strategy.
2128 It does this by giving you a special function draw as the first argu‐
2130 ment, which can be used just like the corresponding method of the
2131 data() strategy within a test. In fact, the implementation is almost
2132 the same - but defining a strategy with @composite makes code reuse
2133 easier, and usually improves the display of failing examples.
2134 For example, the following gives you a list and an index into it:
2136 >>> @composite
2138 ... def list_and_index(draw, elements=integers()):
2139 ... xs = draw(lists(elements, min_size=1))
2140 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2141 ... return (xs, i)
2142 draw(s) is a function that should be thought of as returning s.exam‐
2144 ple(), except that the result is reproducible and will minimize cor‐
2145 rectly. The decorated function has the initial argument removed from
2146 the list, but will accept all the others in the expected order.
2147 Defaults are preserved.
2148 >>> list_and_index()
2150 list_and_index()
2151 >>> list_and_index().example()
2152 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2153 >>> list_and_index(booleans())
2155 list_and_index(elements=booleans())
2156 >>> list_and_index(booleans()).example()
2157 ([True, False], 0)
2158 Note that the repr will work exactly like it does for all the built-in
2160 strategies: it will be a function that you can call to get the strategy
2161 in question, with values provided only if they do not match the
2162 defaults.
2163 You can use assume inside composite functions:
2165 @composite
2167 def distinct_strings_with_common_characters(draw):
2168 x = draw(text(), min_size=1)
2169 y = draw(text(alphabet=x))
2170 assume(x != y)
2171 return (x, y)
2172 This works as assume normally would, filtering out any examples for
2174 which the passed in argument is falsey.
2175 Drawing interactively in tests
2177 There is also the data() strategy, which gives you a means of using
2178 strategies interactively. Rather than having to specify everything up
2179 front in @given you can draw from strategies in the body of your test.
2180 This is similar to @composite, but even more powerful as it allows you
2182 to mix test code with example generation. The downside of this power
2183 is that data() is incompatible with explicit @example(...)s - and the
2184 mixed code is often harder to debug when something goes wrong.
2185 If you need values that are affected by previous draws but which don't
2187 depend on the execution of your test, stick to the simpler @composite.
2188 @given(data())
2190 def test_draw_sequentially(data):
2191 x = data.draw(integers())
2192 y = data.draw(integers(min_value=x))
2193 assert x < y
2194 If the test fails, each draw will be printed with the falsifying exam‐
2196 ple. e.g. the above is wrong (it has a boundary condition error), so
2197 will print:
2198 Falsifying example: test_draw_sequentially(data=data(...))
2200 Draw 1: 0
2201 Draw 2: 0
2202 As you can see, data drawn this way is simplified as usual.
2204 Optionally, you can provide a label to identify values generated by
2206 each call to data.draw(). These labels can be used to identify values
2207 in the output of a falsifying example.
2208 For instance:
2210 @given(data())
2212 def test_draw_sequentially(data):
2213 x = data.draw(integers(), label='First number')
2214 y = data.draw(integers(min_value=x), label='Second number')
2215 assert x < y
2216 will produce the output:
2218 Falsifying example: test_draw_sequentially(data=data(...))
2220 Draw 1 (First number): 0
2221 Draw 2 (Second number): 0
2222
2224 Hypothesis has minimal dependencies (just attrs), to maximise compati‐
2225 bility and make installing Hypothesis as easy as possible.
2226 Our integrations with specific packages are therefore provided by extra
2228 modules that need their individual dependencies installed in order to
2229 work. You can install these dependencies using the setuptools extra
2230 feature as e.g. pip install hypothesis[django]. This will check
2231 installation of compatible versions.
2232 You can also just install hypothesis into a project using them, ignore
2234 the version constraints, and hope for the best.
2235 In general "Which version is Hypothesis compatible with?" is a hard
2237 question to answer and even harder to regularly test. Hypothesis is
2238 always tested against the latest compatible version and each package
2239 will note the expected compatibility range. If you run into a bug with
2240 any of these please specify the dependency version.
2241 There are separate pages for django and numpy.
2243 hypothesis[pytz]
2245 This module provides pytz timezones.
2246 You can use this strategy to make hypothesis.strategies.datetimes() and
2248 hypothesis.strategies.times() produce timezone-aware values.
2249 hypothesis.extra.pytz.timezones()
2251 Any timezone in the Olsen database, as a pytz tzinfo object.
2252 This strategy minimises to UTC, or the smallest possible fixed
2254 offset, and is designed for use with hypothesis.strategies.date‐
2255 times().
2256 hypothesis[dateutil]
2258 This module provides dateutil timezones.
2259 You can use this strategy to make hypothesis.strategies.datetimes() and
2261 hypothesis.strategies.times() produce timezone-aware values.
2262 hypothesis.extra.dateutil.timezones()
2264 Any timezone in dateutil.
2265 This strategy minimises to UTC, or the timezone with the small‐
2267 est offset from UTC as of 2000-01-01, and is designed for use
2268 with datetimes().
2269 Note that the timezones generated by the strategy may vary
2271 depending on the configuration of your machine. See the dateutil
2272 documentation for more information.
2273
2275 Hypothesis offers a number of features specific for Django testing,
2276 available in the hypothesis[django] extra. This is tested against each
2277 supported series with mainstream or extended support - if you're still
2278 getting security patches, you can test with Hypothesis.
2279 class hypothesis.extra.django.TestCase
2281 Using it is quite straightforward: All you need to do is subclass
2283 hypothesis.extra.django.TestCase or
2284 hypothesis.extra.django.TransactionTestCase and you can use @given as
2285 normal, and the transactions will be per example rather than per test
2286 function as they would be if you used @given with a normal django test
2287 suite (this is important because your test function will be called mul‐
2288 tiple times and you don't want them to interfere with each other). Test
2289 cases on these classes that do not use @given will be run as normal.
2290 class hypothesis.extra.django.TransactionTestCase
2292 We recommend avoiding TransactionTestCase unless you really have to run
2294 each test case in a database transaction. Because Hypothesis runs this
2295 in a loop, the performance problems it normally has are significantly
2296 exacerbated and your tests will be really slow. If you are using
2297 TransactionTestCase, you may need to use @settings(sup‐
2298 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2299 example generation.
2300 Having set up a test class, you can now pass @given a strategy for
2302 Django models:
2303 For example, using the trivial django project we have for testing:
2305 >>> from hypothesis.extra.django import from_model
2307 >>> from toystore.models import Customer
2308 >>> c = from_model(Customer).example()
2309 >>> c
2310 <Customer: Customer object>
2311 >>> c.email
2312 'jaime.urbina@gmail.com'
2313 >>> c.name
2314 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2315 >>> c.age
2316 -873375803
2317 Hypothesis has just created this with whatever the relevant type of
2319 data is.
2320 Obviously the customer's age is implausible, which is only possible
2322 because we have not used (eg) MinValueValidator to set the valid range
2323 for this field (or used a PositiveSmallIntegerField, which would only
2324 need a maximum value validator).
2325 If you do have validators attached, Hypothesis will only generate exam‐
2327 ples that pass validation. Sometimes that will mean that we fail a
2328 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2329 egy to skip validation at the strategy level:
2330 NOTE:
2332 Inference from validators will be much more powerful when issue
2333 #1116 is implemented, but there will always be some edge cases that
2334 require you to pass an explicit strategy.
2335 >>> from hypothesis.strategies import integers
2337 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2338 >>> c
2339 <Customer: Customer object>
2340 >>> c.age
2341 5
2342 Tips and tricks
2344 Custom field types
2345 If you have a custom Django field type you can register it with Hypoth‐
2346 esis's model deriving functionality by registering a default strategy
2347 for it:
2348 >>> from toystore.models import CustomishField, Customish
2350 >>> from_model(Customish).example()
2351 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2352 customish for model Customish
2353 >>> from hypothesis.extra.django import register_field_strategy
2354 >>> from hypothesis.strategies import just
2355 >>> register_field_strategy(CustomishField, just("hi"))
2356 >>> x = from_model(Customish).example()
2357 >>> x.customish
2358 'hi'
2359 Note that this mapping is on exact type. Subtypes will not inherit it.
2361 Generating child models
2363 For the moment there's no explicit support in hypothesis-django for
2364 generating dependent models. i.e. a Company model will generate no
2365 Shops. However if you want to generate some dependent models as well,
2366 you can emulate this by using the flatmap function as follows:
2367 from hypothesis.strategies import lists, just
2369 def generate_with_shops(company):
2371 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
2372 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
2374 Lets unpack what this is doing:
2376 The way flatmap works is that we draw a value from the original strat‐
2378 egy, then apply a function to it which gives us a new strategy. We then
2379 draw a value from that strategy. So in this case we're first drawing a
2380 company, and then we're drawing a list of shops belonging to that com‐
2381 pany: The just strategy is a strategy such that drawing it always pro‐
2382 duces the individual value, so from_model(Shop, company=just(company))
2383 is a strategy that generates a Shop belonging to the original company.
2384 So the following code would give us a list of shops all belonging to
2386 the same company:
2387 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
2389 The only difference from this and the above is that we want the com‐
2391 pany, not the shops. This is where the inner map comes in. We build the
2392 list of shops and then throw it away, instead returning the company we
2393 started for. This works because the models that Hypothesis generates
2394 are saved in the database, so we're essentially running the inner
2395 strategy purely for the side effect of creating those children in the
2396 database.
2397 Generating primary key values
2399 If your model includes a custom primary key that you want to generate
2400 using a strategy (rather than a default auto-increment primary key)
2401 then Hypothesis has to deal with the possibility of a duplicate primary
2402 key.
2403 If a model strategy generates a value for the primary key field,
2405 Hypothesis will create the model instance with update_or_create(),
2406 overwriting any existing instance in the database for this test case
2407 with the same primary key.
2408 On the subject of MultiValueField
2410 Django forms feature the MultiValueField which allows for several
2411 fields to be combined under a single named field, the default example
2412 of this is the SplitDateTimeField.
2413 class CustomerForm(forms.Form):
2415 name = forms.CharField()
2416 birth_date_time = forms.SplitDateTimeField()
2417 from_form supports MultiValueField subclasses directly, however if you
2419 want to define your own strategy be forewarned that Django binds data
2420 for a MultiValueField in a peculiar way. Specifically each sub-field is
2421 expected to have its own entry in data addressed by the field name
2422 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
2423 ValueField, so form data for the example above might look like this:
2424 {
2426 'name': 'Samuel John',
2427 'birth_date_time_0': '2018-05-19', # the date, as the first sub-field
2428 'birth_date_time_1': '15:18:00' # the time, as the second sub-field
2429 }
2430 Thus, if you want to define your own strategies for such a field you
2432 must address your sub-fields appropriately:
2433 from_form(CustomerForm, birth_date_time_0=just('2018-05-19'))
2435
2437 numpy
2438 Hypothesis offers a number of strategies for NumPy testing, available
2439 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
2440 package.
2441 The centerpiece is the arrays() strategy, which generates arrays with
2443 any dtype, shape, and contents you can specify or give a strategy for.
2444 To make this as useful as possible, strategies are provided to generate
2445 array shapes and generate all kinds of fixed-size or compound dtypes.
2446 hypothesis.extra.numpy.from_dtype(dtype)
2448 Creates a strategy which can generate any value of the given
2449 dtype.
2450 hypothesis.extra.numpy.arrays(dtype, shape, elements=None, fill=None,
2452 unique=False)
2453 Returns a strategy for generating numpy:numpy.ndarrays.
2454 · dtype may be any valid input to dtype (this includes dtype
2456 objects), or a strategy that generates such values.
2457 · shape may be an integer >= 0, a tuple of such integers, or a
2459 strategy that generates such values.
2460 · elements is a strategy for generating values to put in the
2462 array. If it is None a suitable value will be inferred based
2463 on the dtype, which may give any legal value (including eg NaN
2464 for floats). If you have more specific requirements, you
2465 should supply your own elements strategy.
2466 · fill is a strategy that may be used to generate a single back‐
2468 ground value for the array. If None, a suitable default will
2469 be inferred based on the other arguments. If set to nothing()
2470 then filling behaviour will be disabled entirely and every
2471 element will be generated independently.
2472 · unique specifies if the elements of the array should all be
2474 distinct from one another. Note that in this case multiple NaN
2475 values may still be allowed. If fill is also set, the only
2476 valid values for it to return are NaN values (anything for
2477 which numpy:numpy.isnan returns True. So e.g. for complex num‐
2478 bers (nan+1j) is also a valid fill). Note that if unique is
2479 set to True the generated values must be hashable.
2480 Arrays of specified dtype and shape are generated for example
2482 like this:
2483 >>> import numpy as np
2485 >>> arrays(np.int8, (2, 3)).example()
2486 array([[-8, 6, 3],
2487 [-6, 4, 6]], dtype=int8)
2488 · See What you can generate and how.
2490 >>> import numpy as np
2492 >>> from hypothesis.strategies import floats
2493 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
2494 array([ 0.88974794, 0.77387938, 0.1977879 ])
2495 Array values are generated in two parts:
2497 1. Some subset of the coordinates of the array are populated
2499 with a value drawn from the elements strategy (or its
2500 inferred form).
2501 2. If any coordinates were not assigned in the previous step, a
2503 single value is drawn from the fill strategy and is assigned
2504 to all remaining places.
2505 You can set fill to nothing() if you want to disable this behav‐
2507 iour and draw a value for every element.
2508 If fill is set to None then it will attempt to infer the correct
2510 behaviour automatically: If unique is True, no filling will
2511 occur by default. Otherwise, if it looks safe to reuse the val‐
2512 ues of elements across multiple coordinates (this will be the
2513 case for any inferred strategy, and for most of the builtins,
2514 but is not the case for mutable values or strategies built with
2515 flatmap, map, composite, etc) then it will use the elements
2516 strategy as the fill, else it will default to having no fill.
2517 Having a fill helps Hypothesis craft high quality examples, but
2519 its main importance is when the array generated is large:
2520 Hypothesis is primarily designed around testing small examples.
2521 If you have arrays with hundreds or more elements, having a fill
2522 value is essential if you want your tests to run in reasonable
2523 time.
2524 hypothesis.extra.numpy.array_shapes(min_dims=1, max_dims=None,
2526 min_side=1, max_side=None)
2527 Return a strategy for array shapes (tuples of int >= 1).
2528 hypothesis.extra.numpy.scalar_dtypes()
2530 Return a strategy that can return any non-flexible scalar dtype.
2531 hypothesis.extra.numpy.unsigned_integer_dtypes(endianness='?',
2533 sizes=(8, 16, 32, 64))
2534 Return a strategy for unsigned integer dtypes.
2535 endianness may be < for little-endian, > for big-endian, = for
2537 native byte order, or ? to allow either byte order. This argu‐
2538 ment only applies to dtypes of more than one byte.
2539 sizes must be a collection of integer sizes in bits. The
2541 default (8, 16, 32, 64) covers the full range of sizes.
2542 hypothesis.extra.numpy.integer_dtypes(endianness='?', sizes=(8, 16, 32,
2544 64))
2545 Return a strategy for signed integer dtypes.
2546 endianness and sizes are treated as for
2548 unsigned_integer_dtypes().
2549 hypothesis.extra.numpy.floating_dtypes(endianness='?', sizes=(16, 32,
2551 64))
2552 Return a strategy for floating-point dtypes.
2553 sizes is the size in bits of floating-point number. Some
2555 machines support 96- or 128-bit floats, but these are not gener‐
2556 ated by default.
2557 Larger floats (96 and 128 bit real parts) are not supported on
2559 all platforms and therefore disabled by default. To generate
2560 these dtypes, include these values in the sizes argument.
2561 hypothesis.extra.numpy.complex_number_dtypes(endianness='?', sizes=(64,
2563 128))
2564 Return a strategy for complex-number dtypes.
2565 sizes is the total size in bits of a complex number, which con‐
2567 sists of two floats. Complex halfs (a 16-bit real part) are not
2568 supported by numpy and will not be generated by this strategy.
2569 hypothesis.extra.numpy.datetime64_dtypes(max_period='Y',
2571 min_period='ns', endianness='?')
2572 Return a strategy for datetime64 dtypes, with various precisions
2573 from year to attosecond.
2574 hypothesis.extra.numpy.timedelta64_dtypes(max_period='Y',
2576 min_period='ns', endianness='?')
2577 Return a strategy for timedelta64 dtypes, with various preci‐
2578 sions from year to attosecond.
2579 hypothesis.extra.numpy.byte_string_dtypes(endianness='?', min_len=0,
2581 max_len=16)
2582 Return a strategy for generating bytestring dtypes, of various
2583 lengths and byteorder.
2584 hypothesis.extra.numpy.unicode_string_dtypes(endianness='?', min_len=0,
2586 max_len=16)
2587 Return a strategy for generating unicode string dtypes, of vari‐
2588 ous lengths and byteorder.
2589 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
2591 min_size=1, max_size=5, allow_subarrays=False)
2592 Return a strategy for generating array (compound) dtypes, with
2593 members drawn from the given subtype strategy.
2594 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
2596 max_leaves=10, max_itemsize=None)
2597 Return the most-general dtype strategy.
2598 Elements drawn from this strategy may be simple (from the sub‐
2600 type_strategy), or several such values drawn from array_dtypes()
2601 with allow_subarrays=True. Subdtypes in an array dtype may be
2602 nested to any depth, subject to the max_leaves argument.
2603 hypothesis.extra.numpy.valid_tuple_axes(ndim, min_size=0,
2605 max_size=None)
2606 Return a strategy for generating permissible tuple-values for
2607 the axis argument for a numpy sequential function (e.g.
2608 numpy:numpy.sum()), given an array of the specified dimensional‐
2609 ity.
2610 All tuples will have an length >= min_size and <= max_size. The
2612 default value for max_size is ndim.
2613 Examples from this strategy shrink towards an empty tuple, which
2615 render most sequential functions as no-ops.
2616 The following are some examples drawn from this strategy.
2618 >>> [valid_tuple_axes(3).example() for i in range(4)]
2620 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
2621 valid_tuple_axes can be joined with other strategies to generate
2623 any type of valid axis object, i.e. integers, tuples, and None:
2624 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
2626 hypothesis.extra.numpy.broadcastable_shapes(shape, min_dims=0,
2628 max_dims=None, min_side=1, max_side=None)
2629 Return a strategy for generating shapes that are broadcast-com‐
2630 patible with the provided shape.
2631 Examples from this strategy shrink towards a shape with length
2633 min_dims. The size of an aligned dimension shrinks towards
2634 being a singleton. The size of an unaligned dimension shrink
2635 towards min_side.
2636 · shape a tuple of integers
2638 · min_dims The smallest length that the generated shape can pos‐
2640 sess.
2641 · max_dims The largest length that the generated shape can pos‐
2643 sess. shape can possess. Cannot exceed 32. The default-value
2644 for max_dims is 2 + max(len(shape), min_dims).
2645 · min_side The smallest size that an unaligned dimension can
2647 possess.
2648 · max_side The largest size that an unaligned dimension can pos‐
2650 sess. The default value is 2 +
2651 'size-of-largest-aligned-dimension'.
2652 The following are some examples drawn from this strategy.
2654 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
2656 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
2657 pandas
2659 Hypothesis provides strategies for several of the core pandas data
2660 types: pandas.Index, pandas.Series and pandas.DataFrame.
2661 The general approach taken by the pandas module is that there are mul‐
2663 tiple strategies for generating indexes, and all of the other strate‐
2664 gies take the number of entries they contain from their index strategy
2665 (with sensible defaults). So e.g. a Series is specified by specifying
2666 its numpy.dtype (and/or a strategy for generating elements for it).
2667 class hypothesis.extra.pandas.column(name=None, elements=None,
2669 dtype=None, fill=None, unique=False)
2670 Data object for describing a column in a DataFrame.
2671 Arguments:
2673 · name: the column name, or None to default to the column posi‐
2675 tion. Must be hashable, but can otherwise be any value sup‐
2676 ported as a pandas column name.
2677 · elements: the strategy for generating values in this column,
2679 or None to infer it from the dtype.
2680 · dtype: the dtype of the column, or None to infer it from the
2682 element strategy. At least one of dtype or elements must be
2683 provided.
2684 · fill: A default value for elements of the column. See arrays()
2686 for a full explanation.
2687 · unique: If all values in this column should be distinct.
2689 hypothesis.extra.pandas.columns(names_or_number, dtype=None, ele‐
2691 ments=None, fill=None, unique=False)
2692 A convenience function for producing a list of column objects of
2693 the same general shape.
2694 The names_or_number argument is either a sequence of values, the
2696 elements of which will be used as the name for individual column
2697 objects, or a number, in which case that many unnamed columns
2698 will be created. All other arguments are passed through verbatim
2699 to create the columns.
2700 hypothesis.extra.pandas.data_frames(columns=None, rows=None,
2702 index=None)
2703 Provides a strategy for producing a pandas.DataFrame.
2704 Arguments:
2706 · columns: An iterable of column objects describing the shape of
2708 the generated DataFrame.
2709 · rows: A strategy for generating a row object. Should generate
2711 either dicts mapping column names to values or a sequence map‐
2712 ping column position to the value in that position (note that
2713 unlike the pandas.DataFrame constructor, single values are not
2714 allowed here. Passing e.g. an integer is an error, even if
2715 there is only one column).
2716 At least one of rows and columns must be provided. If both are
2718 provided then the generated rows will be validated against the
2719 columns and an error will be raised if they don't match.
2720 Caveats on using rows:
2722 · In general you should prefer using columns to rows, and only
2724 use rows if the columns interface is insufficiently flexible
2725 to describe what you need - you will get better performance
2726 and example quality that way.
2727 · If you provide rows and not columns, then the shape and
2729 dtype of the resulting DataFrame may vary. e.g. if you have
2730 a mix of int and float in the values for one column in your
2731 row entries, the column will sometimes have an integral
2732 dtype and sometimes a float.
2733 · index: If not None, a strategy for generating indexes for the
2735 resulting DataFrame. This can generate either pandas.Index
2736 objects or any sequence of values (which will be passed to the
2737 Index constructor).
2738 You will probably find it most convenient to use the indexes()
2740 or range_indexes() function to produce values for this argu‐
2741 ment.
2742 Usage:
2744 The expected usage pattern is that you use column and columns()
2746 to specify a fixed shape of the DataFrame you want as follows.
2747 For example the following gives a two column data frame:
2748 >>> from hypothesis.extra.pandas import column, data_frames
2750 >>> data_frames([
2751 ... column('A', dtype=int), column('B', dtype=float)]).example()
2752 A B
2753 0 2021915903 1.793898e+232
2754 1 1146643993 inf
2755 2 -2096165693 1.000000e+07
2756 If you want the values in different columns to interact in some
2758 way you can use the rows argument. For example the following
2759 gives a two column DataFrame where the value in the first column
2760 is always at most the value in the second:
2761 >>> from hypothesis.extra.pandas import column, data_frames
2763 >>> import hypothesis.strategies as st
2764 >>> data_frames(
2765 ... rows=st.tuples(st.floats(allow_nan=False),
2766 ... st.floats(allow_nan=False)).map(sorted)
2767 ... ).example()
2768 0 1
2769 0 -3.402823e+38 9.007199e+15
2770 1 -1.562796e-298 5.000000e-01
2771 You can also combine the two:
2773 >>> from hypothesis.extra.pandas import columns, data_frames
2775 >>> import hypothesis.strategies as st
2776 >>> data_frames(
2777 ... columns=columns(["lo", "hi"], dtype=float),
2778 ... rows=st.tuples(st.floats(allow_nan=False),
2779 ... st.floats(allow_nan=False)).map(sorted)
2780 ... ).example()
2781 lo hi
2782 0 9.314723e-49 4.353037e+45
2783 1 -9.999900e-01 1.000000e+07
2784 2 -2.152861e+134 -1.069317e-73
2785 (Note that the column dtype must still be specified and will not
2787 be inferred from the rows. This restriction may be lifted in
2788 future).
2789 Combining rows and columns has the following behaviour:
2791 · The column names and dtypes will be used.
2793 · If the column is required to be unique, this will be enforced.
2795 · Any values missing from the generated rows will be provided
2797 using the column's fill.
2798 · Any values in the row not present in the column specification
2800 (if dicts are passed, if there are keys with no corresponding
2801 column name, if sequences are passed if there are too many
2802 items) will result in InvalidArgument being raised.
2803 hypothesis.extra.pandas.indexes(elements=None, dtype=None, min_size=0,
2805 max_size=None, unique=True)
2806 Provides a strategy for producing a pandas.Index.
2807 Arguments:
2809 · elements is a strategy which will be used to generate the
2811 individual values of the index. If None, it will be inferred
2812 from the dtype. Note: even if the elements strategy produces
2813 tuples, the generated value will not be a MultiIndex, but
2814 instead be a normal index whose elements are tuples.
2815 · dtype is the dtype of the resulting index. If None, it will be
2817 inferred from the elements strategy. At least one of dtype or
2818 elements must be provided.
2819 · min_size is the minimum number of elements in the index.
2821 · max_size is the maximum number of elements in the index. If
2823 None then it will default to a suitable small size. If you
2824 want larger indexes you should pass a max_size explicitly.
2825 · unique specifies whether all of the elements in the resulting
2827 index should be distinct.
2828 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
2830 Provides a strategy which generates an Index whose values are 0,
2831 1, ..., n for some n.
2832 Arguments:
2834 · min_size is the smallest number of elements the index can
2836 have.
2837 · max_size is the largest number of elements the index can have.
2839 If None it will default to some suitable value based on
2840 min_size.
2841 hypothesis.extra.pandas.series(elements=None, dtype=None, index=None,
2843 fill=None, unique=False)
2844 Provides a strategy for producing a pandas.Series.
2845 Arguments:
2847 · elements: a strategy that will be used to generate the indi‐
2849 vidual values in the series. If None, we will attempt to infer
2850 a suitable default from the dtype.
2851 · dtype: the dtype of the resulting series and may be any value
2853 that can be passed to numpy.dtype. If None, will use pandas's
2854 standard behaviour to infer it from the type of the elements
2855 values. Note that if the type of values that comes out of your
2856 elements strategy varies, then so will the resulting dtype of
2857 the series.
2858 · index: If not None, a strategy for generating indexes for the
2860 resulting Series. This can generate either pandas.Index
2861 objects or any sequence of values (which will be passed to the
2862 Index constructor).
2863 You will probably find it most convenient to use the indexes()
2865 or range_indexes() function to produce values for this argu‐
2866 ment.
2867 Usage:
2869 >>> series(dtype=int).example()
2871 0 -2001747478
2872 1 1153062837
2873 Supported Versions
2875 There is quite a lot of variation between pandas versions. We only com‐
2876 mit to supporting the latest version of pandas, but older minor ver‐
2877 sions are supported on a "best effort" basis. Hypothesis is currently
2878 tested against and confirmed working with Pandas 0.19, 0.20, 0.21,
2879 0.22, and 0.23.
2880 Releases that are not the latest patch release of their minor version
2882 are not tested or officially supported, but will probably also work
2883 unless you hit a pandas bug.
2884
2886 Hypothesis tries to detect common mistakes and things that will cause
2887 difficulty at run time in the form of a number of 'health checks'.
2888 These include detecting and warning about:
2890 · Strategies with very slow data generation
2892 · Strategies which filter out too much
2894 · Recursive strategies which branch too much
2896 · Tests that are unlikely to complete in a reasonable amount of time.
2898 If any of these scenarios are detected, Hypothesis will emit a warning
2900 about them.
2901 The general goal of these health checks is to warn you about things
2903 that you are doing that might appear to work but will either cause
2904 Hypothesis to not work correctly or to perform badly.
2905 To selectively disable health checks, use the suppress_health_check
2907 setting. The argument for this parameter is a list with elements drawn
2908 from any of the class-level attributes of the HealthCheck class. Using
2909 a value of HealthCheck.all() will disable all health checks.
2910 class hypothesis.HealthCheck
2912 Arguments for suppress_health_check.
2913 Each member of this enum is a type of health check to suppress.
2915 all = <bound method HealthCheck.all of <enum 'HealthCheck'>>
2917 data_too_large = 1
2919 Check for when the typical size of the examples you are
2920 generating exceeds the maximum allowed size too often.
2921 filter_too_much = 2
2923 Check for when the test is filtering out too many exam‐
2924 ples, either through use of assume() or filter(), or
2925 occasionally for Hypothesis internal reasons.
2926 too_slow = 3
2928 Check for when your data generation is extremely slow and
2929 likely to hurt testing.
2930 return_value = 5
2932 Checks if your tests return a non-None value (which will
2933 be ignored and is unlikely to do what you want).
2934 hung_test = 6
2936 This health check is deprecated and no longer has any
2937 effect. You can use the max_examples and deadline set‐
2938 tings together to cap the total runtime of your tests,
2939 rather than the previous fixed limit.
2940 large_base_example = 7
2942 Checks if the natural example to shrink towards is very
2943 large.
2944 not_a_test_method = 8
2946 Checks if @given has been applied to a method of
2947 python:unittest.TestCase.
2948 Deprecations
2950 We also use a range of custom exception and warning types, so you can
2951 see exactly where an error came from - or turn only our warnings into
2952 errors.
2953 class hypothesis.errors.HypothesisDeprecationWarning
2955 A deprecation warning issued by Hypothesis.
2956 Actually inherits from FutureWarning, because DeprecationWarning
2958 is hidden by the default warnings filter.
2959 You can configure the Python python:warnings to handle these
2961 warnings differently to others, either turning them into errors
2962 or suppressing them entirely. Obviously we would prefer the
2963 former!
2964 Deprecated features will be continue to emit warnings for at least six
2966 months, and then be removed in the following major release. Note how‐
2967 ever that not all warnings are subject to this grace period; sometimes
2968 we strengthen validation by adding a warning and these may become
2969 errors immediately at a major release.
2970
2972 When Hypothesis finds a bug it stores enough information in its data‐
2973 base to reproduce it. This enables you to have a classic testing work‐
2974 flow of find a bug, fix a bug, and be confident that this is actually
2975 doing the right thing because Hypothesis will start by retrying the
2976 examples that broke things last time.
2977 Limitations
2979 The database is best thought of as a cache that you never need to
2980 invalidate: Information may be lost when you upgrade a Hypothesis ver‐
2981 sion or change your test, so you shouldn't rely on it for correctness -
2982 if there's an example you want to ensure occurs each time then there's
2983 a feature for including them in your source code - but it helps the
2984 development workflow considerably by making sure that the examples
2985 you've just found are reproduced.
2986 The database also records examples that exercise less-used parts of
2988 your code, so the database may update even when no failing examples
2989 were found.
2990 File locations
2992 The default storage format is as a fairly opaque directory structure.
2993 Each test corresponds to a directory, and each example to a file within
2994 that directory. The standard location for it is .hypothesis/examples
2995 in your current working directory. You can override this by setting the
2996 database setting.
2997 If you have not configured a database and the default location is unus‐
2999 able (e.g. because you do not have read/write permission), Hypothesis
3000 will issue a warning and then fall back to an in-memory database.
3001 Upgrading Hypothesis and changing your tests
3003 The design of the Hypothesis database is such that you can put arbi‐
3004 trary data in the database and not get wrong behaviour. When you
3005 upgrade Hypothesis, old data might be invalidated, but this should hap‐
3006 pen transparently. It can never be the case that e.g. changing the
3007 strategy that generates an argument gives you data from the old strat‐
3008 egy.
3009 Sharing your example database
3011 NOTE:
3012 If specific examples are important for correctness you should use
3013 the @example decorator, as the example database may discard entries
3014 due to changes in your code or dependencies. For most users, we
3015 therefore recommend using the example database locally and possibly
3016 persisting it between CI builds, but not tracking it under version
3017 control.
3018 The examples database can be shared simply by checking the directory
3020 into version control, for example with the following .gitignore:
3021 # Ignore files cached by Hypothesis...
3023 .hypothesis/*
3024 # except for the examples directory
3025 !.hypothesis/examples/
3026 Like everything under .hypothesis/, the examples directory will be
3028 transparently created on demand. Unlike the other subdirectories,
3029 examples/ is designed to handle merges, deletes, etc if you just add
3030 the directory into git, mercurial, or any similar version control sys‐
3031 tem.
3032
3034 With @given, your tests are still something that you mostly write your‐
3035 self, with Hypothesis providing some data. With Hypothesis's stateful
3036 testing, Hypothesis instead tries to generate not just data but entire
3037 tests. You specify a number of primitive actions that can be combined
3038 together, and then Hypothesis will try to find sequences of those
3039 actions that result in a failure.
3040 NOTE:
3042 This style of testing is often called model-based testing, but in
3043 Hypothesis is called stateful testing (mostly for historical reasons
3044 - the original implementation of this idea in Hypothesis was more
3045 closely based on ScalaCheck's stateful testing where the name is
3046 more apt). Both of these names are somewhat misleading: You don't
3047 really need any sort of formal model of your code to use this, and
3048 it can be just as useful for pure APIs that don't involve any state
3049 as it is for stateful ones.
3050 It's perhaps best to not take the name of this sort of testing too
3052 seriously. Regardless of what you call it, it is a powerful form of
3053 testing which is useful for most non-trivial APIs.
3054 Hypothesis has two stateful testing APIs: A high level one, providing
3056 what we call rule based state machines, and a low level one, providing
3057 what we call generic state machines.
3058 You probably want to use the rule based state machines - they provide a
3060 high level API for describing the sort of actions you want to perform,
3061 based on a structured representation of actions. However the generic
3062 state machines are more flexible, and are particularly useful if you
3063 want the set of currently possible actions to depend primarily on
3064 external state.
3065 You may not need state machines
3067 The basic idea of stateful testing is to make Hypothesis choose actions
3068 as well as values for your test, and state machines are a great declar‐
3069 ative way to do just that.
3070 For simpler cases though, you might not need them at all - a standard
3072 test with @given might be enough, since you can use data() in branches
3073 or loops. In fact, that's how the state machine explorer works inter‐
3074 nally. For more complex workloads though, where a higher level API
3075 comes into it's own, keep reading!
3076 Rule based state machines
3078 Rule based state machines are the ones you're most likely to want to
3079 use. They're significantly more user friendly and should be good
3080 enough for most things you'd want to do.
3081 class hypothesis.stateful.RuleBasedStateMachine
3083 A RuleBasedStateMachine gives you a more structured way to
3084 define state machines.
3085 The idea is that a state machine carries a bunch of types of
3087 data divided into Bundles, and has a set of rules which may read
3088 data from bundles (or just from normal strategies) and push data
3089 onto bundles. At any given point a random applicable rule will
3090 be executed.
3091 A rule is very similar to a normal @given based test in that it takes
3093 values drawn from strategies and passes them to a user defined test
3094 function. The key difference is that where @given based tests must be
3095 independent, rules can be chained together - a single test run may
3096 involve multiple rule invocations, which may interact in various ways.
3097 Rules can take normal strategies as arguments, or a specific kind of
3099 strategy called a Bundle. A Bundle is a named collection of generated
3100 values that can be reused by other operations in the test. They are
3101 populated with the results of rules, and may be used as arguments to
3102 rules, allowing data to flow from one rule to another, and rules to
3103 work on the results of previous computations or actions.
3104 You can think of each value that gets added to any Bundle as being
3106 assigned to a new variable. Drawing a value from the bundle strategy
3107 means choosing one of the corresponding variables and using that value,
3108 and consumes() as a del statement for that variable. If you can
3109 replace use of Bundles with instance attributes of the class that is
3110 often simpler, but often Bundles are strictly more powerful.
3111 The following rule based state machine example is a simplified version
3113 of a test for Hypothesis's example database implementation. An example
3114 database maps keys to sets of values, and in this test we compare one
3115 implementation of it to a simplified in memory model of its behaviour,
3116 which just stores the same values in a Python dict. The test then runs
3117 operations against both the real database and the in-memory representa‐
3118 tion of it and looks for discrepancies in their behaviour.
3119 import shutil
3121 import tempfile
3122 from collections import defaultdict
3124 import hypothesis.strategies as st
3125 from hypothesis.database import DirectoryBasedExampleDatabase
3126 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
3127 class DatabaseComparison(RuleBasedStateMachine):
3130 def __init__(self):
3131 super(DatabaseComparison, self).__init__()
3132 self.tempd = tempfile.mkdtemp()
3133 self.database = DirectoryBasedExampleDatabase(self.tempd)
3134 self.model = defaultdict(set)
3135 keys = Bundle('keys')
3137 values = Bundle('values')
3138 @rule(target=keys, k=st.binary())
3140 def add_key(self, k):
3141 return k
3142 @rule(target=values, v=st.binary())
3144 def add_value(self, v):
3145 return v
3146 @rule(k=keys, v=values)
3148 def save(self, k, v):
3149 self.model[k].add(v)
3150 self.database.save(k, v)
3151 @rule(k=keys, v=values)
3153 def delete(self, k, v):
3154 self.model[k].discard(v)
3155 self.database.delete(k, v)
3156 @rule(k=keys)
3158 def values_agree(self, k):
3159 assert set(self.database.fetch(k)) == self.model[k]
3160 def teardown(self):
3162 shutil.rmtree(self.tempd)
3163 TestDBComparison = DatabaseComparison.TestCase
3166 In this we declare two bundles - one for keys, and one for values. We
3168 have two trivial rules which just populate them with data (k and v),
3169 and three non-trivial rules: save saves a value under a key and delete
3170 removes a value from a key, in both cases also updating the model of
3171 what should be in the database. values_agree then checks that the con‐
3172 tents of the database agrees with the model for a particular key.
3173 We can then integrate this into our test suite by getting a unittest
3175 TestCase from it:
3176 TestTrees = DatabaseComparison.TestCase
3178 # Or just run with pytest's unittest support
3180 if __name__ == '__main__':
3181 unittest.main()
3182 This test currently passes, but if we comment out the line where we
3184 call self.model[k].discard(v), we would see the following output when
3185 run under pytest:
3186 AssertionError: assert set() == {b''}
3188 ------------ Hypothesis ------------
3190 state = DatabaseComparison()
3192 var1 = state.add_key(k=b'')
3193 var2 = state.add_value(v=var1)
3194 state.save(k=var1, v=var2)
3195 state.delete(k=var1, v=var2)
3196 state.values_agree(k=var1)
3197 state.teardown()
3198 Note how it's printed out a very short program that will demonstrate
3200 the problem. The output from a rule based state machine should gener‐
3201 ally be pretty close to Python code - if you have custom repr implemen‐
3202 tations that don't return valid Python then it might not be, but most
3203 of the time you should just be able to copy and paste the code into a
3204 test to reproduce it.
3205 You can control the detailed behaviour with a settings object on the
3207 TestCase (this is a normal hypothesis settings object using the
3208 defaults at the time the TestCase class was first referenced). For
3209 example if you wanted to run fewer examples with larger programs you
3210 could change the settings to:
3211 DatabaseComparison.TestCase.settings = settings(max_examples=50, stateful_step_count=100)
3213 Which doubles the number of steps each program runs and halves the num‐
3215 ber of test cases that will be run.
3216 Rules
3218 As said earlier, rules are the most common feature used in RuleBased‐
3219 StateMachine. They are defined by applying the rule() decorator on a
3220 function. Note that RuleBasedStateMachine must have at least one rule
3221 defined and that a single function cannot be used to define multiple
3222 rules (this to avoid having multiple rules doing the same things). Due
3223 to the stateful execution method, rules generally cannot take arguments
3224 from other sources such as fixtures or pytest.mark.parametrize - con‐
3225 sider providing them via a strategy such as sampled_from() instead.
3226 hypothesis.stateful.rule(targets=(), target=None, **kwargs)
3228 Decorator for RuleBasedStateMachine. Any name present in target
3229 or targets will define where the end result of this function
3230 should go. If both are empty then the end result will be dis‐
3231 carded.
3232 target must be a Bundle, or if the result should go to multiple
3234 bundles you can pass a tuple of them as the targets argument.
3235 It is invalid to use both arguments for a single rule. If the
3236 result should go to exactly one of several bundles, define a
3237 separate rule for each case.
3238 kwargs then define the arguments that will be passed to the
3240 function invocation. If their value is a Bundle, or if it is
3241 consumes(b) where b is a Bundle, then values that have previ‐
3242 ously been produced for that bundle will be provided. If con‐
3243 sumes is used, the value will also be removed from the bundle.
3244 Any other kwargs should be strategies and values from them will
3246 be provided.
3247 hypothesis.stateful.consumes(bundle)
3249 When introducing a rule in a RuleBasedStateMachine, this func‐
3250 tion can be used to mark bundles from which each value used in a
3251 step with the given rule should be removed. This function
3252 returns a strategy object that can be manipulated and combined
3253 like any other.
3254 For example, a rule declared with
3256 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
3258 sumes(b3)))
3259 will consume a value from Bundle b2 and several values from Bun‐
3261 dle b3 to populate value2 and value3 each time it is executed.
3262 hypothesis.stateful.multiple(*args)
3264 This function can be used to pass multiple results to the tar‐
3265 get(s) of a rule. Just use return multiple(result1, result2,
3266 ...) in your rule.
3267 It is also possible to use return multiple() with no arguments
3269 in order to end a rule without passing any result.
3270 Initializes
3272 Initializes are a special case of rules that are guaranteed to be run
3273 at most once at the beginning of a run (i.e. before any normal rule is
3274 called). Note if multiple initialize rules are defined, they may be
3275 called in any order, and that order will vary from run to run.
3276 Initializes are typically useful to populate bundles:
3278 hypothesis.stateful.initialize(targets=(), target=None, **kwargs)
3280 Decorator for RuleBasedStateMachine.
3281 An initialize decorator behaves like a rule, but the decorated
3283 method is called at most once in a run. All initialize decorated
3284 methods will be called before any rule decorated methods, in an
3285 arbitrary order.
3286 import hypothesis.strategies as st
3288 from hypothesis.stateful import RuleBasedStateMachine, Bundle, rule, initialize
3289 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
3291 class NumberModifier(RuleBasedStateMachine):
3293 folders = Bundle('folders')
3295 files = Bundle('files')
3296 @initialize(target=folders)
3298 def init_folders(self):
3299 return '/'
3300 @rule(target=folders, name=name_strategy)
3302 def create_folder(self, parent, name):
3303 return '%s/%s' % (parent, name)
3304 @rule(target=files, name=name_strategy)
3306 def create_file(self, parent, name):
3307 return '%s/%s' % (parent, name)
3308 Preconditions
3310 While it's possible to use assume() in RuleBasedStateMachine rules, if
3311 you use it in only a few rules you can quickly run into a situation
3312 where few or none of your rules pass their assumptions. Thus, Hypothe‐
3313 sis provides a precondition() decorator to avoid this problem. The
3314 precondition() decorator is used on rule-decorated functions, and must
3315 be given a function that returns True or False based on the RuleBased‐
3316 StateMachine instance.
3317 hypothesis.stateful.precondition(precond)
3319 Decorator to apply a precondition for rules in a RuleBased‐
3320 StateMachine. Specifies a precondition for a rule to be consid‐
3321 ered as a valid step in the state machine. The given function
3322 will be called with the instance of RuleBasedStateMachine and
3323 should return True or False. Usually it will need to look at
3324 attributes on that instance.
3325 For example:
3327 class MyTestMachine(RuleBasedStateMachine):
3329 state = 1
3330 @precondition(lambda self: self.state != 0)
3332 @rule(numerator=integers())
3333 def divide_with(self, numerator):
3334 self.state = numerator / self.state
3335 This is better than using assume in your rule since more valid
3337 rules should be able to be run.
3338 from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
3340 class NumberModifier(RuleBasedStateMachine):
3342 num = 0
3344 @rule()
3346 def add_one(self):
3347 self.num += 1
3348 @precondition(lambda self: self.num != 0)
3350 @rule()
3351 def divide_with_one(self):
3352 self.num = 1 / self.num
3353 By using precondition() here instead of assume(), Hypothesis can filter
3355 the inapplicable rules before running them. This makes it much more
3356 likely that a useful sequence of steps will be generated.
3357 Note that currently preconditions can't access bundles; if you need to
3359 use preconditions, you should store relevant data on the instance
3360 instead.
3361 Invariants
3363 Often there are invariants that you want to ensure are met after every
3364 step in a process. It would be possible to add these as rules that are
3365 run, but they would be run zero or multiple times between other rules.
3366 Hypothesis provides a decorator that marks a function to be run after
3367 every step.
3368 hypothesis.stateful.invariant()
3370 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
3371 chine. The decorated function will be run after every rule and
3372 can raise an exception to indicate failed invariants.
3373 For example:
3375 class MyTestMachine(RuleBasedStateMachine):
3377 state = 1
3378 @invariant()
3380 def is_nonzero(self):
3381 assert self.state != 0
3382 from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
3384 class NumberModifier(RuleBasedStateMachine):
3386 num = 0
3388 @rule()
3390 def add_two(self):
3391 self.num += 2
3392 if self.num > 50:
3393 self.num += 1
3394 @invariant()
3396 def divide_with_one(self):
3397 assert self.num % 2 == 0
3398 NumberTest = NumberModifier.TestCase
3400 Invariants can also have precondition()s applied to them, in which case
3402 they will only be run if the precondition function returns true.
3403 Note that currently invariants can't access bundles; if you need to use
3405 invariants, you should store relevant data on the instance instead.
3406 Generic state machines
3408 The class GenericStateMachine is the underlying machinery of stateful
3409 testing in Hypothesis. Chances are you will want to use the rule based
3410 stateful testing for most things, but the generic state machine func‐
3411 tionality can be useful e.g. if you want to test things where the set
3412 of actions to be taken is more closely tied to the state of the system
3413 you are testing.
3414 NOTE:
3416 GenericStateMachine is a very thin wrapper around @given and data().
3417 Consider using those primitives directly, as the resulting tests are
3418 usually easier to write, understand, and debug. GenericStateMachine
3419 only exists because it predates data(), and will be deprecated and
3420 removed in a future version (issue #1693).
3421 class hypothesis.stateful.GenericStateMachine
3423 A GenericStateMachine is the basic entry point into Hypothesis's
3424 approach to stateful testing.
3425 The intent is for it to be subclassed to provide state machine
3427 descriptions
3428 The way this is used is that Hypothesis will repeatedly execute
3430 something that looks something like:
3431 x = MyStatemachineSubclass()
3433 x.check_invariants()
3434 try:
3435 for _ in range(n_steps):
3436 x.execute_step(x.steps().example())
3437 x.check_invariants()
3438 finally:
3439 x.teardown()
3440 And if this ever produces an error it will shrink it down to a
3442 small sequence of example choices demonstrating that.
3443 steps()
3445 Return a SearchStrategy instance the defines the avail‐
3446 able next steps.
3447 execute_step(step)
3449 Execute a step that has been previously drawn from
3450 self.steps()
3451 teardown()
3453 Called after a run has finished executing to clean up any
3454 necessary state.
3455 Does nothing by default.
3457 check_invariants()
3459 Called after initializing and after executing each step.
3460 For example, here we use stateful testing as a sort of link checker, to
3462 test hypothesis.works for broken links or links that use HTTP instead
3463 of HTTPS.
3464 from hypothesis.stateful import GenericStateMachine
3466 import hypothesis.strategies as st
3467 from requests_html import HTMLSession
3468 class LinkChecker(GenericStateMachine):
3471 def __init__(self):
3472 super(LinkChecker, self).__init__()
3473 self.session = HTMLSession()
3474 self.result = None
3475 def steps(self):
3477 if self.result is None:
3478 # Always start on the home page
3479 return st.just("https://hypothesis.works/")
3480 else:
3481 return st.sampled_from([
3482 l
3483 for l in self.result.html.absolute_links
3484 # Don't try to crawl to other people's sites
3485 if l.startswith("https://hypothesis.works") and
3486 # Avoid Cloudflare's bot protection. We are a bot but we don't
3487 # care about the info it's hiding.
3488 '/cdn-cgi/' not in l
3489 ])
3490 def execute_step(self, step):
3492 self.result = self.session.get(step)
3493 assert self.result.status_code == 200
3495 for l in self.result.html.absolute_links:
3497 # All links should be HTTPS
3498 assert "https://hypothesis.works/" not in l
3499 TestLinks = LinkChecker.TestCase
3502 Running this (at the time of writing this documentation) produced the
3504 following output:
3505 AssertionError: assert 'https://hypothesis.works/' not in 'http://hypoth...test-fixtures/'
3507 'https://hypothesis.works/' is contained here:
3508 https://hypothesis.works/articles/hypothesis-pytest-fixtures/
3509 ? +++++++++++++++++++++++++
3510 ------------ Hypothesis ------------
3512 Step #1: 'https://hypothesis.works/'
3514 Step #2: 'https://hypothesis.works/articles/'
3515 More fine grained control
3517 If you want to bypass the TestCase infrastructure you can invoke these
3518 manually. The stateful module exposes the function
3519 run_state_machine_as_test, which takes an arbitrary function returning
3520 a GenericStateMachine and an optional settings parameter and does the
3521 same as the class based runTest provided.
3522 This is not recommended as it bypasses some important internal func‐
3524 tions, including reporting of statistics such as runtimes and event()
3525 calls. It was originally added to support custom __init__ methods, but
3526 you can now use initialize() rules instead.
3527
3529 Hypothesis does its level best to be compatible with everything you
3530 could possibly need it to be compatible with. Generally you should just
3531 try it and expect it to work. If it doesn't, you can be surprised and
3532 check this document for the details.
3533 Python versions
3535 Hypothesis is supported and tested on CPython 2.7 and CPython 3.5+,
3536 i.e. all versisions of CPython with upstream support,
3537 Hypothesis also supports the latest PyPy for both Python 2 (until 2020)
3539 and Python 3. Hypothesis does not currently work on Jython, though it
3540 probably could (issue #174). IronPython might work but hasn't been
3541 tested. 32-bit and narrow builds should work, though this is currently
3542 only tested on Windows.
3543 In general Hypothesis does not officially support anything except the
3545 latest patch release of any version of Python it supports. Earlier
3546 releases should work and bugs in them will get fixed if reported, but
3547 they're not tested in CI and no guarantees are made.
3548 Operating systems
3550 In theory Hypothesis should work anywhere that Python does. In practice
3551 it is only known to work and regularly tested on OS X, Windows and
3552 Linux, and you may experience issues running it elsewhere.
3553 If you're using something else and it doesn't work, do get in touch and
3555 I'll try to help, but unless you can come up with a way for me to run a
3556 CI server on that operating system it probably won't stay fixed due to
3557 the inevitable march of time.
3558 Testing frameworks
3560 In general Hypothesis goes to quite a lot of effort to generate things
3561 that look like normal Python test functions that behave as closely to
3562 the originals as possible, so it should work sensibly out of the box
3563 with every test framework.
3564 If your testing relies on doing something other than calling a function
3566 and seeing if it raises an exception then it probably won't work out of
3567 the box. In particular things like tests which return generators and
3568 expect you to do something with them (e.g. nose's yield based tests)
3569 will not work. Use a decorator or similar to wrap the test to take this
3570 form, or ask the framework maintainer to support our hooks for insert‐
3571 ing such a wrapper later.
3572 In terms of what's actually known to work:
3574 · Hypothesis integrates as smoothly with pytest and unittest as we
3576 can make it, and this is verified as part of the CI.
3577 · pytest fixtures work in the usual way for tests that have been
3579 decorated with @given - just avoid passing a strategy for each
3580 argument that will be supplied by a fixture. However, each fix‐
3581 ture will run once for the whole function, not once per example.
3582 Decorating a fixture function with @given is meaningless.
3583 · The python:unittest.mock.patch() decorator works with @given, but
3585 we recommend using it as a context manager within the decorated
3586 test to ensure that the mock is per-test-case and avoid poor
3587 interactions with Pytest fixtures.
3588 · Nose works fine with Hypothesis, and this is tested as part of the
3590 CI. yield based tests simply won't work.
3591 · Integration with Django's testing requires use of the hypothe‐
3593 sis-django package. The issue is that in Django's tests' normal
3594 mode of execution it will reset the database once per test rather
3595 than once per example, which is not what you want.
3596 · Coverage works out of the box with Hypothesis; our own test suite
3598 has 100% branch coverage.
3599 Optional Packages
3601 The supported versions of optional packages, for strategies in hypothe‐
3602 sis.extra, are listed in the documentation for that extra. Our general
3603 goal is to support all versions that are supported upstream.
3604 Regularly verifying this
3606 Everything mentioned above as explicitly supported is checked on every
3607 commit with Travis, Appveyor, and CircleCI. Our continous delivery
3608 pipeline runs all of these checks before publishing each release, so
3609 when we say they're supported we really mean it.
3610 Hypothesis versions
3612 Backwards compatibility is better than backporting fixes, so we use
3613 semantic versioning and only support the most recent version of Hypoth‐
3614 esis. See support for more information.
3615
3617 This is a collection of examples of how to use Hypothesis in interest‐
3618 ing ways. It's small for now but will grow over time.
3619 All of these examples are designed to be run under pytest, and nose
3621 should work too.
3622 How not to sort by a partial order
3624 The following is an example that's been extracted and simplified from a
3625 real bug that occurred in an earlier version of Hypothesis. The real
3626 bug was a lot harder to find.
3627 Suppose we've got the following type:
3629 class Node(object):
3631 def __init__(self, label, value):
3632 self.label = label
3633 self.value = tuple(value)
3634 def __repr__(self):
3636 return "Node(%r, %r)" % (self.label, self.value)
3637 def sorts_before(self, other):
3639 if len(self.value) >= len(other.value):
3640 return False
3641 return other.value[:len(self.value)] == self.value
3642 Each node is a label and a sequence of some data, and we have the rela‐
3644 tionship sorts_before meaning the data of the left is an initial seg‐
3645 ment of the right. So e.g. a node with value [1, 2] will sort before a
3646 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
3647 before the other.
3648 We have a list of nodes, and we want to topologically sort them with
3650 respect to this ordering. That is, we want to arrange the list so that
3651 if x.sorts_before(y) then x appears earlier in the list than y. We
3652 naively think that the easiest way to do this is to extend the partial
3653 order defined here to a total order by breaking ties arbitrarily and
3654 then using a normal sorting algorithm. So we define the following code:
3655 from functools import total_ordering
3657 @total_ordering
3660 class TopoKey(object):
3661 def __init__(self, node):
3662 self.value = node
3663 def __lt__(self, other):
3665 if self.value.sorts_before(other.value):
3666 return True
3667 if other.value.sorts_before(self.value):
3668 return False
3669 return self.value.label < other.value.label
3671 def sort_nodes(xs):
3674 xs.sort(key=TopoKey)
3675 This takes the order defined by sorts_before and extends it by breaking
3677 ties by comparing the node labels.
3678 But now we want to test that it works.
3680 First we write a function to verify that our desired outcome holds:
3682 def is_prefix_sorted(xs):
3684 for i in range(len(xs)):
3685 for j in range(i+1, len(xs)):
3686 if xs[j].sorts_before(xs[i]):
3687 return False
3688 return True
3689 This will return false if it ever finds a pair in the wrong order and
3691 return true otherwise.
3692 Given this function, what we want to do with Hypothesis is assert that
3694 for all sequences of nodes, the result of calling sort_nodes on it is
3695 sorted.
3696 First we need to define a strategy for Node:
3698 from hypothesis import settings, strategies
3700 import hypothesis.strategies as s
3701 NodeStrategy = s.builds(
3703 Node,
3704 s.integers(),
3705 s.lists(s.booleans(), max_size=10))
3706 We want to generate short lists of values so that there's a decent
3708 chance of one being a prefix of the other (this is also why the choice
3709 of bool as the elements). We then define a strategy which builds a node
3710 out of an integer and one of those short lists of booleans.
3711 We can now write a test:
3713 from hypothesis import given
3715 @given(s.lists(NodeStrategy))
3717 def test_sorting_nodes_is_prefix_sorted(xs):
3718 sort_nodes(xs)
3719 assert is_prefix_sorted(xs)
3720 this immediately fails with the following example:
3722 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
3724 The reason for this is that because False is not a prefix of (True,
3726 True) nor vice versa, sorting things the first two nodes are equal
3727 because they have equal labels. This makes the whole order non-transi‐
3728 tive and produces basically nonsense results.
3729 But this is pretty unsatisfying. It only works because they have the
3731 same label. Perhaps we actually wanted our labels to be unique. Lets
3732 change the test to do that.
3733 def deduplicate_nodes_by_label(nodes):
3735 table = {node.label: node for node in nodes}
3736 return list(table.values())
3737 We define a function to deduplicate nodes by labels, and can now map
3739 that over a strategy for lists of nodes to give us a strategy for lists
3740 of nodes with unique labels:
3741 @given(s.lists(NodeStrategy).map(deduplicate_nodes_by_label))
3743 def test_sorting_nodes_is_prefix_sorted(xs):
3744 sort_nodes(xs)
3745 assert is_prefix_sorted(xs)
3746 Hypothesis quickly gives us an example of this still being wrong:
3748 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))])
3750 Now this is a more interesting example. None of the nodes will sort
3752 equal. What is happening here is that the first node is strictly less
3753 than the last node because (False,) is a prefix of (False, False). This
3754 is in turn strictly less than the middle node because neither is a pre‐
3755 fix of the other and -2 < -1. The middle node is then less than the
3756 first node because -1 < 0.
3757 So, convinced that our implementation is broken, we write a better one:
3759 def sort_nodes(xs):
3761 for i in hrange(1, len(xs)):
3762 j = i - 1
3763 while j >= 0:
3764 if xs[j].sorts_before(xs[j+1]):
3765 break
3766 xs[j], xs[j+1] = xs[j+1], xs[j]
3767 j -= 1
3768 This is just insertion sort slightly modified - we swap a node back‐
3770 wards until swapping it further would violate the order constraints.
3771 The reason this works is because our order is a partial order already
3772 (this wouldn't produce a valid result for a general topological sorting
3773 - you need the transitivity).
3774 We now run our test again and it passes, telling us that this time
3776 we've successfully managed to sort some nodes without getting it com‐
3777 pletely wrong. Go us.
3778 Time zone arithmetic
3780 This is an example of some tests for pytz which check that various
3781 timezone conversions behave as you would expect them to. These tests
3782 should all pass, and are mostly a demonstration of some useful sorts of
3783 thing to test with Hypothesis, and how the datetimes() strategy works.
3784 >>> from datetime import timedelta
3786 >>> from hypothesis.extra.pytz import timezones
3787 >>> from hypothesis.strategies import datetimes
3788 >>> # The datetimes strategy is naive by default, so tell it to use timezones
3790 >>> aware_datetimes = datetimes(timezones=timezones())
3791 >>> @given(aware_datetimes, timezones(), timezones())
3793 ... def test_convert_via_intermediary(dt, tz1, tz2):
3794 ... """Test that converting between timezones is not affected
3795 ... by a detour via another timezone.
3796 ... """
3797 ... assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
3798 >>> @given(aware_datetimes, timezones())
3800 ... def test_convert_to_and_fro(dt, tz2):
3801 ... """If we convert to a new timezone and back to the old one
3802 ... this should leave the result unchanged.
3803 ... """
3804 ... tz1 = dt.tzinfo
3805 ... assert dt == dt.astimezone(tz2).astimezone(tz1)
3806 >>> @given(aware_datetimes, timezones())
3808 ... def test_adding_an_hour_commutes(dt, tz):
3809 ... """When converting between timezones it shouldn't matter
3810 ... if we add an hour here or add an hour there.
3811 ... """
3812 ... an_hour = timedelta(hours=1)
3813 ... assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
3814 >>> @given(aware_datetimes, timezones())
3816 ... def test_adding_a_day_commutes(dt, tz):
3817 ... """When converting between timezones it shouldn't matter
3818 ... if we add a day here or add a day there.
3819 ... """
3820 ... a_day = timedelta(days=1)
3821 ... assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
3822 >>> # And we can check that our tests pass
3824 >>> test_convert_via_intermediary()
3825 >>> test_convert_to_and_fro()
3826 >>> test_adding_an_hour_commutes()
3827 >>> test_adding_a_day_commutes()
3828 Condorcet's Paradox
3830 A classic paradox in voting theory, called Condorcet's paradox, is that
3831 majority preferences are not transitive. That is, there is a population
3832 and a set of three candidates A, B and C such that the majority of the
3833 population prefer A to B, B to C and C to A.
3834 Wouldn't it be neat if we could use Hypothesis to provide an example of
3836 this?
3837 Well as you can probably guess from the presence of this section, we
3839 can! The main trick is to decide how we want to represent the result
3840 of an election - for this example, we'll use a list of "votes", where
3841 each vote is a list of candidates in the voters preferred order. With‐
3842 out further ado, here is the code:
3843 from hypothesis import given, assume
3845 from hypothesis.strategies import lists, permutations
3846 from collections import Counter
3847 # We need at least three candidates and at least three voters to have a
3849 # paradox; anything less can only lead to victories or at worst ties.
3850 @given(lists(permutations(['A', 'B', 'C']), min_size=3))
3851 def test_elections_are_transitive(election):
3852 all_candidates = {"A", "B", "C"}
3853 # First calculate the pairwise counts of how many prefer each candidate
3855 # to the other
3856 counts = Counter()
3857 for vote in election:
3858 for i in range(len(vote)):
3859 for j in range(i+1, len(vote)):
3860 counts[(vote[i], vote[j])] += 1
3861 # Now look at which pairs of candidates one has a majority over the
3863 # other and store that.
3864 graph = {}
3865 for i in all_candidates:
3866 for j in all_candidates:
3867 if counts[(i, j)] > counts[(j, i)]:
3868 graph.setdefault(i, set()).add(j)
3869 # Now for each triple assert that it is transitive.
3871 for x in all_candidates:
3872 for y in graph.get(x, ()):
3873 for z in graph.get(y, ()):
3874 assert x not in graph.get(z, ())
3875 The example Hypothesis gives me on my first run (your mileage may of
3877 course vary) is:
3878 [['A', 'B', 'C'], ['B', 'C', 'A'], ['C', 'A', 'B']]
3880 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
3882 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
3883 and 2) prefer C to A. This is in fact basically the canonical example
3884 of the voting paradox.
3885 Fuzzing an HTTP API
3887 Hypothesis's support for testing HTTP services is somewhat nascent.
3888 There are plans for some fully featured things around this, but right
3889 now they're probably quite far down the line.
3890 But you can do a lot yourself without any explicit support! Here's a
3892 script I wrote to throw arbitrary data against the API for an entirely
3893 fictitious service called Waspfinder (this is only lightly obfuscated
3894 and you can easily figure out who I'm actually talking about, but I
3895 don't want you to run this code and hammer their API without their per‐
3896 mission).
3897 All this does is use Hypothesis to generate arbitrary JSON data match‐
3899 ing the format their API asks for and check for 500 errors. More
3900 advanced tests which then use the result and go on to do other things
3901 are definitely also possible. The swagger-conformance package provides
3902 an excellent example of this!
3903 import unittest
3905 from hypothesis import given, assume, settings, strategies as st
3906 from collections import namedtuple
3907 import requests
3908 import os
3909 import random
3910 import time
3911 import math
3912 Goal = namedtuple("Goal", ("slug",))
3915 # We just pass in our API credentials via environment variables.
3918 waspfinder_token = os.getenv('WASPFINDER_TOKEN')
3919 waspfinder_user = os.getenv('WASPFINDER_USER')
3920 assert waspfinder_token is not None
3921 assert waspfinder_user is not None
3922 GoalData = st.fixed_dictionaries({
3924 'title': st.text(),
3925 'goal_type': st.sampled_from([
3926 "hustler", "biker", "gainer", "fatloser", "inboxer",
3927 "drinker", "custom"]),
3928 'goaldate': st.one_of(st.none(), st.floats()),
3929 'goalval': st.one_of(st.none(), st.floats()),
3930 'rate': st.one_of(st.none(), st.floats()),
3931 'initval': st.floats(),
3932 'panic': st.floats(),
3933 'secret': st.booleans(),
3934 'datapublic': st.booleans(),
3935 })
3936 needs2 = ['goaldate', 'goalval', 'rate']
3939 class WaspfinderTest(unittest.TestCase):
3942 @given(GoalData)
3944 def test_create_goal_dry_run(self, data):
3945 # We want slug to be unique for each run so that multiple test runs
3946 # don't interfere with each other. If for some reason some slugs trigger
3947 # an error and others don't we'll get a Flaky error, but that's OK.
3948 slug = hex(random.getrandbits(32))[2:]
3949 # Use assume to guide us through validation we know about, otherwise
3951 # we'll spend a lot of time generating boring examples.
3952 # Title must not be empty
3954 assume(data["title"])
3955 # Exactly two of these values should be not None. The other will be
3957 # inferred by the API.
3958 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
3960 for v in data.values():
3961 if isinstance(v, float):
3962 assume(not math.isnan(v))
3963 data["slug"] = slug
3964 # The API nicely supports a dry run option, which means we don't have
3966 # to worry about the user account being spammed with lots of fake goals
3967 # Otherwise we would have to make sure we cleaned up after ourselves
3968 # in this test.
3969 data["dryrun"] = True
3970 data["auth_token"] = waspfinder_token
3971 for d, v in data.items():
3972 if v is None:
3973 data[d] = "null"
3974 else:
3975 data[d] = str(v)
3976 result = requests.post(
3977 "https://waspfinder.example.com/api/v1/users/"
3978 "%s/goals.json" % (waspfinder_user,), data=data)
3979 # Lets not hammer the API too badly. This will of course make the
3981 # tests even slower than they otherwise would have been, but that's
3982 # life.
3983 time.sleep(1.0)
3984 # For the moment all we're testing is that this doesn't generate an
3986 # internal error. If we didn't use the dry run option we could have
3987 # then tried doing more with the result, but this is a good start.
3988 self.assertNotEqual(result.status_code, 500)
3989 if __name__ == '__main__':
3991 unittest.main()
3992
3994 The Hypothesis community is small for the moment but is full of excel‐
3995 lent people who can answer your questions and help you out. Please do
3996 join us. The two major places for community discussion are:
3997 · The mailing list.
3999 · An IRC channel, #hypothesis on freenode, which is more active than
4001 the mailing list.
4002 Feel free to use these to ask for help, provide feedback, or discuss
4004 anything remotely Hypothesis related at all. If you post a question on
4005 StackOverflow, please use the python-hypothesis tag!
4006 Please note that the Hypothesis code of conduct applies in all Hypothe‐
4008 sis community spaces.
4009 If you would like to cite Hypothesis, please consider our sugested
4011 citation.
4012 If you like repo badges, we suggest the following badge, which you can
4014 add with reStructuredText or Markdown, respectively: [image]
4015 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
4017 :alt: Tested with Hypothesis
4018 :target: https://hypothesis.readthedocs.io
4019 [](https://hypothesis.readthedocs.io/)
4021 Finally, we have a beautiful logo which appears online, and often on
4023 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
4024 [image]
4025 As well as being beautiful, dragonflies actively hunt down bugs for a
4027 living! You can find the images and a usage guide in the brand direc‐
4028 tory on GitHub, or find us at conferences where we often have stickers
4029 and sometimes other swag.
4030
4032 What is Hypothesis for?
4033 From the perspective of a user, the purpose of Hypothesis is to make it
4035 easier for you to write better tests.
4036 From my perspective as the author, that is of course also a purpose of
4038 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
4039 mania for a moment), the larger purpose of Hypothesis is to drag the
4040 world kicking and screaming into a new and terrifying age of high qual‐
4041 ity software.
4042 Software is, as they say, eating the world. Software is also terrible.
4044 It's buggy, insecure and generally poorly thought out. This combination
4045 is clearly a recipe for disaster.
4046 And the state of software testing is even worse. Although it's fairly
4048 uncontroversial at this point that you should be testing your code, can
4049 you really say with a straight face that most projects you've worked on
4050 are adequately tested?
4051 A lot of the problem here is that it's too hard to write good tests.
4053 Your tests encode exactly the same assumptions and fallacies that you
4054 had when you wrote the code, so they miss exactly the same bugs that
4055 you missed when you wrote the code.
4056 Meanwhile, there are all sorts of tools for making testing better that
4058 are basically unused. The original Quickcheck is from 1999 and the
4059 majority of developers have not even heard of it, let alone used it.
4060 There are a bunch of half-baked implementations for most languages, but
4061 very few of them are worth using.
4062 The goal of Hypothesis is to bring advanced testing techniques to the
4064 masses, and to provide an implementation that is so high quality that
4065 it is easier to use them than it is not to use them. Where I can, I
4066 will beg, borrow and steal every good idea I can find that someone has
4067 had to make software testing better. Where I can't, I will invent new
4068 ones.
4069 Quickcheck is the start, but I also plan to integrate ideas from fuzz
4071 testing (a planned future feature is to use coverage information to
4072 drive example selection, and the example saving database is already
4073 inspired by the workflows people use for fuzz testing), and am open to
4074 and actively seeking out other suggestions and ideas.
4075 The plan is to treat the social problem of people not using these ideas
4077 as a bug to which there is a technical solution: Does property-based
4078 testing not match your workflow? That's a bug, let's fix it by figur‐
4079 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
4080 tom data for your application? That's a bug. Let's fix it by figuring
4081 out how to make it easier, or how to take something you're already
4082 using to specify your data and derive a generator from that automati‐
4083 cally. Find the explanations of these advanced ideas hopelessly obtuse
4084 and hard to follow? That's a bug. Let's provide you with an easy API
4085 that lets you test your code better without a PhD in software verifica‐
4086 tion.
4087 Grand ambitions, I know, and I expect ultimately the reality will be
4089 somewhat less grand, but so far in about three months of development,
4090 Hypothesis has become the most solid implementation of Quickcheck ever
4091 seen in a mainstream language (as long as we don't count Scala as main‐
4092 stream yet), and at the same time managed to significantly push forward
4093 the state of the art, so I think there's reason to be optimistic.
4094
4096 This is a page for listing people who are using Hypothesis and how
4097 excited they are about that. If that's you and your name is not on the
4098 list, this file is in Git and I'd love it if you sent me a pull request
4099 to fix that.
4100 Stripe
4102 At Stripe we use Hypothesis to test every piece of our machine learning
4103 model training pipeline (powered by scikit). Before we migrated, our
4104 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
4105 resentative at all of our actual very complex data. Because we needed
4106 to craft examples for each test, we took the easy way out and lived
4107 with extremely low test coverage.
4108 Hypothesis changed all that. Once we had our strategies for generating
4110 Dataframes of features it became trivial to slightly customize each
4111 strategy for new tests. Our coverage is now close to 90%.
4112 Full-stop, property-based testing is profoundly more powerful - and has
4114 caught or prevented far more bugs - than our old style of example-based
4115 testing.
4116 Kristian Glass - Director of Technology at LaterPay GmbH
4118 Hypothesis has been brilliant for expanding the coverage of our test
4119 cases, and also for making them much easier to read and understand, so
4120 we're sure we're testing the things we want in the way we want.
4121 Seth Morton
4123 When I first heard about Hypothesis, I knew I had to include it in my
4124 two open-source Python libraries, natsort and fastnumbers . Quite
4125 frankly, I was a little appalled at the number of bugs and "holes" I
4126 found in the code. I can now say with confidence that my libraries are
4127 more robust to "the wild." In addition, Hypothesis gave me the confi‐
4128 dence to expand these libraries to fully support Unicode input, which I
4129 never would have had the stomach for without such thorough testing
4130 capabilities. Thanks!
4131 Sixty North
4133 At Sixty North we use Hypothesis for testing Segpy an open source
4134 Python library for shifting data between Python data structures and SEG
4135 Y files which contain geophysical data from the seismic reflection sur‐
4136 veys used in oil and gas exploration.
4137 This is our first experience of property-based testing – as opposed to
4139 example-based testing. Not only are our tests more powerful, they are
4140 also much better explanations of what we expect of the production code.
4141 In fact, the tests are much closer to being specifications. Hypothesis
4142 has located real defects in our code which went undetected by tradi‐
4143 tional test cases, simply because Hypothesis is more relentlessly devi‐
4144 ous about test case generation than us mere humans! We found Hypothe‐
4145 sis particularly beneficial for Segpy because SEG Y is an antiquated
4146 format that uses legacy text encodings (EBCDIC) and even a legacy
4147 floating point format we implemented from scratch in Python.
4148 Hypothesis is sure to find a place in most of our future Python code‐
4150 bases and many existing ones too.
4151 mulkieran
4153 Just found out about this excellent QuickCheck for Python implementa‐
4154 tion and ran up a few tests for my bytesize package last night. Refuted
4155 a few hypotheses in the process.
4156 Looking forward to using it with a bunch of other projects as well.
4158 Adam Johnson
4160 I have written a small library to serialize dicts to MariaDB's dynamic
4161 columns binary format, mariadb-dyncol. When I first developed it, I
4162 thought I had tested it really well - there were hundreds of test
4163 cases, some of them even taken from MariaDB's test suite itself. I was
4164 ready to release.
4165 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
4167 Wow! It found bug after bug after bug. Even after a first release, I
4168 thought of a way to make the tests do more validation, which revealed a
4169 further round of bugs! Most impressively, Hypothesis found a compli‐
4170 cated off-by-one error in a condition with 4095 versus 4096 bytes of
4171 data - something that I would never have found.
4172 Long live Hypothesis! (Or at least, property-based testing).
4174 Josh Bronson
4176 Adopting Hypothesis improved bidict's test coverage and significantly
4177 increased our ability to make changes to the code with confidence that
4178 correct behavior would be preserved. Thank you, David, for the great
4179 testing tool.
4180 Cory Benfield
4182 Hypothesis is the single most powerful tool in my toolbox for working
4183 with algorithmic code, or any software that produces predictable output
4184 from a wide range of sources. When using it with Priority, Hypothesis
4185 consistently found errors in my assumptions and extremely subtle bugs
4186 that would have taken months of real-world use to locate. In some
4187 cases, Hypothesis found subtle deviations from the correct output of
4188 the algorithm that may never have been noticed at all.
4189 When it comes to validating the correctness of your tools, nothing
4191 comes close to the thoroughness and power of Hypothesis.
4192 Jon Moore
4194 One extremely satisfied user here. Hypothesis is a really solid imple‐
4195 mentation of property-based testing, adapted well to Python, and with
4196 good features such as failure-case shrinkers. I first used it on a
4197 project where we needed to verify that a vendor's Python and non-Python
4198 implementations of an algorithm matched, and it found about a dozen
4199 cases that previous example-based testing and code inspections had not.
4200 Since then I've been evangelizing for it at our firm.
4201 Russel Winder
4203 I am using Hypothesis as an integral part of my Python workshops. Test‐
4204 ing is an integral part of Python programming and whilst unittest and,
4205 better, pytest can handle example-based testing, property-based testing
4206 is increasingly far more important than example-base testing, and
4207 Hypothesis fits the bill.
4208 Wellfire Interactive
4210 We've been using Hypothesis in a variety of client projects, from test‐
4211 ing Django-related functionality to domain-specific calculations. It
4212 both speeds up and simplifies the testing process since there's so much
4213 less tedious and error-prone work to do in identifying edge cases. Test
4214 coverage is nice but test depth is even nicer, and it's much easier to
4215 get meaningful test depth using Hypothesis.
4216 Cody Kochmann
4218 Hypothesis is being used as the engine for random object generation
4219 with my open source function fuzzer battle_tested which maps all behav‐
4220 iors of a function allowing you to minimize the chance of unexpected
4221 crashes when running code in production.
4222 With how efficient Hypothesis is at generating the edge cases that
4224 cause unexpected behavior occur, battle_tested is able to map out the
4225 entire behavior of most functions in less than a few seconds.
4226 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
4228 ing it.
4229 Merchise Autrement
4231 Just minutes after our first use of hypothesis we uncovered a subtle
4232 bug in one of our most used library. Since then, we have increasingly
4233 used hypothesis to improve the quality of our testing in libraries and
4234 applications as well.
4235 Florian Kromer
4237 At Roboception GmbH I use Hypothesis to implement fully automated
4238 stateless and stateful reliability tests for the 3D sensor rc_visard
4239 and robotic software components .
4240 Thank you very much for creating the (probably) most powerful prop‐
4242 erty-based testing framework.
4243 Reposit Power
4245 With a micro-service architecture, testing between services is made
4246 easy using Hypothesis in integration testing. Ensuring everything is
4247 running smoothly is vital to help maintain a secure network of Virtual
4248 Power Plants.
4249 It allows us to find potential bugs and edge cases with relative ease
4251 and minimal overhead. As our architecture relies on services communi‐
4252 cating effectively, Hypothesis allows us to strictly test for the kind
4253 of data which moves around our services, particularly our backend
4254 Python applications.
4255 Your name goes here
4257 I know there are many more, because I keep finding out about new people
4258 I'd never even heard of using Hypothesis. If you're looking to way to
4259 give back to a tool you love, adding your name here only takes a moment
4260 and would really help a lot. As per instructions at the top, just send
4261 me a pull request and I'll add you to the list.
4262
4264 The following is a non-exhaustive list of open source projects I know
4265 are using Hypothesis. If you're aware of any others please add them to
4266 the list! The only inclusion criterion right now is that if it's a
4267 Python library then it should be available on PyPI.
4268 You can find hundreds more from the Hypothesis page at libraries.io,
4270 and over a thousand on GitHub.
4271 · aur
4273 · argon2_cffi
4275 · attrs
4277 · axelrod
4279 · bidict
4281 · binaryornot
4283 · brotlipy
4285 · chardet
4287 · cmph-cffi
4289 · cryptography
4291 · dbus-signature-pyparsing
4293 · fastnumbers
4295 · flocker
4297 · flownetpy
4299 · funsize
4301 · fusion-index
4303 · hyper-h2
4305 · into-dbus-python
4307 · justbases
4309 · justbytes
4311 · loris
4313 · mariadb-dyncol
4315 · mercurial
4317 · natsort
4319 · pretext
4321 · priority
4323 · PyCEbox
4325 · PyPy
4327 · pyrsistent
4329 · python-humble-utils
4331 · pyudev
4333 · qutebrowser
4335 · RubyMarshal
4337 · Segpy
4339 · simoa
4341 · srt
4343 · tchannel
4345 · vdirsyncer
4347 · wcag-contrast-ratio
4349 · yacluster
4351 · yturl
4353
4355 Hypothesis has been eagerly used and extended by the open source commu‐
4356 nity. This page lists extensions and applications; you can find more
4357 or newer packages by searching PyPI by keyword or filter by classifier,
4358 or search libraries.io.
4359 If there's something missing which you think should be here, let us
4361 know!
4362 NOTE:
4364 Being listed on this page does not imply that the Hypothesis main‐
4365 tainers endorse a package.
4366 External Strategies
4368 Some packages provide strategies directly:
4369 · hypothesis-fspaths - strategy to generate filesystem paths.
4371 · hypothesis-geojson - strategy to generate GeoJson.
4373 · hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
4375 · hypothesis_sqlalchemy - strategies to generate SQLAlchemy objects.
4377 · hypothesis-ros - strategies to generate messages and parameters for
4379 the Robot Operating System.
4380 · hypothesis-csv - strategy to generate CSV files.
4382 · hypothesis-networkx - strategy to generate networkx graphs.
4384 Others provide a function to infer a strategy from some other schema:
4386 · hypothesis-jsonschema - infer strategies from JSON schemas.
4388 · lollipop-hypothesis - infer strategies from lollipop schemas.
4390 · hypothesis-drf - infer strategies from a djangorestframework seri‐
4392 aliser.
4393 · hypothesis-mongoengine - infer strategies from a mongoengine model.
4395 · hypothesis-pb - infer strategies from Protocol Buffer schemas.
4397 Other Cool Things
4399 swagger-conformance is powered by Hypothesis and pyswagger. Based on a
4400 Swagger specification, it can build and run an entire test suite to
4401 check that the implementation matches the spec. The command-line ver‐
4402 sion can test apps written in any language, simply by passing the file
4403 or URL path to the schema to check!
4404 Trio is an async framework with "an obsessive focus on usability and
4406 correctness", so naturally it works with Hypothesis! pytest-trio
4407 includes a custom hook that allows @given(...) to work with Trio-style
4408 async test functions, and hypothesis-trio includes stateful testing
4409 extensions to support concurrent programs.
4410 libarchimedes makes it easy to use Hypothesis in the Hy language, a
4412 Lisp embedded in Python.
4413 battle_tested is a fuzzing tool that will show you how your code can
4415 fail - by trying all kinds of inputs and reporting whatever happens.
4416 pytest-subtesthack functions as a workaround for issue #377.
4418 Writing an Extension
4420 See CONTRIBUTING.rst for more information.
4421 New strategies can be added to Hypothesis, or published as an external
4423 package on PyPI - either is fine for most strategies. If in doubt, ask!
4424 It's generally much easier to get things working outside, because
4426 there's more freedom to experiment and fewer requirements in stability
4427 and API style. We're happy to review and help with external packages as
4428 well as pull requests!
4429 If you're thinking about writing an extension, please name it hypothe‐
4431 sis-{something} - a standard prefix makes the community more visible
4432 and searching for extensions easier. And make sure you use the Frame‐
4433 work :: Hypothesis trove classifier!
4434 On the other hand, being inside gets you access to some deeper imple‐
4436 mentation features (if you need them) and better long-term guarantees
4437 about maintenance. We particularly encourage pull requests for new
4438 composable primitives that make implementing other strategies easier,
4439 or for widely used types in the standard library. Strategies for other
4440 things are also welcome; anything with external dependencies just goes
4441 in hypothesis.extra.
4442
4444 This is a record of all past Hypothesis releases and what went into
4445 them, in reverse chronological order. All previous releases should
4446 still be available on pip.
4447 Hypothesis APIs come in three flavours:
4449 · Public: Hypothesis releases since 1.0 are semantically versioned with
4451 respect to these parts of the API. These will not break except
4452 between major version bumps. All APIs mentioned in this documentation
4453 are public unless explicitly noted otherwise.
4454 · Semi-public: These are APIs that are considered ready to use but are
4456 not wholly nailed down yet. They will not break in patch releases and
4457 will usually not break in minor releases, but when necessary minor
4458 releases may break semi-public APIs.
4459 · Internal: These may break at any time and you really should not use
4461 them at all.
4462 You should generally assume that an API is internal unless you have
4464 specific information to the contrary.
4465 4.23.8 - 2019-05-26
4467 This patch has a minor cleanup of the internal engine. There is no
4468 user-visible impact.
4469 4.23.7 - 2019-05-26
4471 This patch clarifies some error messages when the test function signa‐
4472 ture is incompatible with the arguments to @given, especially when the
4473 @settings() decorator is also used (issue #1978).
4474 4.23.6 - 2019-05-19
4476 This release adds the pyupgrade fixer to our code style, for consistent
4477 use of dict and set literals and comprehensions.
4478 4.23.5 - 2019-05-16
4480 This release slightly simplifies a small part of the core engine.
4481 There is no user-visible change.
4482 4.23.4 - 2019-05-09
4484 Fixes a minor formatting issue the docstring of from_type()
4485 4.23.3 - 2019-05-09
4487 Adds a recipe to the docstring of from_type() that describes a means
4488 for drawing values for "everything except" a specified type. This
4489 recipe is especially useful for writing tests that perform input-type
4490 validation.
4491 4.23.2 - 2019-05-08
4493 This patch uses autoflake to remove some pointless pass statements,
4494 which improves our workflow but has no user-visible impact.
4495 4.23.1 - 2019-05-08
4497 This patch fixes an OverflowError in from_type(xrange) on Python 2.
4498 It turns out that not only do the start and stop values have to fit in
4500 a C long, but so does stop - start. We now handle this even on 32bit
4501 platforms, but remind users that Python2 will not be supported after
4502 2019 without specific funding.
4503 4.23.0 - 2019-05-08
4505 This release implements the slices() strategy, to generate slices of a
4506 length-size sequence.
4507 Thanks to Daniel J. West for writing this patch at the PyCon 2019
4509 sprints!
4510 4.22.3 - 2019-05-07
4512 This patch exposes DataObject, solely to support more precise type
4513 hints. Objects of this type are provided by data(), and can be used to
4514 draw examples from strategies intermixed with your test code.
4515 4.22.2 - 2019-05-07
4517 This patch fixes the very rare issue #1798 in array_dtypes(), which
4518 caused an internal error in our tests.
4519 4.22.1 - 2019-05-07
4521 This patch fixes a rare bug in from_type(range).
4522 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
4524 Sprints.
4525 4.22.0 - 2019-05-07
4527 The unique_by argument to lists now accepts a tuple of callables such
4528 that every element of the generated list will be unique with respect to
4529 each callable in the tuple (issue #1916).
4530 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
4532 4.21.1 - 2019-05-06
4534 This patch cleans up the internals of one_of(). You may see a slight
4535 change to the distribution of examples from this strategy but there is
4536 no change to the public API.
4537 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
4539 sprints!
4540 4.21.0 - 2019-05-05
4542 The from_type() strategy now supports python:slice objects.
4543 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
4545 Mentored Sprints.
4546 4.20.0 - 2019-05-05
4548 This release improves the array_shapes() strategy, to choose an appro‐
4549 priate default for max_side based on the min_side, and max_dims based
4550 on the min_dims. An explicit error is raised for dimensions greater
4551 than 32, which are not supported by Numpy, as for other invalid combi‐
4552 nations of arguments.
4553 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
4555 tored Sprints.
4556 4.19.0 - 2019-05-05
4558 The from_type() strategy now supports python:range objects (or xrange
4559 on Python 2).
4560 Thanks to Katrina Durance for writing this feature at the PyCon 2019
4562 Mentored Sprints.
4563 4.18.3 - 2019-04-30
4565 This release fixes a very rare edge case in the test-case mutator,
4566 which could cause an internal error with certain unusual tests.
4567 4.18.2 - 2019-04-30
4569 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
4570 changed the representation of code objects to support positional-only
4571 arguments. Note however that Hypothesis does not (yet) support such
4572 functions as e.g. arguments to builds() or inputs to @given.
4573 Thanks to Paul Ganssle for identifying and fixing this bug.
4575 4.18.1 - 2019-04-29
4577 This patch improves the performance of unique collections such as
4578 sets() when the elements are drawn from a sampled_from() strategy (‐
4579 issue #1115).
4580 4.18.0 - 2019-04-24
4582 This release adds the functions() strategy, which can be used to imi‐
4583 tate your 'real' function for callbacks.
4584 4.17.2 - 2019-04-19
4586 This release refactors stateful rule selection to share the new machin‐
4587 ery with sampled_from() instead of using the original independent
4588 implementation.
4589 4.17.1 - 2019-04-16
4591 This patch allows Hypothesis to try a few more examples after finding
4592 the first bug, in hopes of reporting multiple distinct bugs. The
4593 heuristics described in issue #847 ensure that we avoid wasting time on
4594 fruitless searches, while still surfacing each bug as soon as possible.
4595 4.17.0 - 2019-04-16
4597 This release adds the strategy broadcastable_shapes(), which generates
4598 array shapes that are broadcast-compatible with a provided shape.
4599 4.16.0 - 2019-04-12
4601 This release allows register_type_strategy() to be used with
4602 python:typing.NewType instances. This may be useful to e.g. provide
4603 only positive integers for from_type(UserId) with a UserId = New‐
4604 Type('UserId', int) type.
4605 Thanks to PJCampi for suggesting and writing the patch!
4607 4.15.0 - 2019-04-09
4609 This release supports passing a timedelta as the deadline setting, so
4610 you no longer have to remember that the number is in milliseconds (‐
4611 issue #1900).
4612 Thanks to Damon Francisco for this change!
4614 4.14.7 - 2019-04-09
4616 This patch makes the type annotations on hypothesis.extra.dateutil com‐
4617 patible with mypy 0.700.
4618 4.14.6 - 2019-04-07
4620 This release fixes a bug introduced in Hypothesis 4.14.3 that would
4621 sometimes cause sampled_from(...).filter(...) to encounter an internal
4622 assertion failure when there are three or fewer elements, and every
4623 element is rejected by the filter.
4624 4.14.5 - 2019-04-05
4626 This patch takes the previous efficiency improvements to sam‐
4627 pled_from(...).filter(...) strategies that reject most elements, and
4628 generalises them to also apply to sampled_from(...).filter(...).fil‐
4629 ter(...) and longer chains of filters.
4630 4.14.4 - 2019-04-05
4632 This release fixes a bug that prevented random_module() from correctly
4633 restoring the previous state of the random module.
4634 The random state was instead being restored to a temporary determinis‐
4636 tic state, which accidentally caused subsequent tests to see the same
4637 random values across multiple test runs.
4638 4.14.3 - 2019-04-03
4640 This patch adds an internal special case to make sampled_from(...).fil‐
4641 ter(...) much more efficient when the filter rejects most elements (‐
4642 issue #1885).
4643 4.14.2 - 2019-03-31
4645 This patch improves the error message if the function f in s.flatmap(f)
4646 does not return a strategy.
4647 Thanks to Kai Chen for this change!
4649 4.14.1 - 2019-03-30
4651 This release modifies how Hypothesis selects operations to run during
4652 shrinking, by causing it to deprioritise previously useless classes of
4653 shrink until others have reached a fixed point.
4654 This avoids certain pathological cases where the shrinker gets very
4656 close to finishing and then takes a very long time to finish the last
4657 small changes because it tries many useless shrinks for each useful one
4658 towards the end. It also should cause a more modest improvement (prob‐
4659 ably no more than about 30%) in shrinking performance for most tests.
4660 4.14.0 - 2019-03-19
4662 This release blocks installation of Hypothesis on Python 3.4, which
4663 reached its end of life date on 2019-03-18.
4664 This should not be of interest to anyone but downstream maintainers -
4666 if you are affected, migrate to a secure version of Python as soon as
4667 possible or at least seek commercial support.
4668 4.13.0 - 2019-03-19
4670 This release makes it an explicit error to call floats(min_value=inf,
4671 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
4672 are no possible values that can be generated (issue #1859).
4673 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
4675 -0. and we could thus generate either if comparing by value, violating
4676 the sequence ordering of floats is a special case we don't want or
4677 need.
4678 4.12.1 - 2019-03-18
4680 This release should significantly reduce the amount of memory that
4681 Hypothesis uses for representing large test cases, by storing informa‐
4682 tion in a more compact representation and only unpacking it lazily when
4683 it is first needed.
4684 4.12.0 - 2019-03-18
4686 This update adds the report_multiple_bugs setting, which you can use to
4687 disable multi-bug reporting and only raise whichever bug had the small‐
4688 est minimal example. This is occasionally useful when using a debugger
4689 or tools that annotate tracebacks via introspection.
4690 4.11.7 - 2019-03-18
4692 This change makes a tiny improvement to the core engine's bookkeeping.
4693 There is no user-visible change.
4694 4.11.6 - 2019-03-15
4696 This release changes some of Hypothesis's internal shrinking behaviour
4697 in order to reduce memory usage and hopefully improve performance.
4698 4.11.5 - 2019-03-13
4700 This release adds a micro-optimisation to how Hypothesis handles debug
4701 reporting internally. Hard to shrink test may see a slight performance
4702 improvement, but in most common scenarios it is unlikely to be notice‐
4703 able.
4704 4.11.4 - 2019-03-13
4706 This release removes some redundant code that was no longer needed but
4707 was still running a significant amount of computation and allocation on
4708 the hot path. This should result in a modest speed improvement for
4709 most tests, especially those with large test cases.
4710 4.11.3 - 2019-03-13
4712 This release adds a micro-optimisation to how Hypothesis caches test
4713 cases. This will cause a small improvement in speed and memory usage
4714 for large test cases, but in most common scenarios it is unlikely to be
4715 noticeable.
4716 4.11.2 - 2019-03-13
4718 This release removes some internal code that populates a field that is
4719 no longer used anywhere. This should result in some modest performance
4720 and speed improvements and no other user visible effects.
4721 4.11.1 - 2019-03-13
4723 This is a formatting-only patch, enabled by a new version of isort.
4724 4.11.0 - 2019-03-12
4726 This release deprecates sampled_from() with empty sequences. This
4727 returns nothing(), which gives a clear error if used directly... but
4728 simply vanishes if combined with another strategy.
4729 Tests that silently generate less than expected are a serious problem
4731 for anyone relying on them to find bugs, and we think reliability more
4732 important than convenience in this case.
4733 4.10.0 - 2019-03-11
4735 This release improves Hypothesis's to detect flaky tests, by noticing
4736 when the behaviour of the test changes between runs. In particular
4737 this will notice many new cases where data generation depends on exter‐
4738 nal state (e.g. external sources of randomness) and flag those as flaky
4739 sooner and more reliably.
4740 The basis of this feature is a considerable reengineering of how
4742 Hypothesis stores its history of test cases, so on top of this its mem‐
4743 ory usage should be considerably reduced.
4744 4.9.0 - 2019-03-09
4746 This release adds the strategy valid_tuple_axes(), which generates
4747 tuples of axis-indices that can be passed to the axis argument in
4748 NumPy's sequential functions (e.g. numpy:numpy.sum()).
4749 Thanks to Ryan Soklaski for this strategy.
4751 4.8.0 - 2019-03-06
4753 This release significantly tightens validation in hypothesis.settings.
4754 max_examples, buffer_size, and stateful_step_count must be positive
4755 integers; deadline must be a positive number or None; and derandomize
4756 must be either True or False.
4757 As usual, this replaces existing errors with a more helpful error and
4759 starts new validation checks as deprecation warnings.
4760 4.7.19 - 2019-03-04
4762 This release makes some micro-optimisations to certain calculations
4763 performed in the shrinker. These should particularly speed up large
4764 test cases where the shrinker makes many small changes. It will also
4765 reduce the amount allocated, but most of this is garbage that would
4766 have been immediately thrown away, so you probably won't see much
4767 effect specifically from that.
4768 4.7.18 - 2019-03-03
4770 This patch removes some overhead from arrays() with a constant shape
4771 and dtype. The resulting performance improvement is modest, but worth‐
4772 wile for small arrays.
4773 4.7.17 - 2019-03-01
4775 This release makes some micro-optimisations within Hypothesis's inter‐
4776 nal representation of test cases. This should cause heavily nested
4777 test cases to allocate less during generation and shrinking, which
4778 should speed things up slightly.
4779 4.7.16 - 2019-02-28
4781 This changes the order in which Hypothesis runs certain operations dur‐
4782 ing shrinking. This should significantly decrease memory usage and
4783 speed up shrinking of large examples.
4784 4.7.15 - 2019-02-28
4786 This release allows Hypothesis to calculate a number of attributes of
4787 generated test cases lazily. This should significantly reduce memory
4788 usage and modestly improve performance, especially for large test
4789 cases.
4790 4.7.14 - 2019-02-28
4792 This release reduces the number of operations the shrinker will try
4793 when reordering parts of a test case. This should in some circum‐
4794 stances significantly speed up shrinking. It may result in different
4795 final test cases, and if so usually slightly worse ones, but it should
4796 not generally have much impact on the end result as the operations
4797 removed were typically useless.
4798 4.7.13 - 2019-02-27
4800 This release changes how Hypothesis reorders examples within a test
4801 case during shrinking. This should make shrinking considerably faster.
4802 4.7.12 - 2019-02-27
4804 This release slightly improves the shrinker's ability to replace parts
4805 of a test case with their minimal version, by allowing it to do so in
4806 bulk rather than one at a time. Where this is effective, shrinker per‐
4807 formance should be modestly improved.
4808 4.7.11 - 2019-02-25
4810 This release makes some micro-optimisations to common operations per‐
4811 formed during shrinking. Shrinking should now be slightly faster,
4812 especially for large examples with relatively fast test functions.
4813 4.7.10 - 2019-02-25
4815 This release is a purely internal refactoring of Hypothesis's API for
4816 representing test cases. There should be no user visible effect.
4817 4.7.9 - 2019-02-24
4819 This release changes certain shrink passes to make them more efficient
4820 when they aren't making progress.
4821 4.7.8 - 2019-02-23
4823 This patch removes some unused code, which makes the internals a bit
4824 easier to understand. There is no user-visible impact.
4825 4.7.7 - 2019-02-23
4827 This release reduces the number of operations the shrinker will try
4828 when reordering parts of a test case. This should in some circum‐
4829 stances significantly speed up shrinking. It may result in different
4830 final test cases, and if so usually slightly worse ones, but it should
4831 not generally have much impact on the end result as the operations
4832 removed were typically useless.
4833 4.7.6 - 2019-02-23
4835 This patch removes some unused code from the shrinker. There is no
4836 user-visible change.
4837 4.7.5 - 2019-02-23
4839 This release changes certain shrink passes to make them adaptive - that
4840 is, in cases where they are successfully making progress they may now
4841 do so significantly faster.
4842 4.7.4 - 2019-02-22
4844 This is a docs-only patch, noting that because the lark-parser is under
4845 active development at version 0.x, hypothesis[lark] APIs may break in
4846 minor releases if necessary to keep up with the upstream package.
4847 4.7.3 - 2019-02-22
4849 This changes Hypothesis to no longer import various test frameworks by
4850 default (if they are installed). which will speed up the initial
4851 import hypothesis call.
4852 4.7.2 - 2019-02-22
4854 This release changes Hypothesis's internal representation of a test
4855 case to calculate some expensive structural information on demand
4856 rather than eagerly. This should reduce memory usage a fair bit, and
4857 may make generation somewhat faster.
4858 4.7.1 - 2019-02-21
4860 This release refactors the internal representation of previously run
4861 test cases. The main thing you should see as a result is that Hypothe‐
4862 sis becomes somewhat less memory hungry.
4863 4.7.0 - 2019-02-21
4865 This patch allows array_shapes() to generate shapes with side-length or
4866 even dimension zero, though the minimum still defaults to one. These
4867 shapes are rare and have some odd behavior, but are particularly impor‐
4868 tant to test for just that reason!
4869 In a related bigfix, arrays() now supports generating zero-dimensional
4871 arrays with dtype=object and a strategy for iterable elements. Previ‐
4872 ously, the array element would incorrectly be set to the first item in
4873 the generated iterable.
4874 Thanks to Ryan Turner for continuing to improve our Numpy support.
4876 4.6.1 - 2019-02-19
4878 This release is a trivial micro-optimisation inside Hypothesis which
4879 should result in it using significantly less memory.
4880 4.6.0 - 2019-02-18
4882 This release changes some inconsistent behavior of arrays() from the
4883 Numpy extra when asked for an array of shape=(). arrays() will now
4884 always return a Numpy ndarray, and the array will always be of the
4885 requested dtype.
4886 Thanks to Ryan Turner for this change.
4888 4.5.12 - 2019-02-18
4890 This release fixes a minor typo in an internal comment. There is no
4891 user-visible change.
4892 4.5.11 - 2019-02-15
4894 This release fixes issue #1813, a bug introduced in 3.59.1, which
4895 caused random_module() to no longer affect the body of the test:
4896 Although Hypothesis would claim to be seeding the random module in fact
4897 tests would always run with a seed of zero.
4898 4.5.10 - 2019-02-14
4900 This patch fixes an off-by-one error in the maximum length of emails().
4901 Thanks to Krzysztof Jurewicz for pull request #1812.
4902 4.5.9 - 2019-02-14
4904 This patch removes some unused code from the shrinker. There is no
4905 user-visible change.
4906 4.5.8 - 2019-02-12
4908 This release fixes an internal IndexError in Hypothesis that could
4909 sometimes be triggered during shrinking.
4910 4.5.7 - 2019-02-11
4912 This release modifies the shrinker to interleave different types of
4913 reduction operations, e.g. switching between deleting data and lowering
4914 scalar values rather than trying entirely deletions then entirely low‐
4915 ering.
4916 This may slow things down somewhat in the typical case, but has the
4918 major advantage that many previously difficult to shrink examples
4919 should become much faster, because the shrinker will no longer tend to
4920 stall when trying some ineffective changes to the shrink target but
4921 will instead interleave it with other more effective operations.
4922 4.5.6 - 2019-02-11
4924 This release makes a number of internal changes to the implementation
4925 of hypothesis.extra.lark.from_lark(). These are primarily intended as
4926 a refactoring, but you may see some minor improvements to performance
4927 when generating large strings, and possibly to shrink quality.
4928 4.5.5 - 2019-02-10
4930 This patch prints an explanatory note when issue #1798 is triggered,
4931 because the error message from Numpy is too terse to locate the prob‐
4932 lem.
4933 4.5.4 - 2019-02-08
4935 In Python 2, long integers are not allowed in the shape argument to
4936 arrays(). Thanks to Ryan Turner for fixing this.
4937 4.5.3 - 2019-02-08
4939 This release makes a small internal refactoring to clarify how Hypothe‐
4940 sis instructs tests to stop running when appropriate. There is no
4941 user-visible change.
4942 4.5.2 - 2019-02-06
4944 This release standardises all of the shrinker's internal operations on
4945 running in a random order.
4946 The main effect you will see from this that it should now be much less
4948 common for the shrinker to stall for a long time before making further
4949 progress. In some cases this will correspond to shrinking more slowly,
4950 but on average it should result in faster shrinking.
4951 4.5.1 - 2019-02-05
4953 This patch updates some docstrings, but has no runtime changes.
4954 4.5.0 - 2019-02-03
4956 This release adds exclude_min and exclude_max arguments to floats(), so
4957 that you can easily generate values from open or half-open intervals (‐
4958 issue #1622).
4959 4.4.6 - 2019-02-03
4961 This patch fixes a bug where from_regex() could throw an internal error
4962 if the python:re.IGNORECASE flag was used (issue #1786).
4963 4.4.5 - 2019-02-02
4965 This release removes two shrink passes that Hypothesis runs late in the
4966 process. These were very expensive when the test function was slow and
4967 often didn't do anything useful.
4968 Shrinking should get faster for most failing tests. If you see any
4970 regression in example quality as a result of this release, please let
4971 us know.
4972 4.4.4 - 2019-02-02
4974 This release modifies the way that Hypothesis deletes data during
4975 shrinking. It will primarily be noticeable for very large examples,
4976 which should now shrink faster.
4977 The shrinker is now also able to perform some deletions that it could
4979 not previously, but this is unlikely to be very noticeable.
4980 4.4.3 - 2019-01-25
4982 This release fixes an open file leak that used to cause ResourceWarn‐
4983 ings.
4984 4.4.2 - 2019-01-24
4986 This release changes Hypothesis's internal approach to caching the
4987 results of executing test cases. The result should be that it is now
4988 significantly less memory hungry, especially when shrinking large test
4989 cases.
4990 Some tests may get slower or faster depending on whether the new or old
4992 caching strategy was well suited to them, but any change in speed in
4993 either direction should be minor.
4994 4.4.1 - 2019-01-24
4996 This patch tightens up some of our internal heuristics to deal with
4997 shrinking floating point numbers, which will now run in fewer circum‐
4998 stances.
4999 You are fairly unlikely to see much difference from this, but if you do
5001 you are likely to see shrinking become slightly faster and/or producing
5002 slightly worse results.
5003 4.4.0 - 2019-01-24
5005 This release adds the from_form() function, which allows automatic
5006 testing against Django forms. (issue #35)
5007 Thanks to Paul Stiverson for this feature, which resolves our oldest
5009 open issue!
5010 4.3.0 - 2019-01-24
5012 This release deprecates HealthCheck.hung_test and disables the associ‐
5013 ated runtime check for tests that ran for more than five minutes. Such
5014 a check is redundant now that we enforce the deadline and max_examples
5015 setting, which can be adjusted independently.
5016 4.2.0 - 2019-01-23
5018 This release adds a new module, hypothesis.extra.lark, which you can
5019 use to generate strings matching a context-free grammar.
5020 In this initial version, only lark-parser EBNF grammars are supported,
5022 by the new hypothesis.extra.lark.from_lark() function.
5023 4.1.2 - 2019-01-23
5025 This patch fixes a very rare overflow bug (issue #1748) which could
5026 raise an InvalidArgument error in complex_numbers() even though the
5027 arguments were valid.
5028 4.1.1 - 2019-01-23
5030 This release makes some improvements to internal code organisation and
5031 documentation and has no impact on behaviour.
5032 4.1.0 - 2019-01-22
5034 This release adds register_random(), which registers random.Random
5035 instances or compatible objects to be seeded and reset by Hypothesis to
5036 ensure that test cases are deterministic.
5037 We still recommend explicitly passing a random.Random instance from
5039 randoms() if possible, but registering a framework-global state for
5040 Hypothesis to manage is better than flaky tests!
5041 4.0.2 - 2019-01-22
5043 This patch fixes issue #1387, where bounded integers() with a very
5044 large range would almost always generate very large numbers. Now, we
5045 usually use the same tuned distribution as unbounded integers().
5046 4.0.1 - 2019-01-16
5048 This release randomizes the order in which the shrinker tries some of
5049 its initial normalization operations. You are unlikely to see much
5050 difference as a result unless your generated examples are very large.
5051 In this case you may see some performance improvements in shrinking.
5052 4.0.0 - 2019-01-14
5054 Welcome to the next major version of Hypothesis!
5055 There are no new features here, as we release those in minor versions.
5057 Instead, 4.0 is a chance for us to remove deprecated features (many
5058 already converted into no-ops), and turn a variety of warnings into
5059 errors.
5060 If you were running on the last version of Hypothesis 3.x without any
5062 Hypothesis deprecation warnings (or using private APIs), this will be a
5063 very boring upgrade. In fact, nothing will change for you at all. Per
5064 our deprecation policy, warnings added in the last six months (after
5065 2018-07-05) have not been converted to errors.
5066 Removals
5068 · hypothesis.extra.datetime has been removed, replaced by the core date
5069 and time strategies.
5070 · hypothesis.extra.fakefactory has been removed, replaced by general
5072 expansion of Hypothesis' strategies and the third-party ecosystem.
5073 · The SQLite example database backend has been removed.
5075 Settings
5077 · The deadline is now enforced by default, rather than just emitting a
5078 warning when the default (200 milliseconds per test case) deadline is
5079 exceeded.
5080 · The database_file setting has been removed; use database.
5082 · The perform_health_check setting has been removed; use sup‐
5084 press_health_check.
5085 · The max_shrinks setting has been removed; use phases to disable
5087 shrinking.
5088 · The min_satisfying_examples, max_iterations, strict, timeout, and
5090 use_coverage settings have been removed without user-configurable
5091 replacements.
5092 Strategies
5094 · The elements argument is now required for collection strategies.
5095 · The average_size argument was a no-op and has been removed.
5097 · Date and time strategies now only accept min_value and max_value for
5099 bounds.
5100 · builds() now requires that the thing to build is passed as the first
5102 positional argument.
5103 · Alphabet validation for text() raises errors, not warnings, as does
5105 category validation for characters().
5106 · The choices() strategy has been removed. Instead, you can use data()
5108 with sampled_from(), so choice(elements) becomes data.draw(sam‐
5109 pled_from(elements)).
5110 · The streaming() strategy has been removed. Instead, you can use
5112 data() and replace iterating over the stream with data.draw() calls.
5113 · sampled_from() and permutations() raise errors instead of warnings if
5115 passed a collection that is not a sequence.
5116 Miscellaneous
5118 · Applying @given to a test function multiple times was really ineffi‐
5119 cient, and now it's also an error.
5120 · Using the .example() method of a strategy (intended for interactive
5122 exploration) within another strategy or a test function always weak‐
5123 ened data generation and broke shrinking, and now it's an error too.
5124 · The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
5126 ported, as the database_file setting has been removed.
5127 · The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
5129 ported. You can use the --hypothesis-verbosity pytest argument
5130 instead, or write your own setup code using the settings profile sys‐
5131 tem to replace it.
5132 · Using @seed or derandomize=True now forces database=None to ensure
5134 results are in fact reproducible. If database is not None, doing so
5135 also emits a HypothesisWarning.
5136 · Unused exception types have been removed from hypothesis.errors;
5138 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
5139 ple, Timeout, and WrongFormat.
5140 3.88.3 - 2019-01-11
5142 This changes the order that the shrinker tries certain operations in
5143 its "emergency" phase which runs late in the process. The new order
5144 should be better at avoiding long stalls where the shrinker is failing
5145 to make progress, which may be helpful if you have difficult to shrink
5146 test cases. However this will not be noticeable in the vast majority
5147 of use cases.
5148 3.88.2 - 2019-01-11
5150 This is a pure refactoring release that extracts some logic from the
5151 core Hypothesis engine into its own class and file. It should have no
5152 user visible impact.
5153 3.88.1 - 2019-01-11
5155 This patch fixes some markup in our documentation.
5156 3.88.0 - 2019-01-10
5158 Introduces hypothesis.stateful.multiple(), which allows rules in rule
5159 based state machines to send multiple results at once to their target
5160 Bundle, or none at all.
5161 3.87.0 - 2019-01-10
5163 This release contains a massive cleanup of the Hypothesis for Django
5164 extra:
5165 · hypothesis.extra.django.models.models() is deprecated in favor of
5167 hypothesis.extra.django.from_model().
5168 · hypothesis.extra.django.models.add_default_field_mapping() is depre‐
5170 cated in favor of hypothesis.extra.django.register_field_strategy().
5171 · from_model() does not infer a strategy for nullable fields or fields
5173 with a default unless passed infer, like builds(). models.models()
5174 would usually but not always infer, and a special default_value
5175 marker object was required to disable inference.
5176 3.86.9 - 2019-01-09
5178 This release improves some internal logic about when a test case in
5179 Hypothesis's internal representation could lead to a valid test case.
5180 In some circumstances this can lead to a significant speed up during
5181 shrinking. It may have some minor negative impact on the quality of
5182 the final result due to certain shrink passes now having access to less
5183 information about test cases in some circumstances, but this should
5184 rarely matter.
5185 3.86.8 - 2019-01-09
5187 This release has no user visible changes but updates our URLs to use
5188 HTTPS.
5189 3.86.7 - 2019-01-08
5191 Hypothesis can now automatically generate values for Django models with
5192 a URLfield, thanks to a new provisional strategy for URLs (issue
5193 #1388).
5194 3.86.6 - 2019-01-07
5196 This release is a pure refactoring that extracts some internal code
5197 into its own file. It should have no user visible effect.
5198 3.86.5 - 2019-01-06
5200 This is a docs-only patch, which fixes some typos and removes a few
5201 hyperlinks for deprecated features.
5202 3.86.4 - 2019-01-04
5204 This release changes the order in which the shrinker tries to delete
5205 data. For large and slow tests this may significantly improve the per‐
5206 formance of shrinking.
5207 3.86.3 - 2019-01-04
5209 This release fixes a bug where certain places Hypothesis internal
5210 errors could be raised during shrinking when a user exception occurred
5211 that suppressed an exception Hypothesis uses internally in its genera‐
5212 tion.
5213 The two known ways to trigger this problem were:
5215 · Errors raised in stateful tests' teardown function.
5217 · Errors raised in finally blocks that wrapped a call to data.draw.
5219 These cases will now be handled correctly.
5221 3.86.2 - 2019-01-04
5223 This patch is a docs-only change to fix a broken hyperlink.
5224 3.86.1 - 2019-01-04
5226 This patch fixes issue #1732, where integers() would always return long
5227 values on Python 2.
5228 3.86.0 - 2019-01-03
5230 This release ensures that infinite numbers are never generated by
5231 floats() with allow_infinity=False, which could previously happen in
5232 some cases where one bound was also provided.
5233 The trivially inconsistent min_value=inf, allow_infinity=False now
5235 raises an InvalidArgumentError, as does the inverse with max_value.
5236 You can still use just(inf) to generate inf without violating other
5237 constraints.
5238 3.85.3 - 2019-01-02
5240 Happy new year everyone! This release has no user visible changes but
5241 updates our copyright headers to include 2019.
5242 3.85.2 - 2018-12-31
5244 This release makes a small change to the way the shrinker works. You
5245 may see some improvements to speed of shrinking on especially large and
5246 hard to shrink examples, but most users are unlikely to see much dif‐
5247 ference.
5248 3.85.1 - 2018-12-30
5250 This patch fixes issue #1700, where a line that contained a Unicode
5251 character before a lambda definition would cause an internal exception.
5252 3.85.0 - 2018-12-29
5254 Introduces the hypothesis.stateful.consumes() function. When defining a
5255 rule in stateful testing, it can be used to mark bundles from which
5256 values should be consumed, i. e. removed after use in the rule. This
5257 has been proposed in issue #136.
5258 Thanks to Jochen Müller for this long-awaited feature.
5260 3.84.6 - 2018-12-28
5262 This patch makes a small internal change to fix an issue in Hypothe‐
5263 sis's own coverage tests (issue #1718).
5264 There is no user-visible change.
5266 3.84.5 - 2018-12-21
5268 This patch refactors the hypothesis.strategies module, so that private
5269 names should no longer appear in tab-completion lists. We previously
5270 relied on __all__ for this, but not all editors respect it.
5271 3.84.4 - 2018-12-21
5273 This is a follow-up patch to ensure that the deprecation date is auto‐
5274 matically recorded for any new deprecations. There is no user-visible
5275 effect.
5276 3.84.3 - 2018-12-20
5278 This patch updates the Hypothesis pytest plugin to avoid a recently
5279 deprecated hook interface. There is no user-visible change.
5280 3.84.2 - 2018-12-19
5282 This patch fixes the internals for integers() with one bound. Values
5283 from this strategy now always shrink towards zero instead of towards
5284 the bound, and should shrink much more efficiently too. On Python 2,
5285 providing a bound incorrectly excluded long integers, which can now be
5286 generated.
5287 3.84.1 - 2018-12-18
5289 This patch adds information about when features were deprecated, but
5290 this is only recorded internally and has no user-visible effect.
5291 3.84.0 - 2018-12-18
5293 This release changes the stateful testing backend from find() to use
5294 @given (issue #1300). This doesn't change how you create stateful
5295 tests, but does make them run more like other Hypothesis tests.
5296 @reproduce_failure and @seed now work for stateful tests.
5298 Stateful tests now respect the deadline and suppress_health_check set‐
5300 tings, though they are disabled by default. You can enable them by
5301 using @settings(...) as a class decorator with whatever arguments you
5302 prefer.
5303 3.83.2 - 2018-12-17
5305 Hypothesis has adopted Black as our code formatter (issue #1686).
5306 There are no functional changes to the source, but it's prettier!
5307 3.83.1 - 2018-12-13
5309 This patch increases the variety of examples generated by from_type().
5310 3.83.0 - 2018-12-12
5312 Our pytest plugin now warns you when strategy functions have been col‐
5313 lected as tests, which may happen when e.g. using the @composite deco‐
5314 rator when you should be using @given(st.data()) for inline draws.
5315 Such functions always pass when treated as tests, because the lazy cre‐
5316 ation of strategies mean that the function body is never actually exe‐
5317 cuted!
5318 3.82.6 - 2018-12-11
5320 Hypothesis can now show statistics when running under pytest-xdist.
5321 Previously, statistics were only reported when all tests were run in a
5322 single process (issue #700).
5323 3.82.5 - 2018-12-08
5325 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
5326 to integers() could cause errors on Python 3 due to internal rounding.
5327 3.82.4 - 2018-12-08
5329 Hypothesis now seeds and resets the global state of np.random for each
5330 test case, to ensure that tests are reproducible.
5331 This matches and complements the existing handling of the python:random
5333 module - Numpy simply maintains an independent PRNG for performance
5334 reasons.
5335 3.82.3 - 2018-12-08
5337 This is a no-op release to add the new Framework :: Hypothesis trove
5338 classifier to hypothesis on PyPI.
5339 You can use it as a filter to find Hypothesis-related packages such as
5341 extensions as they add the tag over the coming weeks, or simply visit
5342 our curated list.
5343 3.82.2 - 2018-12-08
5345 The Hypothesis for Pandas extension is now listed in setup.py, so you
5346 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
5347 tion.
5348 3.82.1 - 2018-10-29
5350 This patch fixes from_type() on Python 2 for classes where cls.__init__
5351 is object.__init__. Thanks to ccxcz for reporting issue #1656.
5352 3.82.0 - 2018-10-29
5354 The alphabet argument for text() now uses its default value of charac‐
5355 ters(blacklist_categories=('Cs',)) directly, instead of hiding that
5356 behind alphabet=None and replacing it within the function. Passing
5357 None is therefore deprecated.
5358 3.81.0 - 2018-10-27
5360 GenericStateMachine and RuleBasedStateMachine now raise an explicit
5361 error when instances of settings are assigned to the classes' settings
5362 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
5363 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
5364 tor to handle this automatically.
5365 3.80.0 - 2018-10-25
5367 Since version 3.68.0, arrays() checks that values drawn from the ele‐
5368 ments and fill strategies can be safely cast to the dtype of the array,
5369 and emits a warning otherwise.
5370 This release expands the checks to cover overflow for finite complex64
5372 elements and string truncation caused by too-long elements or trailing
5373 null characters (issue #1591).
5374 3.79.4 - 2018-10-25
5376 Tests using @given now shrink errors raised from pytest helper func‐
5377 tions, instead of reporting the first example found.
5378 This was previously fixed in version 3.56.0, but only for stateful
5380 testing.
5381 3.79.3 - 2018-10-23
5383 Traceback elision is now disabled on Python 2, to avoid an import-time
5384 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
5385 3.79.2: issue #1648).
5386 We encourage all users to upgrade to Python 3 before the end of 2019.
5388 3.79.2 - 2018-10-23
5390 This patch shortens tracebacks from Hypothesis, so you can see exactly
5391 happened in your code without having to skip over irrelevant details
5392 about our internals (issue #848).
5393 In the example test (see pull request #1582), this reduces tracebacks
5395 from nine frames to just three - and for a test with multiple errors,
5396 from seven frames per error to just one!
5397 If you do want to see the internal details, you can disable frame eli‐
5399 sion by setting verbosity to debug.
5400 3.79.1 - 2018-10-22
5402 The abstract number classes Number, Complex, Real, Rational, and Inte‐
5403 gral are now supported by the from_type() strategy. Previously, you
5404 would have to use register_type_strategy() before they could be
5405 resolved (issue #1636)
5406 3.79.0 - 2018-10-18
5408 This release adds a CLI flag for verbosity --hypothesis-verbosity to
5409 the Hypothesis pytest plugin, applied after loading the profile speci‐
5410 fied by --hypothesis-profile. Valid options are the names of verbosity
5411 settings, quiet, normal, verbose or debug.
5412 Thanks to Bex Dunn for writing this patch at the PyCon Australia
5414 sprints!
5415 The pytest header now correctly reports the current profile if
5417 --hypothesis-profile has been used.
5418 Thanks to Mathieu Paturel for the contribution at the Canberra Python
5420 Hacktoberfest.
5421 3.78.0 - 2018-10-16
5423 This release has deprecated the generation of integers, floats and
5424 fractions when the conversion of the upper and/ or lower bound is not
5425 100% exact, e.g. when an integer gets passed a bound that is not a
5426 whole number. (issue #1625)
5427 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
5429 3.77.0 - 2018-10-16
5431 This minor release adds functionality to settings allowing it to be
5432 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
5433 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
5435 3.76.1 - 2018-10-16
5437 This patch fixes some warnings added by recent releases of pydocstyle
5438 and mypy.
5439 3.76.0 - 2018-10-11
5441 This release deprecates using floats for min_size and max_size.
5442 The type hint for average_size arguments has been changed from
5444 Optional[int] to None, because non-None values are always ignored and
5445 deprecated.
5446 3.75.4 - 2018-10-10
5448 This patch adds more internal comments to the core engine's
5449 sequence-length shrinker. There should be no user-visible change.
5450 3.75.3 - 2018-10-09
5452 This patch adds additional comments to some of the core engine's inter‐
5453 nal data structures. There is no user-visible change.
5454 3.75.2 - 2018-10-09
5456 This patch avoids caching a trivial case, fixing issue #493.
5457 3.75.1 - 2018-10-09
5459 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
5460 for this contribution!
5461 3.75.0 - 2018-10-08
5463 This release deprecates the use of min_size=None, setting the default
5464 min_size to 0 (issue #1618).
5465 3.74.3 - 2018-10-08
5467 This patch makes some small internal changes to comply with a new lint
5468 setting in the build. There should be no user-visible change.
5469 3.74.2 - 2018-10-03
5471 This patch fixes issue #1153, where time spent reifying a strategy was
5472 also counted in the time spent generating the first example. Strate‐
5473 gies are now fully constructed and validated before the timer is
5474 started.
5475 3.74.1 - 2018-10-03
5477 This patch fixes some broken formatting and links in the documentation.
5478 3.74.0 - 2018-10-01
5480 This release checks that the value of the print_blob setting is a
5481 PrintSettings instance.
5482 Being able to specify a boolean value was not intended, and is now dep‐
5484 recated. In addition, specifying True will now cause the blob to
5485 always be printed, instead of causing it to be suppressed.
5486 Specifying any value that is not a PrintSettings or a boolean is now an
5488 error.
5489 3.73.5 - 2018-10-01
5491 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
5492 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
5493 eter names min_value and max_value instead of the deprecated names
5494 3.73.4 - 2018-09-30
5496 This patch ensures that Hypothesis deprecation warnings display the
5497 code that emitted them when you're not running in -Werror mode (issue
5498 #652).
5499 3.73.3 - 2018-09-27
5501 Tracebacks involving @composite are now slightly shorter due to some
5502 internal refactoring.
5503 3.73.2 - 2018-09-26
5505 This patch fixes errors in the internal comments for one of the
5506 shrinker passes. There is no user-visible change.
5507 3.73.1 - 2018-09-25
5509 This patch substantially improves the distribution of data generated
5510 with recursive(), and fixes a rare internal error (issue #1502).
5511 3.73.0 - 2018-09-24
5513 This release adds the fulfill() function, which is designed for testing
5514 code that uses dpcontracts 0.4 or later for input validation. This
5515 provides some syntactic sugar around use of assume(), to automatically
5516 filter out and retry calls that cause a precondition check to fail (‐
5517 issue #1474).
5518 3.72.0 - 2018-09-24
5520 This release makes setting attributes of the hypothesis.settings class
5521 an explicit error. This has never had any effect, but could mislead
5522 users who confused it with the current settings instance hypothe‐
5523 sis.settings.default (which is also immutable). You can change the
5524 global settings with settings profiles.
5525 3.71.11 - 2018-09-24
5527 This patch factors out some common code in the shrinker for iterating
5528 over pairs of data blocks. There should be no user-visible change.
5529 3.71.10 - 2018-09-18
5531 This patch allows from_type() to handle the empty tuple type, typ‐
5532 ing.Tuple[()].
5533 3.71.9 - 2018-09-17
5535 This patch updates some internal comments for mypy. There is no
5536 user-visible effect, even for Mypy users.
5537 3.71.8 - 2018-09-17
5539 This patch fixes a rare bug that would cause a particular shrinker pass
5540 to raise an IndexError, if a shrink improvement changed the underlying
5541 data in an unexpected way.
5542 3.71.7 - 2018-09-17
5544 This release fixes the broken cross-references in our docs, and adds a
5545 CI check so we don't add new ones.
5546 3.71.6 - 2018-09-16
5548 This patch fixes two bugs (issue #944 and issue #1521), where messages
5549 about @seed did not check the current verbosity setting, and the wrong
5550 settings were active while executing explicit examples.
5551 3.71.5 - 2018-09-15
5553 This patch fixes a DeprecationWarning added in Python 3.8 (issue
5554 #1576).
5555 Thanks to tirkarthi for this contribution!
5557 3.71.4 - 2018-09-14
5559 This is a no-op release, which implements automatic DOI minting and
5560 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
5561 zon 2020 programme for providing this service!
5562 Check our CITATION file for details, or head right on over to
5564 doi.org/10.5281/zenodo.1412597
5565 3.71.3 - 2018-09-10
5567 This release adds the test name to some deprecation warnings, for eas‐
5568 ier debugging.
5569 Thanks to Sanyam Khurana for the patch!
5571 3.71.2 - 2018-09-10
5573 This release makes Hypothesis's memory usage substantially smaller for
5574 tests with many examples, by bounding the number of past examples it
5575 keeps around.
5576 You will not see much difference unless you are running tests with
5578 max_examples set to well over 1000, but if you do have such tests then
5579 you should see memory usage mostly plateau where previously it would
5580 have grown linearly with time.
5581 3.71.1 - 2018-09-09
5583 This patch adds internal comments to some tree traversals in the core
5584 engine. There is no user-visible change.
5585 3.71.0 - 2018-09-08
5587 This release deprecates the coverage-guided testing functionality, as
5588 it has proven brittle and does not really pull its weight.
5589 We intend to replace it with something more useful in the future, but
5591 the feature in its current form does not seem to be worth the cost of
5592 using, and whatever replaces it will likely look very different.
5593 3.70.4 - 2018-09-08
5595 This patch changes the behaviour of reproduce_failure() so that blobs
5596 are only printed in quiet mode when the print_blob setting is set to
5597 ALWAYS.
5598 Thanks to Cameron McGill for writing this patch at the PyCon Australia
5600 sprints!
5601 3.70.3 - 2018-09-03
5603 This patch removes some unnecessary code from the internals. There is
5604 no user-visible change.
5605 3.70.2 - 2018-09-03
5607 This patch fixes an internal bug where a corrupted argument to @repro‐
5608 duce_failure could raise the wrong type of error. Thanks again to
5609 Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
5610 tently provides excellent bug reports including issue #1558.
5611 3.70.1 - 2018-09-03
5613 This patch updates hypothesis to report its version and settings when
5614 run with pytest. (issue #1223).
5615 Thanks to Jack Massey for this feature.
5617 3.70.0 - 2018-09-01
5619 This release adds a fullmatch argument to from_regex(). When full‐
5620 match=True, the whole example will match the regex pattern as for
5621 python:re.fullmatch().
5622 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
5624 sprints!
5625 3.69.12 - 2018-08-30
5627 This release reverts the changes to logging handling in 3.69.11, which
5628 broke test that use the pytest caplog fixture internally because all
5629 logging was disabled (issue #1546).
5630 3.69.11 - 2018-08-29
5632 This patch will hide all logging messages produced by test cases before
5633 the final, minimal, failing test case (issue #356).
5634 Thanks to Gary Donovan for writing this patch at the PyCon Australia
5636 sprints!
5637 3.69.10 - 2018-08-29
5639 This patch fixes a bug that prevents coverage from reporting unexecuted
5640 Python files (issue #1085).
5641 Thanks to Gary Donovan for writing this patch at the PyCon Australia
5643 sprints!
5644 3.69.9 - 2018-08-28
5646 This patch improves the packaging of the Python package by adding
5647 LICENSE.txt to the sdist (issue #1311), clarifying the minimum sup‐
5648 ported versions of pytz and dateutil (issue #1383), and adds keywords
5649 to the metadata (issue #1520).
5650 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
5652 tralia sprints!
5653 3.69.8 - 2018-08-28
5655 This is an internal change which replaces pickle with json to prevent
5656 possible security issues.
5657 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
5659 sprints!
5660 3.69.7 - 2018-08-28
5662 This patch ensures that note() prints the note for every test case when
5663 the verbosity setting is Verbosity.verbose. At normal verbosity it
5664 only prints from the final test case.
5665 Thanks to Tom McDermott for writing this patch at the PyCon Australia
5667 sprints!
5668 3.69.6 - 2018-08-27
5670 This patch improves the testing of some internal caching. It should
5671 have no user-visible effect.
5672 3.69.5 - 2018-08-27
5674 This change performs a small rename and refactoring in the core engine.
5675 There is no user-visible change.
5676 3.69.4 - 2018-08-27
5678 This change improves the core engine's ability to avoid unnecessary
5679 work, by consulting its cache of previously-tried inputs in more cases.
5680 3.69.3 - 2018-08-27
5682 This patch handles passing an empty python:enum.Enum to from_type() by
5683 returning nothing(), instead of raising an internal python:Assertion‐
5684 Error.
5685 Thanks to Paul Amazona for writing this patch at the PyCon Australia
5687 sprints!
5688 3.69.2 - 2018-08-23
5690 This patch fixes a small mistake in an internal comment. There is no
5691 user-visible change.
5692 3.69.1 - 2018-08-21
5694 This change fixes a small bug in how the core engine consults its cache
5695 of previously-tried inputs. There is unlikely to be any user-visible
5696 change.
5697 3.69.0 - 2018-08-20
5699 This release improves argument validation for stateful testing.
5700 · If the target or targets of a rule() are invalid, we now raise a use‐
5702 ful validation error rather than an internal exception.
5703 · Passing both the target and targets arguments is deprecated - append
5705 the target bundle to the targets tuple of bundles instead.
5706 · Passing the name of a Bundle rather than the Bundle itself is also
5708 deprecated.
5709 3.68.3 - 2018-08-20
5711 This is a docs-only patch, fixing some typos and formatting issues.
5712 3.68.2 - 2018-08-19
5714 This change fixes a small bug in how the core engine caches the results
5715 of previously-tried inputs. The effect is unlikely to be noticeable,
5716 but it might avoid unnecesary work in some cases.
5717 3.68.1 - 2018-08-18
5719 This patch documents the from_dtype() function, which infers a strategy
5720 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
5721 directly when creating e.g. Pandas objects.
5722 3.68.0 - 2018-08-15
5724 arrays() now checks that integer and float values drawn from elements
5725 and fill strategies can be safely cast to the dtype of the array, and
5726 emits a warning otherwise (issue #1385).
5727 Elements in the resulting array could previously violate constraints on
5729 the elements strategy due to floating-point overflow or truncation of
5730 integers to fit smaller types.
5731 3.67.1 - 2018-08-14
5733 This release contains a tiny refactoring of the internals. There is no
5734 user-visible change.
5735 3.67.0 - 2018-08-10
5737 This release adds a width argument to floats(), to generate lower-pre‐
5738 cision floating point numbers for e.g. Numpy arrays.
5739 The generated examples are always instances of Python's native float
5741 type, which is 64bit, but passing width=32 will ensure that all values
5742 can be exactly represented as 32bit floats. This can be useful to
5743 avoid overflow (to +/- infinity), and for efficiency of generation and
5744 shrinking.
5745 Half-precision floats (width=16) are also supported, but require Numpy
5747 if you are running Python 3.5 or earlier.
5748 3.66.33 - 2018-08-10
5750 This release fixes a bug in floats(), where setting allow_infin‐
5751 ity=False and exactly one of min_value and max_value would allow infi‐
5752 nite values to be generated.
5753 3.66.32 - 2018-08-09
5755 This release adds type hints to the example() and seed() decorators,
5756 and fixes the type hint on register_type_strategy(). The second argu‐
5757 ment to register_type_strategy() must either be a SearchStrategy, or a
5758 callable which takes a type and returns a SearchStrategy.
5759 3.66.31 - 2018-08-08
5761 Another set of changes designed to improve the performance of shrinking
5762 on large examples. In particular the shrinker should now spend consid‐
5763 erably less time running useless shrinks.
5764 3.66.30 - 2018-08-06
5766 "Bug fixes and performance improvements".
5767 This release is a fairly major overhaul of the shrinker designed to
5769 improve its behaviour on large examples, especially around stateful
5770 testing. You should hopefully see shrinking become much faster, with
5771 little to no quality degradation (in some cases quality may even
5772 improve).
5773 3.66.29 - 2018-08-05
5775 This release fixes two very minor bugs in the core engine:
5776 · it fixes a corner case that was missing in 3.66.28, which should
5778 cause shrinking to work slightly better.
5779 · it fixes some logic for how shrinking interacts with the database
5781 that was causing Hypothesis to be insufficiently aggressive about
5782 clearing out old keys.
5783 3.66.28 - 2018-08-05
5785 This release improves how Hypothesis handles reducing the size of inte‐
5786 gers' representation. This change should mostly be invisible as it's
5787 purely about the underlying representation and not the generated value,
5788 but it may result in some improvements to shrink performance.
5789 3.66.27 - 2018-08-05
5791 This release changes the order in which Hypothesis chooses parts of the
5792 test case to shrink. For typical usage this should be a significant
5793 performance improvement on large examples. It is unlikely to have a
5794 major impact on example quality, but where it does change the result it
5795 should usually be an improvement.
5796 3.66.26 - 2018-08-05
5798 This release improves the debugging information that the shrinker emits
5799 about the operations it performs, giving better summary statistics
5800 about which passes resulted in test executions and whether they were
5801 successful.
5802 3.66.25 - 2018-08-05
5804 This release fixes several bugs that were introduced to the shrinker in
5805 3.66.24 which would have caused it to behave significantly less well
5806 than advertised. With any luck you should actually see the promised
5807 benefits now.
5808 3.66.24 - 2018-08-03
5810 This release changes how Hypothesis deletes data when shrinking in
5811 order to better handle deletion of large numbers of contiguous
5812 sequences. Most tests should see little change, but this will hopefully
5813 provide a significant speed up for stateful testing.
5814 3.66.23 - 2018-08-02
5816 This release makes some internal changes to enable further improvements
5817 to the shrinker. You may see some changes in the final shrunk examples,
5818 but they are unlikely to be significant.
5819 3.66.22 - 2018-08-01
5821 This release adds some more internal caching to the shrinker. This
5822 should cause a significant speed up for shrinking, especially for
5823 stateful testing and large example sizes.
5824 3.66.21 - 2018-08-01
5826 This patch is for downstream packagers - our tests now pass under
5827 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
5828 of Hypothesis itself.
5829 3.66.20 - 2018-08-01
5831 This release removes some functionality from the shrinker that was tak‐
5832 ing a considerable amount of time and does not appear to be useful any
5833 more due to a number of quality improvements in the shrinker.
5834 You may see some degradation in shrink quality as a result of this, but
5836 mostly shrinking should just get much faster.
5837 3.66.19 - 2018-08-01
5839 This release slightly changes the format of some debugging information
5840 emitted during shrinking, and refactors some of the internal interfaces
5841 around that.
5842 3.66.18 - 2018-07-31
5844 This release is a very small internal refactoring which should have no
5845 user visible impact.
5846 3.66.17 - 2018-07-31
5848 This release fixes a bug that could cause an IndexError to be raised
5849 from inside Hypothesis during shrinking. It is likely that it was
5850 impossible to trigger this bug in practice - it was only made visible
5851 by some currently unreleased work.
5852 3.66.16 - 2018-07-31
5854 This release is a very small internal refactoring which should have no
5855 user visible impact.
5856 3.66.15 - 2018-07-31
5858 This release makes Hypothesis's shrinking faster by removing some
5859 redundant work that it does when minimizing values in its internal rep‐
5860 resentation.
5861 3.66.14 - 2018-07-30
5863 This release expands the deprecation of timeout from 3.16.0 to also
5864 emit the deprecation warning in find or stateful testing.
5865 3.66.13 - 2018-07-30
5867 This release adds an additional shrink pass that is able to reduce the
5868 size of examples in some cases where the transformation is non-obvious.
5869 In particular this will improve the quality of some examples which
5870 would have regressed in 3.66.12.
5871 3.66.12 - 2018-07-28
5873 This release changes how we group data together for shrinking. It
5874 should result in improved shrinker performance, especially in stateful
5875 testing.
5876 3.66.11 - 2018-07-28
5878 This patch modifies how which rule to run is selected during rule based
5879 stateful testing. This should result in a slight performance increase
5880 during generation and a significant performance and quality improvement
5881 when shrinking.
5882 As a result of this change, some state machines which would previously
5884 have thrown an InvalidDefinition are no longer detected as invalid.
5885 3.66.10 - 2018-07-28
5887 This release weakens some minor functionality in the shrinker that had
5888 only modest benefit and made its behaviour much harder to reason about.
5889 This is unlikely to have much user visible effect, but it is possible
5891 that in some cases shrinking may get slightly slower. It is primarily
5892 to make it easier to work on the shrinker and pave the way for future
5893 work.
5894 3.66.9 - 2018-07-26
5896 This release improves the information that Hypothesis emits about its
5897 shrinking when verbosity is set to debug.
5898 3.66.8 - 2018-07-24
5900 This patch includes some minor fixes in the documentation, and updates
5901 the minimum version of pytest to 3.0 (released August 2016).
5902 3.66.7 - 2018-07-24
5904 This release fixes a bug where difficult to shrink tests could some‐
5905 times trigger an internal assertion error inside the shrinker.
5906 3.66.6 - 2018-07-23
5908 This patch ensures that Hypothesis fully supports Python 3.7, by
5909 upgrading from_type() (issue #1264) and fixing some minor issues in our
5910 test suite (issue #1148).
5911 3.66.5 - 2018-07-22
5913 This patch fixes the online docs for various extras, by ensuring that
5914 their dependencies are installed on readthedocs.io (issue #1326).
5915 3.66.4 - 2018-07-20
5917 This release improves the shrinker's ability to reorder examples.
5918 For example, consider the following test:
5920 import hypothesis.strategies as st
5922 from hypothesis import given
5923 @given(st.text(), st.text())
5925 def test_non_equal(x, y):
5926 assert x != y
5927 Previously this could have failed with either of x="", y="0" or x="0",
5929 y="". Now it should always fail with x="", y="0".
5930 This will allow the shrinker to produce more consistent results, espe‐
5932 cially in cases where test cases contain some ordered collection whose
5933 actual order does not matter.
5934 3.66.3 - 2018-07-20
5936 This patch fixes inference in the builds() strategy with subtypes of
5937 python:typing.NamedTuple, where the __init__ method is not useful for
5938 introspection. We now use the field types instead - thanks to James
5939 Uther for identifying this bug.
5940 3.66.2 - 2018-07-19
5942 This release improves the shrinker's ability to handle situations where
5943 there is an additive constraint between two values.
5944 For example, consider the following test:
5946 import hypothesis.strategies as st
5948 from hypothesis import given
5949 @given(st.integers(), st.integers())
5951 def test_does_not_exceed_100(m, n):
5952 assert m + n < 100
5953 Previously this could have failed with almost any pair (m, n) with 0 <=
5955 m <= n and m + n == 100. Now it should almost always fail with m=0,
5956 n=100.
5957 This is a relatively niche specialisation, but can be useful in situa‐
5959 tions where e.g. a bug is triggered by an integer overflow.
5960 3.66.1 - 2018-07-09
5962 This patch fixes a rare bug where an incorrect percentage drawtime
5963 could be displayed for a test, when the system clock was changed during
5964 a test running under Python 2 (we use python:time.monotonic() where it
5965 is available to avoid such problems). It also fixes a possible
5966 zero-division error that can occur when the underlying C library dou‐
5967 ble-rounds an intermediate value in python:math.fsum() and gets the
5968 least significant bit wrong.
5969 3.66.0 - 2018-07-05
5971 This release improves validation of the alphabet argument to the text()
5972 strategy. The following misuses are now deprecated, and will be an
5973 error in a future version:
5974 · passing an unordered collection (such as set('abc')), which violates
5976 invariants about shrinking and reproducibility
5977 · passing an alphabet sequence with elements that are not strings
5979 · passing an alphabet sequence with elements that are not of length
5981 one, which violates any size constraints that may apply
5982 Thanks to Sushobhit for adding these warnings (issue #1329).
5984 3.65.3 - 2018-07-04
5986 This release fixes a mostly theoretical bug where certain usage of the
5987 internal API could trigger an assertion error inside Hypothesis. It is
5988 unlikely that this problem is even possible to trigger through the pub‐
5989 lic API.
5990 3.65.2 - 2018-07-04
5992 This release fixes dependency information for coverage. Previously
5993 Hypothesis would allow installing coverage with any version, but it
5994 only works with coverage 4.0 or later.
5995 We now specify the correct metadata in our setup.py, so Hypothesis will
5997 only allow installation with compatible versions of coverage.
5998 3.65.1 - 2018-07-03
6000 This patch ensures that stateful tests which raise an error from a
6001 pytest helper still print the sequence of steps taken to reach that
6002 point (issue #1372). This reporting was previously broken because the
6003 helpers inherit directly from python:BaseException, and therefore
6004 require special handling to catch without breaking e.g. the use of
6005 ctrl-C to quit the test.
6006 3.65.0 - 2018-06-30
6008 This release deprecates the max_shrinks setting in favor of an internal
6009 heuristic. If you need to avoid shrinking examples, use the phases
6010 setting instead. (issue #1235)
6011 3.64.2 - 2018-06-27
6013 This release fixes a bug where an internal assertion error could some‐
6014 times be triggered while shrinking a failing test.
6015 3.64.1 - 2018-06-27
6017 This patch fixes type-checking errors in our vendored pretty-printer,
6018 which were ignored by our mypy config but visible for anyone else
6019 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
6020 3.64.0 - 2018-06-26
6022 This release adds an interface which can be used to insert a wrapper
6023 between the original test function and @given (issue #1257). This will
6024 be particularly useful for test runner extensions such as pytest-trio,
6025 but is not recommended for direct use by other users of Hypothesis.
6026 3.63.0 - 2018-06-26
6028 This release adds a new mechanism to infer strategies for classes
6029 defined using attrs, based on the the type, converter, or validator of
6030 each attribute. This inference is now built in to builds() and
6031 from_type().
6032 On Python 2, from_type() no longer generates instances of int when
6034 passed long, or vice-versa.
6035 3.62.0 - 2018-06-26
6037 This release adds PEP 484 type hints to Hypothesis on a provisional
6038 basis, using the comment-based syntax for Python 2 compatibility. You
6039 can read more about our type hints here.
6040 It also adds the py.typed marker specified in PEP 561. After you pip
6042 install hypothesis, mypy 0.590 or later will therefore type-check your
6043 use of our public interface!
6044 3.61.0 - 2018-06-24
6046 This release deprecates the use of settings as a context manager, the
6047 use of which is somewhat ambiguous.
6048 Users should define settings with global state or with the @set‐
6050 tings(...) decorator.
6051 3.60.1 - 2018-06-20
6053 Fixed a bug in generating an instance of a Django model from a strategy
6054 where the primary key is generated as part of the strategy. See details
6055 here.
6056 Thanks to Tim Martin for this contribution.
6058 3.60.0 - 2018-06-20
6060 This release adds the @initialize decorator for stateful testing (orig‐
6061 inally discussed in issue #1216). All @initialize rules will be called
6062 once each in an arbitrary order before any normal rule is called.
6063 3.59.3 - 2018-06-19
6065 This is a no-op release to take into account some changes to the
6066 release process. It should have no user visible effect.
6067 3.59.2 - 2018-06-18
6069 This adds support for partially sorting examples which cannot be fully
6070 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
6071 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
6072 0].
6073 Thanks to Luke for contributing.
6075 3.59.1 - 2018-06-16
6077 This patch uses python:random.getstate() and python:random.setstate()
6078 to restore the PRNG state after @given runs deterministic tests. With‐
6079 out restoring state, you might have noticed problems such as issue
6080 #1266. The fix also applies to stateful testing (issue #702).
6081 3.59.0 - 2018-06-14
6083 This release adds the emails() strategy, which generates unicode
6084 strings representing an email address.
6085 Thanks to Sushobhit for moving this to the public API (issue #162).
6087 3.58.1 - 2018-06-13
6089 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
6090 1 2 3.
6091 Thanks to Luke for contributing.
6093 3.58.0 - 2018-06-13
6095 This adds a new extra timezones() strategy that generates dateutil
6096 timezones.
6097 Thanks to Conrad for contributing.
6099 3.57.0 - 2018-05-20
6101 Using an unordered collection with the permutations() strategy has been
6102 deprecated because the order in which e.g. a set shrinks is arbitrary.
6103 This may cause different results between runs.
6104 3.56.10 - 2018-05-16
6106 This release makes hypothesis.settings.define_setting a private method,
6107 which has the effect of hiding it from the documentation.
6108 3.56.9 - 2018-05-11
6110 This is another release with no functionality changes as part of
6111 changes to Hypothesis's new release tagging scheme.
6112 3.56.8 - 2018-05-10
6114 This is a release with no functionality changes that moves Hypothesis
6115 over to a new release tagging scheme.
6116 3.56.7 - 2018-05-10
6118 This release provides a performance improvement for most tests, but in
6119 particular users of sampled_from() who don't have numpy installed
6120 should see a significant performance improvement.
6121 3.56.6 - 2018-05-09
6123 This patch contains further internal work to support Mypy. There are
6124 no user-visible changes... yet.
6125 3.56.5 - 2018-04-22
6127 This patch contains some internal refactoring to run mypy in CI. There
6128 are no user-visible changes.
6129 3.56.4 - 2018-04-21
6131 This release involves some very minor internal clean up and should have
6132 no user visible effect at all.
6133 3.56.3 - 2018-04-20
6135 This release fixes a problem introduced in 3.56.0 where setting the
6136 hypothesis home directory (through currently undocumented means) would
6137 no longer result in the default database location living in the new
6138 home directory.
6139 3.56.2 - 2018-04-20
6141 This release fixes a problem introduced in 3.56.0 where setting
6142 max_examples to 1 would result in tests failing with Unsatisfiable.
6143 This problem could also occur in other harder to trigger circumstances
6144 (e.g. by setting it to a low value, having a hard to satisfy assump‐
6145 tion, and disabling health checks).
6146 3.56.1 - 2018-04-20
6148 This release fixes a problem that was introduced in 3.56.0: Use of the
6149 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
6150 cated, actually broken due to being read before various setup the dep‐
6151 recation path needed was done. It now works correctly (and emits a dep‐
6152 recation warning).
6153 3.56.0 - 2018-04-17
6155 This release deprecates several redundant or internally oriented set‐
6156 tings, working towards an orthogonal set of configuration options that
6157 are widely useful without requiring any knowledge of our internals (‐
6158 issue #535).
6159 · Deprecated settings that no longer have any effect are no longer
6161 shown in the __repr__ unless set to a non-default value.
6162 · hypothesis.settings.perform_health_check is deprecated, as it dupli‐
6164 cates suppress_health_check.
6165 · hypothesis.settings.max_iterations is deprecated and disabled,
6167 because we can usually get better behaviour from an internal heuris‐
6168 tic than a user-controlled setting.
6169 · hypothesis.settings.min_satisfying_examples is deprecated and dis‐
6171 abled, due to overlap with the filter_too_much healthcheck and poor
6172 interaction with max_examples.
6173 · HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
6175 the profile system instead.
6176 · Examples tried by find() are now reported at debug verbosity level
6178 (as well as verbose level).
6179 3.55.6 - 2018-04-14
6181 This release fixes a somewhat obscure condition (issue #1230) under
6182 which you could occasionally see a failing test trigger an assertion
6183 error inside Hypothesis instead of failing normally.
6184 3.55.5 - 2018-04-14
6186 This patch fixes one possible cause of issue #966. When running Python
6187 2 with hash randomisation, passing a python:bytes object to python:ran‐
6188 dom.seed() would use version=1, which broke derandomize (because the
6189 seed depended on a randomised hash). If derandomize is still nondeter‐
6190 ministic for you, please open an issue.
6191 3.55.4 - 2018-04-13
6193 This patch makes a variety of minor improvements to the documentation,
6194 and improves a few validation messages for invalid inputs.
6195 3.55.3 - 2018-04-12
6197 This release updates the URL metadata associated with the PyPI package
6198 (again). It has no other user visible effects.
6199 3.55.2 - 2018-04-11
6201 This release updates the URL metadata associated with the PyPI package.
6202 It has no other user visible effects.
6203 3.55.1 - 2018-04-06
6205 This patch relaxes constraints in our tests on the expected values
6206 returned by the standard library function hypot() and the internal
6207 helper function cathetus, to fix near-exact test failures on some
6208 32-bit systems used by downstream packagers.
6209 3.55.0 - 2018-04-05
6211 This release includes several improvements to the handling of the data‐
6212 base setting.
6213 · The database_file setting was a historical artefact, and you should
6215 just use database directly.
6216 · The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
6218 favor of load_profile() and the database setting.
6219 · If you have not configured the example database at all and the
6221 default location is not usable (due to e.g. permissions issues),
6222 Hypothesis will fall back to an in-memory database. This is not per‐
6223 sisted between sessions, but means that the defaults work on
6224 read-only filesystems.
6225 3.54.0 - 2018-04-04
6227 This release improves the complex_numbers() strategy, which now sup‐
6228 ports min_magnitude and max_magnitude arguments, along with allow_nan
6229 and allow_infinity like for floats().
6230 Thanks to J.J. Green for this feature.
6232 3.53.0 - 2018-04-01
6234 This release removes support for Django 1.8, which reached end of life
6235 on 2018-04-01. You can see Django's release and support schedule on
6236 the Django Project website.
6237 3.52.3 - 2018-04-01
6239 This patch fixes the min_satisfying_examples settings documentation, by
6240 explaining that example shrinking is tracked at the level of the under‐
6241 lying bytestream rather than the output value.
6242 The output from find() in verbose mode has also been adjusted - see the
6244 example session - to avoid duplicating lines when the example repr is
6245 constant, even if the underlying representation has been shrunken.
6246 3.52.2 - 2018-03-30
6248 This release improves the output of failures with rule based stateful
6249 testing in two ways:
6250 · The output from it is now usually valid Python code.
6252 · When the same value has two different names because it belongs to two
6254 different bundles, it will now display with the name associated with
6255 the correct bundle for a rule argument where it is used.
6256 3.52.1 - 2018-03-29
6258 This release improves the behaviour of stateful testing in two ways:
6259 · Previously some runs would run no steps (issue #376). This should no
6261 longer happen.
6262 · RuleBasedStateMachine tests which used bundles extensively would
6264 often shrink terribly. This should now be significantly improved,
6265 though there is likely a lot more room for improvement.
6266 This release also involves a low level change to how ranges of integers
6268 are handles which may result in other improvements to shrink quality in
6269 some cases.
6270 3.52.0 - 2018-03-24
6272 This release deprecates use of @settings(...) as a decorator, on func‐
6273 tions or methods that are not also decorated with @given. You can
6274 still apply these decorators in any order, though you should only do so
6275 once each.
6276 Applying @given twice was already deprecated, and applying @set‐
6278 tings(...) twice is deprecated in this release and will become an error
6279 in a future version. Neither could ever be used twice to good effect.
6280 Using @settings(...) as the sole decorator on a test is completely
6282 pointless, so this common usage error will become an error in a future
6283 version of Hypothesis.
6284 3.51.0 - 2018-03-24
6286 This release deprecates the average_size argument to lists() and other
6287 collection strategies. You should simply delete it wherever it was
6288 used in your tests, as it no longer has any effect.
6289 In early versions of Hypothesis, the average_size argument was treated
6291 as a hint about the distribution of examples from a strategy. Subse‐
6292 quent improvements to the conceptual model and the engine for generat‐
6293 ing and shrinking examples mean it is more effective to simply describe
6294 what constitutes a valid example, and let our internals handle the dis‐
6295 tribution.
6296 3.50.3 - 2018-03-24
6298 This patch contains some internal refactoring so that we can run with
6299 warnings as errors in CI.
6300 3.50.2 - 2018-03-20
6302 This has no user-visible changes except one slight formatting change to
6303 one docstring, to avoid a deprecation warning.
6304 3.50.1 - 2018-03-20
6306 This patch fixes an internal error introduced in 3.48.0, where a check
6307 for the Django test runner would expose import-time errors in Django
6308 configuration (issue #1167).
6309 3.50.0 - 2018-03-19
6311 This release improves validation of numeric bounds for some strategies.
6312 · integers() and floats() now raise InvalidArgument if passed a
6314 min_value or max_value which is not an instance of Real, instead of
6315 various internal errors.
6316 · floats() now converts its bounding values to the nearest float above
6318 or below the min or max bound respectively, instead of just casting
6319 to float. The old behaviour was incorrect in that you could generate
6320 float(min_value), even when this was less than min_value itself (pos‐
6321 sible with eg. fractions).
6322 · When both bounds are provided to floats() but there are no floats in
6324 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
6325 raised.
6326 · decimals() gives a more useful error message if passed a string that
6328 cannot be converted to Decimal in a context where this error is not
6329 trapped.
6330 Code that previously seemed to work may be explicitly broken if there
6332 were no floats between min_value and max_value (only possible with
6333 non-float bounds), or if a bound was not a Real number but still
6334 allowed in python:math.isnan (some custom classes with a __float__
6335 method).
6336 3.49.1 - 2018-03-15
6338 This patch fixes our tests for Numpy dtype strategies on big-endian
6339 platforms, where the strategy behaved correctly but the test assumed
6340 that the native byte order was little-endian.
6341 There is no user impact unless you are running our test suite on
6343 big-endian platforms. Thanks to Graham Inggs for reporting issue
6344 #1164.
6345 3.49.0 - 2018-03-12
6347 This release deprecates passing elements=None to collection strategies,
6348 such as lists().
6349 Requiring lists(nothing()) or builds(list) instead of lists() means
6351 slightly more typing, but also improves the consistency and discover‐
6352 ability of our API - as well as showing how to compose or construct
6353 strategies in ways that still work in more complex situations.
6354 Passing a nonzero max_size to a collection strategy where the elements
6356 strategy contains no values is now deprecated, and will be an error in
6357 a future version. The equivalent with elements=None is already an
6358 error.
6359 3.48.1 - 2018-03-05
6361 This patch will minimize examples that would come out non-minimal in
6362 previous versions. Thanks to Kyle Reeve for this patch.
6363 3.48.0 - 2018-03-05
6365 This release improves some "unhappy paths" when using Hypothesis with
6366 the standard library python:unittest module:
6367 · Applying @given to a non-test method which is overridden from
6369 python:unittest.TestCase, such as setUp, raises a new health check.
6370 (issue #991)
6371 · Using subTest() within a test decorated with @given would leak inter‐
6373 mediate results when tests were run under the python:unittest test
6374 runner. Individual reporting of failing subtests is now disabled
6375 during a test using @given. (issue #1071)
6376 · @given is still not a class decorator, but the error message if you
6378 try using it on a class has been improved.
6379 As a related improvement, using django:django.test.TestCase with @given
6381 instead of hypothesis.extra.django.TestCase raises an explicit error
6382 instead of running all examples in a single database transaction.
6383 3.47.0 - 2018-03-02
6385 register_profile now accepts keyword arguments for specific settings,
6386 and the parent settings object is now optional. Using a name for a
6387 registered profile which is not a string was never suggested, but it is
6388 now also deprecated and will eventually be an error.
6389 3.46.2 - 2018-03-01
6391 This release removes an unnecessary branch from the code, and has no
6392 user-visible impact.
6393 3.46.1 - 2018-03-01
6395 This changes only the formatting of our docstrings and should have no
6396 user-visible effects.
6397 3.46.0 - 2018-02-26
6399 characters() has improved docs about what arguments are valid, and
6400 additional validation logic to raise a clear error early (instead of
6401 e.g. silently ignoring a bad argument). Categories may be specified as
6402 the Unicode 'general category' (eg u'Nd'), or as the 'major category'
6403 (eg [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
6404 In previous versions, general categories were supported and all other
6406 input was silently ignored. Now, major categories are supported in
6407 addition to general categories (which may change the behaviour of some
6408 existing code), and all other input is deprecated.
6409 3.45.5 - 2018-02-26
6411 This patch improves strategy inference in hypothesis.extra.django to
6412 account for some validators in addition to field type - see issue #1116
6413 for ongoing work in this space.
6414 Specifically, if a CharField or TextField has an attached RegexValida‐
6416 tor, we now use from_regex() instead of text() as the underlying strat‐
6417 egy. This allows us to generate examples of the default User model,
6418 closing issue #1112.
6419 3.45.4 - 2018-02-25
6421 This patch improves some internal debugging information, fixes a typo
6422 in a validation error message, and expands the documentation for new
6423 contributors.
6424 3.45.3 - 2018-02-23
6426 This patch may improve example shrinking slightly for some strategies.
6427 3.45.2 - 2018-02-18
6429 This release makes our docstring style more consistent, thanks to
6430 flake8-docstrings. There are no user-visible changes.
6431 3.45.1 - 2018-02-17
6433 This fixes an indentation issue in docstrings for datetimes(), dates(),
6434 times(), and timedeltas().
6435 3.45.0 - 2018-02-13
6437 This release fixes builds() so that target can be used as a keyword
6438 argument for passing values to the target. The target itself can still
6439 be specified as a keyword argument, but that behavior is now depre‐
6440 cated. The target should be provided as the first positional argument.
6441 3.44.26 - 2018-02-06
6443 This release fixes some formatting issues in the Hypothesis source
6444 code. It should have no externally visible effects.
6445 3.44.25 - 2018-02-05
6447 This release changes the way in which Hypothesis tries to shrink the
6448 size of examples. It probably won't have much impact, but might make
6449 shrinking faster in some cases. It is unlikely but not impossible that
6450 it will change the resulting examples.
6451 3.44.24 - 2018-01-27
6453 This release fixes dependency information when installing Hypothesis
6454 from a binary "wheel" distribution.
6455 · The install_requires for enum34 is resolved at install time, rather
6457 than at build time (with potentially different results).
6458 · Django has fixed their python_requires for versions 2.0.0 onward,
6460 simplifying Python2-compatible constraints for downstream projects.
6461 3.44.23 - 2018-01-24
6463 This release improves shrinking in a class of pathological examples
6464 that you are probably never hitting in practice. If you are hitting
6465 them in practice this should be a significant speed up in shrinking. If
6466 you are not, you are very unlikely to notice any difference. You might
6467 see a slight slow down and/or slightly better falsifying examples.
6468 3.44.22 - 2018-01-23
6470 This release fixes a dependency problem. It was possible to install
6471 Hypothesis with an old version of attrs, which would throw a TypeError
6472 as soon as you tried to import hypothesis. Specifically, you need
6473 attrs 16.0.0 or newer.
6474 Hypothesis will now require the correct version of attrs when
6476 installing.
6477 3.44.21 - 2018-01-22
6479 This change adds some additional structural information that Hypothesis
6480 will use to guide its search.
6481 You mostly shouldn't see much difference from this. The two most likely
6483 effects you would notice are:
6484 1. Hypothesis stores slightly more examples in its database for passing
6486 tests.
6487 2. Hypothesis may find new bugs that it was previously missing, but it
6489 probably won't (this is a basic implementation of the feature that
6490 is intended to support future work. Although it is useful on its
6491 own, it's not very useful on its own).
6492 3.44.20 - 2018-01-21
6494 This is a small refactoring release that changes how Hypothesis tracks
6495 some information about the boundary of examples in its internal repre‐
6496 sentation.
6497 You are unlikely to see much difference in behaviour, but memory usage
6499 and run time may both go down slightly during normal test execution,
6500 and when failing Hypothesis might print its failing example slightly
6501 sooner.
6502 3.44.19 - 2018-01-21
6504 This changes how we compute the default average_size for all collection
6505 strategies. Previously setting a max_size without setting an aver‐
6506 age_size would have the seemingly paradoxical effect of making data
6507 generation slower, because it would raise the average size from its
6508 default. Now setting max_size will either leave the default unchanged
6509 or lower it from its default.
6510 If you are currently experiencing this problem, this may make your
6512 tests substantially faster. If you are not, this will likely have no
6513 effect on you.
6514 3.44.18 - 2018-01-20
6516 This is a small refactoring release that changes how Hypothesis detects
6517 when the structure of data generation depends on earlier values gener‐
6518 ated (e.g. when using flatmap or composite()). It should not have any
6519 observable effect on behaviour.
6520 3.44.17 - 2018-01-15
6522 This release fixes a typo in internal documentation, and has no
6523 user-visible impact.
6524 3.44.16 - 2018-01-13
6526 This release improves test case reduction for recursive data struc‐
6527 tures. Hypothesis now guarantees that whenever a strategy calls itself
6528 recursively (usually this will happen because you are using
6529 deferred()), any recursive call may replace the top level value. e.g.
6530 given a tree structure, Hypothesis will always try replacing it with a
6531 subtree.
6532 Additionally this introduces a new heuristic that may in some circum‐
6534 stances significantly speed up test case reduction - Hypothesis should
6535 be better at immediately replacing elements drawn inside another strat‐
6536 egy with their minimal possible value.
6537 3.44.15 - 2018-01-13
6539 from_type() can now resolve recursive types such as binary trees (issue
6540 #1004). Detection of non-type arguments has also improved, leading to
6541 better error messages in many cases involving forward references.
6542 3.44.14 - 2018-01-08
6544 This release fixes a bug in the shrinker that prevented the optimisa‐
6545 tions in 3.44.6 from working in some cases. It would not have worked
6546 correctly when filtered examples were nested (e.g. with a set of inte‐
6547 gers in some range).
6548 This would not have resulted in any correctness problems, but shrinking
6550 may have been slower than it otherwise could be.
6551 3.44.13 - 2018-01-08
6553 This release changes the average bit length of values drawn from inte‐
6554 gers() to be much smaller. Additionally it changes the shrinking order
6555 so that now size is considered before sign - e.g. -1 will be preferred
6556 to +10.
6557 The new internal format for integers required some changes to the mini‐
6559 mizer to make work well, so you may also see some improvements to exam‐
6560 ple quality in unrelated areas.
6561 3.44.12 - 2018-01-07
6563 This changes Hypothesis's internal implementation of weighted sampling.
6564 This will affect example distribution and quality, but you shouldn't
6565 see any other effects.
6566 3.44.11 - 2018-01-06
6568 This is a change to some internals around how Hypothesis handles avoid‐
6569 ing generating duplicate examples and seeking out novel regions of the
6570 search space.
6571 You are unlikely to see much difference as a result of it, but it fixes
6573 a bug where an internal assertion could theoretically be triggered and
6574 has some minor effects on the distribution of examples so could poten‐
6575 tially find bugs that have previously been missed.
6576 3.44.10 - 2018-01-06
6578 This patch avoids creating debug statements when debugging is disabled.
6579 Profiling suggests this is a 5-10% performance improvement (pull
6580 request #1040).
6581 3.44.9 - 2018-01-06
6583 This patch blacklists null characters ('\x00') in automatically created
6584 strategies for Django CharField and TextField, due to a database issue
6585 which was recently fixed upstream (Hypothesis issue #1045).
6586 3.44.8 - 2018-01-06
6588 This release makes the Hypothesis shrinker slightly less greedy in
6589 order to avoid local minima - when it gets stuck, it makes a small
6590 attempt to search around the final example it would previously have
6591 returned to find a new starting point to shrink from. This should
6592 improve example quality in some cases, especially ones where the test
6593 data has dependencies among parts of it that make it difficult for
6594 Hypothesis to proceed.
6595 3.44.7 - 2018-01-04
6597 This release adds support for Django 2 in the hypothesis-django extra.
6598 This release drops support for Django 1.10, as it is no longer sup‐
6600 ported by the Django team.
6601 3.44.6 - 2018-01-02
6603 This release speeds up test case reduction in many examples by being
6604 better at detecting large shrinks it can use to discard redundant parts
6605 of its input. This will be particularly noticeable in examples that
6606 make use of filtering and for some integer ranges.
6607 3.44.5 - 2018-01-02
6609 Happy new year!
6610 This is a no-op release that updates the year range on all of the copy‐
6612 right headers in our source to include 2018.
6613 3.44.4 - 2017-12-23
6615 This release fixes issue #1044, which slowed tests by up to 6% due to
6616 broken caching.
6617 3.44.3 - 2017-12-21
6619 This release improves the shrinker in cases where examples drawn ear‐
6620 lier can affect how much data is drawn later (e.g. when you draw a
6621 length parameter in a composite and then draw that many elements).
6622 Examples found in cases like this should now be much closer to minimal.
6623 3.44.2 - 2017-12-20
6625 This is a pure refactoring release which changes how Hypothesis manages
6626 its set of examples internally. It should have no externally visible
6627 effects.
6628 3.44.1 - 2017-12-18
6630 This release fixes issue #997, in which under some circumstances the
6631 body of tests run under Hypothesis would not show up when run under
6632 coverage even though the tests were run and the code they called out‐
6633 side of the test file would show up normally.
6634 3.44.0 - 2017-12-17
6636 This release adds a new feature: The @reproduce_failure decorator,
6637 designed to make it easy to use Hypothesis's binary format for examples
6638 to reproduce a problem locally without having to share your example
6639 database between machines.
6640 This also changes when seeds are printed:
6642 · They will no longer be printed for normal falsifying examples, as
6644 there are now adequate ways of reproducing those for all cases, so it
6645 just contributes noise.
6646 · They will once again be printed when reusing examples from the data‐
6648 base, as health check failures should now be more reliable in this
6649 scenario so it will almost always work in this case.
6650 This work was funded by Smarkets.
6652 3.43.1 - 2017-12-17
6654 This release fixes a bug with Hypothesis's database management - exam‐
6655 ples that were found in the course of shrinking were saved in a way
6656 that indicated that they had distinct causes, and so they would all be
6657 retried on the start of the next test. The intended behaviour, which is
6658 now what is implemented, is that only a bounded subset of these exam‐
6659 ples would be retried.
6660 3.43.0 - 2017-12-17
6662 HypothesisDeprecationWarning now inherits from python:FutureWarning
6663 instead of python:DeprecationWarning, as recommended by PEP 565 for
6664 user-facing warnings (issue #618). If you have not changed the default
6665 warnings settings, you will now see each distinct HypothesisDepreca‐
6666 tionWarning instead of only the first.
6667 3.42.2 - 2017-12-12
6669 This patch fixes issue #1017, where instances of a list or tuple sub‐
6670 type used as an argument to a strategy would be coerced to tuple.
6671 3.42.1 - 2017-12-10
6673 This release has some internal cleanup, which makes reading the code
6674 more pleasant and may shrink large examples slightly faster.
6675 3.42.0 - 2017-12-09
6677 This release deprecates faker-extra, which was designed as a transition
6678 strategy but does not support example shrinking or coverage-guided dis‐
6679 covery.
6680 3.41.0 - 2017-12-06
6682 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
6683 pling from multi-dimensional ndarrays still results in a deprecation
6684 warning. Thanks to Charlie Tanksley for this patch.
6685 3.40.1 - 2017-12-04
6687 This release makes two changes:
6688 · It makes the calculation of some of the metadata that Hypothesis uses
6690 for shrinking occur lazily. This should speed up performance of test
6691 case generation a bit because it no longer calculates information it
6692 doesn't need.
6693 · It improves the shrinker for certain classes of nested examples. e.g.
6695 when shrinking lists of lists, the shrinker is now able to concate‐
6696 nate two adjacent lists together into a single list. As a result of
6697 this change, shrinking may get somewhat slower when the minimal exam‐
6698 ple found is large.
6699 3.40.0 - 2017-12-02
6701 This release improves how various ways of seeding Hypothesis interact
6702 with the example database:
6703 · Using the example database with seed() is now deprecated. You should
6705 set database=None if you are doing that. This will only warn if you
6706 actually load examples from the database while using @seed.
6707 · The derandomize will behave the same way as @seed.
6709 · Using --hypothesis-seed will disable use of the database.
6711 · If a test used examples from the database, it will not suggest using
6713 a seed to reproduce it, because that won't work.
6714 This work was funded by Smarkets.
6716 3.39.0 - 2017-12-01
6718 This release adds a new health check that checks if the smallest "natu‐
6719 ral" possible example of your test case is very large - this will tend
6720 to cause Hypothesis to generate bad examples and be quite slow.
6721 This work was funded by Smarkets.
6723 3.38.9 - 2017-11-29
6725 This is a documentation release to improve the documentation of shrink‐
6726 ing behaviour for Hypothesis's strategies.
6727 3.38.8 - 2017-11-29
6729 This release improves the performance of characters() when using black‐
6730 list_characters and from_regex() when using negative character classes.
6731 The problems this fixes were found in the course of work funded by
6733 Smarkets.
6734 3.38.7 - 2017-11-29
6736 This is a patch release for from_regex(), which had a bug in handling
6737 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
6738 rectly when parsing regex.
6739 3.38.6 - 2017-11-28
6741 This patch changes a few byte-string literals from double to single
6742 quotes, thanks to an update in unify. There are no user-visible
6743 changes.
6744 3.38.5 - 2017-11-23
6746 This fixes the repr of strategies using lambda that are defined inside
6747 decorators to include the lambda source.
6748 This would mostly have been visible when using the statistics function‐
6750 ality - lambdas used for e.g. filtering would have shown up with a
6751 <unknown> as their body. This can still happen, but it should happen
6752 less often now.
6753 3.38.4 - 2017-11-22
6755 This release updates the reported statistics so that they show approxi‐
6756 mately what fraction of your test run time is spent in data generation
6757 (as opposed to test execution).
6758 This work was funded by Smarkets.
6760 3.38.3 - 2017-11-21
6762 This is a documentation release, which ensures code examples are up to
6763 date by running them as doctests in CI (issue #711).
6764 3.38.2 - 2017-11-21
6766 This release changes the behaviour of the deadline setting when used
6767 with data(): Time spent inside calls to data.draw will no longer be
6768 counted towards the deadline time.
6769 As a side effect of some refactoring required for this work, the way
6771 flaky tests are handled has changed slightly. You are unlikely to see
6772 much difference from this, but some error messages will have changed.
6773 This work was funded by Smarkets.
6775 3.38.1 - 2017-11-21
6777 This patch has a variety of non-user-visible refactorings, removing
6778 various minor warts ranging from indirect imports to typos in comments.
6779 3.38.0 - 2017-11-18
6781 This release overhauls the health check system in a variety of small
6782 ways. It adds no new features, but is nevertheless a minor release
6783 because it changes which tests are likely to fail health checks.
6784 The most noticeable effect is that some tests that used to fail health
6786 checks will now pass, and some that used to pass will fail. These
6787 should all be improvements in accuracy. In particular:
6788 · New failures will usually be because they are now taking into account
6790 things like use of data() and assume() inside the test body.
6791 · New failures may also be because for some classes of example the way
6793 data generation performance was measured was artificially faster than
6794 real data generation (for most examples that are hitting performance
6795 health checks the opposite should be the case).
6796 · Tests that used to fail health checks and now pass do so because the
6798 health check system used to run in a way that was subtly different
6799 than the main Hypothesis data generation and lacked some of its sup‐
6800 port for e.g. large examples.
6801 If your data generation is especially slow, you may also see your tests
6803 get somewhat faster, as there is no longer a separate health check
6804 phase. This will be particularly noticeable when rerunning test fail‐
6805 ures.
6806 This work was funded by Smarkets.
6808 3.37.0 - 2017-11-12
6810 This is a deprecation release for some health check related features.
6811 The following are now deprecated:
6813 · Passing HealthCheck.exception_in_generation to suppress_health_check.
6815 This no longer does anything even when passed - All errors that
6816 occur during data generation will now be immediately reraised rather
6817 than going through the health check mechanism.
6818 · Passing HealthCheck.random_module to suppress_health_check. This
6820 hasn't done anything for a long time, but was never explicitly depre‐
6821 cated. Hypothesis always seeds the random module when running @given
6822 tests, so this is no longer an error and suppressing it doesn't do
6823 anything.
6824 · Passing non-HealthCheck values in suppress_health_check. This was
6826 previously allowed but never did anything useful.
6827 In addition, passing a non-iterable value as suppress_health_check will
6829 now raise an error immediately (it would never have worked correctly,
6830 but it would previously have failed later). Some validation error mes‐
6831 sages have also been updated.
6832 This work was funded by Smarkets.
6834 3.36.1 - 2017-11-10
6836 This is a yak shaving release, mostly concerned with our own tests.
6837 While getfullargspec() was documented as deprecated in Python 3.5, it
6839 never actually emitted a warning. Our code to silence this (nonexis‐
6840 tent) warning has therefore been removed.
6841 We now run our tests with DeprecationWarning as an error, and made some
6843 minor changes to our own tests as a result. This required similar
6844 upstream updates to coverage and execnet (a test-time dependency via
6845 pytest-xdist).
6846 There is no user-visible change in Hypothesis itself, but we encourage
6848 you to consider enabling deprecations as errors in your own tests.
6849 3.36.0 - 2017-11-06
6851 This release adds a setting to the public API, and does some internal
6852 cleanup:
6853 · The derandomize setting is now documented (issue #890)
6855 · Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
6857 #953)
6858 · Documented the strict setting as deprecated, and updated the build so
6860 our docs always match deprecations in the code.
6861 3.35.0 - 2017-11-06
6863 This minor release supports constraining uuids() to generate a particu‐
6864 lar version of UUID (issue #721).
6865 Thanks to Dion Misic for this feature.
6867 3.34.1 - 2017-11-02
6869 This patch updates the documentation to suggest builds(callable)
6870 instead of just(callable()).
6871 3.34.0 - 2017-11-02
6873 Hypothesis now emits deprecation warnings if you apply @given more than
6874 once to a target.
6875 Applying @given repeatedly wraps the target multiple times. Each wrap‐
6877 per will search the space of of possible parameters separately. This
6878 is equivalent but will be much more inefficient than doing it with a
6879 single call to @given.
6880 For example, instead of @given(booleans()) @given(integers()), you
6882 could write @given(booleans(), integers())
6883 3.33.1 - 2017-11-02
6885 This is a bugfix release:
6886 · builds() would try to infer a strategy for required positional argu‐
6888 ments of the target from type hints, even if they had been given to
6889 builds() as positional arguments (issue #946). Now it only infers
6890 missing required arguments.
6891 · An internal introspection function wrongly reported self as a
6893 required argument for bound methods, which might also have affected
6894 builds(). Now it knows better.
6895 3.33.0 - 2017-10-16
6897 This release supports strategy inference for more Django field types -
6898 you can now omit an argument for Date, Time, Duration, Slug, IP
6899 Address, and UUID fields. (issue #642)
6900 Strategy generation for fields with grouped choices now selects choices
6902 from each group, instead of selecting from the group names.
6903 3.32.2 - 2017-10-15
6905 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
6906 an experimental basis. It has never actually worked, and the new
6907 Hypothesis example database is designed to make such a tool unneces‐
6908 sary.
6909 3.32.1 - 2017-10-13
6911 This patch has two improvements for strategies based on enumerations.
6912 · from_type() now handles enumerations correctly, delegating to sam‐
6914 pled_from(). Previously it noted that Enum.__init__ has no required
6915 arguments and therefore delegated to builds(), which would subse‐
6916 quently fail.
6917 · When sampling from an python:enum.Flag, we also generate combinations
6919 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
6920 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
6921 3.32.0 - 2017-10-09
6923 This changes the default value of the use_coverage setting to True when
6924 running on pypy (it was already True on CPython).
6925 It was previously set to False because we expected it to be too slow,
6927 but recent benchmarking shows that actually performance of the feature
6928 on pypy is fairly acceptable - sometimes it's slower than on CPython,
6929 sometimes it's faster, but it's generally within a factor of two either
6930 way.
6931 3.31.6 - 2017-10-08
6933 This patch improves the quality of strategies inferred from Numpy
6934 dtypes:
6935 · Integer dtypes generated examples with the upper half of their
6937 (non-sign) bits set to zero. The inferred strategies can now produce
6938 any representable integer.
6939 · Fixed-width unicode- and byte-string dtypes now cap the internal
6941 example length, which should improve example and shrink quality.
6942 · Numpy arrays can only store fixed-size strings internally, and allow
6944 shorter strings by right-padding them with null bytes. Inferred
6945 string strategies no longer generate such values, as they can never
6946 be retrieved from an array. This improves shrinking performance by
6947 skipping useless values.
6948 This has already been useful in Hypothesis - we found an overflow bug
6950 in our Pandas support, and as a result indexes() and range_indexes()
6951 now check that min_size and max_size are at least zero.
6952 3.31.5 - 2017-10-08
6954 This release fixes a performance problem in tests where the use_cover‐
6955 age setting is True.
6956 Tests experience a slow-down proportionate to the amount of code they
6958 cover. This is still the case, but the factor is now low enough that
6959 it should be unnoticeable. Previously it was large and became much
6960 larger in 3.30.4.
6961 3.31.4 - 2017-10-08
6963 from_type() failed with a very confusing error if passed a NewType() (‐
6964 issue #901). These psudeo-types are now unwrapped correctly, and
6965 strategy inference works as expected.
6966 3.31.3 - 2017-10-06
6968 This release makes some small optimisations to our use of coverage that
6969 should reduce constant per-example overhead. This is probably only
6970 noticeable on examples where the test itself is quite fast. On no-op
6971 tests that don't test anything you may see up to a fourfold speed
6972 increase (which is still significantly slower than without coverage).
6973 On more realistic tests the speed up is likely to be less than that.
6974 3.31.2 - 2017-09-30
6976 This release fixes some formatting and small typos/grammar issues in
6977 the documentation, specifically the page docs/settings.rst, and the
6978 inline docs for the various settings.
6979 3.31.1 - 2017-09-30
6981 This release improves the handling of deadlines so that they act better
6982 with the shrinking process. This fixes issue #892.
6983 This involves two changes:
6985 1. The deadline is raised during the initial generation and shrinking,
6987 and then lowered to the set value for final replay. This restricts
6988 our attention to examples which exceed the deadline by a more sig‐
6989 nificant margin, which increases their reliability.
6990 2. When despite the above a test still becomes flaky because it is sig‐
6992 nificantly faster on rerun than it was on its first run, the error
6993 message is now more explicit about the nature of this problem, and
6994 includes both the initial test run time and the new test run time.
6995 In addition, this release also clarifies the documentation of the dead‐
6997 line setting slightly to be more explicit about where it applies.
6998 This work was funded by Smarkets.
7000 3.31.0 - 2017-09-29
7002 This release blocks installation of Hypothesis on Python 3.3, which
7003 reached its end of life date on 2017-09-29.
7004 This should not be of interest to anyone but downstream maintainers -
7006 if you are affected, migrate to a secure version of Python as soon as
7007 possible or at least seek commercial support.
7008 3.30.4 - 2017-09-27
7010 This release makes several changes:
7011 1. It significantly improves Hypothesis's ability to use coverage
7013 information to find interesting examples.
7014 2. It reduces the default max_examples setting from 200 to 100. This
7016 takes advantage of the improved algorithm meaning fewer examples are
7017 typically needed to get the same testing and is sufficiently better
7018 at covering interesting behaviour, and offsets some of the perfor‐
7019 mance problems of running under coverage.
7020 3. Hypothesis will always try to start its testing with an example that
7022 is near minimized.
7023 The new algorithm for 1 also makes some changes to Hypothesis's low
7025 level data generation which apply even with coverage turned off. They
7026 generally reduce the total amount of data generated, which should
7027 improve test performance somewhat. Between this and 3 you should see a
7028 noticeable reduction in test runtime (how much so depends on your tests
7029 and how much example size affects their performance. On our benchmarks,
7030 where data generation dominates, we saw up to a factor of two perfor‐
7031 mance improvement, but it's unlikely to be that large.
7032 3.30.3 - 2017-09-25
7034 This release fixes some formatting and small typos/grammar issues in
7035 the documentation, specifically the page docs/details.rst, and some
7036 inline docs linked from there.
7037 3.30.2 - 2017-09-24
7039 This release changes Hypothesis's caching approach for functions in
7040 hypothesis.strategies. Previously it would have cached extremely
7041 aggressively and cache entries would never be evicted. Now it adopts a
7042 least-frequently used, least recently used key invalidation policy, and
7043 is somewhat more conservative about which strategies it caches.
7044 Workloads which create strategies based on dynamic values, e.g. by
7046 using flatmap or composite(), will use significantly less memory.
7047 3.30.1 - 2017-09-22
7049 This release fixes a bug where when running with the use_coverage=True
7050 setting inside an existing running instance of coverage, Hypothesis
7051 would frequently put files that the coveragerc excluded in the report
7052 for the enclosing coverage.
7053 3.30.0 - 2017-09-20
7055 This release introduces two new features:
7056 · When a test fails, either with a health check failure or a falsifying
7058 example, Hypothesis will print out a seed that led to that failure,
7059 if the test is not already running with a fixed seed. You can then
7060 recreate that failure using either the @seed decorator or (if you are
7061 running pytest) with --hypothesis-seed.
7062 · pytest users can specify a seed to use for @given based tests by
7064 passing the --hypothesis-seed command line argument.
7065 This work was funded by Smarkets.
7067 3.29.0 - 2017-09-19
7069 This release makes Hypothesis coverage aware. Hypothesis now runs all
7070 test bodies under coverage, and uses this information to guide its
7071 testing.
7072 The use_coverage setting can be used to disable this behaviour if you
7074 want to test code that is sensitive to coverage being enabled (either
7075 because of performance or interaction with the trace function).
7076 The main benefits of this feature are:
7078 · Hypothesis now observes when examples it discovers cover particular
7080 lines or branches and stores them in the database for later.
7081 · Hypothesis will make some use of this information to guide its explo‐
7083 ration of the search space and improve the examples it finds (this is
7084 currently used only very lightly and will likely improve signifi‐
7085 cantly in future releases).
7086 This also has the following side-effects:
7088 · Hypothesis now has an install time dependency on the coverage pack‐
7090 age.
7091 · Tests that are already running Hypothesis under coverage will likely
7093 get faster.
7094 · Tests that are not running under coverage now run their test bodies
7096 under coverage by default.
7097 This feature is only partially supported under pypy. It is signifi‐
7099 cantly slower than on CPython and is turned off by default as a result,
7100 but it should still work correctly if you want to use it.
7101 3.28.3 - 2017-09-18
7103 This release is an internal change that affects how Hypothesis handles
7104 calculating certain properties of strategies.
7105 The primary effect of this is that it fixes a bug where use of
7107 deferred() could sometimes trigger an internal assertion error. However
7108 the fix for this bug involved some moderately deep changes to how
7109 Hypothesis handles certain constructs so you may notice some additional
7110 knock-on effects.
7111 In particular the way Hypothesis handles drawing data from strategies
7113 that cannot generate any values has changed to bail out sooner than it
7114 previously did. This may speed up certain tests, but it is unlikely to
7115 make much of a difference in practice for tests that were not already
7116 failing with Unsatisfiable.
7117 3.28.2 - 2017-09-18
7119 This is a patch release that fixes a bug in the hypothesis.extra.pandas
7120 documentation where it incorrectly referred to column() instead of col‐
7121 umns().
7122 3.28.1 - 2017-09-16
7124 This is a refactoring release. It moves a number of internal uses of
7125 namedtuple() over to using attrs based classes, and removes a couple of
7126 internal namedtuple classes that were no longer in use.
7127 It should have no user visible impact.
7129 3.28.0 - 2017-09-15
7131 This release adds support for testing pandas via the hypothe‐
7132 sis.extra.pandas module.
7133 It also adds a dependency on attrs.
7135 This work was funded by Stripe.
7137 3.27.1 - 2017-09-14
7139 This release fixes some formatting and broken cross-references in the
7140 documentation, which includes editing docstrings - and thus a patch
7141 release.
7142 3.27.0 - 2017-09-13
7144 This release introduces a deadline setting to Hypothesis.
7145 When set this turns slow tests into errors. By default it is unset but
7147 will warn if you exceed 200ms, which will become the default value in a
7148 future release.
7149 This work was funded by Smarkets.
7151 3.26.0 - 2017-09-12
7153 Hypothesis now emits deprecation warnings if you are using the legacy
7154 SQLite example database format, or the tool for merging them. These
7155 were already documented as deprecated, so this doesn't change their
7156 deprecation status, only that we warn about it.
7157 3.25.1 - 2017-09-12
7159 This release fixes a bug with generating numpy datetime and timedelta
7160 types: When inferring the strategy from the dtype, datetime and
7161 timedelta dtypes with sub-second precision would always produce exam‐
7162 ples with one second resolution. Inferring a strategy from a time
7163 dtype will now always produce example with the same precision.
7164 3.25.0 - 2017-09-12
7166 This release changes how Hypothesis shrinks and replays examples to
7167 take into account that it can encounter new bugs while shrinking the
7168 bug it originally found. Previously it would end up replacing the orig‐
7169 inally found bug with the new bug and show you only that one. Now it is
7170 (often) able to recognise when two bugs are distinct and when it finds
7171 more than one will show both.
7172 3.24.2 - 2017-09-11
7174 This release removes the (purely internal and no longer useful) strat‐
7175 egy_test_suite function and the corresponding strategytests module.
7176 3.24.1 - 2017-09-06
7178 This release improves the reduction of examples involving floating
7179 point numbers to produce more human readable examples.
7180 It also has some general purpose changes to the way the minimizer works
7182 internally, which may see some improvement in quality and slow down of
7183 test case reduction in cases that have nothing to do with floating
7184 point numbers.
7185 3.24.0 - 2017-09-05
7187 Hypothesis now emits deprecation warnings if you use some_strat‐
7188 egy.example() inside a test function or strategy definition (this was
7189 never intended to be supported, but is sufficiently widespread that it
7190 warrants a deprecation path).
7191 3.23.3 - 2017-09-05
7193 This is a bugfix release for decimals() with the places argument.
7194 · No longer fails health checks (issue #725, due to internal filtering)
7196 · Specifying a min_value and max_value without any decimals with places
7198 places between them gives a more useful error message.
7199 · Works for any valid arguments, regardless of the decimal precision
7201 context.
7202 3.23.2 - 2017-09-01
7204 This is a small refactoring release that removes a now-unused parameter
7205 to an internal API. It shouldn't have any user visible effect.
7206 3.23.1 - 2017-09-01
7208 Hypothesis no longer propagates the dynamic scope of settings into
7209 strategy definitions.
7210 This release is a small change to something that was never part of the
7212 public API and you will almost certainly not notice any effect unless
7213 you're doing something surprising, but for example the following code
7214 will now give a different answer in some circumstances:
7215 import hypothesis.strategies as st
7217 from hypothesis import settings
7218 CURRENT_SETTINGS = st.builds(lambda: settings.default)
7220 (We don't actually encourage you writing code like this)
7222 Previously this would have generated the settings that were in effect
7224 at the point of definition of CURRENT_SETTINGS. Now it will generate
7225 the settings that are used for the current test.
7226 It is very unlikely to be significant enough to be visible, but you may
7228 also notice a small performance improvement.
7229 3.23.0 - 2017-08-31
7231 This release adds a unique argument to arrays() which behaves the same
7232 ways as the corresponding one for lists(), requiring all of the ele‐
7233 ments in the generated array to be distinct.
7234 3.22.2 - 2017-08-29
7236 This release fixes an issue where Hypothesis would raise a TypeError
7237 when using the datetime-related strategies if running with PYTHONOPTI‐
7238 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
7239 3.22.1 - 2017-08-28
7241 Hypothesis now transparently handles problems with an internal unicode
7242 cache, including file truncation or read-only filesystems (issue #767).
7243 Thanks to Sam Hames for the patch.
7244 3.22.0 - 2017-08-26
7246 This release provides what should be a substantial performance improve‐
7247 ment to numpy arrays generated using provided numpy support, and adds a
7248 new fill_value argument to arrays() to control this behaviour.
7249 This work was funded by Stripe.
7251 3.21.3 - 2017-08-26
7253 This release fixes some extremely specific circumstances that probably
7254 have never occurred in the wild where users of deferred() might have
7255 seen a python:RuntimeError from too much recursion, usually in cases
7256 where no valid example could have been generated anyway.
7257 3.21.2 - 2017-08-25
7259 This release fixes some minor bugs in argument validation:
7260 · hypothesis.extra.numpy dtype strategies would raise an internal
7262 error instead of an InvalidArgument exception when passed an
7263 invalid endianness specification.
7264 · fractions() would raise an internal error instead of an InvalidAr‐
7266 gument if passed float("nan") as one of its bounds.
7267 · The error message for passing float("nan") as a bound to various
7269 strategies has been improved.
7270 · Various bound arguments will now raise InvalidArgument in cases
7272 where they would previously have raised an internal TypeError or
7273 ValueError from the relevant conversion function.
7274 · streaming() would not have emitted a deprecation warning when
7276 called with an invalid argument.
7277 3.21.1 - 2017-08-24
7279 This release fixes a bug where test failures that were the result of an
7280 @example would print an extra stack trace before re-raising the excep‐
7281 tion.
7282 3.21.0 - 2017-08-23
7284 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
7285 sis's deprecation warnings into errors. Similar functionality can be
7286 achieved by using simplefilter('error', HypothesisDeprecationWarning).
7287 3.20.0 - 2017-08-22
7289 This release renames the relevant arguments on the datetimes(),
7290 dates(), times(), and timedeltas() strategies to min_value and
7291 max_value, to make them consistent with the other strategies in the
7292 module.
7293 The old argument names are still supported but will emit a deprecation
7295 warning when used explicitly as keyword arguments. Arguments passed
7296 positionally will go to the new argument names and are not deprecated.
7297 3.19.3 - 2017-08-22
7299 This release provides a major overhaul to the internals of how Hypothe‐
7300 sis handles shrinking.
7301 This should mostly be visible in terms of getting better examples for
7303 tests which make heavy use of composite(), data() or flatmap where the
7304 data drawn depends a lot on previous choices, especially where size
7305 parameters are affected. Previously Hypothesis would have struggled to
7306 reliably produce good examples here. Now it should do much better. Per‐
7307 formance should also be better for examples with a non-zero min_size.
7308 You may see slight changes to example generation (e.g. improved example
7310 diversity) as a result of related changes to internals, but they are
7311 unlikely to be significant enough to notice.
7312 3.19.2 - 2017-08-21
7314 This release fixes two bugs in hypothesis.extra.numpy:
7315 · unicode_string_dtypes() didn't work at all due to an incorrect dtype
7317 specifier. Now it does.
7318 · Various impossible conditions would have been accepted but would
7320 error when they fail to produced any example. Now they raise an
7321 explicit InvalidArgument error.
7322 3.19.1 - 2017-08-21
7324 This is a bugfix release for issue #739, where bounds for fractions()
7325 or floating-point decimals() were not properly converted to integers
7326 before passing them to the integers strategy. This excluded some val‐
7327 ues that should have been possible, and could trigger internal errors
7328 if the bounds lay between adjacent integers.
7329 You can now bound fractions() with two arbitrarily close fractions.
7331 It is now an explicit error to supply a min_value, max_value, and
7333 max_denominator to fractions() where the value bounds do not include a
7334 fraction with denominator at most max_denominator.
7335 3.19.0 - 2017-08-20
7337 This release adds the from_regex() strategy, which generates strings
7338 that contain a match of a regular expression.
7339 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
7341 then helping to upstream it! (issue #662)
7342 3.18.5 - 2017-08-18
7344 This is a bugfix release for integers(). Previously the strategy would
7345 hit an internal assertion if passed non-integer bounds for min_value
7346 and max_value that had no integers between them. The strategy now
7347 raises InvalidArgument instead.
7348 3.18.4 - 2017-08-18
7350 Release to fix a bug where mocks can be used as test runners under cer‐
7351 tain conditions. Specifically, if a mock is injected into a test via
7352 pytest fixtures or patch decorators, and that mock is the first argu‐
7353 ment in the list, hypothesis will think it represents self and turns
7354 the mock into a test runner. If this happens, the affected test always
7355 passes because the mock is executed instead of the test body. Some‐
7356 times, it will also fail health checks.
7357 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
7359 for this bug fix.
7360 3.18.3 - 2017-08-17
7362 This release should improve the performance of some tests which experi‐
7363 enced a slow down as a result of the 3.13.0 release.
7364 Tests most likely to benefit from this are ones that make extensive use
7366 of min_size parameters, but others may see some improvement as well.
7367 3.18.2 - 2017-08-16
7369 This release fixes a bug introduced in 3.18.0. If the arguments
7370 whitelist_characters and blacklist_characters to characters() contained
7371 overlapping elements, then an InvalidArgument exception would be
7372 raised.
7373 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
7375 3.18.1 - 2017-08-14
7377 This is a bug fix release to fix issue #780, where sets() and similar
7378 would trigger health check errors if their element strategy could only
7379 produce one element (e.g. if it was just()).
7380 3.18.0 - 2017-08-13
7382 This is a feature release:
7383 · characters() now accepts whitelist_characters, particular characters
7385 which will be added to those it produces. (issue #668)
7386 · A bug fix for the internal function _union_interval_lists(), and a
7388 rename to _union_intervals(). It now correctly handles all cases
7389 where intervals overlap, and it always returns the result as a tuple
7390 for tuples.
7391 Thanks to Alex Willmer for these.
7393 3.17.0 - 2017-08-07
7395 This release documents the previously undocumented phases feature, mak‐
7396 ing it part of the public API. It also updates how the example database
7397 is used. Principally:
7398 · A Phases.reuse argument will now correctly control whether examples
7400 from the database are run (it previously did exactly the wrong thing
7401 and controlled whether examples would be saved).
7402 · Hypothesis will no longer try to rerun all previously failing exam‐
7404 ples. Instead it will replay the smallest previously failing example
7405 and a selection of other examples that are likely to trigger any
7406 other bugs that will found. This prevents a previous failure from
7407 dominating your tests unnecessarily.
7408 · As a result of the previous change, Hypothesis will be slower about
7410 clearing out old examples from the database that are no longer fail‐
7411 ing (because it can only clear out ones that it actually runs).
7412 3.16.1 - 2017-08-07
7414 This release makes an implementation change to how Hypothesis handles
7415 certain internal constructs.
7416 The main effect you should see is improvement to the behaviour and per‐
7418 formance of collection types, especially ones with a min_size parame‐
7419 ter. Many cases that would previously fail due to being unable to gen‐
7420 erate enough valid examples will now succeed, and other cases should
7421 run slightly faster.
7422 3.16.0 - 2017-08-04
7424 This release introduces a deprecation of the timeout feature. This
7425 results in the following changes:
7426 · Creating a settings object with an explicit timeout will emit a dep‐
7428 recation warning.
7429 · If your test stops because it hits the timeout (and has not found a
7431 bug) then it will emit a deprecation warning.
7432 · There is a new value unlimited which you can import from hypothesis.
7434 settings(timeout=unlimited) will not cause a deprecation warning.
7435 · There is a new health check, hung_test, which will trigger after a
7437 test has been running for five minutes if it is not suppressed.
7438 3.15.0 - 2017-08-04
7440 This release deprecates two strategies, choices() and streaming().
7441 Both of these are somewhat confusing to use and are entirely redundant
7443 since the introduction of the data() strategy for interactive drawing
7444 in tests, and their use should be replaced with direct use of data()
7445 instead.
7446 3.14.2 - 2017-08-03
7448 This fixes a bug where Hypothesis would not work correctly on Python
7449 2.7 if you had the python:typing module backport installed.
7450 3.14.1 - 2017-08-02
7452 This raises the maximum depth at which Hypothesis starts cutting off
7453 data generation to a more reasonable value which it is harder to hit by
7454 accident.
7455 This resolves (issue #751), in which some examples which previously
7457 worked would start timing out, but it will also likely improve the data
7458 generation quality for complex data types.
7459 3.14.0 - 2017-07-23
7461 Hypothesis now understands inline type annotations (issue #293):
7462 · If the target of builds() has type annotations, a default strategy
7464 for missing required arguments is selected based on the type.
7465 Type-based strategy selection will only override a default if you
7466 pass hypothesis.infer as a keyword argument.
7467 · If @given wraps a function with type annotations, you can pass infer
7469 as a keyword argument and the appropriate strategy will be substi‐
7470 tuted.
7471 · You can check what strategy will be inferred for a type with the new
7473 from_type() function.
7474 · register_type_strategy() teaches Hypothesis which strategy to infer
7476 for custom or unknown types. You can provide a strategy, or for more
7477 complex cases a function which takes the type and returns a strategy.
7478 3.13.1 - 2017-07-20
7480 This is a bug fix release for issue #514 - Hypothesis would continue
7481 running examples after a SkipTest exception was raised, including
7482 printing a falsifying example. Skip exceptions from the standard
7483 python:unittest module, and pytest, nose, or unittest2 modules now
7484 abort the test immediately without printing output.
7485 3.13.0 - 2017-07-16
7487 This release has two major aspects to it: The first is the introduction
7488 of deferred(), which allows more natural definition of recursive
7489 (including mutually recursive) strategies.
7490 The second is a number of engine changes designed to support this sort
7492 of strategy better. These should have a knock-on effect of also improv‐
7493 ing the performance of any existing strategies that currently generate
7494 a lot of data or involve heavy nesting by reducing their typical exam‐
7495 ple size.
7496 3.12.0 - 2017-07-07
7498 This release makes some major internal changes to how Hypothesis repre‐
7499 sents data internally, as a prelude to some major engine changes that
7500 should improve data quality. There are no API changes, but it's a sig‐
7501 nificant enough internal change that a minor version bump seemed war‐
7502 ranted.
7503 User facing impact should be fairly mild, but includes:
7505 · All existing examples in the database will probably be invalidated.
7507 Hypothesis handles this automatically, so you don't need to do any‐
7508 thing, but if you see all your examples disappear that's why.
7509 · Almost all data distributions have changed significantly. Possibly
7511 for the better, possibly for the worse. This may result in new bugs
7512 being found, but it may also result in Hypothesis being unable to
7513 find bugs it previously did.
7514 · Data generation may be somewhat faster if your existing bottleneck
7516 was in draw_bytes (which is often the case for large examples).
7517 · Shrinking will probably be slower, possibly significantly.
7519 If you notice any effects you consider to be a significant regression,
7521 please open an issue about them.
7522 3.11.6 - 2017-06-19
7524 This release involves no functionality changes, but is the first to
7525 ship wheels as well as an sdist.
7526 3.11.5 - 2017-06-18
7528 This release provides a performance improvement to shrinking. For cases
7529 where there is some non-trivial "boundary" value (e.g. the bug happens
7530 for all values greater than some other value), shrinking should now be
7531 substantially faster. Other types of bug will likely see improvements
7532 too.
7533 This may also result in some changes to the quality of the final exam‐
7535 ples - it may sometimes be better, but is more likely to get slightly
7536 worse in some edge cases. If you see any examples where this happens in
7537 practice, please report them.
7538 3.11.4 - 2017-06-17
7540 This is a bugfix release: Hypothesis now prints explicit examples when
7541 running in verbose mode. (issue #313)
7542 3.11.3 - 2017-06-11
7544 This is a bugfix release: Hypothesis no longer emits a warning if you
7545 try to use sampled_from() with python:collections.OrderedDict. (issue
7546 #688)
7547 3.11.2 - 2017-06-10
7549 This is a documentation release. Several outdated snippets have been
7550 updated or removed, and many cross-references are now hyperlinks.
7551 3.11.1 - 2017-05-28
7553 This is a minor ergonomics release. Tracebacks shown by pytest no
7554 longer include Hypothesis internals for test functions decorated with
7555 @given.
7556 3.11.0 - 2017-05-23
7558 This is a feature release, adding datetime-related strategies to the
7559 core strategies.
7560 timezones() allows you to sample pytz timezones from the Olsen data‐
7562 base. Use directly in a recipe for tz-aware datetimes, or compose with
7563 none() to allow a mix of aware and naive output.
7564 The new dates(), times(), datetimes(), and timedeltas() strategies are
7566 all constrained by objects of their type. This means that you can gen‐
7567 erate dates bounded by a single day (i.e. a single date), or datetimes
7568 constrained to the microsecond.
7569 times() and datetimes() take an optional timezones= argument, which
7571 defaults to none() for naive times. You can use our extra strategy
7572 based on pytz, or roll your own timezones strategy with dateutil or
7573 even the standard library.
7574 The old dates, times, and datetimes strategies in hypothe‐
7576 sis.extra.datetimes are deprecated in favor of the new core strategies,
7577 which are more flexible and have no dependencies.
7578 3.10.0 - 2017-05-22
7580 Hypothesis now uses python:inspect.getfullargspec() internally. On
7581 Python 2, there are no visible changes.
7582 On Python 3 @given and @composite now preserve PEP 3107 annotations on
7584 the decorated function. Keyword-only arguments are now either handled
7585 correctly (e.g. @composite), or caught in validation instead of
7586 silently discarded or raising an unrelated error later (e.g. @given).
7587 3.9.1 - 2017-05-22
7589 This is a bugfix release: the default field mapping for a DateTimeField
7590 in the Django extra now respects the USE_TZ setting when choosing a
7591 strategy.
7592 3.9.0 - 2017-05-19
7594 This is feature release, expanding the capabilities of the decimals()
7595 strategy.
7596 · The new (optional) places argument allows you to generate decimals
7598 with a certain number of places (e.g. cents, thousandths, satoshis).
7599 · If allow_infinity is None, setting min_bound no longer excludes posi‐
7601 tive infinity and setting max_value no longer excludes negative
7602 infinity.
7603 · All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
7605 True, or if allow_nan is None and min_value or max_value is None.
7606 · min_value and max_value may be given as decimal strings, e.g.
7608 "1.234".
7609 3.8.5 - 2017-05-16
7611 Hypothesis now imports python:sqlite3 when a SQLite database is used,
7612 rather than at module load, improving compatibility with Python imple‐
7613 mentations compiled without SQLite support (such as BSD or Jython).
7614 3.8.4 - 2017-05-16
7616 This is a compatibility bugfix release. sampled_from() no longer
7617 raises a deprecation warning when sampling from an python:enum.Enum, as
7618 all enums have a reliable iteration order.
7619 3.8.3 - 2017-05-09
7621 This release removes a version check for older versions of pytest when
7622 using the Hypothesis pytest plugin. The pytest plugin will now run
7623 unconditionally on all versions of pytest. This breaks compatibility
7624 with any version of pytest prior to 2.7.0 (which is more than two years
7625 old).
7626 The primary reason for this change is that the version check was a fre‐
7628 quent source of breakage when pytest change their versioning scheme. If
7629 you are not working on pytest itself and are not running a very old
7630 version of it, this release probably doesn't affect you.
7631 3.8.2 - 2017-04-26
7633 This is a code reorganisation release that moves some internal test
7634 helpers out of the main source tree so as to not have changes to them
7635 trigger releases in future.
7636 3.8.1 - 2017-04-26
7638 This is a documentation release. Almost all code examples are now
7639 doctests checked in CI, eliminating stale examples.
7640 3.8.0 - 2017-04-23
7642 This is a feature release, adding the iterables() strategy, equivalent
7643 to lists(...).map(iter) but with a much more useful repr. You can use
7644 this strategy to check that code doesn't accidentally depend on
7645 sequence properties such as indexing support or repeated iteration.
7646 3.7.4 - 2017-04-22
7648 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
7649 ture in the same test could cause the test to fail to fill the argu‐
7650 ments, and throw a TypeError.
7651 3.7.3 - 2017-04-21
7653 This release should include no user visible changes and is purely a
7654 refactoring release. This modularises the behaviour of the core given()
7655 function, breaking it up into smaller and more accessible parts, but
7656 its actual behaviour should remain unchanged.
7657 3.7.2 - 2017-04-21
7659 This reverts an undocumented change in 3.7.1 which broke installation
7660 on debian stable: The specifier for the hypothesis[django]
7661 extra_requires had introduced a wild card, which was not supported on
7662 the default version of pip.
7663 3.7.1 - 2017-04-21
7665 This is a bug fix and internal improvements release.
7666 · In particular Hypothesis now tracks a tree of where it has already
7668 explored. This allows it to avoid some classes of duplicate exam‐
7669 ples, and significantly improves the performance of shrinking failing
7670 examples by allowing it to skip some shrinks that it can determine
7671 can't possibly work.
7672 · Hypothesis will no longer seed the global random arbitrarily unless
7674 you have asked it to using random_module()
7675 · Shrinking would previously have not worked correctly in some special
7677 cases on Python 2, and would have resulted in suboptimal examples.
7678 3.7.0 - 2017-03-20
7680 This is a feature release.
7681 New features:
7683 · Rule based stateful testing now has an @invariant decorator that
7685 specifies methods that are run after init and after every step,
7686 allowing you to encode properties that should be true at all times.
7687 Thanks to Tom Prince for this feature.
7688 · The decimals() strategy now supports allow_nan and allow_infinity
7690 flags.
7691 · There are significantly more strategies available for numpy, includ‐
7693 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
7694 for this feature.
7695 · When using the data() strategy you can now add a label as an argument
7697 to draw(), which will be printed along with the value when an example
7698 fails. Thanks to Peter Inglesby for this feature.
7699 Bug fixes:
7701 · Bug fix: composite() now preserves functions' docstrings.
7703 · The build is now reproducible and doesn't depend on the path you
7705 build it from. Thanks to Chris Lamb for this feature.
7706 · numpy strategies for the void data type did not work correctly.
7708 Thanks to Zac Hatfield Dodds for this fix.
7709 There have also been a number of performance optimizations:
7711 · The permutations() strategy is now significantly faster to use for
7713 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
7714 · Shrinking of failing test cases should have got significantly faster
7716 in some circumstances where it was previously struggling for a long
7717 time.
7718 · Example generation now involves less indirection, which results in a
7720 small speedup in some cases (small enough that you won't really
7721 notice it except in pathological cases).
7722 3.6.1 - 2016-12-20
7724 This release fixes a dependency problem and makes some small behind the
7725 scenes improvements.
7726 · The fake-factory dependency was renamed to faker. If you were depend‐
7728 ing on it through hypothesis[django] or hypothesis[fake-factory]
7729 without pinning it yourself then it would have failed to install
7730 properly. This release changes it so that hypothesis[fakefactory]
7731 (which can now also be installed as hypothesis[faker]) will install
7732 the renamed faker package instead.
7733 · This release also removed the dependency of hypothesis[django] on
7735 hypothesis[fakefactory] - it was only being used for emails. These
7736 now use a custom strategy that isn't from fakefactory. As a result
7737 you should also see performance improvements of tests which generated
7738 User objects or other things with email fields, as well as better
7739 shrinking of email addresses.
7740 · The distribution of code using nested calls to one_of() or the |
7742 operator for combining strategies has been improved, as branches are
7743 now flattened to give a more uniform distribution.
7744 · Examples using composite() or .flatmap should now shrink better. In
7746 particular this will affect things which work by first generating a
7747 length and then generating that many items, which have historically
7748 not shrunk very well.
7749 3.6.0 - 2016-10-31
7751 This release reverts Hypothesis to its old pretty printing of lambda
7752 functions based on attempting to extract the source code rather than
7753 decompile the bytecode. This is unfortunately slightly inferior in
7754 some cases and may result in you occasionally seeing things like lambda
7755 x: <unknown> in statistics reports and strategy reprs.
7756 This removes the dependencies on uncompyle6, xdis and spark-parser.
7758 The reason for this is that the new functionality was based on
7760 uncompyle6, which turns out to introduce a hidden GPLed dependency - it
7761 in turn depended on xdis, and although the library was licensed under
7762 the MIT license, it contained some GPL licensed source code and thus
7763 should have been released under the GPL.
7764 My interpretation is that Hypothesis itself was never in violation of
7766 the GPL (because the license it is under, the Mozilla Public License
7767 v2, is fully compatible with being included in a GPL licensed work),
7768 but I have not consulted a lawyer on the subject. Regardless of the
7769 answer to this question, adding a GPLed dependency will likely cause a
7770 lot of users of Hypothesis to inadvertently be in violation of the GPL.
7771 As a result, if you are running Hypothesis 3.5.x you really should
7773 upgrade to this release immediately.
7774 3.5.3 - 2016-10-05
7776 This is a bug fix release.
7777 Bugs fixed:
7779 · If the same test was running concurrently in two processes and there
7781 were examples already in the test database which no longer failed,
7782 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
7783 Python 2) because an example it was trying to read was deleted before
7784 it was read. (issue #372).
7785 · Drawing from an integers() strategy with both a min_value and a
7787 max_value would reject too many examples needlessly. Now it repeat‐
7788 edly redraws until satisfied. (pull request #366. Thanks to Calen
7789 Pennington for the contribution).
7790 3.5.2 - 2016-09-24
7792 This is a bug fix release.
7793 · The Hypothesis pytest plugin broke pytest support for doctests. Now
7795 it doesn't.
7796 3.5.1 - 2016-09-23
7798 This is a bug fix release.
7799 · Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
7801 bytes and unicode.
7802 · python:unittest.TestCase tests would not have shown up in the new
7804 statistics mode. Now they do.
7805 · Similarly, stateful tests would not have shown up in statistics and
7807 now they do.
7808 · Statistics now print with pytest node IDs (the names you'd get in
7810 pytest verbose mode).
7811 3.5.0 - 2016-09-22
7813 This is a feature release.
7814 · fractions() and decimals() strategies now support min_value and
7816 max_value parameters. Thanks go to Anne Mulhern for the development
7817 of this feature.
7818 · The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
7820 tics parameter that gives detailed statistics about the tests that
7821 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
7822 funding the development of this feature.
7823 · There is a new event() function that can be used to add custom sta‐
7825 tistics.
7826 Additionally there have been some minor bug fixes:
7828 · In some cases Hypothesis should produce fewer duplicate examples
7830 (this will mostly only affect cases with a single parameter).
7831 · pytest command line parameters are now under an option group for
7833 Hypothesis (thanks to David Keijser for fixing this)
7834 · Hypothesis would previously error if you used PEP 3107 function anno‐
7836 tations on your tests under Python 3.4.
7837 · The repr of many strategies using lambdas has been improved to
7839 include the lambda body (this was previously supported in many but
7840 not all cases).
7841 3.4.2 - 2016-07-13
7843 This is a bug fix release, fixing a number of problems with the set‐
7844 tings system:
7845 · Test functions defined using @given can now be called from other
7847 threads (issue #337)
7848 · Attempting to delete a settings property would previously have
7850 silently done the wrong thing. Now it raises an AttributeError.
7851 · Creating a settings object with a custom database_file parameter was
7853 silently getting ignored and the default was being used instead. Now
7854 it's not.
7855 3.4.1 - 2016-07-07
7857 This is a bug fix release for a single bug:
7858 · On Windows when running two Hypothesis processes in parallel (e.g.
7860 using pytest-xdist) they could race with each other and one would
7861 raise an exception due to the non-atomic nature of file renaming on
7862 Windows and the fact that you can't rename over an existing file.
7863 This is now fixed.
7864 3.4.0 - 2016-05-27
7866 This release is entirely provided by Lucas Wiman:
7867 Strategies constructed by the Django extra will now respect much more
7869 of Django's validations out of the box. Wherever possible,
7870 full_clean() should succeed.
7871 In particular:
7873 · The max_length, blank and choices kwargs are now respected.
7875 · Add support for DecimalField.
7877 · If a field includes validators, the list of validators are used to
7879 filter the field strategy.
7880 3.3.0 - 2016-05-27
7882 This release went wrong and is functionally equivalent to 3.2.0. Ignore
7883 it.
7884 3.2.0 - 2016-05-19
7886 This is a small single-feature release:
7887 · All tests using @given now fix the global random seed. This removes
7889 the health check for that. If a non-zero seed is required for the
7890 final falsifying example, it will be reported. Otherwise Hypothesis
7891 will assume randomization was not a significant factor for the test
7892 and be silent on the subject. If you use random_module() this will
7893 continue to work and will always display the seed.
7894 3.1.3 - 2016-05-01
7896 Single bug fix release
7897 · Another charmap problem. In 3.1.2 text() and characters() would break
7899 on systems which had /tmp mounted on a different partition than the
7900 Hypothesis storage directory (usually in home). This fixes that.
7901 3.1.2 - 2016-04-30
7903 Single bug fix release:
7904 · Anything which used a text() or characters() strategy was broken on
7906 Windows and I hadn't updated appveyor to use the new repository loca‐
7907 tion so I didn't notice. This is now fixed and windows support should
7908 work correctly.
7909 3.1.1 - 2016-04-29
7911 Minor bug fix release.
7912 · Fix concurrency issue when running tests that use text() from multi‐
7914 ple processes at once (issue #302, thanks to Alex Chan).
7915 · Improve performance of code using lists() with max_size (thanks to
7917 Cristi Cobzarenco).
7918 · Fix install on Python 2 with ancient versions of pip so that it
7920 installs the enum34 backport (thanks to Donald Stufft for telling me
7921 how to do this).
7922 · Remove duplicated __all__ exports from hypothesis.strategies (thanks
7924 to Piët Delport).
7925 · Update headers to point to new repository location.
7927 · Allow use of strategies that can't be used in find() (e.g. choices())
7929 in stateful testing.
7930 3.1.0 - 2016-03-06
7932 · Add a nothing() strategy that never successfully generates values.
7933 · sampled_from() and one_of() can both now be called with an empty
7935 argument list, in which case they also never generate any values.
7936 · one_of() may now be called with a single argument that is a collec‐
7938 tion of strategies as well as as varargs.
7939 · Add a runner() strategy which returns the instance of the current
7941 test object if there is one.
7942 · 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
7944 can be used as such.
7945 · Tests using RuleBasedStateMachine should now shrink significantly
7947 better.
7948 · Hypothesis now uses a pretty-printing library internally, compatible
7950 with IPython's pretty printing protocol (actually using the same
7951 code). This may improve the quality of output in some cases.
7952 · Add a 'phases' setting that allows more fine grained control over
7954 which parts of the process Hypothesis runs
7955 · Add a suppress_health_check setting which allows you to turn off spe‐
7957 cific health checks in a fine grained manner.
7958 · Fix a bug where lists of non fixed size would always draw one more
7960 element than they included. This mostly didn't matter, but if would
7961 cause problems with empty strategies or ones with side effects.
7962 · Add a mechanism to the Django model generator to allow you to explic‐
7964 itly request the default value (thanks to Jeremy Thurgood for this
7965 one).
7966 3.0.5 - 2016-02-25
7968 · Fix a bug where Hypothesis would now error on pytest development ver‐
7969 sions.
7970 3.0.4 - 2016-02-24
7972 · Fix a bug where Hypothesis would error when running on Python 2.7.3
7973 or earlier because it was trying to pass a python:bytearray object to
7974 python:struct.unpack() (which is only supported since 2.7.4).
7975 3.0.3 - 2016-02-23
7977 · Fix version parsing of pytest to work with pytest release candidates
7978 · More general handling of the health check problem where things could
7980 fail because of a cache miss - now one "free" example is generated
7981 before the start of the health check run.
7982 3.0.2 - 2016-02-18
7984 · Under certain circumstances, strategies involving text() buried
7985 inside some other strategy (e.g. text().filter(...) or recur‐
7986 sive(text(), ...)) would cause a test to fail its health checks the
7987 first time it ran. This was caused by having to compute some related
7988 data and cache it to disk. On travis or anywhere else where the
7989 .hypothesis directory was recreated this would have caused the tests
7990 to fail their health check on every run. This is now fixed for all
7991 the known cases, although there could be others lurking.
7992 3.0.1 - 2016-02-18
7994 · Fix a case where it was possible to trigger an "Unreachable" asser‐
7995 tion when running certain flaky stateful tests.
7996 · Improve shrinking of large stateful tests by eliminating a case where
7998 it was hard to delete early steps.
7999 · Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
8001 cantly by provide a custom implementation for fixed size blocks that
8002 can bypass a lot of machinery.
8003 · Set default home directory based on the current working directory at
8005 the point Hypothesis is imported, not whenever the function first
8006 happens to be called.
8007 3.0.0 - 2016-02-17
8009 Codename: This really should have been 2.1.
8010 Externally this looks like a very small release. It has one small
8012 breaking change that probably doesn't affect anyone at all (some behav‐
8013 iour that never really worked correctly is now outright forbidden) but
8014 necessitated a major version bump and one visible new feature.
8015 Internally this is a complete rewrite. Almost nothing other than the
8017 public API is the same.
8018 New features:
8020 · Addition of data() strategy which allows you to draw arbitrary data
8022 interactively within the test.
8023 · New "exploded" database format which allows you to more easily check
8025 the example database into a source repository while supporting merg‐
8026 ing.
8027 · Better management of how examples are saved in the database.
8029 · Health checks will now raise as errors when they fail. It was too
8031 easy to have the warnings be swallowed entirely.
8032 New limitations:
8034 · choices() and streaming() strategies may no longer be used with
8036 find(). Neither may data() (this is the change that necessitated a
8037 major version bump).
8038 Feature removal:
8040 · The ForkingTestCase executor has gone away. It may return in some
8042 more working form at a later date.
8043 Performance improvements:
8045 · A new model which allows flatmap, composite strategies and stateful
8047 testing to perform much better. They should also be more reliable.
8048 · Filtering may in some circumstances have improved significantly. This
8050 will help especially in cases where you have lots of values with
8051 individual filters on them, such as lists(x.filter(...)).
8052 · Modest performance improvements to the general test runner by avoid‐
8054 ing expensive operations
8055 In general your tests should have got faster. If they've instead got
8057 significantly slower, I'm interested in hearing about it.
8058 Data distribution:
8060 The data distribution should have changed significantly. This may
8062 uncover bugs the previous version missed. It may also miss bugs the
8063 previous version could have uncovered. Hypothesis is now producing less
8064 strongly correlated data than it used to, but the correlations are
8065 extended over more of the structure.
8066 Shrinking:
8068 Shrinking quality should have improved. In particular Hypothesis can
8070 now perform simultaneous shrinking of separate examples within a single
8071 test (previously it was only able to do this for elements of a single
8072 collection). In some cases performance will have improved, in some
8073 cases it will have got worse but generally shouldn't have by much.
8074 2.0.0 - 2016-01-10
8076 Codename: A new beginning
8077 This release cleans up all of the legacy that accrued in the course of
8079 Hypothesis 1.0. These are mostly things that were emitting deprecation
8080 warnings in 1.19.0, but there were a few additional changes.
8081 In particular:
8083 · non-strategy values will no longer be converted to strategies when
8085 used in given or find.
8086 · FailedHealthCheck is now an error and not a warning.
8088 · Handling of non-ascii reprs in user types have been simplified by
8090 using raw strings in more places in Python 2.
8091 · given no longer allows mixing positional and keyword arguments.
8093 · given no longer works with functions with defaults.
8095 · given no longer turns provided arguments into defaults - they will
8097 not appear in the argspec at all.
8098 · the basic() strategy no longer exists.
8100 · the n_ary_tree strategy no longer exists.
8102 · the average_list_length setting no longer exists. Note: If you're
8104 using using recursive() this will cause you a significant slow down.
8105 You should pass explicit average_size parameters to collections in
8106 recursive calls.
8107 · @rule can no longer be applied to the same method twice.
8109 · Python 2.6 and 3.3 are no longer officially supported, although in
8111 practice they still work fine.
8112 This also includes two non-deprecation changes:
8114 · given's keyword arguments no longer have to be the rightmost argu‐
8116 ments and can appear anywhere in the method signature.
8117 · The max_shrinks setting would sometimes not have been respected.
8119 1.19.0 - 2016-01-09
8121 Codename: IT COMES
8122 This release heralds the beginning of a new and terrible age of Hypoth‐
8124 esis 2.0.
8125 It's primary purpose is some final deprecations prior to said release.
8127 The goal is that if your code emits no warnings under this release then
8128 it will probably run unchanged under Hypothesis 2.0 (there are some
8129 caveats to this: 2.0 will drop support for some Python versions, and if
8130 you're using internal APIs then as usual that may break without warn‐
8131 ing).
8132 It does have two new features:
8134 · New @seed() decorator which allows you to manually seed a test. This
8136 may be harmlessly combined with and overrides the derandomize set‐
8137 ting.
8138 · settings objects may now be used as a decorator to fix those settings
8140 to a particular @given test.
8141 API changes (old usage still works but is deprecated):
8143 · Settings has been renamed to settings (lower casing) in order to make
8145 the decorator usage more natural.
8146 · Functions for the storage directory that were in hypothesis.settings
8148 are now in a new hypothesis.configuration module.
8149 Additional deprecations:
8151 · the average_list_length setting has been deprecated in favour of
8153 being explicit.
8154 · the basic() strategy has been deprecated as it is impossible to sup‐
8156 port it under a Conjecture based model, which will hopefully be
8157 implemented at some point in the 2.x series.
8158 · the n_ary_tree strategy (which was never actually part of the public
8160 API) has been deprecated.
8161 · Passing settings or random as keyword arguments to given is depre‐
8163 cated (use the new functionality instead)
8164 Bug fixes:
8166 · No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
8168 tion in streaming() values.
8169 · When running in health check mode with non strict, don't print quite
8171 so many errors for an exception in reify.
8172 · When an assumption made in a test or a filter is flaky, tests will
8174 now raise Flaky instead of UnsatisfiedAssumption.
8175 1.18.1 - 2015-12-22
8177 Two behind the scenes changes:
8178 · Hypothesis will no longer write generated code to the file system.
8180 This will improve performance on some systems (e.g. if you're using
8181 PythonAnywhere which is running your code from NFS) and prevent some
8182 annoying interactions with auto-restarting systems.
8183 · Hypothesis will cache the creation of some strategies. This can sig‐
8185 nificantly improve performance for code that uses flatmap or compos‐
8186 ite and thus has to instantiate strategies a lot.
8187 1.18.0 - 2015-12-21
8189 Features:
8190 · Tests and find are now explicitly seeded off the global random mod‐
8192 ule. This means that if you nest one inside the other you will now
8193 get a health check error. It also means that you can control global
8194 randomization by seeding random.
8195 · There is a new random_module() strategy which seeds the global random
8197 module for you and handles things so that you don't get a health
8198 check warning if you use it inside your tests.
8199 · floats() now accepts two new arguments: allow_nan and allow_infinity.
8201 These default to the old behaviour, but when set to False will do
8202 what the names suggest.
8203 Bug fixes:
8205 · Fix a bug where tests that used text() on Python 3.4+ would not actu‐
8207 ally be deterministic even when explicitly seeded or using the deran‐
8208 domize mode, because generation depended on dictionary iteration
8209 order which was affected by hash randomization.
8210 · Fix a bug where with complicated strategies the timing of the initial
8212 health check could affect the seeding of the subsequent test, which
8213 would also render supposedly deterministic tests non-deterministic in
8214 some scenarios.
8215 · In some circumstances flatmap() could get confused by two struc‐
8217 turally similar things it could generate and would produce a flaky
8218 test where the first time it produced an error but the second time it
8219 produced the other value, which was not an error. The same bug was
8220 presumably also possible in composite().
8221 · flatmap() and composite() initial generation should now be moderately
8223 faster. This will be particularly noticeable when you have many val‐
8224 ues drawn from the same strategy in a single run, e.g. constructs
8225 like lists(s.flatmap(f)). Shrinking performance may have suffered,
8226 but this didn't actually produce an interestingly worse result in any
8227 of the standard scenarios tested.
8228 1.17.1 - 2015-12-16
8230 A small bug fix release, which fixes the fact that the 'note' function
8231 could not be used on tests which used the @example decorator to provide
8232 explicit examples.
8233 1.17.0 - 2015-12-15
8235 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
8236 because it contains the following additional change that was not
8237 intended to be in a patch release (it's perfectly stable, but is a
8238 larger change that should have required a minor version bump):
8239 · Hypothesis will now perform a series of "health checks" as part of
8241 running your tests. These detect and warn about some common error
8242 conditions that people often run into which wouldn't necessarily have
8243 caused the test to fail but would cause e.g. degraded performance or
8244 confusing results.
8245 1.16.1 - 2015-12-14
8247 Note: This release has been removed.
8248 A small bugfix release that allows bdists for Hypothesis to be built
8250 under 2.7 - the compat3.py file which had Python 3 syntax wasn't
8251 intended to be loaded under Python 2, but when building a bdist it was.
8252 In particular this would break running setup.py test.
8253 1.16.0 - 2015-12-08
8255 There are no public API changes in this release but it includes a be‐
8256 haviour change that I wasn't comfortable putting in a patch release.
8257 · Functions from hypothesis.strategies will no longer raise InvalidAr‐
8259 gument on bad arguments. Instead the same errors will be raised when
8260 a test using such a strategy is run. This may improve startup time in
8261 some cases, but the main reason for it is so that errors in strate‐
8262 gies won't cause errors in loading, and it can interact correctly
8263 with things like pytest.mark.skipif.
8264 · Errors caused by accidentally invoking the legacy API are now much
8266 less confusing, although still throw NotImplementedError.
8267 · hypothesis.extra.django is 1.9 compatible.
8269 · When tests are run with max_shrinks=0 this will now still rerun the
8271 test on failure and will no longer print "Trying example:" before
8272 each run. Additionally note() will now work correctly when used with
8273 max_shrinks=0.
8274 1.15.0 - 2015-11-24
8276 A release with two new features.
8277 · A 'characters' strategy for more flexible generation of text with
8279 particular character ranges and types, kindly contributed by
8280 Alexander Shorin.
8281 · Add support for preconditions to the rule based stateful testing.
8283 Kindly contributed by Christopher Armstrong
8284 1.14.0 - 2015-11-01
8286 New features:
8287 · Add 'note' function which lets you include additional information in
8289 the final test run's output.
8290 · Add 'choices' strategy which gives you a choice function that emu‐
8292 lates random.choice.
8293 · Add 'uuid' strategy that generates UUIDs'
8295 · Add 'shared' strategy that lets you create a strategy that just gen‐
8297 erates a single shared value for each test run
8298 Bugs:
8300 · Using strategies of the form streaming(x.flatmap(f)) with find or in
8302 stateful testing would have caused InvalidArgument errors when the
8303 resulting values were used (because code that expected to only be
8304 called within a test context would be invoked).
8305 1.13.0 - 2015-10-29
8307 This is quite a small release, but deprecates some public API functions
8308 and removes some internal API functionality so gets a minor version
8309 bump.
8310 · All calls to the 'strategy' function are now deprecated, even ones
8312 which pass just a SearchStrategy instance (which is still a no-op).
8313 · Never documented hypothesis.extra entry_points mechanism has now been
8315 removed ( it was previously how hypothesis.extra packages were loaded
8316 and has been deprecated and unused for some time)
8317 · Some corner cases that could previously have produced an OverflowEr‐
8319 ror when simplifying failing cases using hypothesis.extra.datetimes
8320 (or dates or times) have now been fixed.
8321 · Hypothesis load time for first import has been significantly reduced
8323 - it used to be around 250ms (on my SSD laptop) and now is around
8324 100-150ms. This almost never matters but was slightly annoying when
8325 using it in the console.
8326 · hypothesis.strategies.randoms was previously missing from __all__.
8328 1.12.0 - 2015-10-18
8330 · Significantly improved performance of creating strategies using the
8331 functions from the hypothesis.strategies module by deferring the cal‐
8332 culation of their repr until it was needed. This is unlikely to have
8333 been an performance issue for you unless you were using flatmap, com‐
8334 posite or stateful testing, but for some cases it could be quite a
8335 significant impact.
8336 · A number of cases where the repr of strategies build from lambdas is
8338 improved
8339 · Add dates() and times() strategies to hypothesis.extra.datetimes
8341 · Add new 'profiles' mechanism to the settings system
8343 · Deprecates mutability of Settings, both the Settings.default top
8345 level property and individual settings.
8346 · A Settings object may now be directly initialized from a parent Set‐
8348 tings.
8349 · @given should now give a better error message if you attempt to use
8351 it with a function that uses destructuring arguments (it still won't
8352 work, but it will error more clearly),
8353 · A number of spelling corrections in error messages
8355 · pytest should no longer display the intermediate modules Hypothesis
8357 generates when running in verbose mode
8358 · Hypothesis should now correctly handle printing objects with
8360 non-ascii reprs on python 3 when running in a locale that cannot han‐
8361 dle ascii printing to stdout.
8362 · Add a unique=True argument to lists(). This is equivalent to
8364 unique_by=lambda x: x, but offers a more convenient syntax.
8365 1.11.4 - 2015-09-27
8367 · Hide modifications Hypothesis needs to make to sys.path by undoing
8368 them after we've imported the relevant modules. This is a workaround
8369 for issues cryptography experienced on windows.
8370 · Slightly improved performance of drawing from sampled_from on large
8372 lists of alternatives.
8373 · Significantly improved performance of drawing from one_of or strate‐
8375 gies using | (note this includes a lot of strategies internally -
8376 floats() and integers() both fall into this category). There turned
8377 out to be a massive performance regression introduced in 1.10.0
8378 affecting these which probably would have made tests using Hypothesis
8379 significantly slower than they should have been.
8380 1.11.3 - 2015-09-23
8382 · Better argument validation for datetimes() strategy - previously set‐
8383 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
8384 would not have raised an InvalidArgument error and instead would have
8385 behaved confusingly.
8386 · Compatibility with being run on pytest < 2.7 (achieved by disabling
8388 the plugin).
8389 1.11.2 - 2015-09-23
8391 Bug fixes:
8392 · Settings(database=my_db) would not be correctly inherited when used
8394 as a default setting, so that newly created settings would use the
8395 database_file setting and create an SQLite example database.
8396 · Settings.default.database = my_db would previously have raised an
8398 error and now works.
8399 · Timeout could sometimes be significantly exceeded if during simplifi‐
8401 cation there were a lot of examples tried that didn't trigger the
8402 bug.
8403 · When loading a heavily simplified example using a basic() strategy
8405 from the database this could cause Python to trigger a recursion
8406 error.
8407 · Remove use of deprecated API in pytest plugin so as to not emit warn‐
8409 ing
8410 Misc:
8412 · hypothesis-pytest is now part of hypothesis core. This should have no
8414 externally visible consequences, but you should update your dependen‐
8415 cies to remove hypothesis-pytest and depend on only Hypothesis.
8416 · Better repr for hypothesis.extra.datetimes() strategies.
8418 · Add .close() method to abstract base class for Backend (it was
8420 already present in the main implementation).
8421 1.11.1 - 2015-09-16
8423 Bug fixes:
8424 · When running Hypothesis tests in parallel (e.g. using pytest-xdist)
8426 there was a race condition caused by code generation.
8427 · Example databases are now cached per thread so as to not use sqlite
8429 connections from multiple threads. This should make Hypothesis now
8430 entirely thread safe.
8431 · floats() with only min_value or max_value set would have had a very
8433 bad distribution.
8434 · Running on 3.5, Hypothesis would have emitted deprecation warnings
8436 because of use of inspect.getargspec
8437 1.11.0 - 2015-08-31
8439 · text() with a non-string alphabet would have used the repr() of the
8440 the alphabet instead of its contexts. This is obviously silly. It now
8441 works with any sequence of things convertible to unicode strings.
8442 · @given will now work on methods whose definitions contains no
8444 explicit positional arguments, only varargs (issue #118). This may
8445 have some knock on effects because it means that @given no longer
8446 changes the argspec of functions other than by adding defaults.
8447 · Introduction of new @composite feature for more natural definition of
8449 strategies you'd previously have used flatmap for.
8450 1.10.6 - 2015-08-26
8452 Fix support for fixtures on Django 1.7.
8453 1.10.4 - 2015-08-21
8455 Tiny bug fix release:
8456 · If the database_file setting is set to None, this would have resulted
8458 in an error when running tests. Now it does the same as setting data‐
8459 base to None.
8460 1.10.3 - 2015-08-19
8462 Another small bug fix release.
8463 · lists(elements, unique_by=some_function, min_size=n) would have
8465 raised a ValidationError if n > Settings.default.average_list_length
8466 because it would have wanted to use an average list length shorter
8467 than the minimum size of the list, which is impossible. Now it
8468 instead defaults to twice the minimum size in these circumstances.
8469 · basic() strategy would have only ever produced at most ten distinct
8471 values per run of the test (which is bad if you e.g. have it inside a
8472 list). This was obviously silly. It will now produce a much better
8473 distribution of data, both duplicated and non duplicated.
8474 1.10.2 - 2015-08-19
8476 This is a small bug fix release:
8477 · star imports from hypothesis should now work correctly.
8479 · example quality for examples using flatmap will be better, as the way
8481 it had previously been implemented was causing problems where Hypoth‐
8482 esis was erroneously labelling some examples as being duplicates.
8483 1.10.0 - 2015-08-04
8485 This is just a bugfix and performance release, but it changes some
8486 semi-public APIs, hence the minor version bump.
8487 · Significant performance improvements for strategies which are
8489 one_of() many branches. In particular this included recursive()
8490 strategies. This should take the case where you use one recursive()
8491 strategy as the base strategy of another from unusably slow (tens of
8492 seconds per generated example) to reasonably fast.
8493 · Better handling of just() and sampled_from() for values which have an
8495 incorrect __repr__ implementation that returns non-ASCII unicode on
8496 Python 2.
8497 · Better performance for flatmap from changing the internal morpher API
8499 to be significantly less general purpose.
8500 · Introduce a new semi-public BuildContext/cleanup API. This allows
8502 strategies to register cleanup activities that should run once the
8503 example is complete. Note that this will interact somewhat weirdly
8504 with find.
8505 · Better simplification behaviour for streaming strategies.
8507 · Don't error on lambdas which use destructuring arguments in Python 2.
8509 · Add some better reprs for a few strategies that were missing good
8511 ones.
8512 · The Random instances provided by randoms() are now copyable.
8514 · Slightly more debugging information about simplify when using a debug
8516 verbosity level.
8517 · Support using given for functions with varargs, but not passing argu‐
8519 ments to it as positional.
8520 1.9.0 - 2015-07-27
8522 Codename: The great bundling.
8523 This release contains two fairly major changes.
8525 The first is the deprecation of the hypothesis-extra mechanism. From
8527 now on all the packages that were previously bundled under it other
8528 than hypothesis-pytest (which is a different beast and will remain sep‐
8529 arate). The functionality remains unchanged and you can still import
8530 them from exactly the same location, they just are no longer separate
8531 packages.
8532 The second is that this introduces a new way of building strategies
8534 which lets you build up strategies recursively from other strategies.
8535 It also contains the minor change that calling .example() on a strategy
8537 object will give you examples that are more representative of the
8538 actual data you'll get. There used to be some logic in there to make
8539 the examples artificially simple but this proved to be a bad idea.
8540 1.8.5 - 2015-07-24
8542 This contains no functionality changes but fixes a mistake made with
8543 building the previous package that would have broken installation on
8544 Windows.
8545 1.8.4 - 2015-07-20
8547 Bugs fixed:
8548 · When a call to floats() had endpoints which were not floats but
8550 merely convertible to one (e.g. integers), these would be included in
8551 the generated data which would cause it to generate non-floats.
8552 · Splitting lambdas used in the definition of flatmap, map or filter
8554 over multiple lines would break the repr, which would in turn break
8555 their usage.
8556 1.8.3 - 2015-07-20
8558 "Falsifying example" would not have been printed when the failure came
8559 from an explicit example.
8560 1.8.2 - 2015-07-18
8562 Another small bugfix release:
8563 · When using ForkingTestCase you would usually not get the falsifying
8565 example printed if the process exited abnormally (e.g. due to
8566 os._exit).
8567 · Improvements to the distribution of characters when using text() with
8569 a default alphabet. In particular produces a better distribution of
8570 ascii and whitespace in the alphabet.
8571 1.8.1 - 2015-07-17
8573 This is a small release that contains a workaround for people who have
8574 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
8575 for Hypothesis per se because that's not a thing that is actually sup‐
8576 posed to work, but Hypothesis leans more heavily on repr than is typi‐
8577 cal so it's worth having a workaround for.
8578 1.8.0 - 2015-07-16
8580 New features:
8581 · Much more sensible reprs for strategies, especially ones that come
8583 from hypothesis.strategies. These should now have as reprs python
8584 code that would produce the same strategy.
8585 · lists() accepts a unique_by argument which forces the generated lists
8587 to be only contain elements unique according to some function key
8588 (which must return a hashable value).
8589 · Better error messages from flaky tests to help you debug things.
8591 Mostly invisible implementation details that may result in finding new
8593 bugs in your code:
8594 · Sets and dictionary generation should now produce a better range of
8596 results.
8597 · floats with bounds now focus more on 'critical values', trying to
8599 produce values at edge cases.
8600 · flatmap should now have better simplification for complicated cases,
8602 as well as generally being (I hope) more reliable.
8603 Bug fixes:
8605 · You could not previously use assume() if you were using the forking
8607 executor.
8608 1.7.2 - 2015-07-10
8610 This is purely a bug fix release:
8611 · When using floats() with stale data in the database you could some‐
8613 times get values in your tests that did not respect min_value or
8614 max_value.
8615 · When getting a Flaky error from an unreliable test it would have
8617 incorrectly displayed the example that caused it.
8618 · 2.6 dependency on backports was incorrectly specified. This would
8620 only have caused you problems if you were building a universal wheel
8621 from Hypothesis, which is not how Hypothesis ships, so unless you're
8622 explicitly building wheels for your dependencies and support Python
8623 2.6 plus a later version of Python this probably would never have
8624 affected you.
8625 · If you use flatmap in a way that the strategy on the right hand side
8627 depends sensitively on the left hand side you may have occasionally
8628 seen Flaky errors caused by producing unreliable examples when mini‐
8629 mizing a bug. This use case may still be somewhat fraught to be hon‐
8630 est. This code is due a major rearchitecture for 1.8, but in the
8631 meantime this release fixes the only source of this error that I'm
8632 aware of.
8633 1.7.1 - 2015-06-29
8635 Codename: There is no 1.7.0.
8636 A slight technical hitch with a premature upload means there's was a
8638 yanked 1.7.0 release. Oops.
8639 The major feature of this release is Python 2.6 support. Thanks to Jeff
8641 Meadows for doing most of the work there.
8642 Other minor features
8644 · strategies now has a permutations() function which returns a strategy
8646 yielding permutations of values from a given collection.
8647 · if you have a flaky test it will print the exception that it last saw
8649 before failing with Flaky, even if you do not have verbose reporting
8650 on.
8651 · Slightly experimental git merge script available as "python -m
8653 hypothesis.tools.mergedbs". Instructions on how to use it in the doc‐
8654 string of that file.
8655 Bug fixes:
8657 · Better performance from use of filter. In particular tests which
8659 involve large numbers of heavily filtered strategies should perform a
8660 lot better.
8661 · floats() with a negative min_value would not have worked correctly
8663 (worryingly, it would have just silently failed to run any examples).
8664 This is now fixed.
8665 · tests using sampled_from would error if the number of sampled ele‐
8667 ments was smaller than min_satisfying_examples.
8668 1.6.2 - 2015-06-08
8670 This is just a few small bug fixes:
8671 · Size bounds were not validated for values for a binary() strategy
8673 when reading examples from the database.
8674 · sampled_from is now in __all__ in hypothesis.strategies
8676 · floats no longer consider negative integers to be simpler than posi‐
8678 tive non-integers
8679 · Small floating point intervals now correctly count members, so if you
8681 have a floating point interval so narrow there are only a handful of
8682 values in it, this will no longer cause an error when Hypothesis runs
8683 out of values.
8684 1.6.1 - 2015-05-21
8686 This is a small patch release that fixes a bug where 1.6.0 broke the
8687 use of flatmap with the deprecated API and assumed the passed in func‐
8688 tion returned a SearchStrategy instance rather than converting it to a
8689 strategy.
8690 1.6.0 - 2015-05-21
8692 This is a smallish release designed to fix a number of bugs and smooth
8693 out some weird behaviours.
8694 · Fix a critical bug in flatmap where it would reuse old strategies. If
8696 all your flatmap code was pure you're fine. If it's not, I'm sur‐
8697 prised it's working at all. In particular if you want to use flatmap
8698 with django models, you desperately need to upgrade to this version.
8699 · flatmap simplification performance should now be better in some cases
8701 where it previously had to redo work.
8702 · Fix for a bug where invalid unicode data with surrogates could be
8704 generated during simplification (it was already filtered out during
8705 actual generation).
8706 · The Hypothesis database is now keyed off the name of the test instead
8708 of the type of data. This makes much more sense now with the new
8709 strategies API and is generally more robust. This means you will lose
8710 old examples on upgrade.
8711 · The database will now not delete values which fail to deserialize
8713 correctly, just skip them. This is to handle cases where multiple
8714 incompatible strategies share the same key.
8715 · find now also saves and loads values from the database, keyed off a
8717 hash of the function you're finding from.
8718 · Stateful tests now serialize and load values from the database. They
8720 should have before, really. This was a bug.
8721 · Passing a different verbosity level into a test would not have worked
8723 entirely correctly, leaving off some messages. This is now fixed.
8724 · Fix a bug where derandomized tests with unicode characters in the
8726 function body would error on Python 2.7.
8727 1.5.0 - 2015-05-14
8729 Codename: Strategic withdrawal.
8730 The purpose of this release is a radical simplification of the API for
8732 building strategies. Instead of the old approach of @strategy.extend
8733 and things that get converted to strategies, you just build strategies
8734 directly.
8735 The old method of defining strategies will still work until Hypothesis
8737 2.0, because it's a major breaking change, but will now emit depreca‐
8738 tion warnings.
8739 The new API is also a lot more powerful as the functions for defining
8741 strategies give you a lot of dials to turn. See the updated data sec‐
8742 tion for details.
8743 Other changes:
8745 · Mixing keyword and positional arguments in a call to @given is
8747 deprecated as well.
8748 · There is a new setting called 'strict'. When set to True, Hypothe‐
8750 sis will raise warnings instead of merely printing them. Turning
8751 it on by default is inadvisable because it means that Hypothesis
8752 minor releases can break your code, but it may be useful for mak‐
8753 ing sure you catch all uses of deprecated APIs.
8754 · max_examples in settings is now interpreted as meaning the maximum
8756 number of unique (ish) examples satisfying assumptions. A new set‐
8757 ting max_iterations which defaults to a larger value has the old
8758 interpretation.
8759 · Example generation should be significantly faster due to a new
8761 faster parameter selection algorithm. This will mostly show up for
8762 simple data types - for complex ones the parameter selection is
8763 almost certainly dominated.
8764 · Simplification has some new heuristics that will tend to cut down
8766 on cases where it could previously take a very long time.
8767 · timeout would previously not have been respected in cases where
8769 there were a lot of duplicate examples. You probably wouldn't have
8770 previously noticed this because max_examples counted duplicates,
8771 so this was very hard to hit in a way that mattered.
8772 · A number of internal simplifications to the SearchStrategy API.
8774 · You can now access the current Hypothesis version as hypothe‐
8776 sis.__version__.
8777 · A top level function is provided for running the stateful tests
8779 without the TestCase infrastructure.
8780 1.4.0 - 2015-05-04
8782 Codename: What a state.
8783 The big feature of this release is the new and slightly experimental
8785 stateful testing API. You can read more about that in the appropriate
8786 section.
8787 Two minor features the were driven out in the course of developing
8789 this:
8790 · You can now set settings.max_shrinks to limit the number of times
8792 Hypothesis will try to shrink arguments to your test. If this is set
8793 to <= 0 then Hypothesis will not rerun your test and will just raise
8794 the failure directly. Note that due to technical limitations if
8795 max_shrinks is <= 0 then Hypothesis will print every example it calls
8796 your test with rather than just the failing one. Note also that I
8797 don't consider settings max_shrinks to zero a sensible way to run
8798 your tests and it should really be considered a debug feature.
8799 · There is a new debug level of verbosity which is even more verbose
8801 than verbose. You probably don't want this.
8802 Breakage of semi-public SearchStrategy API:
8804 · It is now a required invariant of SearchStrategy that if u simplifies
8806 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
8807 plifying should not increase the complexity even though it is not
8808 required to decrease it. Enforcing this invariant lead to finding
8809 some bugs where simplifying of integers, floats and sets was subopti‐
8810 mal.
8811 · Integers in basic data are now required to fit into 64 bits. As a
8813 result python integer types are now serialized as strings, and some
8814 types have stopped using quite so needlessly large random seeds.
8815 Hypothesis Stateful testing was then turned upon Hypothesis itself,
8817 which lead to an amazing number of minor bugs being found in Hypothesis
8818 itself.
8819 Bugs fixed (most but not all from the result of stateful testing)
8821 include:
8822 · Serialization of streaming examples was flaky in a way that you would
8824 probably never notice: If you generate a template, simplify it, seri‐
8825 alize it, deserialize it, serialize it again and then deserialize it
8826 you would get the original stream instead of the simplified one.
8827 · If you reduced max_examples below the number of examples already
8829 saved in the database, you would have got a ValueError. Additionally,
8830 if you had more than max_examples in the database all of them would
8831 have been considered.
8832 · @given will no longer count duplicate examples (which it never called
8834 your function with) towards max_examples. This may result in your
8835 tests running slower, but that's probably just because they're trying
8836 more examples.
8837 · General improvements to example search which should result in better
8839 performance and higher quality examples. In particular parameters
8840 which have a history of producing useless results will be more
8841 aggressively culled. This is useful both because it decreases the
8842 chance of useless examples and also because it's much faster to not
8843 check parameters which we were unlikely to ever pick!
8844 · integers_from and lists of types with only one value (e.g. [None])
8846 would previously have had a very high duplication rate so you were
8847 probably only getting a handful of examples. They now have a much
8848 lower duplication rate, as well as the improvements to search making
8849 this less of a problem in the first place.
8850 · You would sometimes see simplification taking significantly longer
8852 than your defined timeout. This would happen because timeout was only
8853 being checked after each successful simplification, so if Hypothesis
8854 was spending a lot of time unsuccessfully simplifying things it
8855 wouldn't stop in time. The timeout is now applied for unsuccessful
8856 simplifications too.
8857 · In Python 2.7, integers_from strategies would have failed during sim‐
8859 plification with an OverflowError if their starting point was at or
8860 near to the maximum size of a 64-bit integer.
8861 · flatmap and map would have failed if called with a function without a
8863 __name__ attribute.
8864 · If max_examples was less than min_satisfying_examples this would
8866 always error. Now min_satisfying_examples is capped to max_examples.
8867 Note that if you have assumptions to satisfy here this will still
8868 cause an error.
8869 Some minor quality improvements:
8871 · Lists of streams, flatmapped strategies and basic strategies should
8873 now now have slightly better simplification.
8874 1.3.0 - 2015-05-22
8876 New features:
8877 · New verbosity level API for printing intermediate results and excep‐
8879 tions.
8880 · New specifier for strings generated from a specified alphabet.
8882 · Better error messages for tests that are failing because of a lack of
8884 enough examples.
8885 Bug fixes:
8887 · Fix error where use of ForkingTestCase would sometimes result in too
8889 many open files.
8890 · Fix error where saving a failing example that used flatmap could
8892 error.
8893 · Implement simplification for sampled_from, which apparently never
8895 supported it previously. Oops.
8896 General improvements:
8898 · Better range of examples when using one_of or sampled_from.
8900 · Fix some pathological performance issues when simplifying lists of
8902 complex values.
8903 · Fix some pathological performance issues when simplifying examples
8905 that require unicode strings with high codepoints.
8906 · Random will now simplify to more readable examples.
8908 1.2.1 - 2015-04-16
8910 A small patch release for a bug in the new executors feature. Tests
8911 which require doing something to their result in order to fail would
8912 have instead reported as flaky.
8913 1.2.0 - 2015-04-15
8915 Codename: Finders keepers.
8916 A bunch of new features and improvements.
8918 · Provide a mechanism for customizing how your tests are executed.
8920 · Provide a test runner that forks before running each example. This
8922 allows better support for testing native code which might trigger a
8923 segfault or a C level assertion failure.
8924 · Support for using Hypothesis to find examples directly rather than as
8926 just as a test runner.
8927 · New streaming type which lets you generate infinite lazily loaded
8929 streams of data - perfect for if you need a number of examples but
8930 don't know how many.
8931 · Better support for large integer ranges. You can now use inte‐
8933 gers_in_range with ranges of basically any size. Previously large
8934 ranges would have eaten up all your memory and taken forever.
8935 · Integers produce a wider range of data than before - previously they
8937 would only rarely produce integers which didn't fit into a machine
8938 word. Now it's much more common. This percolates to other numeric
8939 types which build on integers.
8940 · Better validation of arguments to @given. Some situations that would
8942 previously have caused silently wrong behaviour will now raise an
8943 error.
8944 · Include +/- sys.float_info.max in the set of floating point edge
8946 cases that Hypothesis specifically tries.
8947 · Fix some bugs in floating point ranges which happen when given +/-
8949 sys.float_info.max as one of the endpoints... (really any two floats
8950 that are sufficiently far apart so that x, y are finite but y - x is
8951 infinite). This would have resulted in generating infinite values
8952 instead of ones inside the range.
8953 1.1.1 - 2015-04-07
8955 Codename: Nothing to see here
8956 This is just a patch release put out because it fixed some internal
8958 bugs that would block the Django integration release but did not actu‐
8959 ally affect anything anyone could previously have been using. It also
8960 contained a minor quality fix for floats that I'd happened to have fin‐
8961 ished in time.
8962 · Fix some internal bugs with object lifecycle management that were
8964 impossible to hit with the previously released versions but broke
8965 hypothesis-django.
8966 · Bias floating point numbers somewhat less aggressively towards very
8968 small numbers
8969 1.1.0 - 2015-04-06
8971 Codename: No-one mention the M word.
8972 · Unicode strings are more strongly biased towards ascii characters.
8974 Previously they would generate all over the space. This is mostly so
8975 that people who try to shape their unicode strings with assume() have
8976 less of a bad time.
8977 · A number of fixes to data deserialization code that could theoreti‐
8979 cally have caused mysterious bugs when using an old version of a
8980 Hypothesis example database with a newer version. To the best of my
8981 knowledge a change that could have triggered this bug has never actu‐
8982 ally been seen in the wild. Certainly no-one ever reported a bug of
8983 this nature.
8984 · Out of the box support for Decimal and Fraction.
8986 · new dictionary specifier for dictionaries with variable keys.
8988 · Significantly faster and higher quality simplification, especially
8990 for collections of data.
8991 · New filter() and flatmap() methods on Strategy for better ways of
8993 building strategies out of other strategies.
8994 · New BasicStrategy class which allows you to define your own strate‐
8996 gies from scratch without needing an existing matching strategy or
8997 being exposed to the full horror or non-public nature of the Search‐
8998 Strategy interface.
8999 1.0.0 - 2015-03-27
9001 Codename: Blast-off!
9002 There are no code changes in this release. This is precisely the 0.9.2
9004 release with some updated documentation.
9005 0.9.2 - 2015-03-26
9007 Codename: T-1 days.
9008 · floats_in_range would not actually have produced floats_in_range
9010 unless that range happened to be (0, 1). Fix this.
9011 0.9.1 - 2015-03-25
9013 Codename: T-2 days.
9014 · Fix a bug where if you defined a strategy using map on a lambda then
9016 the results would not be saved in the database.
9017 · Significant performance improvements when simplifying examples using
9019 lists, strings or bounded integer ranges.
9020 0.9.0 - 2015-03-23
9022 Codename: The final countdown
9023 This release could also be called 1.0-RC1.
9025 It contains a teeny tiny bugfix, but the real point of this release is
9027 to declare feature freeze. There will be zero functionality changes
9028 between 0.9.0 and 1.0 unless something goes really really wrong. No new
9029 features will be added, no breaking API changes will occur, etc. This
9030 is the final shakedown before I declare Hypothesis stable and ready to
9031 use and throw a party to celebrate.
9032 Bug bounty for any bugs found between now and 1.0: I will buy you a
9034 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
9035 we ever find ourselves in the same city at the same time.
9036 The one tiny bugfix:
9038 · Under pypy, databases would fail to close correctly when garbage col‐
9040 lected, leading to a memory leak and a confusing error message if you
9041 were repeatedly creating databases and not closing them. It is very
9042 unlikely you were doing this and the chances of you ever having
9043 noticed this bug are very low.
9044 0.7.2 - 2015-03-22
9046 Codename: Hygienic macros or bust
9047 · You can now name an argument to @given 'f' and it won't break (issue
9049 #38)
9050 · strategy_test_suite is now named strategy_test_suite as the documen‐
9052 tation claims and not in fact strategy_test_suitee
9053 · Settings objects can now be used as a context manager to temporarily
9055 override the default values inside their context.
9056 0.7.1 - 2015-03-21
9058 Codename: Point releases go faster
9059 · Better string generation by parametrizing by a limited alphabet
9061 · Faster string simplification - previously if simplifying a string
9063 with high range unicode characters it would try every unicode charac‐
9064 ter smaller than that. This was pretty pointless. Now it stops after
9065 it's a short range (it can still reach smaller ones through recursive
9066 calls because of other simplifying operations).
9067 · Faster list simplification by first trying a binary chop down the
9069 middle
9070 · Simultaneous simplification of identical elements in a list. So if a
9072 bug only triggers when you have duplicates but you drew e.g. [-17,
9073 -17], this will now simplify to [0, 0].
9074 0.7.0, - 2015-03-20
9076 Codename: Starting to look suspiciously real
9077 This is probably the last minor release prior to 1.0. It consists of
9079 stability improvements, a few usability things designed to make Hypoth‐
9080 esis easier to try out, and filing off some final rough edges from the
9081 API.
9082 · Significant speed and memory usage improvements
9084 · Add an example() method to strategy objects to give an example of the
9086 sort of data that the strategy generates.
9087 · Remove .descriptor attribute of strategies
9089 · Rename descriptor_test_suite to strategy_test_suite
9091 · Rename the few remaining uses of descriptor to specifier (descriptor
9093 already has a defined meaning in Python)
9094 0.6.0 - 2015-03-13
9096 Codename: I'm sorry, were you using that API?
9097 This is primarily a "simplify all the weird bits of the API" release.
9099 As a result there are a lot of breaking changes. If you just use @given
9100 with core types then you're probably fine.
9101 In particular:
9103 · Stateful testing has been removed from the API
9105 · The way the database is used has been rendered less useful (sorry).
9107 The feature for reassembling values saved from other tests doesn't
9108 currently work. This will probably be brought back in post 1.0.
9109 · SpecificationMapper is no longer a thing. Instead there is an
9111 ExtMethod called strategy which you extend to specify how to convert
9112 other types to strategies.
9113 · Settings are now extensible so you can add your own for configuring a
9115 strategy
9116 · MappedSearchStrategy no longer needs an unpack method
9118 · Basically all the SearchStrategy internals have changed massively. If
9120 you implemented SearchStrategy directly rather than using Mapped‐
9121 SearchStrategy talk to me about fixing it.
9122 · Change to the way extra packages work. You now specify the package.
9124 This must have a load() method. Additionally any modules in the pack‐
9125 age will be loaded in under hypothesis.extra
9126 Bug fixes:
9128 · Fix for a bug where calling falsify on a lambda with a non-ascii
9130 character in its body would error.
9131 Hypothesis Extra:
9133 hypothesis-fakefactory: An extension for using faker data in hypothe‐
9135 sis. Depends
9136 on fake-factory.
9137 0.5.0 - 2015-02-10
9139 Codename: Read all about it.
9140 Core hypothesis:
9142 · Add support back in for pypy and python 3.2
9144 · @given functions can now be invoked with some arguments explicitly
9146 provided. If all arguments that hypothesis would have provided are
9147 passed in then no falsification is run.
9148 · Related to the above, this means that you can now use pytest fixtures
9150 and mark.parametrize with Hypothesis without either interfering with
9151 the other.
9152 · Breaking change: @given no longer works for functions with varargs
9154 (varkwargs are fine). This might be added back in at a later date.
9155 · Windows is now fully supported. A limited version (just the tests
9157 with none of the extras) of the test suite is run on windows with
9158 each commit so it is now a first class citizen of the Hypothesis
9159 world.
9160 · Fix a bug for fuzzy equality of equal complex numbers with different
9162 reprs (this can happen when one coordinate is zero). This shouldn't
9163 affect users - that feature isn't used anywhere public facing.
9164 · Fix generation of floats on windows and 32-bit builds of python. I
9166 was using some struct.pack logic that only worked on certain word
9167 sizes.
9168 · When a test times out and hasn't produced enough examples this now
9170 raises a Timeout subclass of Unfalsifiable.
9171 · Small search spaces are better supported. Previously something like a
9173 @given(bool, bool) would have failed because it couldn't find enough
9174 examples. Hypothesis is now aware of the fact that these are small
9175 search spaces and will not error in this case.
9176 · Improvements to parameter search in the case of hard to satisfy
9178 assume. Hypothesis will now spend less time exploring parameters that
9179 are unlikely to provide anything useful.
9180 · Increase chance of generating "nasty" floats
9182 · Fix a bug that would have caused unicode warnings if you had a sam‐
9184 pled_from that was mixing unicode and byte strings.
9185 · Added a standard test suite that you can use to validate a custom
9187 strategy you've defined is working correctly.
9188 Hypothesis extra:
9190 First off, introducing Hypothesis extra packages!
9192 These are packages that are separated out from core Hypothesis because
9194 they have one or more dependencies. Every hypothesis-extra package is
9195 pinned to a specific point release of Hypothesis and will have some
9196 version requirements on its dependency. They use entry_points so you
9197 will usually not need to explicitly import them, just have them
9198 installed on the path.
9199 This release introduces two of them:
9201 hypothesis-datetime:
9203 Does what it says on the tin: Generates datetimes for Hypothesis. Just
9205 install the package and datetime support will start working.
9206 Depends on pytz for timezone support
9208 hypothesis-pytest:
9210 A very rudimentary pytest plugin. All it does right now is hook the
9212 display of falsifying examples into pytest reporting.
9213 Depends on pytest.
9215 0.4.3 - 2015-02-05
9217 Codename: TIL narrow Python builds are a thing
9218 This just fixes the one bug.
9220 · Apparently there is such a thing as a "narrow python build" and OS X
9222 ships with these by default for python 2.7. These are builds where
9223 you only have two bytes worth of unicode. As a result, generating
9224 unicode was completely broken on OS X. Fix this by only generating
9225 unicode codepoints in the range supported by the system.
9226 0.4.2 - 2015-02-04
9228 Codename: O(dear)
9229 This is purely a bugfix release:
9231 · Provide sensible external hashing for all core types. This will sig‐
9233 nificantly improve performance of tracking seen examples which hap‐
9234 pens in literally every falsification run. For Hypothesis fixing this
9235 cut 40% off the runtime of the test suite. The behaviour is quadratic
9236 in the number of examples so if you're running the default configura‐
9237 tion this will be less extreme (Hypothesis's test suite runs at a
9238 higher number of examples than default), but you should still see a
9239 significant improvement.
9240 · Fix a bug in formatting of complex numbers where the string could get
9242 incorrectly truncated.
9243 0.4.1 - 2015-02-03
9245 Codename: Cruel and unusual edge cases
9246 This release is mostly about better test case generation.
9248 Enhancements:
9250 · Has a cool release name
9252 · text_type (str in python 3, unicode in python 2) example generation
9254 now actually produces interesting unicode instead of boring ascii
9255 strings.
9256 · floating point numbers are generated over a much wider range, with
9258 particular attention paid to generating nasty numbers - nan, infin‐
9259 ity, large and small values, etc.
9260 · examples can be generated using pieces of examples previously saved
9262 in the database. This allows interesting behaviour that has previ‐
9263 ously been discovered to be propagated to other examples.
9264 · improved parameter exploration algorithm which should allow it to
9266 more reliably hit interesting edge cases.
9267 · Timeout can now be disabled entirely by setting it to any value <= 0.
9269 Bug fixes:
9271 · The descriptor on a OneOfStrategy could be wrong if you had descrip‐
9273 tors which were equal but should not be coalesced. e.g. a strategy
9274 for one_of((frozenset({int}), {int})) would have reported its
9275 descriptor as {int}. This is unlikely to have caused you any problems
9276 · If you had strategies that could produce NaN (which float previously
9278 couldn't but e.g. a Just(float('nan')) could) then this would have
9279 sent hypothesis into an infinite loop that would have only been ter‐
9280 minated when it hit the timeout.
9281 · Given elements that can take a long time to minimize, minimization of
9283 floats or tuples could be quadratic or worse in the that value. You
9284 should now see much better performance for simplification, albeit at
9285 some cost in quality.
9286 Other:
9288 · A lot of internals have been been rewritten. This shouldn't affect
9290 you at all, but it opens the way for certain of hypothesis's oddities
9291 to be a lot more extensible by users. Whether this is a good thing
9292 may be up for debate...
9293 0.4.0 - 2015-01-21
9295 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
9296 stores data in a local SQLite database and will reuse it for all tests
9297 of the same type.
9298 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
9300 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
9301 ther notice. The previous license remains applicable to all code prior
9302 to 0.4.0.
9303 Enhancements:
9305 · Printing of failing examples. I was finding that the pytest runner
9307 was not doing a good job of displaying these, and that Hypothesis
9308 itself could do much better.
9309 · Drop dependency on six for cross-version compatibility. It was easy
9311 enough to write the shim for the small set of features that we care
9312 about and this lets us avoid a moderately complex dependency.
9313 · Some improvements to statistical distribution of selecting from small
9315 (<= 3 elements)
9316 · Improvements to parameter selection for finding examples.
9318 Bugs fixed:
9320 · could_have_produced for lists, dicts and other collections would not
9322 have examined the elements and thus when using a union of different
9323 types of list this could result in Hypothesis getting confused and
9324 passing a value to the wrong strategy. This could potentially result
9325 in exceptions being thrown from within simplification.
9326 · sampled_from would not work correctly on a single element list.
9328 · Hypothesis could get very confused by values which are equal despite
9330 having different types being used in descriptors. Hypothesis now has
9331 its own more specific version of equality it uses for descriptors and
9332 tracking. It is always more fine grained than Python equality: Things
9333 considered != are not considered equal by hypothesis, but some things
9334 that are considered == are distinguished. If your test suite uses
9335 both frozenset and set tests this bug is probably affecting you.
9336 0.3.2 - 2015-01-16
9338 · Fix a bug where if you specified floats_in_range with integer argu‐
9339 ments Hypothesis would error in example simplification.
9340 · Improve the statistical distribution of the floats you get for the
9342 floats_in_range strategy. I'm not sure whether this will affect users
9343 in practice but it took my tests for various conditions from flaky to
9344 rock solid so it at the very least improves discovery of the artifi‐
9345 cial cases I'm looking for.
9346 · Improved repr() for strategies and RandomWithSeed instances.
9348 · Add detection for flaky test cases where hypothesis managed to find
9350 an example which breaks it but on the final invocation of the test it
9351 does not raise an error. This will typically happen with too much
9352 recursion errors but could conceivably happen in other circumstances
9353 too.
9354 · Provide a "derandomized" mode. This allows you to run hypothesis with
9356 zero real randomization, making your build nice and deterministic.
9357 The tests run with a seed calculated from the function they're test‐
9358 ing so you should still get a good distribution of test cases.
9359 · Add a mechanism for more conveniently defining tests which just sam‐
9361 ple from some collection.
9362 · Fix for a really subtle bug deep in the internals of the strategy ta‐
9364 ble. In some circumstances if you were to define instance strategies
9365 for both a parent class and one or more of its subclasses you would
9366 under some circumstances get the strategy for the wrong superclass of
9367 an instance. It is very unlikely anyone has ever encountered this in
9368 the wild, but it is conceivably possible given that a mix of namedtu‐
9369 ple and tuple are used fairly extensively inside hypothesis which do
9370 exhibit this pattern of strategy.
9371 0.3.1 - 2015-01-13
9373 · Support for generation of frozenset and Random values
9374 · Correct handling of the case where a called function mutates it argu‐
9376 ment. This involved introducing a notion of a strategies knowing how
9377 to copy their argument. The default method should be entirely accept‐
9378 able and the worst case is that it will continue to have the old be‐
9379 haviour if you don't mark your strategy as mutable, so this shouldn't
9380 break anything.
9381 · Fix for a bug where some strategies did not correctly implement
9383 could_have_produced. It is very unlikely that any of these would have
9384 been seen in the wild, and the consequences if they had been would
9385 have been minor.
9386 · Re-export the @given decorator from the main hypothesis namespace.
9388 It's still available at the old location too.
9389 · Minor performance optimisation for simplifying long lists.
9391 0.3.0 - 2015-01-12
9393 · Complete redesign of the data generation system. Extreme breaking
9394 change for anyone who was previously writing their own SearchStrategy
9395 implementations. These will not work any more and you'll need to mod‐
9396 ify them.
9397 · New settings system allowing more global and modular control of Veri‐
9399 fier behaviour.
9400 · Decouple SearchStrategy from the StrategyTable. This leads to much
9402 more composable code which is a lot easier to understand.
9403 · A significant amount of internal API renaming and moving. This may
9405 also break your code.
9406 · Expanded available descriptors, allowing for generating integers or
9408 floats in a specific range.
9409 · Significantly more robust. A very large number of small bug fixes,
9411 none of which anyone is likely to have ever noticed.
9412 · Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
9414 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
9415 these to remain officially supported for a very long time. I would
9416 not be surprised to add pypy support back in later but I'm not going
9417 to do so until I know someone cares about it. In the meantime it will
9418 probably still work.
9419 0.2.2 - 2015-01-08
9421 · Fix an embarrassing complete failure of the installer caused by my
9422 being bad at version control
9423 0.2.1 - 2015-01-07
9425 · Fix a bug in the new stateful testing feature where you could make
9426 __init__ a @requires method. Simplification would not always work if
9427 the prune method was able to successfully shrink the test.
9428 0.2.0 - 2015-01-07
9430 · It's aliiive.
9431 · Improve python 3 support using six.
9433 · Distinguish between byte and unicode types.
9435 · Fix issues where FloatStrategy could raise.
9437 · Allow stateful testing to request constructor args.
9439 · Fix for issue where test annotations would timeout based on when the
9441 module was loaded instead of when the test started
9442 0.1.4 - 2013-12-14
9444 · Make verification runs time bounded with a configurable timeout
9445 0.1.3 - 2013-05-03
9447 · Bugfix: Stateful testing behaved incorrectly with subclassing.
9448 · Complex number support
9450 · support for recursive strategies
9452 · different error for hypotheses with unsatisfiable assumptions
9454 0.1.2 - 2013-03-24
9456 · Bugfix: Stateful testing was not minimizing correctly and could throw
9457 exceptions.
9458 · Better support for recursive strategies.
9460 · Support for named tuples.
9462 · Much faster integer generation.
9464 0.1.1 - 2013-03-24
9466 · Python 3.x support via 2to3.
9467 · Use new style classes (oops).
9469 0.1.0 - 2013-03-23
9471 · Introduce stateful testing.
9472 · Massive rewrite of internals to add flags and strategies.
9474 0.0.5 - 2013-03-13
9476 · No changes except trying to fix packaging
9477 0.0.4 - 2013-03-13
9479 · No changes except that I checked in a failing test case for 0.0.3 so
9480 had to replace the release. Doh
9481 0.0.3 - 2013-03-13
9483 · Improved a few internals.
9484 · Opened up creating generators from instances as a general API.
9486 · Test integration.
9488 0.0.2 - 2013-03-12
9490 · Starting to tighten up on the internals.
9491 · Change API to allow more flexibility in configuration.
9493 · More testing.
9495 0.0.1 - 2013-03-10
9497 · Initial release.
9498 · Basic working prototype. Demonstrates idea, probably shouldn't be
9500 used.
9501
9503 Hypothesis development is managed by me, David R. MacIver. I am the
9504 primary author of Hypothesis.
9505 However, I no longer do unpaid feature development on Hypothesis. My
9507 roles as leader of the project are:
9508 1. Helping other people do feature development on Hypothesis
9510 2. Fixing bugs and other code health issues
9512 3. Improving documentation
9514 4. General release management work
9516 5. Planning the general roadmap of the project
9518 6. Doing sponsored development on tasks that are too large or in depth
9520 for other people to take on
9521 So all new features must either be sponsored or implemented by someone
9523 else. That being said, the maintenance team takes an active role in
9524 shepherding pull requests and helping people write a new feature (see
9525 CONTRIBUTING.rst for details and pull request #154 for an example of
9526 how the process goes). This isn't "patches welcome", it's "we will help
9527 you write a patch".
9528 Release Policy
9530 Hypothesis releases follow semantic versioning.
9531 We maintain backwards-compatibility wherever possible, and use depreca‐
9533 tion warnings to mark features that have been superseded by a newer
9534 alternative. If you want to detect this, you can upgrade warnings to
9535 errors in the usual ways.
9536 We use continuous deployment to ensure that you can always use our new‐
9538 est and shiniest features - every change to the source tree is automat‐
9539 ically built and published on PyPI as soon as it's merged onto master,
9540 after code review and passing our extensive test suite.
9541 Project Roadmap
9543 Hypothesis does not have a long-term release plan. However some visi‐
9544 bility into our plans for future compatibility may be useful:
9545 · We value compatibility, and maintain it as far as practical. This
9547 generally excludes things which are end-of-life upstream, or have an
9548 unstable API.
9549 · We would like to drop Python 2 support when it reaches end of life in
9551 2020. Ongoing support is likely to depend on commercial funding.
9552
9554 For questions you are happy to ask in public, the Hypothesis community
9555 is a friendly place where I or others will be more than happy to help
9556 you out. You're also welcome to ask questions on Stack Overflow. If you
9557 do, please tag them with 'python-hypothesis' so someone sees them.
9558 For bugs and enhancements, please file an issue on the GitHub issue
9560 tracker. Note that as per the development policy, enhancements will
9561 probably not get implemented unless you're willing to pay for develop‐
9562 ment or implement them yourself (with assistance from me). Bugs will
9563 tend to get fixed reasonably promptly, though it is of course on a best
9564 effort basis.
9565 To see the versions of Python, optional dependencies, test runners, and
9567 operating systems Hypothesis supports (meaning incompatibility is
9568 treated as a bug), see supported.
9569 If you need to ask questions privately or want more of a guarantee of
9571 bugs being fixed promptly, please contact me on
9572 hypothesis-support@drmaciver.com to talk about availability of support
9573 contracts.
9574
9576 Downstream packagers often want to package Hypothesis. Here are some
9577 guidelines.
9578 The primary guideline is this: If you are not prepared to keep up with
9580 the Hypothesis release schedule, don't. You will annoy me and are doing
9581 your users a disservice.
9582 Hypothesis has a very frequent release schedule. It's rare that it goes
9584 a week without a release, and there are often multiple releases in a
9585 given week.
9586 If you are prepared to keep up with this schedule, you might find the
9588 rest of this document useful.
9589 Release tarballs
9591 These are available from the GitHub releases page. The tarballs on PyPI
9592 are intended for installation from a Python tool such as pip and should
9593 not be considered complete releases. Requests to include additional
9594 files in them will not be granted. Their absence is not a bug.
9595 Dependencies
9597 Python versions
9598 Hypothesis is designed to work with a range of Python versions. We
9599 always support all versisions of CPython with upstream support, and
9600 plan to drop Python 2 at EOL in 2020. We also support the latest ver‐
9601 sions of PyPy for Python 3, and for Python 2 until the CPython 2 EOL.
9602 If you feel the need to have separate Python 3 and Python 2 packages
9604 you can, but Hypothesis works unmodified on either.
9605 Other Python libraries
9607 Hypothesis has mandatory dependencies on the following libraries:
9608 · attrs
9610 · enum34 is required on Python 2.7
9612 Hypothesis has optional dependencies on the following libraries:
9614 · pytz (almost any version should work)
9616 · Django, all supported versions
9618 · numpy, 1.10 or later (earlier versions will probably work fine)
9620 · pandas, 1.19 or later
9622 · pytest (3.0 or greater). This is a mandatory dependency for testing
9624 Hypothesis itself but optional for users.
9625 The way this works when installing Hypothesis normally is that these
9627 features become available if the relevant library is installed.
9628 Testing Hypothesis
9630 If you want to test Hypothesis as part of your packaging you will prob‐
9631 ably not want to use the mechanisms Hypothesis itself uses for running
9632 its tests, because it has a lot of logic for installing and testing
9633 against different versions of Python.
9634 The tests must be run with pytest >= 3.0; check the requirements/
9636 directory for details.
9637 The organisation of the tests is described in the
9639 hypothesis-python/tests/README.rst.
9640 Examples
9642 · arch linux
9643 · fedora
9645 · gentoo
9647
9649 One of the things that is often concerning for people using randomized
9650 testing is the question of how to reproduce failing test cases.
9651 NOTE:
9653 It is better to think about the data Hypothesis generates as being
9654 arbitrary, rather than random. We deliberately generate any valid
9655 data that seems likely to cause errors, so you shouldn't rely on any
9656 expected distribution of or relationships between generated data.
9657 You can read about "swarm testing" and "coverage guided fuzzing" if
9658 you're interested, because you don't need to know for Hypothesis!
9659 Fortunately Hypothesis has a number of features to support reproducing
9661 test failures. The one you will use most commonly when developing
9662 locally is the example database, which means that you shouldn't have to
9663 think about the problem at all for local use - test failures will just
9664 automatically reproduce without you having to do anything.
9665 The example database is perfectly suitable for sharing between
9667 machines, but there currently aren't very good work flows for that, so
9668 Hypothesis provides a number of ways to make examples reproducible by
9669 adding them to the source code of your tests. This is particularly use‐
9670 ful when e.g. you are trying to run an example that has failed on your
9671 CI, or otherwise share them between machines.
9672 Providing explicit examples
9674 The simplest way to reproduce a failed test is to ask Hypothesis to run
9675 the failing example it printed. For example, if Falsifying example:
9676 test(n=1) was printed you can decorate test with @example(n=1).
9677 @example can also be used to ensure a specific example is always exe‐
9679 cuted as a regression test or to cover some edge case - basically com‐
9680 bining a Hypothesis test and a traditional parametrized test.
9681 hypothesis.example(*args, **kwargs)
9683 A decorator which ensures a specific example is always tested.
9684 Hypothesis will run all examples you've asked for first. If any of them
9686 fail it will not go on to look for more examples.
9687 It doesn't matter whether you put the example decorator before or after
9689 given. Any permutation of the decorators in the above will do the same
9690 thing.
9691 Note that examples can be positional or keyword based. If they're posi‐
9693 tional then they will be filled in from the right when calling, so
9694 either of the following styles will work as expected:
9695 @given(text())
9697 @example("Hello world")
9698 @example(x="Some very long string")
9699 def test_some_code(x):
9700 assert True
9701 from unittest import TestCase
9703 class TestThings(TestCase):
9705 @given(text())
9706 @example("Hello world")
9707 @example(x="Some very long string")
9708 def test_some_code(self, x):
9709 assert True
9710 As with @given, it is not permitted for a single example to be a mix of
9712 positional and keyword arguments. Either are fine, and you can use one
9713 in one example and the other in another example if for some reason you
9714 really want to, but a single example must be consistent.
9715 Reproducing a test run with @seed
9717 hypothesis.seed(seed)
9718 seed: Start the test execution from a specific seed.
9719 May be any hashable object. No exact meaning for seed is pro‐
9721 vided other than that for a fixed seed value Hypothesis will try
9722 the same actions (insofar as it can given external sources of
9723 non- determinism. e.g. timing and hash randomization).
9724 Overrides the derandomize setting, which is designed to enable
9726 deterministic builds rather than reproducing observed failures.
9727 When a test fails unexpectedly, usually due to a health check failure,
9729 Hypothesis will print out a seed that led to that failure, if the test
9730 is not already running with a fixed seed. You can then recreate that
9731 failure using either the @seed decorator or (if you are running pytest)
9732 with --hypothesis-seed.
9733 The seed will not be printed if you could simply use @example instead.
9735 Reproducing an example with @reproduce_failure
9737 Hypothesis has an opaque binary representation that it uses for all
9738 examples it generates. This representation is not intended to be stable
9739 across versions or with respect to changes in the test, but can be used
9740 to to reproduce failures with the @reproduce_example decorator.
9741 hypothesis.reproduce_failure(version, blob)
9743 Run the example that corresponds to this data blob in order to
9744 reproduce a failure.
9745 A test with this decorator always runs only one example and
9747 always fails. If the provided example does not cause a failure,
9748 or is in some way invalid for this test, then this will fail
9749 with a DidNotReproduce error.
9750 This decorator is not intended to be a permanent addition to
9752 your test suite. It's simply some code you can add to ease
9753 reproduction of a problem in the event that you don't have
9754 access to the test database. Because of this, no compatibility
9755 guarantees are made between different versions of Hypothesis -
9756 its API may change arbitrarily from version to version.
9757 The intent is that you should never write this decorator by hand, but
9759 it is instead provided by Hypothesis. When a test fails with a falsi‐
9760 fying example, Hypothesis may print out a suggestion to use @repro‐
9761 duce_failure on the test to recreate the problem as follows:
9762 >>> from hypothesis import settings, given, PrintSettings
9764 >>> import hypothesis.strategies as st
9765 >>> @given(st.floats())
9766 ... @settings(print_blob=PrintSettings.ALWAYS)
9767 ... def test(f):
9768 ... assert f == f
9769 ...
9770 >>> try:
9771 ... test()
9772 ... except AssertionError:
9773 ... pass
9774 Falsifying example: test(f=nan)
9775 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
9777 Adding the suggested decorator to the test should reproduce the failure
9779 (as long as everything else is the same - changing the versions of
9780 Python or anything else involved, might of course affect the behaviour
9781 of the test! Note that changing the version of Hypothesis will result
9782 in a different error - each @reproduce_failure invocation is specific
9783 to a Hypothesis version).
9784 When to do this is controlled by the print_blob setting, which may be
9786 one of the following values:
9787 class hypothesis.PrintSettings
9789 Flags to determine whether or not to print a detailed example
9790 blob to use with reproduce_failure() for failing test cases.
9791 NEVER = 0
9793 Never print a blob.
9794 INFER = 1
9796 Make an educated guess as to whether it would be appro‐
9797 priate to print the blob.
9798 The current rules are that this will print if:
9800 1. The output from Hypothesis appears to be unsuitable
9802 for use with example(), and
9803 2. The output is not too long, and
9805 3. Verbosity is at least normal.
9807 ALWAYS = 2
9809 Always print a blob on failure.
9810
9812 David R. MacIver
9813
9815 2013-2020, David R. MacIver
98164.23.8 Jan 30, 2020 HYPOTHESIS(1)
