1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property-based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the fu‐
41 ture.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 • Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 • If you delete an object, it is no longer visible.
53 • If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ""
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ""
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Let's see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127 Anyway, this test immediately finds a bug in the code:
129 Falsifying example: test_decode_inverts_encode(s='')
131 UnboundLocalError: local variable 'character' referenced before assignment
133 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136 If we fix that by just adding the following code to the beginning of
138 the function then Hypothesis tells us the code is correct (by doing
139 nothing as you'd expect a passing test to).
140 if not input_string:
142 return []
143 If we wanted to make sure this example was always checked we could add
145 it in explicitly by using the @example decorator:
146 from hypothesis import example, given, strategies as st
148 @given(st.text())
151 @example("")
152 def test_decode_inverts_encode(s):
153 assert decode(encode(s)) == s
154 This can be useful to show other developers (or your future self) what
156 kinds of data are valid inputs, or to ensure that particular edge cases
157 such as "" are tested every time. It's also great for regression tests
158 because although Hypothesis will remember failing examples, we don't
159 recommend distributing that database.
160 It's also worth noting that both example and given support keyword ar‐
162 guments as well as positional. The following would have worked just as
163 well:
164 @given(s=st.text())
166 @example(s="")
167 def test_decode_inverts_encode(s):
168 assert decode(encode(s)) == s
169 Suppose we had a more interesting bug and forgot to reset the count
171 each time. Say we missed a line in our encode method:
172 def encode(input_string):
174 count = 1
175 prev = ""
176 lst = []
177 for character in input_string:
178 if character != prev:
179 if prev:
180 entry = (prev, count)
181 lst.append(entry)
182 # count = 1 # Missing reset operation
183 prev = character
184 else:
185 count += 1
186 entry = (character, count)
187 lst.append(entry)
188 return lst
189 Hypothesis quickly informs us of the following example:
191 Falsifying example: test_decode_inverts_encode(s='001')
193 Note that the example provided is really quite simple. Hypothesis
195 doesn't just find any counter-example to your tests, it knows how to
196 simplify the examples it finds to produce small easy to understand
197 ones. In this case, two identical values are enough to set the count to
198 a number different from one, followed by another distinct value which
199 should have reset the count but in this case didn't.
200 The examples Hypothesis provides are valid Python code you can run. Any
202 arguments that you explicitly provide when calling the function are not
203 generated by Hypothesis, and if you explicitly provide all the argu‐
204 ments Hypothesis will just call the underlying function once rather
205 than running it multiple times.
206 Installing
208 Hypothesis is available on PyPI as "hypothesis". You can install it
209 with:
210 pip install hypothesis
212 You can install the dependencies for optional extensions with e.g. pip
214 install hypothesis[pandas,django].
215 If you want to install directly from the source code (e.g. because you
217 want to make changes and install the changed version), check out the
218 instructions in CONTRIBUTING.rst.
219 Running tests
221 In our example above we just let pytest discover and run our tests, but
222 we could also have run it explicitly ourselves:
223 if __name__ == "__main__":
225 test_decode_inverts_encode()
226 We could also have done this as a python:unittest.TestCase:
228 import unittest
230 class TestEncoding(unittest.TestCase):
233 @given(text())
234 def test_decode_inverts_encode(self, s):
235 self.assertEqual(decode(encode(s)), s)
236 if __name__ == "__main__":
239 unittest.main()
240 A detail: This works because Hypothesis ignores any arguments it hasn't
242 been told to provide (positional arguments start from the right), so
243 the self argument to the test is simply ignored and works as normal.
244 This also means that Hypothesis will play nicely with other ways of pa‐
245 rameterizing tests. e.g it works fine if you use pytest fixtures for
246 some arguments and Hypothesis for others.
247 Writing tests
249 A test in Hypothesis consists of two parts: A function that looks like
250 a normal test in your test framework of choice but with some additional
251 arguments, and a @given decorator that specifies how to provide those
252 arguments.
253 Here are some other examples of how you could use that:
255 from hypothesis import given, strategies as st
257 @given(st.integers(), st.integers())
260 def test_ints_are_commutative(x, y):
261 assert x + y == y + x
262 @given(x=st.integers(), y=st.integers())
265 def test_ints_cancel(x, y):
266 assert (x + y) - y == x
267 @given(st.lists(st.integers()))
270 def test_reversing_twice_gives_same_list(xs):
271 # This will generate lists of arbitrary length (usually between 0 and
272 # 100 elements) whose elements are integers.
273 ys = list(xs)
274 ys.reverse()
275 ys.reverse()
276 assert xs == ys
277 @given(st.tuples(st.booleans(), st.text()))
280 def test_look_tuples_work_too(t):
281 # A tuple is generated as the one you provided, with the corresponding
282 # types in those positions.
283 assert len(t) == 2
284 assert isinstance(t[0], bool)
285 assert isinstance(t[1], str)
286 Note that as we saw in the above example you can pass arguments to
288 @given either as positional or as keywords.
289 Where to start
291 You should now know enough of the basics to write some tests for your
292 code using Hypothesis. The best way to learn is by doing, so go have a
293 try.
294 If you're stuck for ideas for how to use this sort of test for your
296 code, here are some good starting points:
297 1. Try just calling functions with appropriate arbitrary data and see
299 if they crash. You may be surprised how often this works. e.g. note
300 that the first bug we found in the encoding example didn't even get
301 as far as our assertion: It crashed because it couldn't handle the
302 data we gave it, not because it did the wrong thing.
303 2. Look for duplication in your tests. Are there any cases where you're
305 testing the same thing with multiple different examples? Can you
306 generalise that to a single test using Hypothesis?
307 3. This piece is designed for an F# implementation, but is still very
309 good advice which you may find helps give you good ideas for using
310 Hypothesis.
311 If you have any trouble getting started, don't feel shy about asking
313 for help.
314
316 This is an account of slightly less common Hypothesis features that you
317 don't need to get started but will nevertheless make your life easier.
318 Additional test output
320 Normally the output of a failing test will look something like:
321 Falsifying example: test_a_thing(x=1, y="foo")
323 With the repr of each keyword argument being printed.
325 Sometimes this isn't enough, either because you have a value with a
327 __repr__() method that isn't very descriptive or because you need to
328 see the output of some intermediate steps of your test. That's where
329 the note function comes in:
330 hypothesis.note(value)
332 Report this value for the minimal failing example.
333 >>> from hypothesis import given, note, strategies as st
335 >>> @given(st.lists(st.integers()), st.randoms())
336 ... def test_shuffle_is_noop(ls, r):
337 ... ls2 = list(ls)
338 ... r.shuffle(ls2)
339 ... note(f"Shuffle: {ls2!r}")
340 ... assert ls == ls2
341 ...
342 >>> try:
343 ... test_shuffle_is_noop()
344 ... except AssertionError:
345 ... print("ls != ls2")
346 ...
347 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
348 Shuffle: [1, 0]
349 ls != ls2
350 The note is printed for the minimal failing example of the test in or‐
352 der to include any additional information you might need in your test.
353 Test statistics
355 If you are using pytest you can see a number of statistics about the
356 executed tests by passing the command line argument --hypothe‐
357 sis-show-statistics. This will include some general statistics about
358 the test:
359 For example if you ran the following with --hypothesis-show-statistics:
361 from hypothesis import given, strategies as st
363 @given(st.integers())
366 def test_integers(i):
367 pass
368 You would see:
370 - during generate phase (0.06 seconds):
372 - Typical runtimes: < 1ms, ~ 47% in data generation
373 - 100 passing examples, 0 failing examples, 0 invalid examples
374 - Stopped because settings.max_examples=100
375 The final "Stopped because" line is particularly important to note: It
377 tells you the setting value that determined when the test should stop
378 trying new examples. This can be useful for understanding the behaviour
379 of your tests. Ideally you'd always want this to be max_examples.
380 In some cases (such as filtered and recursive strategies) you will see
382 events mentioned which describe some aspect of the data generation:
383 from hypothesis import given, strategies as st
385 @given(st.integers().filter(lambda x: x % 2 == 0))
388 def test_even_integers(i):
389 pass
390 You would see something like:
392 test_even_integers:
394 - during generate phase (0.08 seconds):
396 - Typical runtimes: < 1ms, ~ 57% in data generation
397 - 100 passing examples, 0 failing examples, 12 invalid examples
398 - Events:
399 * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
400 * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
401 - Stopped because settings.max_examples=100
402 You can also mark custom events in a test using the event function:
404 hypothesis.event(value)
406 Record an event that occurred this test. Statistics on number of
407 test runs with each event will be reported at the end if you run
408 Hypothesis in statistics reporting mode.
409 Events should be strings or convertible to them.
411 from hypothesis import event, given, strategies as st
413 @given(st.integers().filter(lambda x: x % 2 == 0))
416 def test_even_integers(i):
417 event(f"i mod 3 = {i%3}")
418 You will then see output like:
420 test_even_integers:
422 - during generate phase (0.09 seconds):
424 - Typical runtimes: < 1ms, ~ 59% in data generation
425 - 100 passing examples, 0 failing examples, 32 invalid examples
426 - Events:
427 * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
428 * 31.06%, i mod 3 = 2
429 * 28.79%, i mod 3 = 0
430 * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
431 * 15.91%, i mod 3 = 1
432 - Stopped because settings.max_examples=100
433 Arguments to event can be any hashable type, but two events will be
435 considered the same if they are the same when converted to a string
436 with python:str.
437 Making assumptions
439 Sometimes Hypothesis doesn't give you exactly the right sort of data
440 you want - it's mostly of the right shape, but some examples won't work
441 and you don't want to care about them. You can just ignore these by
442 aborting the test early, but this runs the risk of accidentally testing
443 a lot less than you think you are. Also it would be nice to spend less
444 time on bad examples - if you're running 100 examples per test (the de‐
445 fault) and it turns out 70 of those examples don't match your needs,
446 that's a lot of wasted time.
447 hypothesis.assume(condition)
449 Calling assume is like an assert that marks the example as bad,
450 rather than failing the test.
451 This allows you to specify properties that you assume will be
453 true, and let Hypothesis try to avoid similar examples in fu‐
454 ture.
455 For example suppose you had the following test:
457 @given(floats())
459 def test_negation_is_self_inverse(x):
460 assert x == -(-x)
461 Running this gives us:
463 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
465 AssertionError
466 This is annoying. We know about NaN and don't really care about it, but
468 as soon as Hypothesis finds a NaN example it will get distracted by
469 that and tell us about it. Also the test will fail and we want it to
470 pass.
471 So let's block off this particular example:
473 from math import isnan
475 @given(floats())
478 def test_negation_is_self_inverse_for_non_nan(x):
479 assume(not isnan(x))
480 assert x == -(-x)
481 And this passes without a problem.
483 In order to avoid the easy trap where you assume a lot more than you
485 intended, Hypothesis will fail a test when it can't find enough exam‐
486 ples passing the assumption.
487 If we'd written:
489 @given(floats())
491 def test_negation_is_self_inverse_for_non_nan(x):
492 assume(False)
493 assert x == -(-x)
494 Then on running we'd have got the exception:
496 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
498 How good is assume?
500 Hypothesis has an adaptive exploration strategy to try to avoid things
501 which falsify assumptions, which should generally result in it still
502 being able to find examples in hard to find situations.
503 Suppose we had the following:
505 @given(lists(integers()))
507 def test_sum_is_positive(xs):
508 assert sum(xs) > 0
509 Unsurprisingly this fails and gives the falsifying example [].
511 Adding assume(xs) to this removes the trivial empty example and gives
513 us [0].
514 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
516 of positive integers is positive.
517 The reason that this should be surprising is not that it doesn't find a
519 counter-example, but that it finds enough examples at all.
520 In order to make sure something interesting is happening, suppose we
522 wanted to try this for long lists. e.g. suppose we added an as‐
523 sume(len(xs) > 10) to it. This should basically never find an example:
524 a naive strategy would find fewer than one in a thousand examples, be‐
525 cause if each element of the list is negative with probability
526 one-half, you'd have to have ten of these go the right way by chance.
527 In the default configuration Hypothesis gives up long before it's tried
528 1000 examples (by default it tries 200).
529 Here's what happens if we try to run this:
531 @given(lists(integers()))
533 def test_sum_is_positive(xs):
534 assume(len(xs) > 10)
535 assume(all(x > 0 for x in xs))
536 print(xs)
537 assert sum(xs) > 0
538 In: test_sum_is_positive()
540 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
542 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
543 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
544 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
545 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
546 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
547 As you can see, Hypothesis doesn't find many examples here, but it
549 finds some - enough to keep it happy.
550 In general if you can shape your strategies better to your tests you
552 should - for example integers(1, 1000) is a lot better than assume(1 <=
553 x <= 1000), but assume will take you a long way if you can't.
554 Defining strategies
556 The type of object that is used to explore the examples given to your
557 test function is called a SearchStrategy. These are created using the
558 functions exposed in the hypothesis.strategies module.
559 Many of these strategies expose a variety of arguments you can use to
561 customize generation. For example for integers you can specify min and
562 max values of integers you want. If you want to see exactly what a
563 strategy produces you can ask for an example:
564 >>> integers(min_value=0, max_value=10).example()
566 1
567 Many strategies are built out of other strategies. For example, if you
569 want to define a tuple you need to say what goes in each element:
570 >>> from hypothesis.strategies import tuples
572 >>> tuples(integers(), integers()).example()
573 (-24597, 12566)
574 Further details are available in a separate document.
576 The gory details of given parameters
578 hypothesis.given(*_given_arguments, **_given_kwargs)
579 A decorator for turning a test function that accepts arguments
580 into a randomized test.
581 This is the main entry point to Hypothesis.
583 The @given decorator may be used to specify which arguments of a func‐
585 tion should be parametrized over. You can use either positional or key‐
586 word arguments, but not a mixture of both.
587 For example all of the following are valid uses:
589 @given(integers(), integers())
591 def a(x, y):
592 pass
593 @given(integers())
596 def b(x, y):
597 pass
598 @given(y=integers())
601 def c(x, y):
602 pass
603 @given(x=integers())
606 def d(x, y):
607 pass
608 @given(x=integers(), y=integers())
611 def e(x, **kwargs):
612 pass
613 @given(x=integers(), y=integers())
616 def f(x, *args, **kwargs):
617 pass
618 class SomeTest(TestCase):
621 @given(integers())
622 def test_a_thing(self, x):
623 pass
624 The following are not:
626 @given(integers(), integers(), integers())
628 def g(x, y):
629 pass
630 @given(integers())
633 def h(x, *args):
634 pass
635 @given(integers(), x=integers())
638 def i(x, y):
639 pass
640 @given()
643 def j(x, y):
644 pass
645 The rules for determining what are valid uses of given are as follows:
647 1. You may pass any keyword argument to given.
649 2. Positional arguments to given are equivalent to the rightmost named
651 arguments for the test function.
652 3. Positional arguments may not be used if the underlying test function
654 has varargs, arbitrary keywords, or keyword-only arguments.
655 4. Functions tested with given may not have any defaults.
657 The reason for the "rightmost named arguments" behaviour is so that us‐
659 ing @given with instance methods works: self will be passed to the
660 function as normal and not be parametrized over.
661 The function returned by given has all the same arguments as the origi‐
663 nal test, minus those that are filled in by @given. Check the notes on
664 framework compatibility to see how this affects other testing libraries
665 you may be using.
666 Targeted example generation
668 Targeted property-based testing combines the advantages of both
669 search-based and property-based testing. Instead of being completely
670 random, T-PBT uses a search-based component to guide the input genera‐
671 tion towards values that have a higher probability of falsifying a
672 property. This explores the input space more effectively and requires
673 fewer tests to find a bug or achieve a high confidence in the system
674 being tested than random PBT. (Löscher and Sagonas)
675 This is not always a good idea - for example calculating the search
677 metric might take time better spent running more uniformly-random test
678 cases, or your target metric might accidentally lead Hypothesis away
679 from bugs - but if there is a natural metric like "floating-point er‐
680 ror", "load factor" or "queue length", we encourage you to experiment
681 with targeted testing.
682 hypothesis.target(observation, *, label='')
684 Calling this function with an int or float observation gives it
685 feedback with which to guide our search for inputs that will
686 cause an error, in addition to all the usual heuristics. Obser‐
687 vations must always be finite.
688 Hypothesis will try to maximize the observed value over several
690 examples; almost any metric will work so long as it makes sense
691 to increase it. For example, -abs(error) is a metric that in‐
692 creases as error approaches zero.
693 Example metrics:
695 • Number of elements in a collection, or tasks in a queue
697 • Mean or maximum runtime of a task (or both, if you use label)
699 • Compression ratio for data (perhaps per-algorithm or
701 per-level)
702 • Number of steps taken by a state machine
704 The optional label argument can be used to distinguish between
706 and therefore separately optimise distinct observations, such as
707 the mean and standard deviation of a dataset. It is an error to
708 call target() with any label more than once per test case.
709 NOTE:
711 The more examples you run, the better this technique works.
712 As a rule of thumb, the targeting effect is noticeable above
714 max_examples=1000, and immediately obvious by around ten
715 thousand examples per label used by your test.
716 Test statistics include the best score seen for each label,
718 which can help avoid the threshold problem when the minimal ex‐
719 ample shrinks right down to the threshold of failure (issue
720 #2180).
721 We recommend that users also skim the papers introducing targeted PBT;
723 from ISSTA 2017 and ICST 2018. For the curious, the initial implemen‐
724 tation in Hypothesis uses hill-climbing search via a mutating fuzzer,
725 with some tactics inspired by simulated annealing to avoid getting
726 stuck and endlessly mutating a local maximum.
727 Custom function execution
729 Hypothesis provides you with a hook that lets you control how it runs
730 examples.
731 This lets you do things like set up and tear down around each example,
733 run examples in a subprocess, transform coroutine tests into normal
734 tests, etc. For example, TransactionTestCase in the Django extra runs
735 each example in a separate database transaction.
736 The way this works is by introducing the concept of an executor. An ex‐
738 ecutor is essentially a function that takes a block of code and run it.
739 The default executor is:
740 def default_executor(function):
742 return function()
743 You define executors by defining a method execute_example on a class.
745 Any test methods on that class with @given used on them will use
746 self.execute_example as an executor with which to run tests. For exam‐
747 ple, the following executor runs all its code twice:
748 from unittest import TestCase
750 class TestTryReallyHard(TestCase):
753 @given(integers())
754 def test_something(self, i):
755 perform_some_unreliable_operation(i)
756 def execute_example(self, f):
758 f()
759 return f()
760 Note: The functions you use in map, etc. will run inside the executor.
762 i.e. they will not be called until you invoke the function passed to
763 execute_example.
764 An executor must be able to handle being passed a function which re‐
766 turns None, otherwise it won't be able to run normal test cases. So for
767 example the following executor is invalid:
768 from unittest import TestCase
770 class TestRunTwice(TestCase):
773 def execute_example(self, f):
774 return f()()
775 and should be rewritten as:
777 from unittest import TestCase
779 class TestRunTwice(TestCase):
782 def execute_example(self, f):
783 result = f()
784 if callable(result):
785 result = result()
786 return result
787 An alternative hook is provided for use by test runner extensions such
789 as pytest-trio, which cannot use the execute_example method. This is
790 not recommended for end-users - it is better to write a complete test
791 function directly, perhaps by using a decorator to perform the same
792 transformation before applying @given.
793 @given(x=integers())
795 @pytest.mark.trio
796 async def test(x):
797 ...
798 # Illustrative code, inside the pytest-trio plugin
801 test.hypothesis.inner_test = lambda x: trio.run(test, x)
802 For authors of test runners however, assigning to the inner_test attri‐
804 bute of the hypothesis attribute of the test will replace the interior
805 test.
806 NOTE:
808 The new inner_test must accept and pass through all the *args and
809 **kwargs expected by the original test.
810 If the end user has also specified a custom executor using the exe‐
812 cute_example method, it - and all other execution-time logic - will be
813 applied to the new inner test assigned by the test runner.
814 Making random code deterministic
816 While Hypothesis' example generation can be used for nondeterministic
817 tests, debugging anything nondeterministic is usually a very frustrat‐
818 ing exercise. To make things worse, our example shrinking relies on
819 the same input causing the same failure each time - though we show the
820 un-shrunk failure and a decent error message if it doesn't.
821 By default, Hypothesis will handle the global random and numpy.random
823 random number generators for you, and you can register others:
824 hypothesis.register_random(r)
826 Register the given Random instance for management by Hypothesis.
827 You can pass random.Random instances (or other objects with
829 seed, getstate, and setstate methods) to register_random(r) to
830 have their states seeded and restored in the same way as the
831 global PRNGs from the random and numpy.random modules.
832 All global PRNGs, from e.g. simulation or scheduling frameworks,
834 should be registered to prevent flaky tests. Hypothesis will
835 ensure that the PRNG state is consistent for all test runs, or
836 reproducibly varied if you choose to use the random_module()
837 strategy.
838 Inferred strategies
840 In some cases, Hypothesis can work out what to do when you omit argu‐
841 ments. This is based on introspection, not magic, and therefore has
842 well-defined limits.
843 builds() will check the signature of the target (using signature()).
845 If there are required arguments with type annotations and no strategy
846 was passed to builds(), from_type() is used to fill them in. You can
847 also pass the value ... (Ellipsis) as a keyword argument, to force this
848 inference for arguments with a default value.
849 >>> def func(a: int, b: str):
851 ... return [a, b]
852 ...
853 >>> builds(func).example()
854 [-6993, '']
855 hypothesis.infer
857 @given does not perform any implicit inference for required arguments,
859 as this would break compatibility with pytest fixtures. ...
860 (python:Ellipsis), can be used as a keyword argument to explicitly fill
861 in an argument from its type annotation. You can also use the hypothe‐
862 sis.infer alias if writing a literal ... seems too weird.
863 @given(a=...) # or @given(a=infer)
865 def test(a: int):
866 pass
867 # is equivalent to
870 @given(a=from_type(int))
871 def test(a):
872 pass
873 @given(...) can also be specified to fill all arguments from their type
875 annotations.
876 @given(...)
878 def test(a: int, b: str):
879 pass
880 # is equivalent to
883 @given(a=..., b=...)
884 def test(a, b):
885 pass
886 Limitations
888 Hypothesis does not inspect PEP 484 type comments at runtime. While
889 from_type() will work as usual, inference in builds() and @given will
890 only work if you manually create the __annotations__ attribute (e.g. by
891 using @annotations(...) and @returns(...) decorators).
892 The python:typing module changes between different Python releases, in‐
894 cluding at minor versions. These are all supported on a best-effort
895 basis, but you may encounter problems. Please report them to us, and
896 consider updating to a newer version of Python as a workaround.
897 Type annotations in Hypothesis
899 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
900 patible tool, the type checker should automatically pick up our type
901 hints.
902 NOTE:
904 Hypothesis' type hints may make breaking changes between minor re‐
905 leases.
906 Upstream tools and conventions about type hints remain in flux - for
908 example the python:typing module itself is provisional, and Mypy has
909 not yet reached version 1.0 - and we plan to support the latest ver‐
910 sion of this ecosystem, as well as older versions where practical.
911 We may also find more precise ways to describe the type of various
913 interfaces, or change their type and runtime behaviour together in a
914 way which is otherwise backwards-compatible. We often omit type
915 hints for deprecated features or arguments, as an additional form of
916 warning.
917 There are known issues inferring the type of examples generated by
919 deferred(), recursive(), one_of(), dictionaries(), and
920 fixed_dictionaries(). We will fix these, and require correspondingly
921 newer versions of Mypy for type hinting, as the ecosystem improves.
922 Writing downstream type hints
924 Projects that provide Hypothesis strategies and use type hints may wish
925 to annotate their strategies too. This is a supported use-case, again
926 on a best-effort provisional basis. For example:
927 def foo_strategy() -> SearchStrategy[Foo]:
929 ...
930 class hypothesis.strategies.SearchStrategy
932 SearchStrategy is the type of all strategy objects. It is a generic
934 type, and covariant in the type of the examples it creates. For exam‐
935 ple:
936 • integers() is of type SearchStrategy[int].
938 • lists(integers()) is of type SearchStrategy[List[int]].
940 • SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
942 a subtype of Animal (as seems likely).
943 WARNING:
945 SearchStrategy should only be used in type hints. Please do not in‐
946 herit from, compare to, or otherwise use it in any way outside of
947 type hints. The only supported way to construct objects of this
948 type is to use the functions provided by the hypothesis.strategies
949 module!
950 The Hypothesis pytest plugin
952 Hypothesis includes a tiny plugin to improve integration with pytest,
953 which is activated by default (but does not affect other test runners).
954 It aims to improve the integration between Hypothesis and Pytest by
955 providing extra information and convenient access to config options.
956 • pytest --hypothesis-show-statistics can be used to display test and
958 data generation statistics.
959 • pytest --hypothesis-profile=<profile name> can be used to load a set‐
961 tings profile. pytest --hypothesis-verbosity=<level name> can be
962 used to override the current verbosity level.
963 • pytest --hypothesis-seed=<an int> can be used to reproduce a failure
965 with a particular seed.
966 • pytest --hypothesis-explain can be used to temporarily enable the ex‐
968 plain phase.
969 Finally, all tests that are defined with Hypothesis automatically have
971 @pytest.mark.hypothesis applied to them. See here for information on
972 working with markers.
973 NOTE:
975 Pytest will load the plugin automatically if Hypothesis is in‐
976 stalled. You don't need to do anything at all to use it.
977 Use with external fuzzers
979 TIP:
980 Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
981 Use HypoFuzz to fuzz your whole test suite,
982 and find more bugs without more tests!
983 Sometimes, you might want to point a traditional fuzzer such as
986 python-afl, pythonfuzz, or Google's atheris (for Python and native ex‐
987 tensions) at your code. Wouldn't it be nice if you could use any of
988 your @given tests as fuzz targets, instead of converting bytestrings
989 into your objects by hand?
990 @given(st.text())
992 def test_foo(s):
993 ...
994 # This is a traditional fuzz target - call it with a bytestring,
997 # or a binary IO object, and it runs the test once.
998 fuzz_target = test_foo.hypothesis.fuzz_one_input
999 # For example:
1001 fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
1002 fuzz_target(io.BytesIO(...))
1003 Depending on the input to fuzz_one_input, one of three things will hap‐
1005 pen:
1006 • If the bytestring was invalid, for example because it was too short
1008 or failed a filter or assume() too many times, fuzz_one_input returns
1009 None.
1010 • If the bytestring was valid and the test passed, fuzz_one_input re‐
1012 turns a canonicalised and pruned buffer which will replay that test
1013 case. This is provided as an option to improve the performance of
1014 mutating fuzzers, but can safely be ignored.
1015 • If the test failed, i.e. raised an exception, fuzz_one_input will add
1017 the pruned buffer to the Hypothesis example database and then
1018 re-raise that exception. All you need to do to reproduce, minimize,
1019 and de-duplicate all the failures found via fuzzing is run your test
1020 suite!
1021 Note that the interpretation of both input and output bytestrings is
1023 specific to the exact version of Hypothesis you are using and the
1024 strategies given to the test, just like the example database and
1025 @reproduce_failure decorator.
1026 Interaction with settings
1028 fuzz_one_input uses just enough of Hypothesis' internals to drive your
1029 test function with a fuzzer-provided bytestring, and most settings
1030 therefore have no effect in this mode. We recommend running your tests
1031 the usual way before fuzzing to get the benefits of healthchecks, as
1032 well as afterwards to replay, shrink, deduplicate, and report whatever
1033 errors were discovered.
1034 • The database setting is used by fuzzing mode - adding failures to the
1036 database to be replayed when you next run your tests is our preferred
1037 reporting mechanism and response to the 'fuzzer taming' problem.
1038 • The verbosity and stateful_step_count settings work as usual.
1040 The deadline, derandomize, max_examples, phases, print_blob, re‐
1042 port_multiple_bugs, and suppress_health_check settings do not affect
1043 fuzzing mode.
1044 Thread-Safety Policy
1046 As discussed in issue #2719, Hypothesis is not truly thread-safe and
1047 that's unlikely to change in the future. This policy therefore de‐
1048 scribes what you can expect if you use Hypothesis with multiple
1049 threads.
1050 Running tests in multiple processes, e.g. with pytest -n auto, is fully
1052 supported and we test this regularly in CI - thanks to process isola‐
1053 tion, we only need to ensure that DirectoryBasedExampleDatabase can't
1054 tread on its own toes too badly. If you find a bug here we will fix it
1055 ASAP.
1056 Running separate tests in multiple threads is not something we design
1058 or test for, and is not formally supported. That said, anecdotally it
1059 does mostly work and we would like it to keep working - we accept rea‐
1060 sonable patches and low-priority bug reports. The main risks here are
1061 global state, shared caches, and cached strategies.
1062 Using multiple threads within a single test , or running a single test
1064 simultaneously in multiple threads, makes it pretty easy to trigger in‐
1065 ternal errors. We usually accept patches for such issues unless read‐
1066 ability or single-thread performance suffer.
1067 Hypothesis assumes that tests are single-threaded, or do a suffi‐
1069 ciently-good job of pretending to be single-threaded. Tests that use
1070 helper threads internally should be OK, but the user must be careful to
1071 ensure that test outcomes are still deterministic. In particular it
1072 counts as nondeterministic if helper-thread timing changes the sequence
1073 of dynamic draws using e.g. the data().
1074 Interacting with any Hypothesis APIs from helper threads might do
1076 weird/bad things, so avoid that too - we rely on thread-local variables
1077 in a few places, and haven't explicitly tested/audited how they respond
1078 to cross-thread API calls. While data() and equivalents are the most
1079 obvious danger, other APIs might also be subtly affected.
1080
1082 Hypothesis tries to have good defaults for its behaviour, but sometimes
1083 that's not enough and you need to tweak it.
1084 The mechanism for doing this is the settings object. You can set up a
1086 @given based test to use this using a settings decorator:
1087 @given invocation is as follows:
1089 from hypothesis import given, settings
1091 @given(integers())
1094 @settings(max_examples=500)
1095 def test_this_thoroughly(x):
1096 pass
1097 This uses a settings object which causes the test to receive a much
1099 larger set of examples than normal.
1100 This may be applied either before or after the given and the results
1102 are the same. The following is exactly equivalent:
1103 from hypothesis import given, settings
1105 @settings(max_examples=500)
1108 @given(integers())
1109 def test_this_thoroughly(x):
1110 pass
1111 Available settings
1113 class hypothesis.settings(parent=None, *, max_examples=not_set, deran‐
1114 domize=not_set, database=not_set, verbosity=not_set, phases=not_set,
1115 stateful_step_count=not_set, report_multiple_bugs=not_set, sup‐
1116 press_health_check=not_set, deadline=not_set, print_blob=not_set)
1117 A settings object controls a variety of parameters that are used
1118 in falsification. These may control both the falsification
1119 strategy and the details of the data that is generated.
1120 Default values are picked up from the settings.default object
1122 and changes made there will be picked up in newly created set‐
1123 tings.
1124 database
1126 An instance of ExampleDatabase that will be used to save
1127 examples to and load previous examples from. May be None
1128 in which case no storage will be used.
1129 See the example database documentation for a list of
1131 built-in example database implementations, and how to de‐
1132 fine custom implementations.
1133 default value: (dynamically calculated)
1135 deadline
1137 If set, a duration (as timedelta, or integer or float
1138 number of milliseconds) that each individual example
1139 (i.e. each time your test function is called, not the
1140 whole decorated test) within a test is not allowed to ex‐
1141 ceed. Tests which take longer than that may be converted
1142 into errors (but will not necessarily be if close to the
1143 deadline, to allow some variability in test run time).
1144 Set this to None to disable this behaviour entirely.
1146 default value: timedelta(milliseconds=200)
1148 derandomize
1150 If True, seed Hypothesis' random number generator using a
1151 hash of the test function, so that every run will test
1152 the same set of examples until you update Hypothesis,
1153 Python, or the test function.
1154 This allows you to check for regressions and look for
1156 bugs using separate settings profiles - for example run‐
1157 ning quick deterministic tests on every commit, and a
1158 longer non-deterministic nightly testing run.
1159 default value: False
1161 max_examples
1163 Once this many satisfying examples have been considered
1164 without finding any counter-example, falsification will
1165 terminate.
1166 The default value is chosen to suit a workflow where the
1168 test will be part of a suite that is regularly executed
1169 locally or on a CI server, balancing total running time
1170 against the chance of missing a bug.
1171 If you are writing one-off tests, running tens of thou‐
1173 sands of examples is quite reasonable as Hypothesis may
1174 miss uncommon bugs with default settings. For very com‐
1175 plex code, we have observed Hypothesis finding novel bugs
1176 after several million examples while testing SymPy. If
1177 you are running more than 100k examples for a test, con‐
1178 sider using our integration for coverage-guided fuzzing -
1179 it really shines when given minutes or hours to run.
1180 default value: 100
1182 phases Control which phases should be run. See the full documen‐
1184 tation for more details
1185 default value: (Phase.explicit, Phase.reuse, Phase.gener‐
1187 ate, Phase.target, Phase.shrink)
1188 print_blob
1190 If set to True, Hypothesis will print code for failing
1191 examples that can be used with @reproduce_failure to re‐
1192 produce the failing example. The default is True if the
1193 CI or TF_BUILD env vars are set, False otherwise.
1194 default value: (dynamically calculated)
1196 report_multiple_bugs
1198 Because Hypothesis runs the test many times, it can some‐
1199 times find multiple bugs in a single run. Reporting all
1200 of them at once is usually very useful, but replacing the
1201 exceptions can occasionally clash with debuggers. If
1202 disabled, only the exception with the smallest minimal
1203 example is raised.
1204 default value: True
1206 stateful_step_count
1208 Number of steps to run a stateful program for before giv‐
1209 ing up on it breaking.
1210 default value: 50
1212 suppress_health_check
1214 A list of HealthCheck items to disable.
1215 default value: ()
1217 verbosity
1219 Control the verbosity level of Hypothesis messages
1220 default value: Verbosity.normal
1222 Controlling what runs
1224 Hypothesis divides tests into logically distinct phases:
1225 1. Running explicit examples provided with the @example decorator.
1227 2. Rerunning a selection of previously failing examples to reproduce a
1229 previously seen error
1230 3. Generating new examples.
1232 4. Mutating examples for targeted property-based testing.
1234 5. Attempting to shrink an example found in previous phases (other than
1236 phase 1 - explicit examples cannot be shrunk). This turns poten‐
1237 tially large and complicated examples which may be hard to read into
1238 smaller and simpler ones.
1239 6. Attempting to explain the cause of the failure, by identifying sus‐
1241 picious lines of code (e.g. the earliest lines which are never run
1242 on passing inputs, and always run on failures). This relies on
1243 python:sys.settrace(), and is therefore automatically disabled on
1244 PyPy or if you are using coverage or a debugger. If there are no
1245 clearly suspicious lines of code, we refuse the temptation to guess.
1246 The phases setting provides you with fine grained control over which of
1248 these run, with each phase corresponding to a value on the Phase enum:
1249 class hypothesis.Phase(value, names=None, *, module=None, qual‐
1251 name=None, type=None, start=1, boundary=None)
1252 explicit = 0
1254 controls whether explicit examples are run.
1255 reuse = 1
1257 controls whether previous examples will be reused.
1258 generate = 2
1260 controls whether new examples will be generated.
1261 target = 3
1263 controls whether examples will be mutated for targeting.
1264 shrink = 4
1266 controls whether examples will be shrunk.
1267 explain = 5
1269 controls whether Hypothesis attempts to explain test
1270 failures.
1271 The phases argument accepts a collection with any subset of these. e.g.
1273 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1274 ples and shrink them, but will not run explicit examples or reuse pre‐
1275 vious failures, while settings(phases=[Phase.explicit]) will only run
1276 the explicit examples.
1277 Seeing intermediate result
1279 To see what's going on while Hypothesis runs your tests, you can turn
1280 up the verbosity setting.
1281 >>> from hypothesis import find, settings, Verbosity
1283 >>> from hypothesis.strategies import lists, integers
1284 >>> @given(lists(integers()))
1285 ... @settings(verbosity=Verbosity.verbose)
1286 ... def f(x):
1287 ... assert not any(x)
1288 ... f()
1289 Trying example: []
1290 Falsifying example: [-1198601713, -67, 116, -29578]
1291 Shrunk example to [-1198601713]
1292 Shrunk example to [-1198601600]
1293 Shrunk example to [-1191228800]
1294 Shrunk example to [-8421504]
1295 Shrunk example to [-32896]
1296 Shrunk example to [-128]
1297 Shrunk example to [64]
1298 Shrunk example to [32]
1299 Shrunk example to [16]
1300 Shrunk example to [8]
1301 Shrunk example to [4]
1302 Shrunk example to [3]
1303 Shrunk example to [2]
1304 Shrunk example to [1]
1305 [1]
1306 The four levels are quiet, normal, verbose and debug. normal is the de‐
1308 fault, while in quiet mode Hypothesis will not print anything out, not
1309 even the final falsifying example. debug is basically verbose but a bit
1310 more so. You probably don't want it.
1311 If you are using pytest, you may also need to disable output capturing
1313 for passing tests.
1314 Building settings objects
1316 Settings can be created by calling settings with any of the available
1317 settings values. Any absent ones will be set to defaults:
1318 >>> from hypothesis import settings
1320 >>> settings().max_examples
1321 100
1322 >>> settings(max_examples=10).max_examples
1323 10
1324 You can also pass a 'parent' settings object as the first argument, and
1326 any settings you do not specify as keyword arguments will be copied
1327 from the parent settings:
1328 >>> parent = settings(max_examples=10)
1330 >>> child = settings(parent, deadline=None)
1331 >>> parent.max_examples == child.max_examples == 10
1332 True
1333 >>> parent.deadline
1334 200
1335 >>> child.deadline is None
1336 True
1337 Default settings
1339 At any given point in your program there is a current default settings,
1340 available as settings.default. As well as being a settings object in
1341 its own right, all newly created settings objects which are not explic‐
1342 itly based off another settings are based off the default, so will in‐
1343 herit any values that are not explicitly set from it.
1344 You can change the defaults by using profiles.
1346 Settings profiles
1348 Depending on your environment you may want different default settings.
1349 For example: during development you may want to lower the number of ex‐
1350 amples to speed up the tests. However, in a CI environment you may want
1351 more examples so you are more likely to find bugs.
1352 Hypothesis allows you to define different settings profiles. These pro‐
1354 files can be loaded at any time.
1355 static settings.register_profile(name, parent=None, **kwargs)
1357 Registers a collection of values to be used as a settings pro‐
1358 file.
1359 Settings profiles can be loaded by name - for example, you might
1361 create a 'fast' profile which runs fewer examples, keep the 'de‐
1362 fault' profile, and create a 'ci' profile that increases the
1363 number of examples and uses a different database to store fail‐
1364 ures.
1365 The arguments to this method are exactly as for settings: op‐
1367 tional parent settings, and keyword arguments for each setting
1368 that will be set differently to parent (or settings.default, if
1369 parent is None).
1370 static settings.get_profile(name)
1372 Return the profile with the given name.
1373 static settings.load_profile(name)
1375 Loads in the settings defined in the profile provided.
1376 If the profile does not exist, InvalidArgument will be raised.
1378 Any setting not defined in the profile will be the library de‐
1379 fined default for that setting.
1380 Loading a profile changes the default settings but will not change the
1382 behaviour of tests that explicitly change the settings.
1383 >>> from hypothesis import settings
1385 >>> settings.register_profile("ci", max_examples=1000)
1386 >>> settings().max_examples
1387 100
1388 >>> settings.load_profile("ci")
1389 >>> settings().max_examples
1390 1000
1391 Instead of loading the profile and overriding the defaults you can re‐
1393 trieve profiles for specific tests.
1394 >>> settings.get_profile("ci").max_examples
1396 1000
1397 Optionally, you may define the environment variable to load a profile
1399 for you. This is the suggested pattern for running your tests on CI.
1400 The code below should run in a conftest.py or any setup/initialization
1401 section of your test suite. If this variable is not defined the Hy‐
1402 pothesis defined defaults will be loaded.
1403 >>> import os
1405 >>> from hypothesis import settings, Verbosity
1406 >>> settings.register_profile("ci", max_examples=1000)
1407 >>> settings.register_profile("dev", max_examples=10)
1408 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1409 >>> settings.load_profile(os.getenv(u"HYPOTHESIS_PROFILE", "default"))
1410 If you are using the hypothesis pytest plugin and your profiles are
1412 registered by your conftest you can load one with the command line op‐
1413 tion --hypothesis-profile.
1414 $ pytest tests --hypothesis-profile <profile-name>
1416
1418 Most things should be easy to generate and everything should be possi‐
1419 ble.
1420 To support this principle Hypothesis provides strategies for most
1422 built-in types with arguments to constrain or adjust the output, as
1423 well as higher-order strategies that can be composed to generate more
1424 complex types.
1425 This document is a guide to what strategies are available for generat‐
1427 ing data and how to build them. Strategies have a variety of other im‐
1428 portant internal features, such as how they simplify, but the data they
1429 can generate is the only public part of their API.
1430 Core strategies
1432 Functions for building strategies are all available in the hypothe‐
1433 sis.strategies module. The salient functions from it are as follows:
1434 hypothesis.strategies.binary(*, min_size=0, max_size=None)
1436 Generates python:bytes.
1437 The generated python:bytes will have a length of at least
1439 min_size and at most max_size. If max_size is None there is no
1440 upper limit.
1441 Examples from this strategy shrink towards smaller strings and
1443 lower byte values.
1444 hypothesis.strategies.booleans()
1446 Returns a strategy which generates instances of python:bool.
1447 Examples from this strategy will shrink towards False (i.e.
1449 shrinking will replace True with False where possible).
1450 hypothesis.strategies.builds(target, /, *args, **kwargs)
1452 Generates values by drawing from args and kwargs and passing
1453 them to the callable (provided as the first positional argument)
1454 in the appropriate argument position.
1455 e.g. builds(target, integers(), flag=booleans()) would draw an
1457 integer i and a boolean b and call target(i, flag=b).
1458 If the callable has type annotations, they will be used to infer
1460 a strategy for required arguments that were not passed to
1461 builds. You can also tell builds to infer a strategy for an op‐
1462 tional argument by passing ... (python:Ellipsis) as a keyword
1463 argument to builds, instead of a strategy for that argument to
1464 the callable.
1465 If the callable is a class defined with attrs, missing required
1467 arguments will be inferred from the attribute on a best-effort
1468 basis, e.g. by checking attrs standard validators. Dataclasses
1469 are handled natively by the inference from type hints.
1470 Examples from this strategy shrink by shrinking the argument
1472 values to the callable.
1473 hypothesis.strategies.characters(*, whitelist_categories=None, black‐
1475 list_categories=None, blacklist_characters=None, min_codepoint=None,
1476 max_codepoint=None, whitelist_characters=None)
1477 Generates characters, length-one python:strings, following spec‐
1478 ified filtering rules.
1479 • When no filtering rules are specified, any character can be
1481 produced.
1482 • If min_codepoint or max_codepoint is specified, then only
1484 characters having a codepoint in that range will be produced.
1485 • If whitelist_categories is specified, then only characters
1487 from those Unicode categories will be produced. This is a fur‐
1488 ther restriction, characters must also satisfy min_codepoint
1489 and max_codepoint.
1490 • If blacklist_categories is specified, then any character from
1492 those categories will not be produced. Any overlap between
1493 whitelist_categories and blacklist_categories will raise an
1494 exception, as each character can only belong to a single
1495 class.
1496 • If whitelist_characters is specified, then any additional
1498 characters in that list will also be produced.
1499 • If blacklist_characters is specified, then any characters in
1501 that list will be not be produced. Any overlap between
1502 whitelist_characters and blacklist_characters will raise an
1503 exception.
1504 The _codepoint arguments must be integers between zero and
1506 python:sys.maxunicode. The _characters arguments must be col‐
1507 lections of length-one unicode strings, such as a unicode
1508 string.
1509 The _categories arguments must be used to specify either the
1511 one-letter Unicode major category or the two-letter Unicode
1512 general category. For example, ('Nd', 'Lu') signifies "Number,
1513 decimal digit" and "Letter, uppercase". A single letter ('major
1514 category') can be given to match all corresponding categories,
1515 for example 'P' for characters in any punctuation category.
1516 Examples from this strategy shrink towards the codepoint for
1518 '0', or the first allowable codepoint after it if '0' is ex‐
1519 cluded.
1520 hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magni‐
1522 tude=None, allow_infinity=None, allow_nan=None)
1523 Returns a strategy that generates complex numbers.
1524 This strategy draws complex numbers with constrained magnitudes.
1526 The min_magnitude and max_magnitude parameters should be
1527 non-negative Real numbers; a value of None corresponds an infi‐
1528 nite upper bound.
1529 If min_magnitude is nonzero or max_magnitude is finite, it is an
1531 error to enable allow_nan. If max_magnitude is finite, it is an
1532 error to enable allow_infinity.
1533 The magnitude constraints are respected up to a relative error
1535 of (around) floating-point epsilon, due to implementation via
1536 the system sqrt function.
1537 Examples from this strategy shrink by shrinking their real and
1539 imaginary parts, as floats().
1540 If you need to generate complex numbers with particular real and
1542 imaginary parts or relationships between parts, consider using
1543 builds(complex, ...) or @composite respectively.
1544 hypothesis.strategies.composite(f)
1546 Defines a strategy that is built out of potentially arbitrarily
1547 many other strategies.
1548 This is intended to be used as a decorator. See the full docu‐
1550 mentation for more details about how to use this function.
1551 Examples from this strategy shrink by shrinking the output of
1553 each draw call.
1554 hypothesis.strategies.data()
1556 This isn't really a normal strategy, but instead gives you an
1557 object which can be used to draw data interactively from other
1558 strategies.
1559 See the rest of the documentation for more complete information.
1561 Examples from this strategy do not shrink (because there is only
1563 one), but the result of calls to each draw() call shrink as they
1564 normally would.
1565 class hypothesis.strategies.DataObject(data)
1567 This type only exists so that you can write type hints for tests
1568 using the data() strategy. Do not use it directly!
1569 hypothesis.strategies.dates(min_value=datetime.date.min,
1571 max_value=datetime.date.max)
1572 A strategy for dates between min_value and max_value.
1573 Examples from this strategy shrink towards January 1st 2000.
1575 hypothesis.strategies.datetimes(min_value=datetime.datetime.min,
1577 max_value=datetime.datetime.max, *, timezones=none(), allow_imagi‐
1578 nary=True)
1579 A strategy for generating datetimes, which may be time‐
1580 zone-aware.
1581 This strategy works by drawing a naive datetime between
1583 min_value and max_value, which must both be naive (have no time‐
1584 zone).
1585 timezones must be a strategy that generates either None, for
1587 naive datetimes, or tzinfo objects for 'aware' datetimes. You
1588 can construct your own, though we recommend using the dateutil
1589 package and hypothesis.extra.dateutil.timezones() strategy, and
1590 also provide hypothesis.extra.pytz.timezones().
1591 You may pass allow_imaginary=False to filter out "imaginary"
1593 datetimes which did not (or will not) occur due to daylight sav‐
1594 ings, leap seconds, timezone and calendar adjustments, etc.
1595 Imaginary datetimes are allowed by default, because malformed
1596 timestamps are a common source of bugs.
1597 Examples from this strategy shrink towards midnight on January
1599 1st 2000, local time.
1600 hypothesis.strategies.decimals(min_value=None, max_value=None, *, al‐
1602 low_nan=None, allow_infinity=None, places=None)
1603 Generates instances of python:decimal.Decimal, which may be:
1604 • A finite rational number, between min_value and max_value.
1606 • Not a Number, if allow_nan is True. None means "allow NaN,
1608 unless min_value and max_value are not None".
1609 • Positive or negative infinity, if max_value and min_value re‐
1611 spectively are None, and allow_infinity is not False. None
1612 means "allow infinity, unless excluded by the min and max val‐
1613 ues".
1614 Note that where floats have one NaN value, Decimals have four:
1616 signed, and either quiet or signalling. See the decimal module
1617 docs for more information on special values.
1618 If places is not None, all finite values drawn from the strategy
1620 will have that number of digits after the decimal place.
1621 Examples from this strategy do not have a well defined shrink
1623 order but try to maximize human readability when shrinking.
1624 hypothesis.strategies.deferred(definition)
1626 A deferred strategy allows you to write a strategy that refer‐
1627 ences other strategies that have not yet been defined. This al‐
1628 lows for the easy definition of recursive and mutually recursive
1629 strategies.
1630 The definition argument should be a zero-argument function that
1632 returns a strategy. It will be evaluated the first time the
1633 strategy is used to produce an example.
1634 Example usage:
1636 >>> import hypothesis.strategies as st
1638 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1639 >>> x.example()
1640 (((False, (True, True)), (False, True)), (True, True))
1641 >>> x.example()
1642 True
1643 Mutual recursion also works fine:
1645 >>> a = st.deferred(lambda: st.booleans() | b)
1647 >>> b = st.deferred(lambda: st.tuples(a, a))
1648 >>> a.example()
1649 True
1650 >>> b.example()
1651 (False, (False, ((False, True), False)))
1652 Examples from this strategy shrink as they normally would from
1654 the strategy returned by the definition.
1655 hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class
1657 'dict'>, min_size=0, max_size=None)
1658 Generates dictionaries of type dict_class with keys drawn from
1659 the keys argument and values drawn from the values argument.
1660 The size parameters have the same interpretation as for lists().
1662 Examples from this strategy shrink by trying to remove keys from
1664 the generated dictionary, and by shrinking each generated key
1665 and value.
1666 class hypothesis.strategies.DrawFn
1668 This type only exists so that you can write type hints for func‐
1669 tions decorated with @composite.
1670 @composite
1672 def list_and_index(draw: DrawFn) -> Tuple[int, str]:
1673 i = draw(integers()) # type inferred as 'int'
1674 s = draw(text()) # type inferred as 'str'
1675 hypothesis.strategies.emails()
1677 A strategy for generating email addresses as unicode strings.
1678 The address format is specified in RFC 5322#section-3.4.1. Val‐
1679 ues shrink towards shorter local-parts and host domains.
1680 This strategy is useful for generating "user data" for tests, as
1682 mishandling of email addresses is a common source of bugs.
1683 hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)
1685 Generates a dictionary of the same type as mapping with a fixed
1686 set of keys mapping to strategies. mapping must be a dict sub‐
1687 class.
1688 Generated values have all keys present in mapping, in iteration
1690 order, with the corresponding values drawn from mapping[key].
1691 If optional is passed, the generated value may or may not con‐
1693 tain each key from optional and a value drawn from the corre‐
1694 sponding strategy. Generated values may contain optional keys
1695 in an arbitrary order.
1696 Examples from this strategy shrink by shrinking each individual
1698 value in the generated dictionary, and omitting optional
1699 key-value pairs.
1700 hypothesis.strategies.floats(min_value=None, max_value=None, *, al‐
1702 low_nan=None, allow_infinity=None, allow_subnormal=None, width=64, ex‐
1703 clude_min=False, exclude_max=False)
1704 Returns a strategy which generates floats.
1705 • If min_value is not None, all values will be >= min_value (or
1707 > min_value if exclude_min).
1708 • If max_value is not None, all values will be <= max_value (or
1710 < max_value if exclude_max).
1711 • If min_value or max_value is not None, it is an error to en‐
1713 able allow_nan.
1714 • If both min_value and max_value are not None, it is an error
1716 to enable allow_infinity.
1717 • If inferred values range does not include subnormal values, it
1719 is an error to enable allow_subnormal.
1720 Where not explicitly ruled out by the bounds, subnormals, in‐
1722 finities, and NaNs are possible values generated by this strat‐
1723 egy.
1724 The width argument specifies the maximum number of bits of pre‐
1726 cision required to represent the generated float. Valid values
1727 are 16, 32, or 64. Passing width=32 will still use the builtin
1728 64-bit float class, but always for values which can be exactly
1729 represented as a 32-bit float.
1730 The exclude_min and exclude_max argument can be used to generate
1732 numbers from open or half-open intervals, by excluding the re‐
1733 spective endpoints. Excluding either signed zero will also ex‐
1734 clude the other. Attempting to exclude an endpoint which is
1735 None will raise an error; use allow_infinity=False to generate
1736 finite floats. You can however use e.g. min_value=-math.inf,
1737 exclude_min=True to exclude only one infinite endpoint.
1738 Examples from this strategy have a complicated and hard to ex‐
1740 plain shrinking behaviour, but it tries to improve "human read‐
1741 ability". Finite numbers will be preferred to infinity and in‐
1742 finity will be preferred to NaN.
1743 hypothesis.strategies.fractions(min_value=None, max_value=None, *,
1745 max_denominator=None)
1746 Returns a strategy which generates Fractions.
1747 If min_value is not None then all generated values are no less
1749 than min_value. If max_value is not None then all generated
1750 values are no greater than max_value. min_value and max_value
1751 may be anything accepted by the Fraction constructor.
1752 If max_denominator is not None then the denominator of any gen‐
1754 erated values is no greater than max_denominator. Note that
1755 max_denominator must be None or a positive integer.
1756 Examples from this strategy shrink towards smaller denominators,
1758 then closer to zero.
1759 hypothesis.strategies.from_regex(regex, *, fullmatch=False)
1761 Generates strings that contain a match for the given regex (i.e.
1762 ones for which python:re.search() will return a non-None re‐
1763 sult).
1764 regex may be a pattern or compiled regex. Both byte-strings and
1766 unicode strings are supported, and will generate examples of the
1767 same type.
1768 You can use regex flags such as python:re.IGNORECASE or
1770 python:re.DOTALL to control generation. Flags can be passed ei‐
1771 ther in compiled regex or inside the pattern with a (?iLmsux)
1772 group.
1773 Some regular expressions are only partly supported - the under‐
1775 lying strategy checks local matching and relies on filtering to
1776 resolve context-dependent expressions. Using too many of these
1777 constructs may cause health-check errors as too many examples
1778 are filtered out. This mainly includes (positive or negative)
1779 lookahead and lookbehind groups.
1780 If you want the generated string to match the whole regex you
1782 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1783 gle character string, while "." will return any string, and
1784 r"\A.$" will return a single character optionally followed by a
1785 "\n". Alternatively, passing fullmatch=True will ensure that
1786 the whole string is a match, as if you had used the \A and \Z
1787 markers.
1788 Examples from this strategy shrink towards shorter strings and
1790 lower character values, with exact behaviour that may depend on
1791 the pattern.
1792 hypothesis.strategies.from_type(thing)
1794 Looks up the appropriate search strategy for the given type.
1795 from_type is used internally to fill in missing arguments to
1797 builds() and can be used interactively to explore what strate‐
1798 gies are available or to debug type resolution.
1799 You can use register_type_strategy() to handle your custom
1801 types, or to globally redefine certain strategies - for example
1802 excluding NaN from floats, or use timezone-aware instead of
1803 naive time and datetime strategies.
1804 The resolution logic may be changed in a future version, but
1806 currently tries these five options:
1807 1. If thing is in the default lookup mapping or user-registered
1809 lookup, return the corresponding strategy. The default
1810 lookup covers all types with Hypothesis strategies, including
1811 extras where possible.
1812 2. If thing is from the python:typing module, return the corre‐
1814 sponding strategy (special logic).
1815 3. If thing has one or more subtypes in the merged lookup, re‐
1817 turn the union of the strategies for those types that are not
1818 subtypes of other elements in the lookup.
1819 4. Finally, if thing has type annotations for all required argu‐
1821 ments, and is not an abstract class, it is resolved via
1822 builds().
1823 5. Because abstract types cannot be instantiated, we treat ab‐
1825 stract types as the union of their concrete subclasses. Note
1826 that this lookup works via inheritance but not via register,
1827 so you may still need to use register_type_strategy().
1828 There is a valuable recipe for leveraging from_type() to gener‐
1830 ate "everything except" values from a specified type. I.e.
1831 def everything_except(excluded_types):
1833 return (
1834 from_type(type)
1835 .flatmap(from_type)
1836 .filter(lambda x: not isinstance(x, excluded_types))
1837 )
1838 For example, everything_except(int) returns a strategy that can
1840 generate anything that from_type() can ever generate, except for
1841 instances of python:int, and excluding instances of types added
1842 via register_type_strategy().
1843 This is useful when writing tests which check that invalid input
1845 is rejected in a certain way.
1846 hypothesis.strategies.frozensets(elements, *, min_size=0,
1848 max_size=None)
1849 This is identical to the sets function but instead returns
1850 frozensets.
1851 hypothesis.strategies.functions(*, like=lambda : ..., returns=...,
1853 pure=False)
1854 A strategy for functions, which can be used in callbacks.
1855 The generated functions will mimic the interface of like, which
1857 must be a callable (including a class, method, or function).
1858 The return value for the function is drawn from the returns ar‐
1859 gument, which must be a strategy. If returns is not passed, we
1860 attempt to infer a strategy from the return-type annotation if
1861 present, falling back to none().
1862 If pure=True, all arguments passed to the generated function
1864 must be hashable, and if passed identical arguments the original
1865 return value will be returned again - not regenerated, so beware
1866 mutable values.
1867 If pure=False, generated functions do not validate their argu‐
1869 ments, and may return a different value if called again with the
1870 same arguments.
1871 Generated functions can only be called within the scope of the
1873 @given which created them. This strategy does not support .ex‐
1874 ample().
1875 hypothesis.strategies.integers(min_value=None, max_value=None)
1877 Returns a strategy which generates integers.
1878 If min_value is not None then all values will be >= min_value.
1880 If max_value is not None then all values will be <= max_value
1881 Examples from this strategy will shrink towards zero, and nega‐
1883 tive values will also shrink towards positive (i.e. -n may be
1884 replaced by +n).
1885 hypothesis.strategies.ip_addresses(*, v=None, network=None)
1887 Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Ad‐
1888 dresses, or leave unspecified to allow both versions.
1889 network may be an IPv4Network or IPv6Network, or a string repre‐
1891 senting a network such as "127.0.0.0/24" or "2001:db8::/32". As
1892 well as generating addresses within a particular routable net‐
1893 work, this can be used to generate addresses from a reserved
1894 range listed in the IANA registries.
1895 If you pass both v and network, they must be for the same ver‐
1897 sion.
1898 hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None,
1900 unique_by=None, unique=False)
1901 This has the same behaviour as lists, but returns iterables in‐
1902 stead.
1903 Some iterables cannot be indexed (e.g. sets) and some do not
1905 have a fixed length (e.g. generators). This strategy produces
1906 iterators, which cannot be indexed and do not have a fixed
1907 length. This ensures that you do not accidentally depend on se‐
1908 quence behaviour.
1909 hypothesis.strategies.just(value)
1911 Return a strategy which only generates value.
1912 Note: value is not copied. Be wary of using mutable values.
1914 If value is the result of a callable, you can use
1916 builds(callable) instead of just(callable()) to get a fresh
1917 value each time.
1918 Examples from this strategy do not shrink (because there is only
1920 one).
1921 hypothesis.strategies.lists(elements, *, min_size=0, max_size=None,
1923 unique_by=None, unique=False)
1924 Returns a list containing values drawn from elements with length
1925 in the interval [min_size, max_size] (no bounds in that direc‐
1926 tion if these are None). If max_size is 0, only the empty list
1927 will be drawn.
1928 If unique is True (or something that evaluates to True), we com‐
1930 pare direct object equality, as if unique_by was lambda x: x.
1931 This comparison only works for hashable types.
1932 If unique_by is not None it must be a callable or tuple of
1934 callables returning a hashable type when given a value drawn
1935 from elements. The resulting list will satisfy the condition
1936 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1937 If unique_by is a tuple of callables the uniqueness will be re‐
1939 spective to each callable.
1940 For example, the following will produce two columns of integers
1942 with both columns being unique respectively.
1943 >>> twoints = st.tuples(st.integers(), st.integers())
1945 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1946 Examples from this strategy shrink by trying to remove elements
1948 from the list, and by shrinking each individual element of the
1949 list.
1950 hypothesis.strategies.none()
1952 Return a strategy which only generates None.
1953 Examples from this strategy do not shrink (because there is only
1955 one).
1956 hypothesis.strategies.nothing()
1958 This strategy never successfully draws a value and will always
1959 reject on an attempt to draw.
1960 Examples from this strategy do not shrink (because there are
1962 none).
1963 hypothesis.strategies.one_of(*args)
1965 Return a strategy which generates values from any of the argu‐
1966 ment strategies.
1967 This may be called with one iterable argument instead of multi‐
1969 ple strategy arguments, in which case one_of(x) and one_of(*x)
1970 are equivalent.
1971 Examples from this strategy will generally shrink to ones that
1973 come from strategies earlier in the list, then shrink according
1974 to behaviour of the strategy that produced them. In order to get
1975 good shrinking behaviour, try to put simpler strategies first.
1976 e.g. one_of(none(), text()) is better than one_of(text(),
1977 none()).
1978 This is especially important when using recursive strategies.
1980 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1981 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1982 st.none()) will shrink very badly indeed.
1983 hypothesis.strategies.permutations(values)
1985 Return a strategy which returns permutations of the ordered col‐
1986 lection values.
1987 Examples from this strategy shrink by trying to become closer to
1989 the original order of values.
1990 hypothesis.strategies.random_module()
1992 The Hypothesis engine handles PRNG state for the stdlib and
1993 Numpy random modules internally, always seeding them to zero and
1994 restoring the previous state after the test.
1995 If having a fixed seed would unacceptably weaken your tests, and
1997 you cannot use a random.Random instance provided by randoms(),
1998 this strategy calls python:random.seed() with an arbitrary inte‐
1999 ger and passes you an opaque object whose repr displays the seed
2000 value for debugging. If numpy.random is available, that state
2001 is also managed.
2002 Examples from these strategy shrink to seeds closer to zero.
2004 hypothesis.strategies.randoms(*, note_method_calls=False, use_true_ran‐
2006 dom=False)
2007 Generates instances of random.Random. The generated Random in‐
2008 stances are of a special HypothesisRandom subclass.
2009 • If note_method_calls is set to True, Hypothesis will print the
2011 randomly drawn values in any falsifying test case. This can be
2012 helpful for debugging the behaviour of randomized algorithms.
2013 • If use_true_random is set to True then values will be drawn
2015 from their usual distribution, otherwise they will actually be
2016 Hypothesis generated values (and will be shrunk accordingly
2017 for any failing test case). Setting use_true_random=False will
2018 tend to expose bugs that would occur with very low probability
2019 when it is set to True, and this flag should only be set to
2020 True when your code relies on the distribution of values for
2021 correctness.
2022 hypothesis.strategies.recursive(base, extend, *, max_leaves=100)
2024 base: A strategy to start from.
2025 extend: A function which takes a strategy and returns a new
2027 strategy.
2028 max_leaves: The maximum number of elements to be drawn from base
2030 on a given run.
2031 This returns a strategy S such that S = extend(base | S). That
2033 is, values may be drawn from base, or from any strategy reach‐
2034 able by mixing applications of | and extend.
2035 An example may clarify: recursive(booleans(), lists) would re‐
2037 turn a strategy that may return arbitrarily nested and mixed
2038 lists of booleans. So e.g. False, [True], [False, []], and
2039 [[[[True]]]] are all valid values to be drawn from that strat‐
2040 egy.
2041 Examples from this strategy shrink by trying to reduce the
2043 amount of recursion and by shrinking according to the shrinking
2044 behaviour of base and the result of extend.
2045 hypothesis.strategies.register_type_strategy(custom_type, strategy)
2047 Add an entry to the global type-to-strategy lookup.
2048 This lookup is used in builds() and @given.
2050 builds() will be used automatically for classes with type anno‐
2052 tations on __init__ , so you only need to register a strategy if
2053 one or more arguments need to be more tightly defined than their
2054 type-based default, or if you want to supply a strategy for an
2055 argument with a default value.
2056 strategy may be a search strategy, or a function that takes a
2058 type and returns a strategy (useful for generic types).
2059 Note that you may not register a parametrised generic type (such
2061 as MyCollection[int]) directly, because the resolution logic
2062 does not handle this case correctly. Instead, you may register
2063 a function for MyCollection and inspect the type parameters
2064 within that function.
2065 hypothesis.strategies.runner(*, default=not_set)
2067 A strategy for getting "the current test runner", whatever that
2068 may be. The exact meaning depends on the entry point, but it
2069 will usually be the associated 'self' value for it.
2070 If there is no current test runner and a default is provided,
2072 return that default. If no default is provided, raises Invali‐
2073 dArgument.
2074 Examples from this strategy do not shrink (because there is only
2076 one).
2077 hypothesis.strategies.sampled_from(elements)
2079 Returns a strategy which generates any value present in ele‐
2080 ments.
2081 Note that as with just(), values will not be copied and thus you
2083 should be careful of using mutable data.
2084 sampled_from supports ordered collections, as well as Enum ob‐
2086 jects. Flag objects may also generate any combination of their
2087 members.
2088 Examples from this strategy shrink by replacing them with values
2090 earlier in the list. So e.g. sampled_from([10, 1]) will shrink
2091 by trying to replace 1 values with 10, and sampled_from([1, 10])
2092 will shrink by trying to replace 10 values with 1.
2093 It is an error to sample from an empty sequence, because return‐
2095 ing nothing() makes it too easy to silently drop parts of com‐
2096 pound strategies. If you need that behaviour, use sam‐
2097 pled_from(seq) if seq else nothing().
2098 hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)
2100 This has the same behaviour as lists, but returns sets instead.
2101 Note that Hypothesis cannot tell if values are drawn from ele‐
2103 ments are hashable until running the test, so you can define a
2104 strategy for sets of an unhashable type but it will fail at test
2105 time.
2106 Examples from this strategy shrink by trying to remove elements
2108 from the set, and by shrinking each individual element of the
2109 set.
2110 hypothesis.strategies.shared(base, *, key=None)
2112 Returns a strategy that draws a single shared value per run,
2113 drawn from base. Any two shared instances with the same key will
2114 share the same value, otherwise the identity of this strategy
2115 will be used. That is:
2116 >>> s = integers() # or any other strategy
2118 >>> x = shared(s)
2119 >>> y = shared(s)
2120 In the above x and y may draw different (or potentially the
2122 same) values. In the following they will always draw the same:
2123 >>> x = shared(s, key="hi")
2125 >>> y = shared(s, key="hi")
2126 Examples from this strategy shrink as per their base strategy.
2128 hypothesis.strategies.slices(size)
2130 Generates slices that will select indices up to the supplied
2131 size
2132 Generated slices will have start and stop indices that range
2134 from -size to size - 1 and will step in the appropriate direc‐
2135 tion. Slices should only produce an empty selection if the start
2136 and end are the same.
2137 Examples from this strategy shrink toward 0 and smaller values
2139 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
2141 gories=('Cs',)), *, min_size=0, max_size=None)
2142 Generates strings with characters drawn from alphabet, which
2143 should be a collection of length one strings or a strategy gen‐
2144 erating such strings.
2145 The default alphabet strategy can generate the full unicode
2147 range but excludes surrogate characters because they are invalid
2148 in the UTF-8 encoding. You can use characters() without argu‐
2149 ments to find surrogate-related bugs such as bpo-34454.
2150 min_size and max_size have the usual interpretations. Note that
2152 Python measures string length by counting codepoints: U+00C5 Å
2153 is a single character, while U+0041 U+030A Å is two - the A,
2154 and a combining ring above.
2155 Examples from this strategy shrink towards shorter strings, and
2157 with the characters in the text shrinking as per the alphabet
2158 strategy. This strategy does not normalize() examples, so gen‐
2159 erated strings may be in any or none of the 'normal forms'.
2160 hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min,
2162 max_value=datetime.timedelta.max)
2163 A strategy for timedeltas between min_value and max_value.
2164 Examples from this strategy shrink towards zero.
2166 hypothesis.strategies.times(min_value=datetime.time.min,
2168 max_value=datetime.time.max, *, timezones=none())
2169 A strategy for times between min_value and max_value.
2170 The timezones argument is handled as for datetimes().
2172 Examples from this strategy shrink towards midnight, with the
2174 timezone component shrinking as for the strategy that provided
2175 it.
2176 hypothesis.strategies.timezone_keys(*, allow_prefix=True)
2178 A strategy for IANA timezone names.
2179 As well as timezone names like "UTC", "Australia/Sydney", or
2181 "America/New_York", this strategy can generate:
2182 • Aliases such as "Antarctica/McMurdo", which links to "Pa‐
2184 cific/Auckland".
2185 • Deprecated names such as "Antarctica/South_Pole", which also
2187 links to "Pacific/Auckland". Note that most but not all dep‐
2188 recated timezone names are also aliases.
2189 • Timezone names with the "posix/" or "right/" prefixes, unless
2191 allow_prefix=False.
2192 These strings are provided separately from Tzinfo objects - such
2194 as ZoneInfo instances from the timezones() strategy - to facili‐
2195 tate testing of timezone logic without needing workarounds to
2196 access non-canonical names.
2197 NOTE:
2199 The python:zoneinfo module is new in Python 3.9, so you will
2200 need to install the backports.zoneinfo module on earlier ver‐
2201 sions.
2202 On Windows, you will also need to install the tzdata package.
2204 pip install hypothesis[zoneinfo] will install these condi‐
2206 tional dependencies if and only if they are needed.
2207 On Windows, you may need to access IANA timezone data via the
2209 tzdata package. For non-IANA timezones, such as Windows-native
2210 names or GNU TZ strings, we recommend using sampled_from() with
2211 the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().
2212 hypothesis.strategies.timezones(*, no_cache=False)
2214 A strategy for python:zoneinfo.ZoneInfo objects.
2215 If no_cache=True, the generated instances are constructed using
2217 ZoneInfo.no_cache instead of the usual constructor. This may
2218 change the semantics of your datetimes in surprising ways, so
2219 only use it if you know that you need to!
2220 NOTE:
2222 The python:zoneinfo module is new in Python 3.9, so you will
2223 need to install the backports.zoneinfo module on earlier ver‐
2224 sions.
2225 On Windows, you will also need to install the tzdata package.
2227 pip install hypothesis[zoneinfo] will install these condi‐
2229 tional dependencies if and only if they are needed.
2230 hypothesis.strategies.tuples(*args)
2232 Return a strategy which generates a tuple of the same length as
2233 args by generating the value at index i from args[i].
2234 e.g. tuples(integers(), integers()) would generate a tuple of
2236 length two with both values an integer.
2237 Examples from this strategy shrink by shrinking their component
2239 parts.
2240 hypothesis.strategies.uuids(*, version=None)
2242 Returns a strategy that generates UUIDs.
2243 If the optional version argument is given, value is passed
2245 through to UUID and only UUIDs of that version will be gener‐
2246 ated.
2247 All returned values from this will be unique, so e.g. if you do
2249 lists(uuids()) the resulting list will never contain duplicates.
2250 Examples from this strategy don't have any meaningful shrink or‐
2252 der.
2253 Provisional strategies
2255 This module contains various provisional APIs and strategies.
2256 It is intended for internal use, to ease code reuse, and is not stable.
2258 Point releases may move or break the contents at any time!
2259 Internet strategies should conform to RFC 3986 or the authoritative
2261 definitions it links to. If not, report the bug!
2262 hypothesis.provisional.domains(*, max_length=255, max_ele‐
2264 ment_length=63)
2265 Generate RFC 1035 compliant fully qualified domain names.
2266 hypothesis.provisional.urls()
2268 A strategy for RFC 3986, generating http/https URLs.
2269 Shrinking
2271 When using strategies it is worth thinking about how the data shrinks.
2272 Shrinking is the process by which Hypothesis tries to produce human
2273 readable examples when it finds a failure - it takes a complex example
2274 and turns it into a simpler one.
2275 Each strategy defines an order in which it shrinks - you won't usually
2277 need to care about this much, but it can be worth being aware of as it
2278 can affect what the best way to write your own strategies is.
2279 The exact shrinking behaviour is not a guaranteed part of the API, but
2281 it doesn't change that often and when it does it's usually because we
2282 think the new way produces nicer examples.
2283 Possibly the most important one to be aware of is one_of(), which has a
2285 preference for values produced by strategies earlier in its argument
2286 list. Most of the others should largely "do the right thing" without
2287 you having to think about it.
2288 Adapting strategies
2290 Often it is the case that a strategy doesn't produce exactly what you
2291 want it to and you need to adapt it. Sometimes you can do this in the
2292 test, but this hurts reuse because you then have to repeat the adaption
2293 in every test.
2294 Hypothesis gives you ways to build strategies from other strategies
2296 given functions for transforming the data.
2297 Mapping
2299 map is probably the easiest and most useful of these to use. If you
2300 have a strategy s and a function f, then an example s.map(f).example()
2301 is f(s.example()), i.e. we draw an example from s and then apply f to
2302 it.
2303 e.g.:
2305 >>> lists(integers()).map(sorted).example()
2307 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2308 Note that many things that you might use mapping for can also be done
2310 with builds(), and if you find yourself indexing into a tuple within
2311 .map() it's probably time to use that instead.
2312 Filtering
2314 filter lets you reject some examples. s.filter(f).example() is some ex‐
2315 ample of s such that f(example) is truthy.
2316 >>> integers().filter(lambda x: x > 11).example()
2318 26126
2319 >>> integers().filter(lambda x: x > 11).example()
2320 23324
2321 It's important to note that filter isn't magic and if your condition is
2323 too hard to satisfy then this can fail:
2324 >>> integers().filter(lambda x: False).example()
2326 Traceback (most recent call last):
2327 ...
2328 hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries
2329 In general you should try to use filter only to avoid corner cases that
2331 you don't want rather than attempting to cut out a large chunk of the
2332 search space.
2333 A technique that often works well here is to use map to first transform
2335 the data and then use filter to remove things that didn't work out. So
2336 for example if you wanted pairs of integers (x,y) such that x < y you
2337 could do the following:
2338 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2340 [-8543729478746591815, 3760495307320535691]
2341 Chaining strategies together
2343 Finally there is flatmap. flatmap draws an example, then turns that ex‐
2344 ample into a strategy, then draws an example from that strategy.
2345 It may not be obvious why you want this at first, but it turns out to
2347 be quite useful because it lets you generate different types of data
2348 with relationships to each other.
2349 For example suppose we wanted to generate a list of lists of the same
2351 length:
2352 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2354 ... lambda n: lists(lists(integers(), min_size=n, max_size=n))
2355 ... )
2356 >>> rectangle_lists.example()
2357 []
2358 >>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
2359 [[], [], [], [], [], [], [], [], [], []]
2360 >>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
2361 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2362 >>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
2363 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2364 In this example we first choose a length for our tuples, then we build
2366 a strategy which generates lists containing lists precisely of that
2367 length. The finds show what simple examples for this look like.
2368 Most of the time you probably don't want flatmap, but unlike filter and
2370 map which are just conveniences for things you could just do in your
2371 tests, flatmap allows genuinely new data generation that you wouldn't
2372 otherwise be able to easily do.
2373 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2375 don't know Haskell, ignore everything in these parentheses. You do not
2376 need to understand anything about monads to use this, or anything else
2377 in Hypothesis).
2378 Recursive data
2380 Sometimes the data you want to generate has a recursive definition.
2381 e.g. if you wanted to generate JSON data, valid JSON is:
2382 1. Any float, any boolean, any unicode string.
2384 2. Any list of valid JSON data
2386 3. Any dictionary mapping unicode strings to valid JSON data.
2388 The problem is that you cannot call a strategy recursively and expect
2390 it to not just blow up and eat all your memory. The other problem here
2391 is that not all unicode strings display consistently on different ma‐
2392 chines, so we'll restrict them in our doctest.
2393 The way Hypothesis handles this is with the recursive() strategy which
2395 you pass in a base case and a function that, given a strategy for your
2396 data type, returns a new strategy for it. So for example:
2397 >>> from string import printable
2399 ... from pprint import pprint
2400 >>> json = recursive(
2401 ... none() | booleans() | floats() | text(printable),
2402 ... lambda children: lists(children) | dictionaries(text(printable), children),
2403 ... )
2404 >>> pprint(json.example())
2405 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2406 >>> pprint(json.example())
2407 {'de(l': None,
2408 'nK': {'(Rt)': None,
2409 '+hoZh1YU]gy8': True,
2410 '8z]EIFA06^li^': 'LFE{Q',
2411 '9,': 'l{cA=/'}}
2412 That is, we start with our leaf data and then we augment it by allowing
2414 lists and dictionaries of anything we can generate as JSON data.
2415 The size control of this works by limiting the maximum number of values
2417 that can be drawn from the base strategy. So for example if we wanted
2418 to only generate really small JSON we could do this as:
2419 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2421 >>> small_lists.example()
2422 True
2423 >>> small_lists.example()
2424 [False]
2425 Composite strategies
2427 The @composite decorator lets you combine other strategies in more or
2428 less arbitrary ways. It's probably the main thing you'll want to use
2429 for complicated custom strategies.
2430 The composite decorator works by converting a function that returns one
2432 example into a function that returns a strategy that produces such ex‐
2433 amples - which you can pass to @given, modify with .map or .filter, and
2434 generally use like any other strategy.
2435 It does this by giving you a special function draw as the first argu‐
2437 ment, which can be used just like the corresponding method of the
2438 data() strategy within a test. In fact, the implementation is almost
2439 the same - but defining a strategy with @composite makes code reuse
2440 easier, and usually improves the display of failing examples.
2441 For example, the following gives you a list and an index into it:
2443 >>> @composite
2445 ... def list_and_index(draw, elements=integers()):
2446 ... xs = draw(lists(elements, min_size=1))
2447 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2448 ... return (xs, i)
2449 ...
2450 draw(s) is a function that should be thought of as returning s.exam‐
2452 ple(), except that the result is reproducible and will minimize cor‐
2453 rectly. The decorated function has the initial argument removed from
2454 the list, but will accept all the others in the expected order. De‐
2455 faults are preserved.
2456 >>> list_and_index()
2458 list_and_index()
2459 >>> list_and_index().example()
2460 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2461 >>> list_and_index(booleans())
2463 list_and_index(elements=booleans())
2464 >>> list_and_index(booleans()).example()
2465 ([True, False], 0)
2466 Note that the repr will work exactly like it does for all the built-in
2468 strategies: it will be a function that you can call to get the strategy
2469 in question, with values provided only if they do not match the de‐
2470 faults.
2471 You can use assume inside composite functions:
2473 @composite
2475 def distinct_strings_with_common_characters(draw):
2476 x = draw(text(min_size=1))
2477 y = draw(text(alphabet=x))
2478 assume(x != y)
2479 return (x, y)
2480 This works as assume normally would, filtering out any examples for
2482 which the passed in argument is falsey.
2483 Take care that your function can cope with adversarial draws, or ex‐
2485 plicitly rejects them using the .filter() method or assume() - our mu‐
2486 tation and shrinking logic can do some strange things, and a naive im‐
2487 plementation might lead to serious performance problems. For example:
2488 @composite
2490 def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
2491 # The bad way: if Hypothesis keeps generating e.g. zero,
2492 # we'll keep looping for a very long time.
2493 result = set()
2494 while len(result) < size:
2495 result.add(draw(elements))
2496 # The good way: use a filter, so Hypothesis can tell what's valid!
2497 for _ in range(size):
2498 result.add(draw(elements.filter(lambda x: x not in result)))
2499 return result
2500 If @composite is used to decorate a method or classmethod, the draw ar‐
2502 gument must come before self or cls. While we therefore recommend
2503 writing strategies as standalone functions and using the
2504 register_type_strategy() function to associate them with a class, meth‐
2505 ods are supported and the @composite decorator may be applied either
2506 before or after @classmethod or @staticmethod. See issue #2578 and
2507 pull request #2634 for more details.
2508 Drawing interactively in tests
2510 There is also the data() strategy, which gives you a means of using
2511 strategies interactively. Rather than having to specify everything up
2512 front in @given you can draw from strategies in the body of your test.
2513 This is similar to @composite, but even more powerful as it allows you
2515 to mix test code with example generation. The downside of this power
2516 is that data() is incompatible with explicit @example(...)s - and the
2517 mixed code is often harder to debug when something goes wrong.
2518 If you need values that are affected by previous draws but which don't
2520 depend on the execution of your test, stick to the simpler @composite.
2521 @given(data())
2523 def test_draw_sequentially(data):
2524 x = data.draw(integers())
2525 y = data.draw(integers(min_value=x))
2526 assert x < y
2527 If the test fails, each draw will be printed with the falsifying exam‐
2529 ple. e.g. the above is wrong (it has a boundary condition error), so
2530 will print:
2531 Falsifying example: test_draw_sequentially(data=data(...))
2533 Draw 1: 0
2534 Draw 2: 0
2535 As you can see, data drawn this way is simplified as usual.
2537 Optionally, you can provide a label to identify values generated by
2539 each call to data.draw(). These labels can be used to identify values
2540 in the output of a falsifying example.
2541 For instance:
2543 @given(data())
2545 def test_draw_sequentially(data):
2546 x = data.draw(integers(), label="First number")
2547 y = data.draw(integers(min_value=x), label="Second number")
2548 assert x < y
2549 will produce the output:
2551 Falsifying example: test_draw_sequentially(data=data(...))
2553 Draw 1 (First number): 0
2554 Draw 2 (Second number): 0
2555
2557 Hypothesis has minimal dependencies, to maximise compatibility and make
2558 installing Hypothesis as easy as possible.
2559 Our integrations with specific packages are therefore provided by extra
2561 modules that need their individual dependencies installed in order to
2562 work. You can install these dependencies using the setuptools extra
2563 feature as e.g. pip install hypothesis[django]. This will check in‐
2564 stallation of compatible versions.
2565 You can also just install hypothesis into a project using them, ignore
2567 the version constraints, and hope for the best.
2568 In general "Which version is Hypothesis compatible with?" is a hard
2570 question to answer and even harder to regularly test. Hypothesis is al‐
2571 ways tested against the latest compatible version and each package will
2572 note the expected compatibility range. If you run into a bug with any
2573 of these please specify the dependency version.
2574 There are separate pages for Hypothesis for Django users and Hypothesis
2576 for the scientific stack.
2577 hypothesis[cli]
2579 $ hypothesis --help
2580 Usage: hypothesis [OPTIONS] COMMAND [ARGS]...
2581 Options:
2583 --version Show the version and exit.
2584 -h, --help Show this message and exit.
2585 Commands:
2587 codemod `hypothesis codemod` refactors deprecated or inefficient code.
2588 fuzz [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
2589 write `hypothesis write` writes property-based tests for you!
2590 This module requires the click package, and provides Hypothesis' com‐
2592 mand-line interface, for e.g. 'ghostwriting' tests via the terminal.
2593 It's also where HypoFuzz adds the hypothesis fuzz command (learn more
2594 about that here).
2595 hypothesis[lark]
2597 This extra can be used to generate strings matching any context-free
2598 grammar, using the Lark parser library.
2599 It currently only supports Lark's native EBNF syntax, but we plan to
2601 extend this to support other common syntaxes such as ANTLR and RFC 5234
2602 ABNF. Lark already supports loading grammars from nearley.js, so you
2603 may not have to write your own at all.
2604 Note that as Lark is at version 0.x, this module may break API compati‐
2606 bility in minor releases if supporting the latest version of Lark would
2607 otherwise be infeasible. We may also be quite aggressive in bumping
2608 the minimum version of Lark, unless someone volunteers to either fund
2609 or do the maintenance.
2610 hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None)
2612 A strategy for strings accepted by the given context-free gram‐
2613 mar.
2614 grammar must be a Lark object, which wraps an EBNF specifica‐
2616 tion. The Lark EBNF grammar reference can be found here.
2617 from_lark will automatically generate strings matching the non‐
2619 terminal start symbol in the grammar, which was supplied as an
2620 argument to the Lark class. To generate strings matching a dif‐
2621 ferent symbol, including terminals, you can override this by
2622 passing the start argument to from_lark. Note that Lark may re‐
2623 move unreachable productions when the grammar is compiled, so
2624 you should probably pass the same value for start to both.
2625 Currently from_lark does not support grammars that need custom
2627 lexing. Any lexers will be ignored, and any undefined terminals
2628 from the use of %declare will result in generation errors. To
2629 define strategies for such terminals, pass a dictionary mapping
2630 their name to a corresponding strategy as the explicit argument.
2631 The hypothesmith project includes a strategy for Python source,
2633 based on a grammar and careful post-processing.
2634 Example grammars, which may provide a useful starting point for your
2636 tests, can be found in the Lark repository and in this third-party col‐
2637 lection.
2638 hypothesis[pytz]
2640 This module provides pytz timezones.
2641 You can use this strategy to make hypothesis.strategies.datetimes() and
2643 hypothesis.strategies.times() produce timezone-aware values.
2644 hypothesis.extra.pytz.timezones()
2646 Any timezone in the Olsen database, as a pytz tzinfo object.
2647 This strategy minimises to UTC, or the smallest possible fixed
2649 offset, and is designed for use with
2650 hypothesis.strategies.datetimes().
2651 hypothesis[dateutil]
2653 This module provides dateutil timezones.
2654 You can use this strategy to make datetimes() and times() produce time‐
2656 zone-aware values.
2657 hypothesis.extra.dateutil.timezones()
2659 Any timezone from dateutil.
2660 This strategy minimises to UTC, or the timezone with the small‐
2662 est offset from UTC as of 2000-01-01, and is designed for use
2663 with datetimes().
2664 Note that the timezones generated by the strategy may vary de‐
2666 pending on the configuration of your machine. See the dateutil
2667 documentation for more information.
2668
2670 Writing tests with Hypothesis frees you from the tedium of deciding on
2671 and writing out specific inputs to test. Now, the hypothesis.ex‐
2672 tra.ghostwriter module can write your test functions for you too!
2673 The idea is to provide an easy way to start property-based testing, and
2675 a seamless transition to more complex test code - because ghostwritten
2676 tests are source code that you could have written for yourself.
2677 So just pick a function you'd like tested, and feed it to one of the
2679 functions below. They follow imports, use but do not require type an‐
2680 notations, and generally do their best to write you a useful test. You
2681 can also use our command-line interface:
2682 $ hypothesis write --help
2684 Usage: hypothesis write [OPTIONS] FUNC...
2685 `hypothesis write` writes property-based tests for you!
2687 Type annotations are helpful but not required for our advanced
2689 introspection and templating logic. Try running the examples below to see
2690 how it works:
2691 hypothesis write gzip
2693 hypothesis write numpy.matmul
2694 hypothesis write re.compile --except re.error
2695 hypothesis write --equivalent ast.literal_eval eval
2696 hypothesis write --roundtrip json.dumps json.loads
2697 hypothesis write --style=unittest --idempotent sorted
2698 hypothesis write --binary-op operator.add
2699 Options:
2701 --roundtrip start by testing write/read or encode/decode!
2702 --equivalent very useful when optimising or refactoring code
2703 --errors-equivalent --equivalent, but also allows consistent errors
2704 --idempotent check that f(x) == f(f(x))
2705 --binary-op associativity, commutativity, identity element
2706 --style [pytest|unittest] pytest-style function, or unittest-style method?
2707 -e, --except OBJ_NAME dotted name of exception(s) to ignore
2708 -h, --help Show this message and exit.
2709 TIP:
2711 Using a light theme? Hypothesis respects NO_COLOR and DJANGO_COL‐
2712 ORS=light.
2713 NOTE:
2715 The ghostwriter requires black, but the generated code only requires
2716 Hypothesis itself.
2717 NOTE:
2719 Legal questions? While the ghostwriter fragments and logic is under
2720 the MPL-2.0 license like the rest of Hypothesis, the output from the
2721 ghostwriter is made available under the Creative Commons Zero (CC0)
2722 public domain dedication, so you can use it without any restric‐
2723 tions.
2724 hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(),
2726 style='pytest')
2727 Guess which ghostwriters to use, for a module or collection of
2728 functions.
2729 As for all ghostwriters, the except_ argument should be an
2731 python:Exception or tuple of exceptions, and style may be either
2732 "pytest" to write test functions or "unittest" to write test
2733 methods and TestCase.
2734 After finding the public functions attached to any modules, the
2736 magic ghostwriter looks for pairs of functions to pass to
2737 roundtrip(), then checks for binary_operation() and ufunc()
2738 functions, and any others are passed to fuzz().
2739 For example, try hypothesis write gzip on the command line!
2741 hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest')
2743 Write source code for a property-based test of func.
2744 The resulting test checks that valid input only leads to ex‐
2746 pected exceptions. For example:
2747 from re import compile, error
2749 from hypothesis.extra import ghostwriter
2751 ghostwriter.fuzz(compile, except_=error)
2753 Gives:
2755 # This test code was written by the `hypothesis.extra.ghostwriter` module
2757 # and is provided under the Creative Commons Zero public domain dedication.
2758 import re
2759 from hypothesis import given, reject, strategies as st
2761 # TODO: replace st.nothing() with an appropriate strategy
2763 @given(pattern=st.nothing(), flags=st.just(0))
2766 def test_fuzz_compile(pattern, flags):
2767 try:
2768 re.compile(pattern=pattern, flags=flags)
2769 except re.error:
2770 reject()
2771 Note that it includes all the required imports. Because the
2773 pattern parameter doesn't have annotations or a default argu‐
2774 ment, you'll need to specify a strategy - for example text() or
2775 binary(). After that, you have a test!
2776 hypothesis.extra.ghostwriter.idempotent(func, *, except_=(),
2778 style='pytest')
2779 Write source code for a property-based test of func.
2780 The resulting test checks that if you call func on it's own out‐
2782 put, the result does not change. For example:
2783 from typing import Sequence
2785 from hypothesis.extra import ghostwriter
2787 def timsort(seq: Sequence[int]) -> Sequence[int]:
2790 return sorted(seq)
2791 ghostwriter.idempotent(timsort)
2794 Gives:
2796 # This test code was written by the `hypothesis.extra.ghostwriter` module
2798 # and is provided under the Creative Commons Zero public domain dedication.
2799 from hypothesis import given, strategies as st
2801 @given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
2804 def test_idempotent_timsort(seq):
2805 result = timsort(seq=seq)
2806 repeat = timsort(seq=result)
2807 assert result == repeat, (result, repeat)
2808 hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(),
2810 style='pytest')
2811 Write source code for a property-based test of funcs.
2812 The resulting test checks that if you call the first function,
2814 pass the result to the second (and so on), the final result is
2815 equal to the first input argument.
2816 This is a very powerful property to test, especially when the
2818 config options are varied along with the object to round-trip.
2819 For example, try ghostwriting a test for python:json.dumps() -
2820 would you have thought of all that?
2821 hypothesis write --roundtrip json.dumps json.loads
2823 hypothesis.extra.ghostwriter.equivalent(*funcs, allow_same_er‐
2825 rors=False, except_=(), style='pytest')
2826 Write source code for a property-based test of funcs.
2827 The resulting test checks that calling each of the functions re‐
2829 turns an equal value. This can be used as a classic 'oracle',
2830 such as testing a fast sorting algorithm against the
2831 python:sorted() builtin, or for differential testing where none
2832 of the compared functions are fully trusted but any difference
2833 indicates a bug (e.g. running a function on different numbers of
2834 threads, or simply multiple times).
2835 The functions should have reasonably similar signatures, as only
2837 the common parameters will be passed the same arguments - any
2838 other parameters will be allowed to vary.
2839 If allow_same_errors is True, then the test will pass if calling
2841 each of the functions returns an equal value, or if the first
2842 function raises an exception and each of the others raises an
2843 exception of the same type. This relaxed mode can be useful for
2844 code synthesis projects.
2845 hypothesis.extra.ghostwriter.binary_operation(func, *, associa‐
2847 tive=True, commutative=True, identity=Ellipsis, distributes_over=None,
2848 except_=(), style='pytest')
2849 Write property tests for the binary operation func.
2850 While binary operations are not particularly common, they have
2852 such nice properties to test that it seems a shame not to demon‐
2853 strate them with a ghostwriter. For an operator f, test that:
2854 • if associative, f(a, f(b, c)) == f(f(a, b), c)
2856 • if commutative, f(a, b) == f(b, a)
2858 • if identity is not None, f(a, identity) == a
2860 • if distributes_over is +, f(a, b) + f(a, c) == f(a, b+c)
2862 For example:
2864 ghostwriter.binary_operation(
2866 operator.mul,
2867 identity=1,
2868 distributes_over=operator.add,
2869 style="unittest",
2870 )
2871 hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest')
2873 Write a property-based test for the array ufunc func.
2874 The resulting test checks that your ufunc or gufunc has the ex‐
2876 pected broadcasting and dtype casting behaviour. You will prob‐
2877 ably want to add extra assertions, but as with the other ghost‐
2878 writers this gives you a great place to start.
2879 hypothesis write numpy.matmul
2881
2883 Hypothesis offers a number of features specific for Django testing,
2884 available in the hypothesis[django] extra. This is tested against each
2885 supported series with mainstream or extended support - if you're still
2886 getting security patches, you can test with Hypothesis.
2887 class hypothesis.extra.django.TestCase
2889 Using it is quite straightforward: All you need to do is subclass
2891 hypothesis.extra.django.TestCase or
2892 hypothesis.extra.django.TransactionTestCase or LiveServerTestCase or
2893 StaticLiveServerTestCase and you can use @given as normal, and the
2894 transactions will be per example rather than per test function as they
2895 would be if you used @given with a normal django test suite (this is
2896 important because your test function will be called multiple times and
2897 you don't want them to interfere with each other). Test cases on these
2898 classes that do not use @given will be run as normal.
2899 class hypothesis.extra.django.TransactionTestCase
2901 class hypothesis.extra.django.LiveServerTestCase
2903 class hypothesis.extra.django.StaticLiveServerTestCase
2905 We recommend avoiding TransactionTestCase unless you really have to run
2907 each test case in a database transaction. Because Hypothesis runs this
2908 in a loop, the performance problems it normally has are significantly
2909 exacerbated and your tests will be really slow. If you are using
2910 TransactionTestCase, you may need to use @settings(sup‐
2911 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2912 example generation.
2913 Having set up a test class, you can now pass @given a strategy for
2915 Django models:
2916 hypothesis.extra.django.from_model(model, /, **field_strategies)
2918 Return a strategy for examples of model.
2919 WARNING:
2921 Hypothesis creates saved models. This will run inside your
2922 testing transaction when using the test runner, but if you
2923 use the dev console this will leave debris in your database.
2924 model must be an subclass of Model. Strategies for fields may
2926 be passed as keyword arguments, for example
2927 is_staff=st.just(False). In order to support models with fields
2928 named "model", this is a positional-only parameter.
2929 Hypothesis can often infer a strategy based the field type and
2931 validators, and will attempt to do so for any required fields.
2932 No strategy will be inferred for an AutoField, nullable field,
2933 foreign key, or field for which a keyword argument is passed to
2934 from_model(). For example, a Shop type with a foreign key to
2935 Company could be generated with:
2936 shop_strategy = from_model(Shop, company=from_model(Company))
2938 Like for builds(), you can pass ... (python:Ellipsis) as a key‐
2940 word argument to infer a strategy for a field which has a de‐
2941 fault value instead of using the default.
2942 For example, using the trivial django project we have for testing:
2944 >>> from hypothesis.extra.django import from_model
2946 >>> from toystore.models import Customer
2947 >>> c = from_model(Customer).example()
2948 >>> c
2949 <Customer: Customer object>
2950 >>> c.email
2951 'jaime.urbina@gmail.com'
2952 >>> c.name
2953 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2954 >>> c.age
2955 -873375803
2956 Hypothesis has just created this with whatever the relevant type of
2958 data is.
2959 Obviously the customer's age is implausible, which is only possible be‐
2961 cause we have not used (eg) MinValueValidator to set the valid range
2962 for this field (or used a PositiveSmallIntegerField, which would only
2963 need a maximum value validator).
2964 If you do have validators attached, Hypothesis will only generate exam‐
2966 ples that pass validation. Sometimes that will mean that we fail a
2967 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2968 egy to skip validation at the strategy level:
2969 NOTE:
2971 Inference from validators will be much more powerful when issue
2972 #1116 is implemented, but there will always be some edge cases that
2973 require you to pass an explicit strategy.
2974 >>> from hypothesis.strategies import integers
2976 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2977 >>> c
2978 <Customer: Customer object>
2979 >>> c.age
2980 5
2981 hypothesis.extra.django.from_form(form, form_kwargs=None,
2983 **field_strategies)
2984 Return a strategy for examples of form.
2985 form must be an subclass of Form. Strategies for fields may be
2987 passed as keyword arguments, for example
2988 is_staff=st.just(False).
2989 Hypothesis can often infer a strategy based the field type and
2991 validators, and will attempt to do so for any required fields.
2992 No strategy will be inferred for a disabled field or field for
2993 which a keyword argument is passed to from_form().
2994 This function uses the fields of an unbound form instance to de‐
2996 termine field strategies, any keyword arguments needed to in‐
2997 stantiate the unbound form instance can be passed into
2998 from_form() as a dict with the keyword form_kwargs. E.g.:
2999 shop_strategy = from_form(Shop, form_kwargs={"company_id": 5})
3001 Like for builds(), you can pass ... (python:Ellipsis) as a key‐
3003 word argument to infer a strategy for a field which has a de‐
3004 fault value instead of using the default.
3005 Tips and tricks
3007 Custom field types
3008 If you have a custom Django field type you can register it with Hypoth‐
3009 esis's model deriving functionality by registering a default strategy
3010 for it:
3011 >>> from toystore.models import CustomishField, Customish
3013 >>> from_model(Customish).example()
3014 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
3015 customish for model Customish
3016 >>> from hypothesis.extra.django import register_field_strategy
3017 >>> from hypothesis.strategies import just
3018 >>> register_field_strategy(CustomishField, just("hi"))
3019 >>> x = from_model(Customish).example()
3020 >>> x.customish
3021 'hi'
3022 Note that this mapping is on exact type. Subtypes will not inherit it.
3024 hypothesis.extra.django.register_field_strategy(field_type, strategy)
3026 Add an entry to the global field-to-strategy lookup used by
3027 from_field().
3028 field_type must be a subtype of django.db.models.Field or
3030 django.forms.Field, which must not already be registered.
3031 strategy must be a SearchStrategy.
3032 hypothesis.extra.django.from_field(field)
3034 Return a strategy for values that fit the given field.
3035 This function is used by from_form() and from_model() for any
3037 fields that require a value, or for which you passed ...
3038 (python:Ellipsis) to infer a strategy from an annotation.
3039 It's pretty similar to the core from_type() function, with a
3041 subtle but important difference: from_field takes a Field in‐
3042 stance, rather than a Field subtype, so that it has access to
3043 instance attributes such as string length and validators.
3044 Generating child models
3046 For the moment there's no explicit support in hypothesis-django for
3047 generating dependent models. i.e. a Company model will generate no
3048 Shops. However if you want to generate some dependent models as well,
3049 you can emulate this by using the flatmap function as follows:
3050 from hypothesis.strategies import just, lists
3052 def generate_with_shops(company):
3055 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
3056 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
3059 Let's unpack what this is doing:
3061 The way flatmap works is that we draw a value from the original strat‐
3063 egy, then apply a function to it which gives us a new strategy. We then
3064 draw a value from that strategy. So in this case we're first drawing a
3065 company, and then we're drawing a list of shops belonging to that com‐
3066 pany: The just strategy is a strategy such that drawing it always pro‐
3067 duces the individual value, so from_model(Shop, company=just(company))
3068 is a strategy that generates a Shop belonging to the original company.
3069 So the following code would give us a list of shops all belonging to
3071 the same company:
3072 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
3074 The only difference from this and the above is that we want the com‐
3076 pany, not the shops. This is where the inner map comes in. We build the
3077 list of shops and then throw it away, instead returning the company we
3078 started for. This works because the models that Hypothesis generates
3079 are saved in the database, so we're essentially running the inner
3080 strategy purely for the side effect of creating those children in the
3081 database.
3082 Generating primary key values
3084 If your model includes a custom primary key that you want to generate
3085 using a strategy (rather than a default auto-increment primary key)
3086 then Hypothesis has to deal with the possibility of a duplicate primary
3087 key.
3088 If a model strategy generates a value for the primary key field, Hy‐
3090 pothesis will create the model instance with update_or_create(), over‐
3091 writing any existing instance in the database for this test case with
3092 the same primary key.
3093 On the subject of MultiValueField
3095 Django forms feature the MultiValueField which allows for several
3096 fields to be combined under a single named field, the default example
3097 of this is the SplitDateTimeField.
3098 class CustomerForm(forms.Form):
3100 name = forms.CharField()
3101 birth_date_time = forms.SplitDateTimeField()
3102 from_form supports MultiValueField subclasses directly, however if you
3104 want to define your own strategy be forewarned that Django binds data
3105 for a MultiValueField in a peculiar way. Specifically each sub-field is
3106 expected to have its own entry in data addressed by the field name
3107 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
3108 ValueField, so form data for the example above might look like this:
3109 {
3111 "name": "Samuel John",
3112 "birth_date_time_0": "2018-05-19", # the date, as the first sub-field
3113 "birth_date_time_1": "15:18:00", # the time, as the second sub-field
3114 }
3115 Thus, if you want to define your own strategies for such a field you
3117 must address your sub-fields appropriately:
3118 from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))
3120
3122 numpy
3123 Hypothesis offers a number of strategies for NumPy testing, available
3124 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
3125 package.
3126 The centerpiece is the arrays() strategy, which generates arrays with
3128 any dtype, shape, and contents you can specify or give a strategy for.
3129 To make this as useful as possible, strategies are provided to generate
3130 array shapes and generate all kinds of fixed-size or compound dtypes.
3131 hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0,
3133 max_size=None, min_value=None, max_value=None, allow_nan=None, al‐
3134 low_infinity=None, allow_subnormal=None, exclude_min=None, ex‐
3135 clude_max=None)
3136 Creates a strategy which can generate any value of the given
3137 dtype.
3138 Compatible **kwargs are passed to the inferred strategy function
3140 for integers, floats, and strings. This allows you to customise
3141 the min and max values, control the length or contents of
3142 strings, or exclude non-finite numbers. This is particularly
3143 useful when kwargs are passed through from arrays() which allow
3144 a variety of numeric dtypes, as it seamlessly handles the width
3145 or representable bounds for you. See issue #2552 for more de‐
3146 tail.
3147 hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None,
3149 fill=None, unique=False)
3150 Returns a strategy for generating numpy:numpy.ndarrays.
3151 • dtype may be any valid input to dtype (this includes dtype ob‐
3153 jects), or a strategy that generates such values.
3154 • shape may be an integer >= 0, a tuple of such integers, or a
3156 strategy that generates such values.
3157 • elements is a strategy for generating values to put in the ar‐
3159 ray. If it is None a suitable value will be inferred based on
3160 the dtype, which may give any legal value (including eg NaN
3161 for floats). If a mapping, it will be passed as **kwargs to
3162 from_dtype()
3163 • fill is a strategy that may be used to generate a single back‐
3165 ground value for the array. If None, a suitable default will
3166 be inferred based on the other arguments. If set to nothing()
3167 then filling behaviour will be disabled entirely and every el‐
3168 ement will be generated independently.
3169 • unique specifies if the elements of the array should all be
3171 distinct from one another. Note that in this case multiple NaN
3172 values may still be allowed. If fill is also set, the only
3173 valid values for it to return are NaN values (anything for
3174 which numpy:numpy.isnan returns True. So e.g. for complex num‐
3175 bers (nan+1j) is also a valid fill). Note that if unique is
3176 set to True the generated values must be hashable.
3177 Arrays of specified dtype and shape are generated for example
3179 like this:
3180 >>> import numpy as np
3182 >>> arrays(np.int8, (2, 3)).example()
3183 array([[-8, 6, 3],
3184 [-6, 4, 6]], dtype=int8)
3185 • See What you can generate and how.
3187 >>> import numpy as np
3189 >>> from hypothesis.strategies import floats
3190 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
3191 array([ 0.88974794, 0.77387938, 0.1977879 ])
3192 Array values are generated in two parts:
3194 1. Some subset of the coordinates of the array are populated
3196 with a value drawn from the elements strategy (or its in‐
3197 ferred form).
3198 2. If any coordinates were not assigned in the previous step, a
3200 single value is drawn from the fill strategy and is assigned
3201 to all remaining places.
3202 You can set fill to nothing() if you want to disable this behav‐
3204 iour and draw a value for every element.
3205 If fill is set to None then it will attempt to infer the correct
3207 behaviour automatically: If unique is True, no filling will oc‐
3208 cur by default. Otherwise, if it looks safe to reuse the values
3209 of elements across multiple coordinates (this will be the case
3210 for any inferred strategy, and for most of the builtins, but is
3211 not the case for mutable values or strategies built with
3212 flatmap, map, composite, etc) then it will use the elements
3213 strategy as the fill, else it will default to having no fill.
3214 Having a fill helps Hypothesis craft high quality examples, but
3216 its main importance is when the array generated is large: Hy‐
3217 pothesis is primarily designed around testing small examples. If
3218 you have arrays with hundreds or more elements, having a fill
3219 value is essential if you want your tests to run in reasonable
3220 time.
3221 hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None,
3223 min_side=1, max_side=None)
3224 Return a strategy for array shapes (tuples of int >= 1).
3225 • min_dims is the smallest length that the generated shape can
3227 possess.
3228 • max_dims is the largest length that the generated shape can
3230 possess, defaulting to min_dims + 2.
3231 • min_side is the smallest size that a dimension can possess.
3233 • max_side is the largest size that a dimension can possess, de‐
3235 faulting to min_side + 5.
3236 hypothesis.extra.numpy.scalar_dtypes()
3238 Return a strategy that can return any non-flexible scalar dtype.
3239 hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?',
3241 sizes=(8, 16, 32, 64))
3242 Return a strategy for unsigned integer dtypes.
3243 endianness may be < for little-endian, > for big-endian, = for
3245 native byte order, or ? to allow either byte order. This argu‐
3246 ment only applies to dtypes of more than one byte.
3247 sizes must be a collection of integer sizes in bits. The de‐
3249 fault (8, 16, 32, 64) covers the full range of sizes.
3250 hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16,
3252 32, 64))
3253 Return a strategy for signed integer dtypes.
3254 endianness and sizes are treated as for
3256 unsigned_integer_dtypes().
3257 hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16,
3259 32, 64))
3260 Return a strategy for floating-point dtypes.
3261 sizes is the size in bits of floating-point number. Some ma‐
3263 chines support 96- or 128-bit floats, but these are not gener‐
3264 ated by default.
3265 Larger floats (96 and 128 bit real parts) are not supported on
3267 all platforms and therefore disabled by default. To generate
3268 these dtypes, include these values in the sizes argument.
3269 hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?',
3271 sizes=(64, 128))
3272 Return a strategy for complex-number dtypes.
3273 sizes is the total size in bits of a complex number, which con‐
3275 sists of two floats. Complex halves (a 16-bit real part) are
3276 not supported by numpy and will not be generated by this strat‐
3277 egy.
3278 hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_pe‐
3280 riod='ns', endianness='?')
3281 Return a strategy for datetime64 dtypes, with various precisions
3282 from year to attosecond.
3283 hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_pe‐
3285 riod='ns', endianness='?')
3286 Return a strategy for timedelta64 dtypes, with various preci‐
3287 sions from year to attosecond.
3288 hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1,
3290 max_len=16)
3291 Return a strategy for generating bytestring dtypes, of various
3292 lengths and byteorder.
3293 While Hypothesis' string strategies can generate empty strings,
3295 string dtypes with length 0 indicate that size is still to be
3296 determined, so the minimum length for string dtypes is 1.
3297 hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?',
3299 min_len=1, max_len=16)
3300 Return a strategy for generating unicode string dtypes, of vari‐
3301 ous lengths and byteorder.
3302 While Hypothesis' string strategies can generate empty strings,
3304 string dtypes with length 0 indicate that size is still to be
3305 determined, so the minimum length for string dtypes is 1.
3306 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
3308 *, min_size=1, max_size=5, allow_subarrays=False)
3309 Return a strategy for generating array (compound) dtypes, with
3310 members drawn from the given subtype strategy.
3311 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
3313 *, max_leaves=10, max_itemsize=None)
3314 Return the most-general dtype strategy.
3315 Elements drawn from this strategy may be simple (from the sub‐
3317 type_strategy), or several such values drawn from array_dtypes()
3318 with allow_subarrays=True. Subdtypes in an array dtype may be
3319 nested to any depth, subject to the max_leaves argument.
3320 hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0,
3322 max_size=None)
3323 Return a strategy for generating permissible tuple-values for
3324 the axis argument for a numpy sequential function (e.g.
3325 numpy:numpy.sum()), given an array of the specified dimensional‐
3326 ity.
3327 All tuples will have a length >= min_size and <= max_size. The
3329 default value for max_size is ndim.
3330 Examples from this strategy shrink towards an empty tuple, which
3332 render most sequential functions as no-ops.
3333 The following are some examples drawn from this strategy.
3335 >>> [valid_tuple_axes(3).example() for i in range(4)]
3337 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3338 valid_tuple_axes can be joined with other strategies to generate
3340 any type of valid axis object, i.e. integers, tuples, and None:
3341 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3343 hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0,
3345 max_dims=None, min_side=1, max_side=None)
3346 Return a strategy for shapes that are broadcast-compatible with
3347 the provided shape.
3348 Examples from this strategy shrink towards a shape with length
3350 min_dims. The size of an aligned dimension shrinks towards size
3351 1. The size of an unaligned dimension shrink towards min_side.
3352 • shape is a tuple of integers.
3354 • min_dims is the smallest length that the generated shape can
3356 possess.
3357 • max_dims is the largest length that the generated shape can
3359 possess, defaulting to max(len(shape), min_dims) + 2.
3360 • min_side is the smallest size that an unaligned dimension can
3362 possess.
3363 • max_side is the largest size that an unaligned dimension can
3365 possess, defaulting to 2 plus the size of the largest aligned
3366 dimension.
3367 The following are some examples drawn from this strategy.
3369 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3371 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3372 hypothesis.extra.numpy.mutually_broadcastable_shapes(*,
3374 num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0,
3375 max_dims=None, min_side=1, max_side=None)
3376 Return a strategy for a specified number of shapes N that are
3377 mutually-broadcastable with one another and with the provided
3378 base shape.
3379 • num_shapes is the number of mutually broadcast-compatible
3381 shapes to generate.
3382 • base_shape is the shape against which all generated shapes can
3384 broadcast. The default shape is empty, which corresponds to a
3385 scalar and thus does not constrain broadcasting at all.
3386 • min_dims is the smallest length that the generated shape can
3388 possess.
3389 • max_dims is the largest length that the generated shape can
3391 possess, defaulting to max(len(shape), min_dims) + 2.
3392 • min_side is the smallest size that an unaligned dimension can
3394 possess.
3395 • max_side is the largest size that an unaligned dimension can
3397 possess, defaulting to 2 plus the size of the largest aligned
3398 dimension.
3399 The strategy will generate a python:typing.NamedTuple contain‐
3401 ing:
3402 • input_shapes as a tuple of the N generated shapes.
3404 • result_shape as the resulting shape produced by broadcasting
3406 the N shapes with the base shape.
3407 The following are some examples drawn from this strategy.
3409 >>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
3411 ... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
3412 >>> for _ in range(5):
3413 ... print(strat.example())
3414 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3415 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3416 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
3417 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
3418 BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))
3419 Use with Generalised Universal Function signatures
3421 A universal function (or ufunc for short) is a function that op‐
3423 erates on ndarrays in an element-by-element fashion, supporting
3424 array broadcasting, type casting, and several other standard
3425 features. A generalised ufunc operates on sub-arrays rather
3426 than elements, based on the "signature" of the function. Com‐
3427 pare e.g. numpy.add() (ufunc) to numpy.matmul() (gufunc).
3428 To generate shapes for a gufunc, you can pass the signature ar‐
3430 gument instead of num_shapes. This must be a gufunc signature
3431 string; which you can write by hand or access as e.g. np.mat‐
3432 mul.signature on generalised ufuncs.
3433 In this case, the side arguments are applied to the 'core dimen‐
3435 sions' as well, ignoring any frozen dimensions. base_shape and
3436 the dims arguments are applied to the 'loop dimensions', and if
3437 necessary, the dimensionality of each shape is silently capped
3438 to respect the 32-dimension limit.
3439 The generated result_shape is the real result shape of applying
3441 the gufunc to arrays of the generated input_shapes, even where
3442 this is different to broadcasting the loop dimensions.
3443 gufunc-compatible shapes shrink their loop dimensions as above,
3445 towards omitting optional core dimensions, and smaller-size core
3446 dimensions.
3447 >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
3449 >>> for _ in range(3):
3450 ... mutually_broadcastable_shapes(signature=np.matmul.signature).example()
3451 BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
3452 BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
3453 BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))
3454 hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0,
3456 max_dims=None, allow_newaxis=False, allow_ellipsis=True)
3457 Return a strategy for basic indexes of arrays with the specified
3458 shape, which may include dimensions of size zero.
3459 It generates tuples containing some mix of integers,
3461 python:slice objects, ... (an Ellipsis), and None. When a
3462 length-one tuple would be generated, this strategy may instead
3463 return the element which will index the first axis, e.g. 5 in‐
3464 stead of (5,).
3465 • shape is the shape of the array that will be indexed, as a tu‐
3467 ple of positive integers. This must be at least two-dimen‐
3468 sional for a tuple to be a valid index; for one-dimensional
3469 arrays use slices() instead.
3470 • min_dims is the minimum dimensionality of the resulting array
3472 from use of the generated index. When min_dims == 0, scalars
3473 and zero-dimensional arrays are both allowed.
3474 • max_dims is the the maximum dimensionality of the resulting
3476 array, defaulting to len(shape) if not allow_newaxis else
3477 max(len(shape), min_dims) + 2.
3478 • allow_newaxis specifies whether None is allowed in the index.
3480 • allow_ellipsis specifies whether ... is allowed in the index.
3482 hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=ar‐
3484 ray_shapes(), dtype='int')
3485 Return a search strategy for tuples of integer-arrays that, when
3486 used to index into an array of shape shape, given an array whose
3487 shape was drawn from result_shape.
3488 Examples from this strategy shrink towards the tuple of in‐
3490 dex-arrays:
3491 len(shape) * (np.zeros(drawn_result_shape, dtype), )
3493 • shape a tuple of integers that indicates the shape of the ar‐
3495 ray, whose indices are being generated.
3496 • result_shape a strategy for generating tuples of integers,
3498 which describe the shape of the resulting index arrays. The
3499 default is array_shapes(). The shape drawn from this strategy
3500 determines the shape of the array that will be produced when
3501 the corresponding example from integer_array_indices is used
3502 as an index.
3503 • dtype the integer data type of the generated index-arrays.
3505 Negative integer indices can be generated if a signed integer
3506 type is specified.
3507 Recall that an array can be indexed using a tuple of integer-ar‐
3509 rays to access its members in an arbitrary order, producing an
3510 array with an arbitrary shape. For example:
3511 >>> from numpy import array
3513 >>> x = array([-0, -1, -2, -3, -4])
3514 >>> ind = (array([[4, 0], [0, 1]]),) # a tuple containing a 2D integer-array
3515 >>> x[ind] # the resulting array is commensurate with the indexing array(s)
3516 array([[-4, 0],
3517 [0, -1]])
3518 Note that this strategy does not accommodate all variations of
3520 so-called 'advanced indexing', as prescribed by NumPy's nomen‐
3521 clature. Combinations of basic and advanced indexes are too
3522 complex to usefully define in a standard strategy; we leave ap‐
3523 plication-specific strategies to the user. Advanced-boolean in‐
3524 dexing can be defined as arrays(shape=..., dtype=bool), and is
3525 similarly left to the user.
3526 pandas
3528 Hypothesis provides strategies for several of the core pandas data
3529 types: pandas.Index, pandas.Series and pandas.DataFrame.
3530 The general approach taken by the pandas module is that there are mul‐
3532 tiple strategies for generating indexes, and all of the other strate‐
3533 gies take the number of entries they contain from their index strategy
3534 (with sensible defaults). So e.g. a Series is specified by specifying
3535 its numpy.dtype (and/or a strategy for generating elements for it).
3536 hypothesis.extra.pandas.indexes(*, elements=None, dtype=None,
3538 min_size=0, max_size=None, unique=True, name=none())
3539 Provides a strategy for producing a pandas.Index.
3540 Arguments:
3542 • elements is a strategy which will be used to generate the in‐
3544 dividual values of the index. If None, it will be inferred
3545 from the dtype. Note: even if the elements strategy produces
3546 tuples, the generated value will not be a MultiIndex, but in‐
3547 stead be a normal index whose elements are tuples.
3548 • dtype is the dtype of the resulting index. If None, it will be
3550 inferred from the elements strategy. At least one of dtype or
3551 elements must be provided.
3552 • min_size is the minimum number of elements in the index.
3554 • max_size is the maximum number of elements in the index. If
3556 None then it will default to a suitable small size. If you
3557 want larger indexes you should pass a max_size explicitly.
3558 • unique specifies whether all of the elements in the resulting
3560 index should be distinct.
3561 • name is a strategy for strings or None, which will be passed
3563 to the pandas.Index constructor.
3564 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
3566 Provides a strategy which generates an Index whose values are 0,
3567 1, ..., n for some n.
3568 Arguments:
3570 • min_size is the smallest number of elements the index can
3572 have.
3573 • max_size is the largest number of elements the index can have.
3575 If None it will default to some suitable value based on
3576 min_size.
3577 hypothesis.extra.pandas.series(*, elements=None, dtype=None, in‐
3579 dex=None, fill=None, unique=False, name=none())
3580 Provides a strategy for producing a pandas.Series.
3581 Arguments:
3583 • elements: a strategy that will be used to generate the indi‐
3585 vidual values in the series. If None, we will attempt to infer
3586 a suitable default from the dtype.
3587 • dtype: the dtype of the resulting series and may be any value
3589 that can be passed to numpy.dtype. If None, will use pandas's
3590 standard behaviour to infer it from the type of the elements
3591 values. Note that if the type of values that comes out of your
3592 elements strategy varies, then so will the resulting dtype of
3593 the series.
3594 • index: If not None, a strategy for generating indexes for the
3596 resulting Series. This can generate either pandas.Index ob‐
3597 jects or any sequence of values (which will be passed to the
3598 Index constructor).
3599 You will probably find it most convenient to use the indexes()
3601 or range_indexes() function to produce values for this argu‐
3602 ment.
3603 • name: is a strategy for strings or None, which will be passed
3605 to the pandas.Series constructor.
3606 Usage:
3608 >>> series(dtype=int).example()
3610 0 -2001747478
3611 1 1153062837
3612 class hypothesis.extra.pandas.column(name=None, elements=None,
3614 dtype=None, fill=None, unique=False)
3615 Data object for describing a column in a DataFrame.
3616 Arguments:
3618 • name: the column name, or None to default to the column posi‐
3620 tion. Must be hashable, but can otherwise be any value sup‐
3621 ported as a pandas column name.
3622 • elements: the strategy for generating values in this column,
3624 or None to infer it from the dtype.
3625 • dtype: the dtype of the column, or None to infer it from the
3627 element strategy. At least one of dtype or elements must be
3628 provided.
3629 • fill: A default value for elements of the column. See arrays()
3631 for a full explanation.
3632 • unique: If all values in this column should be distinct.
3634 hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, ele‐
3636 ments=None, fill=None, unique=False)
3637 A convenience function for producing a list of column objects of
3638 the same general shape.
3639 The names_or_number argument is either a sequence of values, the
3641 elements of which will be used as the name for individual column
3642 objects, or a number, in which case that many unnamed columns
3643 will be created. All other arguments are passed through verbatim
3644 to create the columns.
3645 hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, in‐
3647 dex=None)
3648 Provides a strategy for producing a pandas.DataFrame.
3649 Arguments:
3651 • columns: An iterable of column objects describing the shape of
3653 the generated DataFrame.
3654 • rows: A strategy for generating a row object. Should generate
3656 either dicts mapping column names to values or a sequence map‐
3657 ping column position to the value in that position (note that
3658 unlike the pandas.DataFrame constructor, single values are not
3659 allowed here. Passing e.g. an integer is an error, even if
3660 there is only one column).
3661 At least one of rows and columns must be provided. If both are
3663 provided then the generated rows will be validated against the
3664 columns and an error will be raised if they don't match.
3665 Caveats on using rows:
3667 • In general you should prefer using columns to rows, and only
3669 use rows if the columns interface is insufficiently flexible
3670 to describe what you need - you will get better performance
3671 and example quality that way.
3672 • If you provide rows and not columns, then the shape and
3674 dtype of the resulting DataFrame may vary. e.g. if you have
3675 a mix of int and float in the values for one column in your
3676 row entries, the column will sometimes have an integral
3677 dtype and sometimes a float.
3678 • index: If not None, a strategy for generating indexes for the
3680 resulting DataFrame. This can generate either pandas.Index ob‐
3681 jects or any sequence of values (which will be passed to the
3682 Index constructor).
3683 You will probably find it most convenient to use the indexes()
3685 or range_indexes() function to produce values for this argu‐
3686 ment.
3687 Usage:
3689 The expected usage pattern is that you use column and columns()
3691 to specify a fixed shape of the DataFrame you want as follows.
3692 For example the following gives a two column data frame:
3693 >>> from hypothesis.extra.pandas import column, data_frames
3695 >>> data_frames([
3696 ... column('A', dtype=int), column('B', dtype=float)]).example()
3697 A B
3698 0 2021915903 1.793898e+232
3699 1 1146643993 inf
3700 2 -2096165693 1.000000e+07
3701 If you want the values in different columns to interact in some
3703 way you can use the rows argument. For example the following
3704 gives a two column DataFrame where the value in the first column
3705 is always at most the value in the second:
3706 >>> from hypothesis.extra.pandas import column, data_frames
3708 >>> import hypothesis.strategies as st
3709 >>> data_frames(
3710 ... rows=st.tuples(st.floats(allow_nan=False),
3711 ... st.floats(allow_nan=False)).map(sorted)
3712 ... ).example()
3713 0 1
3714 0 -3.402823e+38 9.007199e+15
3715 1 -1.562796e-298 5.000000e-01
3716 You can also combine the two:
3718 >>> from hypothesis.extra.pandas import columns, data_frames
3720 >>> import hypothesis.strategies as st
3721 >>> data_frames(
3722 ... columns=columns(["lo", "hi"], dtype=float),
3723 ... rows=st.tuples(st.floats(allow_nan=False),
3724 ... st.floats(allow_nan=False)).map(sorted)
3725 ... ).example()
3726 lo hi
3727 0 9.314723e-49 4.353037e+45
3728 1 -9.999900e-01 1.000000e+07
3729 2 -2.152861e+134 -1.069317e-73
3730 (Note that the column dtype must still be specified and will not
3732 be inferred from the rows. This restriction may be lifted in fu‐
3733 ture).
3734 Combining rows and columns has the following behaviour:
3736 • The column names and dtypes will be used.
3738 • If the column is required to be unique, this will be enforced.
3740 • Any values missing from the generated rows will be provided
3742 using the column's fill.
3743 • Any values in the row not present in the column specification
3745 (if dicts are passed, if there are keys with no corresponding
3746 column name, if sequences are passed if there are too many
3747 items) will result in InvalidArgument being raised.
3748 Supported versions
3750 There is quite a lot of variation between pandas versions. We only com‐
3751 mit to supporting the latest version of pandas, but older minor ver‐
3752 sions are supported on a "best effort" basis. Hypothesis is currently
3753 tested against and confirmed working with every Pandas minor version
3754 from 0.25 through to 1.3.
3755 Releases that are not the latest patch release of their minor version
3757 are not tested or officially supported, but will probably also work un‐
3758 less you hit a pandas bug.
3759 Array API
3761 TIP:
3762 The Array API standard is not yet finalised, so this module will
3763 make breaking changes if necessary to support newer versions of the
3764 standard.
3765 Hypothesis offers strategies for Array API adopting libraries in the
3767 hypothesis.extra.array_api package. See issue #3037 for more details.
3768 If you want to test with CuPy, Dask, JAX, MXNet, PyTorch, TensorFlow,
3769 or Xarray - or just numpy.array_api - this is the extension for you!
3770 hypothesis.extra.array_api.make_strategies_namespace(xp)
3772 Creates a strategies namespace for the given array module.
3773 • xp is the Array API library to automatically pass to the
3775 namespaced methods.
3776 A python:types.SimpleNamespace is returned which contains all
3778 the strategy methods in this module but without requiring the xp
3779 argument. Creating and using a strategies namespace for NumPy's
3780 Array API implementation would go like this:
3781 >>> from numpy import array_api as xp
3783 >>> xps = make_strategies_namespace(xp)
3784 >>> x = xps.arrays(xp.int8, (2, 3)).example()
3785 >>> x
3786 Array([[-8, 6, 3],
3787 [-6, 4, 6]], dtype=int8)
3788 >>> x.__array_namespace__() is xp
3789 True
3790 The resulting namespace contains all our familiar strategies like
3792 arrays() and from_dtype(), but based on the Array API standard seman‐
3793 tics and returning objects from the xp module:
3794 xps.from_dtype(dtype, *, min_value=None, max_value=None, al‐
3796 low_nan=None, allow_infinity=None, allow_subnormal=None, ex‐
3797 clude_min=None, exclude_max=None)
3798 Return a strategy for any value of the given dtype.
3799 Values generated are of the Python scalar which is promotable to
3801 dtype, where the values do not exceed its bounds.
3802 • dtype may be a dtype object or the string name of a valid
3804 dtype.
3805 Compatible **kwargs are passed to the inferred strategy function
3807 for integers and floats. This allows you to customise the min
3808 and max values, and exclude non-finite numbers. This is particu‐
3809 larly useful when kwargs are passed through from arrays(), as it
3810 seamlessly handles the width or other representable bounds for
3811 you.
3812 xps.arrays(dtype, shape, *, elements=None, fill=None, unique=False)
3814 Returns a strategy for arrays.
3815 • dtype may be a valid dtype object or name, or a strategy that
3817 generates such values.
3818 • shape may be an integer >= 0, a tuple of such integers, or a
3820 strategy that generates such values.
3821 • elements is a strategy for values to put in the array. If None
3823 then a suitable value will be inferred based on the dtype,
3824 which may give any legal value (including e.g. NaN for
3825 floats). If a mapping, it will be passed as **kwargs to
3826 from_dtype() when inferring based on the dtype.
3827 • fill is a strategy that may be used to generate a single back‐
3829 ground value for the array. If None, a suitable default will
3830 be inferred based on the other arguments. If set to nothing()
3831 then filling behaviour will be disabled entirely and every el‐
3832 ement will be generated independently.
3833 • unique specifies if the elements of the array should all be
3835 distinct from one another; if fill is also set, the only valid
3836 values for fill to return are NaN values.
3837 Arrays of specified dtype and shape are generated for example
3839 like this:
3840 >>> from numpy import array_api as xp
3842 >>> xps.arrays(xp, xp.int8, (2, 3)).example()
3843 Array([[-8, 6, 3],
3844 [-6, 4, 6]], dtype=int8)
3845 Specifying element boundaries by a python:dict of the kwargs to
3847 pass to from_dtype() will ensure dtype bounds will be respected.
3848 >>> xps.arrays(xp, xp.int8, 3, elements={"min_value": 10}).example()
3850 Array([125, 13, 79], dtype=int8)
3851 Refer to What you can generate and how for passing your own ele‐
3853 ments strategy.
3854 >>> xps.arrays(xp, xp.float32, 3, elements=floats(0, 1, width=32)).example()
3856 Array([ 0.88974794, 0.77387938, 0.1977879 ], dtype=float32)
3857 Array values are generated in two parts:
3859 1. A single value is drawn from the fill strategy and is used to
3861 create a filled array.
3862 2. Some subset of the coordinates of the array are populated
3864 with a value drawn from the elements strategy (or its in‐
3865 ferred form).
3866 You can set fill to nothing() if you want to disable this behav‐
3868 iour and draw a value for every element.
3869 By default arrays will attempt to infer the correct fill behav‐
3871 iour: if unique is also True, no filling will occur. Otherwise,
3872 if it looks safe to reuse the values of elements across multiple
3873 coordinates (this will be the case for any inferred strategy,
3874 and for most of the builtins, but is not the case for mutable
3875 values or strategies built with flatmap, map, composite, etc.)
3876 then it will use the elements strategy as the fill, else it will
3877 default to having no fill.
3878 Having a fill helps Hypothesis craft high quality examples, but
3880 its main importance is when the array generated is large: Hy‐
3881 pothesis is primarily designed around testing small examples. If
3882 you have arrays with hundreds or more elements, having a fill
3883 value is essential if you want your tests to run in reasonable
3884 time.
3885 xps.array_shapes(*, min_dims=1, max_dims=None, min_side=1,
3887 max_side=None)
3888 Return a strategy for array shapes (tuples of int >= 1).
3889 • min_dims is the smallest length that the generated shape can
3891 possess.
3892 • max_dims is the largest length that the generated shape can
3894 possess, defaulting to min_dims + 2.
3895 • min_side is the smallest size that a dimension can possess.
3897 • max_side is the largest size that a dimension can possess, de‐
3899 faulting to min_side + 5.
3900 xps.scalar_dtypes()
3902 Return a strategy for all valid dtype objects.
3903 xps.boolean_dtypes()
3905 Return a strategy for just the boolean dtype object.
3906 xps.numeric_dtypes()
3908 Return a strategy for all numeric dtype objects.
3909 xps.integer_dtypes(*, sizes=(8, 16, 32, 64))
3911 Return a strategy for signed integer dtype objects.
3912 sizes contains the signed integer sizes in bits, defaulting to
3914 (8, 16, 32, 64) which covers all valid sizes.
3915 xps.unsigned_integer_dtypes(*, sizes=(8, 16, 32, 64))
3917 Return a strategy for unsigned integer dtype objects.
3918 sizes contains the unsigned integer sizes in bits, defaulting to
3920 (8, 16, 32, 64) which covers all valid sizes.
3921 xps.floating_dtypes(*, sizes=(32, 64))
3923 Return a strategy for floating-point dtype objects.
3924 sizes contains the floating-point sizes in bits, defaulting to
3926 (32, 64) which covers all valid sizes.
3927 xps.valid_tuple_axes(ndim, *, min_size=0, max_size=None)
3929 Return a strategy for permissible tuple-values for the axis ar‐
3930 gument in Array API sequential methods e.g. sum, given the spec‐
3931 ified dimensionality.
3932 All tuples will have a length >= min_size and <= max_size. The
3934 default value for max_size is ndim.
3935 Examples from this strategy shrink towards an empty tuple, which
3937 render most sequential functions as no-ops.
3938 The following are some examples drawn from this strategy.
3940 >>> [valid_tuple_axes(3).example() for i in range(4)]
3942 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3943 valid_tuple_axes can be joined with other strategies to generate
3945 any type of valid axis object, i.e. integers, tuples, and None:
3946 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3948 xps.broadcastable_shapes(shape, *, min_dims=0, max_dims=None,
3950 min_side=1, max_side=None)
3951 Return a strategy for shapes that are broadcast-compatible with
3952 the provided shape.
3953 Examples from this strategy shrink towards a shape with length
3955 min_dims. The size of an aligned dimension shrinks towards size
3956 1. The size of an unaligned dimension shrink towards min_side.
3957 • shape is a tuple of integers.
3959 • min_dims is the smallest length that the generated shape can
3961 possess.
3962 • max_dims is the largest length that the generated shape can
3964 possess, defaulting to max(len(shape), min_dims) + 2.
3965 • min_side is the smallest size that an unaligned dimension can
3967 possess.
3968 • max_side is the largest size that an unaligned dimension can
3970 possess, defaulting to 2 plus the size of the largest aligned
3971 dimension.
3972 The following are some examples drawn from this strategy.
3974 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3976 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3977 xps.mutually_broadcastable_shapes(num_shapes, *, base_shape=(),
3979 min_dims=0, max_dims=None, min_side=1, max_side=None)
3980 Return a strategy for a specified number of shapes N that are
3981 mutually-broadcastable with one another and with the provided
3982 base shape.
3983 • num_shapes is the number of mutually broadcast-compatible
3985 shapes to generate.
3986 • base_shape is the shape against which all generated shapes can
3988 broadcast. The default shape is empty, which corresponds to a
3989 scalar and thus does not constrain broadcasting at all.
3990 • min_dims is the smallest length that the generated shape can
3992 possess.
3993 • max_dims is the largest length that the generated shape can
3995 possess, defaulting to max(len(shape), min_dims) + 2.
3996 • min_side is the smallest size that an unaligned dimension can
3998 possess.
3999 • max_side is the largest size that an unaligned dimension can
4001 possess, defaulting to 2 plus the size of the largest aligned
4002 dimension.
4003 The strategy will generate a python:typing.NamedTuple contain‐
4005 ing:
4006 • input_shapes as a tuple of the N generated shapes.
4008 • result_shape as the resulting shape produced by broadcasting
4010 the N shapes with the base shape.
4011 The following are some examples drawn from this strategy.
4013 >>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
4015 ... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
4016 >>> for _ in range(5):
4017 ... print(strat.example())
4018 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
4019 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
4020 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
4021 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
4022 BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))
4023 xps.indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False,
4025 allow_ellipsis=True)
4026 Return a strategy for valid indices of arrays with the specified
4027 shape, which may include dimensions of size zero.
4028 It generates tuples containing some mix of integers,
4030 python:slice objects, ... (an Ellipsis), and None. When a
4031 length-one tuple would be generated, this strategy may instead
4032 return the element which will index the first axis, e.g. 5 in‐
4033 stead of (5,).
4034 • shape is the shape of the array that will be indexed, as a tu‐
4036 ple of integers >= 0. This must be at least two-dimensional
4037 for a tuple to be a valid index; for one-dimensional arrays
4038 use slices() instead.
4039 • min_dims is the minimum dimensionality of the resulting array
4041 from use of the generated index.
4042 • max_dims is the the maximum dimensionality of the resulting
4044 array, defaulting to len(shape) if not allow_newaxis else
4045 max(len(shape), min_dims) + 2.
4046 • allow_ellipsis specifies whether None is allowed in the index.
4048 • allow_ellipsis specifies whether ... is allowed in the index.
4050
4052 Hypothesis tries to detect common mistakes and things that will cause
4053 difficulty at run time in the form of a number of 'health checks'.
4054 These include detecting and warning about:
4056 • Strategies with very slow data generation
4058 • Strategies which filter out too much
4060 • Recursive strategies which branch too much
4062 • Tests that are unlikely to complete in a reasonable amount of time.
4064 If any of these scenarios are detected, Hypothesis will emit a warning
4066 about them.
4067 The general goal of these health checks is to warn you about things
4069 that you are doing that might appear to work but will either cause Hy‐
4070 pothesis to not work correctly or to perform badly.
4071 To selectively disable health checks, use the suppress_health_check
4073 setting. The argument for this parameter is a list with elements drawn
4074 from any of the class-level attributes of the HealthCheck class. Using
4075 a value of HealthCheck.all() will disable all health checks.
4076 class hypothesis.HealthCheck(value, names=None, *, module=None, qual‐
4078 name=None, type=None, start=1, boundary=None)
4079 Arguments for suppress_health_check.
4080 Each member of this enum is a type of health check to suppress.
4082 data_too_large = 1
4084 Checks if too many examples are aborted for being too
4085 large.
4086 This is measured by the number of random choices that Hy‐
4088 pothesis makes in order to generate something, not the
4089 size of the generated object. For example, choosing a
4090 100MB object from a predefined list would take only a few
4091 bits, while generating 10KB of JSON from scratch might
4092 trigger this health check.
4093 filter_too_much = 2
4095 Check for when the test is filtering out too many exam‐
4096 ples, either through use of assume() or filter(), or oc‐
4097 casionally for Hypothesis internal reasons.
4098 too_slow = 3
4100 Check for when your data generation is extremely slow and
4101 likely to hurt testing.
4102 return_value = 5
4104 Checks if your tests return a non-None value (which will
4105 be ignored and is unlikely to do what you want).
4106 large_base_example = 7
4108 Checks if the natural example to shrink towards is very
4109 large.
4110 not_a_test_method = 8
4112 Checks if @given has been applied to a method defined by
4113 python:unittest.TestCase (i.e. not a test).
4114 function_scoped_fixture = 9
4116 Checks if @given has been applied to a test with a pytest
4117 function-scoped fixture. Function-scoped fixtures run
4118 once for the whole function, not once per example, and
4119 this is usually not what you want.
4120 Because of this limitation, tests that need to set up or
4122 reset state for every example need to do so manually
4123 within the test itself, typically using an appropriate
4124 context manager.
4125 Suppress this health check only in the rare case that you
4127 are using a function-scoped fixture that does not need to
4128 be reset between individual examples, but for some reason
4129 you cannot use a wider fixture scope (e.g. session scope,
4130 module scope, class scope).
4131 This check requires the Hypothesis pytest plugin, which
4133 is enabled by default when running Hypothesis inside
4134 pytest.
4135 Deprecations
4137 We also use a range of custom exception and warning types, so you can
4138 see exactly where an error came from - or turn only our warnings into
4139 errors.
4140 class hypothesis.errors.HypothesisDeprecationWarning
4142 A deprecation warning issued by Hypothesis.
4143 Actually inherits from FutureWarning, because DeprecationWarning
4145 is hidden by the default warnings filter.
4146 You can configure the Python python:warnings to handle these
4148 warnings differently to others, either turning them into errors
4149 or suppressing them entirely. Obviously we would prefer the
4150 former!
4151 Deprecated features will be continue to emit warnings for at least six
4153 months, and then be removed in the following major release. Note how‐
4154 ever that not all warnings are subject to this grace period; sometimes
4155 we strengthen validation by adding a warning and these may become er‐
4156 rors immediately at a major release.
4157
4159 When Hypothesis finds a bug it stores enough information in its data‐
4160 base to reproduce it. This enables you to have a classic testing work‐
4161 flow of find a bug, fix a bug, and be confident that this is actually
4162 doing the right thing because Hypothesis will start by retrying the ex‐
4163 amples that broke things last time.
4164 Limitations
4166 The database is best thought of as a cache that you never need to in‐
4167 validate: Information may be lost when you upgrade a Hypothesis version
4168 or change your test, so you shouldn't rely on it for correctness - if
4169 there's an example you want to ensure occurs each time then there's a
4170 feature for including them in your source code - but it helps the de‐
4171 velopment workflow considerably by making sure that the examples you've
4172 just found are reproduced.
4173 The database also records examples that exercise less-used parts of
4175 your code, so the database may update even when no failing examples
4176 were found.
4177 Upgrading Hypothesis and changing your tests
4179 The design of the Hypothesis database is such that you can put arbi‐
4180 trary data in the database and not get wrong behaviour. When you up‐
4181 grade Hypothesis, old data might be invalidated, but this should happen
4182 transparently. It can never be the case that e.g. changing the strategy
4183 that generates an argument gives you data from the old strategy.
4184 ExampleDatabase implementations
4186 Hypothesis' default database setting creates a
4187 DirectoryBasedExampleDatabase in your current working directory, under
4188 .hypothesis/examples. If this location is unusable, e.g. because you
4189 do not have read or write permissions, Hypothesis will emit a warning
4190 and fall back to an InMemoryExampleDatabase.
4191 Hypothesis provides the following ExampleDatabase implementations:
4193 class hypothesis.database.InMemoryExampleDatabase
4195 A non-persistent example database, implemented in terms of a
4196 dict of sets.
4197 This can be useful if you call a test function several times in
4199 a single session, or for testing other database implementations,
4200 but because it does not persist between runs we do not recommend
4201 it for general use.
4202 class hypothesis.database.DirectoryBasedExampleDatabase(path)
4204 Use a directory to store Hypothesis examples as files.
4205 Each test corresponds to a directory, and each example to a file
4207 within that directory. While the contents are fairly opaque, a
4208 DirectoryBasedExampleDatabase can be shared by checking the di‐
4209 rectory into version control, for example with the following
4210 .gitignore:
4211 # Ignore files cached by Hypothesis...
4213 .hypothesis/*
4214 # except for the examples directory
4215 !.hypothesis/examples/
4216 Note however that this only makes sense if you also pin to an
4218 exact version of Hypothesis, and we would usually recommend im‐
4219 plementing a shared database with a network datastore - see
4220 ExampleDatabase, and the MultiplexedDatabase helper.
4221 class hypothesis.database.ReadOnlyDatabase(db)
4223 A wrapper to make the given database read-only.
4224 The implementation passes through fetch, and turns save, delete,
4226 and move into silent no-ops.
4227 Note that this disables Hypothesis' automatic discarding of
4229 stale examples. It is designed to allow local machines to ac‐
4230 cess a shared database (e.g. from CI servers), without propagat‐
4231 ing changes back from a local or in-development branch.
4232 class hypothesis.database.MultiplexedDatabase(*dbs)
4234 A wrapper around multiple databases.
4235 Each save, fetch, move, or delete operation will be run against
4237 all of the wrapped databases. fetch does not yield duplicate
4238 values, even if the same value is present in two or more of the
4239 wrapped databases.
4240 This combines well with a ReadOnlyDatabase, as follows:
4242 local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
4244 shared = CustomNetworkDatabase()
4245 settings.register_profile("ci", database=shared)
4247 settings.register_profile(
4248 "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
4249 )
4250 settings.load_profile("ci" if os.environ.get("CI") else "dev")
4251 So your CI system or fuzzing runs can populate a central shared
4253 database; while local runs on development machines can reproduce
4254 any failures from CI but will only cache their own failures lo‐
4255 cally and cannot remove examples from the shared database.
4256 class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_af‐
4258 ter=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:')
4259 Store Hypothesis examples as sets in the given Redis datastore.
4260 This is particularly useful for shared databases, as per the
4262 recipe for a MultiplexedDatabase.
4263 NOTE:
4265 If a test has not been run for expire_after, those examples
4266 will be allowed to expire. The default time-to-live persists
4267 examples between weekly runs.
4268 Defining your own ExampleDatabase
4270 You can define your ExampleDatabase, for example to use a shared datas‐
4271 tore, with just a few methods:
4272 class hypothesis.database.ExampleDatabase(*args, **kwargs)
4274 An abstract base class for storing examples in Hypothesis' in‐
4275 ternal format.
4276 An ExampleDatabase maps each bytes key to many distinct bytes
4278 values, like a Mapping[bytes, AbstractSet[bytes]].
4279 abstract save(key, value)
4281 Save value under key.
4282 If this value is already present for this key, silently
4284 do nothing.
4285 abstract fetch(key)
4287 Return an iterable over all values matching this key.
4288 abstract delete(key, value)
4290 Remove this value from this key.
4291 If this value is not present, silently do nothing.
4293 move(src, dest, value)
4295 Move value from key src to key dest. Equivalent to
4296 delete(src, value) followed by save(src, value), but may
4297 have a more efficient implementation.
4298 Note that value will be inserted at dest regardless of
4300 whether it is currently present at src.
4301
4303 With @given, your tests are still something that you mostly write your‐
4304 self, with Hypothesis providing some data. With Hypothesis's stateful
4305 testing, Hypothesis instead tries to generate not just data but entire
4306 tests. You specify a number of primitive actions that can be combined
4307 together, and then Hypothesis will try to find sequences of those ac‐
4308 tions that result in a failure.
4309 TIP:
4311 Before reading this reference documentation, we recommend reading
4312 How not to Die Hard with Hypothesis and An Introduction to
4313 Rule-Based Stateful Testing, in that order. The implementation de‐
4314 tails will make more sense once you've seen them used in practice,
4315 and know why each method or decorator is available.
4316 NOTE:
4318 This style of testing is often called model-based testing, but in
4319 Hypothesis is called stateful testing (mostly for historical reasons
4320 - the original implementation of this idea in Hypothesis was more
4321 closely based on ScalaCheck's stateful testing where the name is
4322 more apt). Both of these names are somewhat misleading: You don't
4323 really need any sort of formal model of your code to use this, and
4324 it can be just as useful for pure APIs that don't involve any state
4325 as it is for stateful ones.
4326 It's perhaps best to not take the name of this sort of testing too
4328 seriously. Regardless of what you call it, it is a powerful form of
4329 testing which is useful for most non-trivial APIs.
4330 You may not need state machines
4332 The basic idea of stateful testing is to make Hypothesis choose actions
4333 as well as values for your test, and state machines are a great declar‐
4334 ative way to do just that.
4335 For simpler cases though, you might not need them at all - a standard
4337 test with @given might be enough, since you can use data() in branches
4338 or loops. In fact, that's how the state machine explorer works inter‐
4339 nally. For more complex workloads though, where a higher level API
4340 comes into it's own, keep reading!
4341 Rule-based state machines
4343 class hypothesis.stateful.RuleBasedStateMachine
4344 A RuleBasedStateMachine gives you a structured way to define
4345 state machines.
4346 The idea is that a state machine carries a bunch of types of
4348 data divided into Bundles, and has a set of rules which may read
4349 data from bundles (or just from normal strategies) and push data
4350 onto bundles. At any given point a random applicable rule will
4351 be executed.
4352 A rule is very similar to a normal @given based test in that it takes
4354 values drawn from strategies and passes them to a user defined test
4355 function. The key difference is that where @given based tests must be
4356 independent, rules can be chained together - a single test run may in‐
4357 volve multiple rule invocations, which may interact in various ways.
4358 Rules can take normal strategies as arguments, or a specific kind of
4360 strategy called a Bundle. A Bundle is a named collection of generated
4361 values that can be reused by other operations in the test. They are
4362 populated with the results of rules, and may be used as arguments to
4363 rules, allowing data to flow from one rule to another, and rules to
4364 work on the results of previous computations or actions.
4365 You can think of each value that gets added to any Bundle as being as‐
4367 signed to a new variable. Drawing a value from the bundle strategy
4368 means choosing one of the corresponding variables and using that value,
4369 and consumes() as a del statement for that variable. If you can re‐
4370 place use of Bundles with instance attributes of the class that is of‐
4371 ten simpler, but often Bundles are strictly more powerful.
4372 The following rule based state machine example is a simplified version
4374 of a test for Hypothesis's example database implementation. An example
4375 database maps keys to sets of values, and in this test we compare one
4376 implementation of it to a simplified in memory model of its behaviour,
4377 which just stores the same values in a Python dict. The test then runs
4378 operations against both the real database and the in-memory representa‐
4379 tion of it and looks for discrepancies in their behaviour.
4380 import shutil
4382 import tempfile
4383 from collections import defaultdict
4384 import hypothesis.strategies as st
4386 from hypothesis.database import DirectoryBasedExampleDatabase
4387 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
4388 class DatabaseComparison(RuleBasedStateMachine):
4391 def __init__(self):
4392 super().__init__()
4393 self.tempd = tempfile.mkdtemp()
4394 self.database = DirectoryBasedExampleDatabase(self.tempd)
4395 self.model = defaultdict(set)
4396 keys = Bundle("keys")
4398 values = Bundle("values")
4399 @rule(target=keys, k=st.binary())
4401 def add_key(self, k):
4402 return k
4403 @rule(target=values, v=st.binary())
4405 def add_value(self, v):
4406 return v
4407 @rule(k=keys, v=values)
4409 def save(self, k, v):
4410 self.model[k].add(v)
4411 self.database.save(k, v)
4412 @rule(k=keys, v=values)
4414 def delete(self, k, v):
4415 self.model[k].discard(v)
4416 self.database.delete(k, v)
4417 @rule(k=keys)
4419 def values_agree(self, k):
4420 assert set(self.database.fetch(k)) == self.model[k]
4421 def teardown(self):
4423 shutil.rmtree(self.tempd)
4424 TestDBComparison = DatabaseComparison.TestCase
4427 In this we declare two bundles - one for keys, and one for values. We
4429 have two trivial rules which just populate them with data (k and v),
4430 and three non-trivial rules: save saves a value under a key and delete
4431 removes a value from a key, in both cases also updating the model of
4432 what should be in the database. values_agree then checks that the con‐
4433 tents of the database agrees with the model for a particular key.
4434 We can then integrate this into our test suite by getting a unittest
4436 TestCase from it:
4437 TestTrees = DatabaseComparison.TestCase
4439 # Or just run with pytest's unittest support
4441 if __name__ == "__main__":
4442 unittest.main()
4443 This test currently passes, but if we comment out the line where we
4445 call self.model[k].discard(v), we would see the following output when
4446 run under pytest:
4447 AssertionError: assert set() == {b''}
4449 ------------ Hypothesis ------------
4451 state = DatabaseComparison()
4453 var1 = state.add_key(k=b'')
4454 var2 = state.add_value(v=var1)
4455 state.save(k=var1, v=var2)
4456 state.delete(k=var1, v=var2)
4457 state.values_agree(k=var1)
4458 state.teardown()
4459 Note how it's printed out a very short program that will demonstrate
4461 the problem. The output from a rule based state machine should gener‐
4462 ally be pretty close to Python code - if you have custom repr implemen‐
4463 tations that don't return valid Python then it might not be, but most
4464 of the time you should just be able to copy and paste the code into a
4465 test to reproduce it.
4466 You can control the detailed behaviour with a settings object on the
4468 TestCase (this is a normal hypothesis settings object using the de‐
4469 faults at the time the TestCase class was first referenced). For exam‐
4470 ple if you wanted to run fewer examples with larger programs you could
4471 change the settings to:
4472 DatabaseComparison.TestCase.settings = settings(
4474 max_examples=50, stateful_step_count=100
4475 )
4476 Which doubles the number of steps each program runs and halves the num‐
4478 ber of test cases that will be run.
4479 Rules
4481 As said earlier, rules are the most common feature used in RuleBased‐
4482 StateMachine. They are defined by applying the rule() decorator on a
4483 function. Note that RuleBasedStateMachine must have at least one rule
4484 defined and that a single function cannot be used to define multiple
4485 rules (this to avoid having multiple rules doing the same things). Due
4486 to the stateful execution method, rules generally cannot take arguments
4487 from other sources such as fixtures or pytest.mark.parametrize - con‐
4488 sider providing them via a strategy such as sampled_from() instead.
4489 hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)
4491 Decorator for RuleBasedStateMachine. Any Bundle present in tar‐
4492 get or targets will define where the end result of this function
4493 should go. If both are empty then the end result will be dis‐
4494 carded.
4495 target must be a Bundle, or if the result should go to multiple
4497 bundles you can pass a tuple of them as the targets argument.
4498 It is invalid to use both arguments for a single rule. If the
4499 result should go to exactly one of several bundles, define a
4500 separate rule for each case.
4501 kwargs then define the arguments that will be passed to the
4503 function invocation. If their value is a Bundle, or if it is
4504 consumes(b) where b is a Bundle, then values that have previ‐
4505 ously been produced for that bundle will be provided. If con‐
4506 sumes is used, the value will also be removed from the bundle.
4507 Any other kwargs should be strategies and values from them will
4509 be provided.
4510 hypothesis.stateful.consumes(bundle)
4512 When introducing a rule in a RuleBasedStateMachine, this func‐
4513 tion can be used to mark bundles from which each value used in a
4514 step with the given rule should be removed. This function re‐
4515 turns a strategy object that can be manipulated and combined
4516 like any other.
4517 For example, a rule declared with
4519 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
4521 sumes(b3)))
4522 will consume a value from Bundle b2 and several values from Bun‐
4524 dle b3 to populate value2 and value3 each time it is executed.
4525 hypothesis.stateful.multiple(*args)
4527 This function can be used to pass multiple results to the tar‐
4528 get(s) of a rule. Just use return multiple(result1, result2,
4529 ...) in your rule.
4530 It is also possible to use return multiple() with no arguments
4532 in order to end a rule without passing any result.
4533 Initializes
4535 Initializes are a special case of rules that are guaranteed to be run
4536 at most once at the beginning of a run (i.e. before any normal rule is
4537 called). Note if multiple initialize rules are defined, they may be
4538 called in any order, and that order will vary from run to run.
4539 Initializes are typically useful to populate bundles:
4541 hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)
4543 Decorator for RuleBasedStateMachine.
4544 An initialize decorator behaves like a rule, but all @initial‐
4546 ize() decorated methods will be called before any @rule() deco‐
4547 rated methods, in an arbitrary order. Each @initialize() method
4548 will be called exactly once per run, unless one raises an excep‐
4549 tion - after which only the .teardown() method will be run.
4550 @initialize() methods may not have preconditions.
4551 import hypothesis.strategies as st
4553 from hypothesis.stateful import Bundle, RuleBasedStateMachine, initialize, rule
4554 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
4556 class NumberModifier(RuleBasedStateMachine):
4559 folders = Bundle("folders")
4561 files = Bundle("files")
4562 @initialize(target=folders)
4564 def init_folders(self):
4565 return "/"
4566 @rule(target=folders, name=name_strategy)
4568 def create_folder(self, parent, name):
4569 return f"{parent}/{name}"
4570 @rule(target=files, name=name_strategy)
4572 def create_file(self, parent, name):
4573 return f"{parent}/{name}"
4574 Preconditions
4576 While it's possible to use assume() in RuleBasedStateMachine rules, if
4577 you use it in only a few rules you can quickly run into a situation
4578 where few or none of your rules pass their assumptions. Thus, Hypothe‐
4579 sis provides a precondition() decorator to avoid this problem. The
4580 precondition() decorator is used on rule-decorated functions, and must
4581 be given a function that returns True or False based on the RuleBased‐
4582 StateMachine instance.
4583 hypothesis.stateful.precondition(precond)
4585 Decorator to apply a precondition for rules in a RuleBased‐
4586 StateMachine. Specifies a precondition for a rule to be consid‐
4587 ered as a valid step in the state machine, which is more effi‐
4588 cient than using assume() within the rule. The precond function
4589 will be called with the instance of RuleBasedStateMachine and
4590 should return True or False. Usually it will need to look at at‐
4591 tributes on that instance.
4592 For example:
4594 class MyTestMachine(RuleBasedStateMachine):
4596 state = 1
4597 @precondition(lambda self: self.state != 0)
4599 @rule(numerator=integers())
4600 def divide_with(self, numerator):
4601 self.state = numerator / self.state
4602 If multiple preconditions are applied to a single rule, it is
4604 only considered a valid step when all of them return True. Pre‐
4605 conditions may be applied to invariants as well as rules.
4606 from hypothesis.stateful import RuleBasedStateMachine, precondition, rule
4608 class NumberModifier(RuleBasedStateMachine):
4611 num = 0
4613 @rule()
4615 def add_one(self):
4616 self.num += 1
4617 @precondition(lambda self: self.num != 0)
4619 @rule()
4620 def divide_with_one(self):
4621 self.num = 1 / self.num
4622 By using precondition() here instead of assume(), Hypothesis can filter
4624 the inapplicable rules before running them. This makes it much more
4625 likely that a useful sequence of steps will be generated.
4626 Note that currently preconditions can't access bundles; if you need to
4628 use preconditions, you should store relevant data on the instance in‐
4629 stead.
4630 Invariants
4632 Often there are invariants that you want to ensure are met after every
4633 step in a process. It would be possible to add these as rules that are
4634 run, but they would be run zero or multiple times between other rules.
4635 Hypothesis provides a decorator that marks a function to be run after
4636 every step.
4637 hypothesis.stateful.invariant(*, check_during_init=False)
4639 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
4640 chine. The decorated function will be run after every rule and
4641 can raise an exception to indicate failed invariants.
4642 For example:
4644 class MyTestMachine(RuleBasedStateMachine):
4646 state = 1
4647 @invariant()
4649 def is_nonzero(self):
4650 assert self.state != 0
4651 By default, invariants are only checked after all @initialize()
4653 rules have been run. Pass check_during_init=True for invariants
4654 which can also be checked during initialization.
4655 from hypothesis.stateful import RuleBasedStateMachine, invariant, rule
4657 class NumberModifier(RuleBasedStateMachine):
4660 num = 0
4662 @rule()
4664 def add_two(self):
4665 self.num += 2
4666 if self.num > 50:
4667 self.num += 1
4668 @invariant()
4670 def divide_with_one(self):
4671 assert self.num % 2 == 0
4672 NumberTest = NumberModifier.TestCase
4675 Invariants can also have precondition()s applied to them, in which case
4677 they will only be run if the precondition function returns true.
4678 Note that currently invariants can't access bundles; if you need to use
4680 invariants, you should store relevant data on the instance instead.
4681 More fine grained control
4683 If you want to bypass the TestCase infrastructure you can invoke these
4684 manually. The stateful module exposes the function run_state_ma‐
4685 chine_as_test, which takes an arbitrary function returning a RuleBased‐
4686 StateMachine and an optional settings parameter and does the same as
4687 the class based runTest provided.
4688 This is not recommended as it bypasses some important internal func‐
4690 tions, including reporting of statistics such as runtimes and event()
4691 calls. It was originally added to support custom __init__ methods, but
4692 you can now use initialize() rules instead.
4693
4695 Hypothesis does its level best to be compatible with everything you
4696 could possibly need it to be compatible with. Generally you should just
4697 try it and expect it to work. If it doesn't, you can be surprised and
4698 check this document for the details.
4699 Hypothesis versions
4701 Backwards compatibility is better than backporting fixes, so we use
4702 semantic versioning and only support the most recent version of Hypoth‐
4703 esis. See Help and support for more information.
4704 Documented APIs will not break except between major version bumps. All
4706 APIs mentioned in this documentation are public unless explicitly noted
4707 as provisional, in which case they may be changed in minor releases.
4708 Undocumented attributes, modules, and behaviour may include breaking
4709 changes in patch releases.
4710 Python versions
4712 Hypothesis is supported and tested on CPython 3.7+, i.e. all versions
4713 of CPython with upstream support, along with PyPy for the same ver‐
4714 sions. 32-bit builds of CPython also work, though we only test them on
4715 Windows.
4716 In general Hypothesis does not officially support anything except the
4718 latest patch release of any version of Python it supports. Earlier re‐
4719 leases should work and bugs in them will get fixed if reported, but
4720 they're not tested in CI and no guarantees are made.
4721 Operating systems
4723 In theory Hypothesis should work anywhere that Python does. In practice
4724 it is only known to work and regularly tested on OS X, Windows and
4725 Linux, and you may experience issues running it elsewhere.
4726 If you're using something else and it doesn't work, do get in touch and
4728 I'll try to help, but unless you can come up with a way for me to run a
4729 CI server on that operating system it probably won't stay fixed due to
4730 the inevitable march of time.
4731 Testing frameworks
4733 In general Hypothesis goes to quite a lot of effort to generate things
4734 that look like normal Python test functions that behave as closely to
4735 the originals as possible, so it should work sensibly out of the box
4736 with every test framework.
4737 If your testing relies on doing something other than calling a function
4739 and seeing if it raises an exception then it probably won't work out of
4740 the box. In particular things like tests which return generators and
4741 expect you to do something with them (e.g. nose's yield based tests)
4742 will not work. Use a decorator or similar to wrap the test to take this
4743 form, or ask the framework maintainer to support our hooks for insert‐
4744 ing such a wrapper later.
4745 In terms of what's actually known to work:
4747 • Hypothesis integrates as smoothly with pytest and unittest as we
4749 can make it, and this is verified as part of the CI.
4750 • pytest fixtures work in the usual way for tests that have been
4752 decorated with @given - just avoid passing a strategy for each ar‐
4753 gument that will be supplied by a fixture. However, each fixture
4754 will run once for the whole function, not once per example. Deco‐
4755 rating a fixture function with @given is meaningless.
4756 • The python:unittest.mock.patch() decorator works with @given, but
4758 we recommend using it as a context manager within the decorated
4759 test to ensure that the mock is per-test-case and avoid poor in‐
4760 teractions with Pytest fixtures.
4761 • Nose works fine with Hypothesis, and this is tested as part of the
4763 CI. yield based tests simply won't work.
4764 • Integration with Django's testing requires use of the Hypothesis
4766 for Django users extra. The issue is that in Django's tests' nor‐
4767 mal mode of execution it will reset the database once per test
4768 rather than once per example, which is not what you want.
4769 • Coverage works out of the box with Hypothesis; our own test suite
4771 has 100% branch coverage.
4772 Optional packages
4774 The supported versions of optional packages, for strategies in hypothe‐
4775 sis.extra, are listed in the documentation for that extra. Our general
4776 goal is to support all versions that are supported upstream.
4777 Regularly verifying this
4779 Everything mentioned above as explicitly supported is checked on every
4780 commit with GitHub Actions. Our continuous delivery pipeline runs all
4781 of these checks before publishing each release, so when we say they're
4782 supported we really mean it.
4783
4785 This is a collection of examples of how to use Hypothesis in interest‐
4786 ing ways. It's small for now but will grow over time.
4787 All of these examples are designed to be run under pytest, and nose
4789 should work too.
4790 How not to sort by a partial order
4792 The following is an example that's been extracted and simplified from a
4793 real bug that occurred in an earlier version of Hypothesis. The real
4794 bug was a lot harder to find.
4795 Suppose we've got the following type:
4797 class Node:
4799 def __init__(self, label, value):
4800 self.label = label
4801 self.value = tuple(value)
4802 def __repr__(self):
4804 return f"Node({self.label!r}, {self.value!r})"
4805 def sorts_before(self, other):
4807 if len(self.value) >= len(other.value):
4808 return False
4809 return other.value[: len(self.value)] == self.value
4810 Each node is a label and a sequence of some data, and we have the rela‐
4812 tionship sorts_before meaning the data of the left is an initial seg‐
4813 ment of the right. So e.g. a node with value [1, 2] will sort before a
4814 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
4815 before the other.
4816 We have a list of nodes, and we want to topologically sort them with
4818 respect to this ordering. That is, we want to arrange the list so that
4819 if x.sorts_before(y) then x appears earlier in the list than y. We
4820 naively think that the easiest way to do this is to extend the partial
4821 order defined here to a total order by breaking ties arbitrarily and
4822 then using a normal sorting algorithm. So we define the following code:
4823 from functools import total_ordering
4825 @total_ordering
4828 class TopoKey:
4829 def __init__(self, node):
4830 self.value = node
4831 def __lt__(self, other):
4833 if self.value.sorts_before(other.value):
4834 return True
4835 if other.value.sorts_before(self.value):
4836 return False
4837 return self.value.label < other.value.label
4839 def sort_nodes(xs):
4842 xs.sort(key=TopoKey)
4843 This takes the order defined by sorts_before and extends it by breaking
4845 ties by comparing the node labels.
4846 But now we want to test that it works.
4848 First we write a function to verify that our desired outcome holds:
4850 def is_prefix_sorted(xs):
4852 for i in range(len(xs)):
4853 for j in range(i + 1, len(xs)):
4854 if xs[j].sorts_before(xs[i]):
4855 return False
4856 return True
4857 This will return false if it ever finds a pair in the wrong order and
4859 return true otherwise.
4860 Given this function, what we want to do with Hypothesis is assert that
4862 for all sequences of nodes, the result of calling sort_nodes on it is
4863 sorted.
4864 First we need to define a strategy for Node:
4866 import hypothesis.strategies as st
4868 NodeStrategy = st.builds(Node, st.integers(), st.lists(st.booleans(), max_size=10))
4870 We want to generate short lists of values so that there's a decent
4872 chance of one being a prefix of the other (this is also why the choice
4873 of bool as the elements). We then define a strategy which builds a node
4874 out of an integer and one of those short lists of booleans.
4875 We can now write a test:
4877 from hypothesis import given
4879 @given(st.lists(NodeStrategy))
4882 def test_sorting_nodes_is_prefix_sorted(xs):
4883 sort_nodes(xs)
4884 assert is_prefix_sorted(xs)
4885 this immediately fails with the following example:
4887 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
4889 The reason for this is that because False is not a prefix of (True,
4891 True) nor vice versa, sorting things the first two nodes are equal be‐
4892 cause they have equal labels. This makes the whole order non-transi‐
4893 tive and produces basically nonsense results.
4894 But this is pretty unsatisfying. It only works because they have the
4896 same label. Perhaps we actually wanted our labels to be unique. Let's
4897 change the test to do that.
4898 def deduplicate_nodes_by_label(nodes):
4900 table = {node.label: node for node in nodes}
4901 return list(table.values())
4902 We define a function to deduplicate nodes by labels, and can now map
4904 that over a strategy for lists of nodes to give us a strategy for lists
4905 of nodes with unique labels:
4906 @given(st.lists(NodeStrategy).map(deduplicate_nodes_by_label))
4908 def test_sorting_nodes_is_prefix_sorted(xs):
4909 sort_nodes(xs)
4910 assert is_prefix_sorted(xs)
4911 Hypothesis quickly gives us an example of this still being wrong:
4913 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]
4915 Now this is a more interesting example. None of the nodes will sort
4917 equal. What is happening here is that the first node is strictly less
4918 than the last node because (False,) is a prefix of (False, False). This
4919 is in turn strictly less than the middle node because neither is a pre‐
4920 fix of the other and -2 < -1. The middle node is then less than the
4921 first node because -1 < 0.
4922 So, convinced that our implementation is broken, we write a better one:
4924 def sort_nodes(xs):
4926 for i in range(1, len(xs)):
4927 j = i - 1
4928 while j >= 0:
4929 if xs[j].sorts_before(xs[j + 1]):
4930 break
4931 xs[j], xs[j + 1] = xs[j + 1], xs[j]
4932 j -= 1
4933 This is just insertion sort slightly modified - we swap a node back‐
4935 wards until swapping it further would violate the order constraints.
4936 The reason this works is because our order is a partial order already
4937 (this wouldn't produce a valid result for a general topological sorting
4938 - you need the transitivity).
4939 We now run our test again and it passes, telling us that this time
4941 we've successfully managed to sort some nodes without getting it com‐
4942 pletely wrong. Go us.
4943 Time zone arithmetic
4945 This is an example of some tests for pytz which check that various
4946 timezone conversions behave as you would expect them to. These tests
4947 should all pass, and are mostly a demonstration of some useful sorts of
4948 thing to test with Hypothesis, and how the datetimes() strategy works.
4949 from datetime import timedelta
4951 # The datetimes strategy is naive by default, so tell it to use timezones
4953 aware_datetimes = st.datetimes(timezones=st.timezones())
4954 @given(aware_datetimes, st.timezones(), st.timezones())
4957 def test_convert_via_intermediary(dt, tz1, tz2):
4958 """Test that converting between timezones is not affected
4959 by a detour via another timezone.
4960 """
4961 assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
4962 @given(aware_datetimes, st.timezones())
4965 def test_convert_to_and_fro(dt, tz2):
4966 """If we convert to a new timezone and back to the old one
4967 this should leave the result unchanged.
4968 """
4969 tz1 = dt.tzinfo
4970 assert dt == dt.astimezone(tz2).astimezone(tz1)
4971 @given(aware_datetimes, st.timezones())
4974 def test_adding_an_hour_commutes(dt, tz):
4975 """When converting between timezones it shouldn't matter
4976 if we add an hour here or add an hour there.
4977 """
4978 an_hour = timedelta(hours=1)
4979 assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
4980 @given(aware_datetimes, st.timezones())
4983 def test_adding_a_day_commutes(dt, tz):
4984 """When converting between timezones it shouldn't matter
4985 if we add a day here or add a day there.
4986 """
4987 a_day = timedelta(days=1)
4988 assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
4989 Condorcet's paradox
4991 A classic paradox in voting theory, called Condorcet's paradox, is that
4992 majority preferences are not transitive. That is, there is a population
4993 and a set of three candidates A, B and C such that the majority of the
4994 population prefer A to B, B to C and C to A.
4995 Wouldn't it be neat if we could use Hypothesis to provide an example of
4997 this?
4998 Well as you can probably guess from the presence of this section, we
5000 can! The main trick is to decide how we want to represent the result
5001 of an election - for this example, we'll use a list of "votes", where
5002 each vote is a list of candidates in the voters preferred order. With‐
5003 out further ado, here is the code:
5004 from collections import Counter
5006 from hypothesis import given
5008 from hypothesis.strategies import lists, permutations
5009 # We need at least three candidates and at least three voters to have a
5012 # paradox; anything less can only lead to victories or at worst ties.
5013 @given(lists(permutations(["A", "B", "C"]), min_size=3))
5014 def test_elections_are_transitive(election):
5015 all_candidates = {"A", "B", "C"}
5016 # First calculate the pairwise counts of how many prefer each candidate
5018 # to the other
5019 counts = Counter()
5020 for vote in election:
5021 for i in range(len(vote)):
5022 for j in range(i + 1, len(vote)):
5023 counts[(vote[i], vote[j])] += 1
5024 # Now look at which pairs of candidates one has a majority over the
5026 # other and store that.
5027 graph = {}
5028 for i in all_candidates:
5029 for j in all_candidates:
5030 if counts[(i, j)] > counts[(j, i)]:
5031 graph.setdefault(i, set()).add(j)
5032 # Now for each triple assert that it is transitive.
5034 for x in all_candidates:
5035 for y in graph.get(x, ()):
5036 for z in graph.get(y, ()):
5037 assert x not in graph.get(z, ())
5038 The example Hypothesis gives me on my first run (your mileage may of
5040 course vary) is:
5041 [["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]
5043 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
5045 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
5046 and 2) prefer C to A. This is in fact basically the canonical example
5047 of the voting paradox.
5048 Fuzzing an HTTP API
5050 Hypothesis's support for testing HTTP services is somewhat nascent.
5051 There are plans for some fully featured things around this, but right
5052 now they're probably quite far down the line.
5053 But you can do a lot yourself without any explicit support! Here's a
5055 script I wrote to throw arbitrary data against the API for an entirely
5056 fictitious service called Waspfinder (this is only lightly obfuscated
5057 and you can easily figure out who I'm actually talking about, but I
5058 don't want you to run this code and hammer their API without their per‐
5059 mission).
5060 All this does is use Hypothesis to generate arbitrary JSON data match‐
5062 ing the format their API asks for and check for 500 errors. More ad‐
5063 vanced tests which then use the result and go on to do other things are
5064 definitely also possible. The schemathesis package provides an excel‐
5065 lent example of this!
5066 import math
5068 import os
5069 import random
5070 import time
5071 import unittest
5072 from collections import namedtuple
5073 import requests
5075 from hypothesis import assume, given, strategies as st
5077 Goal = namedtuple("Goal", ("slug",))
5079 # We just pass in our API credentials via environment variables.
5082 waspfinder_token = os.getenv("WASPFINDER_TOKEN")
5083 waspfinder_user = os.getenv("WASPFINDER_USER")
5084 assert waspfinder_token is not None
5085 assert waspfinder_user is not None
5086 GoalData = st.fixed_dictionaries(
5088 {
5089 "title": st.text(),
5090 "goal_type": st.sampled_from(
5091 ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
5092 ),
5093 "goaldate": st.one_of(st.none(), st.floats()),
5094 "goalval": st.one_of(st.none(), st.floats()),
5095 "rate": st.one_of(st.none(), st.floats()),
5096 "initval": st.floats(),
5097 "panic": st.floats(),
5098 "secret": st.booleans(),
5099 "datapublic": st.booleans(),
5100 }
5101 )
5102 needs2 = ["goaldate", "goalval", "rate"]
5105 class WaspfinderTest(unittest.TestCase):
5108 @given(GoalData)
5109 def test_create_goal_dry_run(self, data):
5110 # We want slug to be unique for each run so that multiple test runs
5111 # don't interfere with each other. If for some reason some slugs trigger
5112 # an error and others don't we'll get a Flaky error, but that's OK.
5113 slug = hex(random.getrandbits(32))[2:]
5114 # Use assume to guide us through validation we know about, otherwise
5116 # we'll spend a lot of time generating boring examples.
5117 # Title must not be empty
5119 assume(data["title"])
5120 # Exactly two of these values should be not None. The other will be
5122 # inferred by the API.
5123 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
5125 for v in data.values():
5126 if isinstance(v, float):
5127 assume(not math.isnan(v))
5128 data["slug"] = slug
5129 # The API nicely supports a dry run option, which means we don't have
5131 # to worry about the user account being spammed with lots of fake goals
5132 # Otherwise we would have to make sure we cleaned up after ourselves
5133 # in this test.
5134 data["dryrun"] = True
5135 data["auth_token"] = waspfinder_token
5136 for d, v in data.items():
5137 if v is None:
5138 data[d] = "null"
5139 else:
5140 data[d] = str(v)
5141 result = requests.post(
5142 "https://waspfinder.example.com/api/v1/users/"
5143 "%s/goals.json" % (waspfinder_user,),
5144 data=data,
5145 )
5146 # Let's not hammer the API too badly. This will of course make the
5148 # tests even slower than they otherwise would have been, but that's
5149 # life.
5150 time.sleep(1.0)
5151 # For the moment all we're testing is that this doesn't generate an
5153 # internal error. If we didn't use the dry run option we could have
5154 # then tried doing more with the result, but this is a good start.
5155 self.assertNotEqual(result.status_code, 500)
5156 if __name__ == "__main__":
5159 unittest.main()
5160
5162 The Hypothesis community is small for the moment but is full of excel‐
5163 lent people who can answer your questions and help you out. Please do
5164 join us. The major place for community discussion is the mailing list.
5165 Feel free to use it to ask for help, provide feedback, or discuss any‐
5167 thing remotely Hypothesis related at all. If you post a question on
5168 Stack Overflow, please use the python-hypothesis tag!
5169 Please note that the Hypothesis code of conduct applies in all Hypothe‐
5171 sis community spaces.
5172 If you would like to cite Hypothesis, please consider our suggested ci‐
5174 tation.
5175 If you like repo badges, we suggest the following badge, which you can
5177 add with reStructuredText or Markdown, respectively: [image]
5178 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
5180 :alt: Tested with Hypothesis
5181 :target: https://hypothesis.readthedocs.io
5182 [](https://hypothesis.readthedocs.io/)
5184 Finally, we have a beautiful logo which appears online, and often on
5186 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
5187 [image]
5188 As well as being beautiful, dragonflies actively hunt down bugs for a
5190 living! You can find the images and a usage guide in the brand direc‐
5191 tory on GitHub, or find us at conferences where we often have stickers
5192 and sometimes other swag.
5193
5195 What is Hypothesis for?
5196 From the perspective of a user, the purpose of Hypothesis is to make it
5198 easier for you to write better tests.
5199 From my perspective as the author, that is of course also a purpose of
5201 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
5202 mania for a moment), the larger purpose of Hypothesis is to drag the
5203 world kicking and screaming into a new and terrifying age of high qual‐
5204 ity software.
5205 Software is, as they say, eating the world. Software is also terrible.
5207 It's buggy, insecure and generally poorly thought out. This combination
5208 is clearly a recipe for disaster.
5209 And the state of software testing is even worse. Although it's fairly
5211 uncontroversial at this point that you should be testing your code, can
5212 you really say with a straight face that most projects you've worked on
5213 are adequately tested?
5214 A lot of the problem here is that it's too hard to write good tests.
5216 Your tests encode exactly the same assumptions and fallacies that you
5217 had when you wrote the code, so they miss exactly the same bugs that
5218 you missed when you wrote the code.
5219 Meanwhile, there are all sorts of tools for making testing better that
5221 are basically unused. The original Quickcheck is from 1999 and the ma‐
5222 jority of developers have not even heard of it, let alone used it.
5223 There are a bunch of half-baked implementations for most languages, but
5224 very few of them are worth using.
5225 The goal of Hypothesis is to bring advanced testing techniques to the
5227 masses, and to provide an implementation that is so high quality that
5228 it is easier to use them than it is not to use them. Where I can, I
5229 will beg, borrow and steal every good idea I can find that someone has
5230 had to make software testing better. Where I can't, I will invent new
5231 ones.
5232 Quickcheck is the start, but I also plan to integrate ideas from fuzz
5234 testing (a planned future feature is to use coverage information to
5235 drive example selection, and the example saving database is already in‐
5236 spired by the workflows people use for fuzz testing), and am open to
5237 and actively seeking out other suggestions and ideas.
5238 The plan is to treat the social problem of people not using these ideas
5240 as a bug to which there is a technical solution: Does property-based
5241 testing not match your workflow? That's a bug, let's fix it by figur‐
5242 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
5243 tom data for your application? That's a bug. Let's fix it by figuring
5244 out how to make it easier, or how to take something you're already us‐
5245 ing to specify your data and derive a generator from that automati‐
5246 cally. Find the explanations of these advanced ideas hopelessly obtuse
5247 and hard to follow? That's a bug. Let's provide you with an easy API
5248 that lets you test your code better without a PhD in software verifica‐
5249 tion.
5250 Grand ambitions, I know, and I expect ultimately the reality will be
5252 somewhat less grand, but so far in about three months of development,
5253 Hypothesis has become the most solid implementation of Quickcheck ever
5254 seen in a mainstream language (as long as we don't count Scala as main‐
5255 stream yet), and at the same time managed to significantly push forward
5256 the state of the art, so I think there's reason to be optimistic.
5257
5259 This is a page for listing people who are using Hypothesis and how ex‐
5260 cited they are about that. If that's you and your name is not on the
5261 list, this file is in Git and I'd love it if you sent me a pull request
5262 to fix that.
5263 Stripe
5265 At Stripe we use Hypothesis to test every piece of our machine learning
5266 model training pipeline (powered by scikit). Before we migrated, our
5267 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
5268 resentative at all of our actual very complex data. Because we needed
5269 to craft examples for each test, we took the easy way out and lived
5270 with extremely low test coverage.
5271 Hypothesis changed all that. Once we had our strategies for generating
5273 Dataframes of features it became trivial to slightly customize each
5274 strategy for new tests. Our coverage is now close to 90%.
5275 Full-stop, property-based testing is profoundly more powerful - and has
5277 caught or prevented far more bugs - than our old style of example-based
5278 testing.
5279 Kristian Glass - Director of Technology at LaterPay GmbH
5281 Hypothesis has been brilliant for expanding the coverage of our test
5282 cases, and also for making them much easier to read and understand, so
5283 we're sure we're testing the things we want in the way we want.
5284 Seth Morton
5286 When I first heard about Hypothesis, I knew I had to include it in my
5287 two open-source Python libraries, natsort and fastnumbers . Quite
5288 frankly, I was a little appalled at the number of bugs and "holes" I
5289 found in the code. I can now say with confidence that my libraries are
5290 more robust to "the wild." In addition, Hypothesis gave me the confi‐
5291 dence to expand these libraries to fully support Unicode input, which I
5292 never would have had the stomach for without such thorough testing ca‐
5293 pabilities. Thanks!
5294 Sixty North
5296 At Sixty North we use Hypothesis for testing Segpy an open source
5297 Python library for shifting data between Python data structures and SEG
5298 Y files which contain geophysical data from the seismic reflection sur‐
5299 veys used in oil and gas exploration.
5300 This is our first experience of property-based testing – as opposed to
5302 example-based testing. Not only are our tests more powerful, they are
5303 also much better explanations of what we expect of the production code.
5304 In fact, the tests are much closer to being specifications. Hypothesis
5305 has located real defects in our code which went undetected by tradi‐
5306 tional test cases, simply because Hypothesis is more relentlessly devi‐
5307 ous about test case generation than us mere humans! We found Hypothe‐
5308 sis particularly beneficial for Segpy because SEG Y is an antiquated
5309 format that uses legacy text encodings (EBCDIC) and even a legacy
5310 floating point format we implemented from scratch in Python.
5311 Hypothesis is sure to find a place in most of our future Python code‐
5313 bases and many existing ones too.
5314 mulkieran
5316 Just found out about this excellent QuickCheck for Python implementa‐
5317 tion and ran up a few tests for my bytesize package last night. Refuted
5318 a few hypotheses in the process.
5319 Looking forward to using it with a bunch of other projects as well.
5321 Adam Johnson
5323 I have written a small library to serialize dicts to MariaDB's dynamic
5324 columns binary format, mariadb-dyncol. When I first developed it, I
5325 thought I had tested it really well - there were hundreds of test
5326 cases, some of them even taken from MariaDB's test suite itself. I was
5327 ready to release.
5328 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
5330 Wow! It found bug after bug after bug. Even after a first release, I
5331 thought of a way to make the tests do more validation, which revealed a
5332 further round of bugs! Most impressively, Hypothesis found a compli‐
5333 cated off-by-one error in a condition with 4095 versus 4096 bytes of
5334 data - something that I would never have found.
5335 Long live Hypothesis! (Or at least, property-based testing).
5337 Josh Bronson
5339 Adopting Hypothesis improved bidict's test coverage and significantly
5340 increased our ability to make changes to the code with confidence that
5341 correct behavior would be preserved. Thank you, David, for the great
5342 testing tool.
5343 Cory Benfield
5345 Hypothesis is the single most powerful tool in my toolbox for working
5346 with algorithmic code, or any software that produces predictable output
5347 from a wide range of sources. When using it with Priority, Hypothesis
5348 consistently found errors in my assumptions and extremely subtle bugs
5349 that would have taken months of real-world use to locate. In some
5350 cases, Hypothesis found subtle deviations from the correct output of
5351 the algorithm that may never have been noticed at all.
5352 When it comes to validating the correctness of your tools, nothing
5354 comes close to the thoroughness and power of Hypothesis.
5355 Jon Moore
5357 One extremely satisfied user here. Hypothesis is a really solid imple‐
5358 mentation of property-based testing, adapted well to Python, and with
5359 good features such as failure-case shrinkers. I first used it on a
5360 project where we needed to verify that a vendor's Python and non-Python
5361 implementations of an algorithm matched, and it found about a dozen
5362 cases that previous example-based testing and code inspections had not.
5363 Since then I've been evangelizing for it at our firm.
5364 Russel Winder
5366 I am using Hypothesis as an integral part of my Python workshops. Test‐
5367 ing is an integral part of Python programming and whilst unittest and,
5368 better, pytest can handle example-based testing, property-based testing
5369 is increasingly far more important than example-base testing, and Hy‐
5370 pothesis fits the bill.
5371 Wellfire Interactive
5373 We've been using Hypothesis in a variety of client projects, from test‐
5374 ing Django-related functionality to domain-specific calculations. It
5375 both speeds up and simplifies the testing process since there's so much
5376 less tedious and error-prone work to do in identifying edge cases. Test
5377 coverage is nice but test depth is even nicer, and it's much easier to
5378 get meaningful test depth using Hypothesis.
5379 Cody Kochmann
5381 Hypothesis is being used as the engine for random object generation
5382 with my open source function fuzzer battle_tested which maps all behav‐
5383 iors of a function allowing you to minimize the chance of unexpected
5384 crashes when running code in production.
5385 With how efficient Hypothesis is at generating the edge cases that
5387 cause unexpected behavior occur, battle_tested is able to map out the
5388 entire behavior of most functions in less than a few seconds.
5389 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
5391 ing it.
5392 Merchise Autrement
5394 Just minutes after our first use of hypothesis we uncovered a subtle
5395 bug in one of our most used library. Since then, we have increasingly
5396 used hypothesis to improve the quality of our testing in libraries and
5397 applications as well.
5398 Florian Kromer
5400 At Roboception GmbH I use Hypothesis to implement fully automated
5401 stateless and stateful reliability tests for the 3D sensor rc_visard
5402 and robotic software components .
5403 Thank you very much for creating the (probably) most powerful prop‐
5405 erty-based testing framework.
5406 Reposit Power
5408 With a micro-service architecture, testing between services is made
5409 easy using Hypothesis in integration testing. Ensuring everything is
5410 running smoothly is vital to help maintain a secure network of Virtual
5411 Power Plants.
5412 It allows us to find potential bugs and edge cases with relative ease
5414 and minimal overhead. As our architecture relies on services communi‐
5415 cating effectively, Hypothesis allows us to strictly test for the kind
5416 of data which moves around our services, particularly our backend
5417 Python applications.
5418 Your name goes here
5420 I know there are many more, because I keep finding out about new people
5421 I'd never even heard of using Hypothesis. If you're looking to way to
5422 give back to a tool you love, adding your name here only takes a moment
5423 and would really help a lot. As per instructions at the top, just send
5424 me a pull request and I'll add you to the list.
5425
5427 The following is a non-exhaustive list of open source projects I know
5428 are using Hypothesis. If you're aware of any others please add them to
5429 the list! The only inclusion criterion right now is that if it's a
5430 Python library then it should be available on PyPI.
5431 You can find hundreds more from the Hypothesis page at libraries.io,
5433 and thousands on GitHub. Hypothesis has over 100,000 downloads per
5434 week, and was used by more than 4% of Python users surveyed by the PSF
5435 in 2020.
5436 • aur
5438 • argon2_cffi
5440 • attrs
5442 • axelrod
5444 • bidict
5446 • binaryornot
5448 • brotlicffi
5450 • chardet
5452 • cmph-cffi
5454 • cryptography
5456 • dbus-signature-pyparsing
5458 • dry-python/returns
5460 • fastnumbers
5462 • flocker
5464 • flownetpy
5466 • funsize
5468 • fusion-index
5470 • hyper-h2
5472 • into-dbus-python
5474 • justbases
5476 • justbytes
5478 • loris
5480 • mariadb-dyncol
5482 • mercurial
5484 • natsort
5486 • poliastro
5488 • pretext
5490 • priority
5492 • PyCEbox
5494 • PyPy
5496 • pyrsistent
5498 • python-humble-utils
5500 • pyudev
5502 • qutebrowser
5504 • RubyMarshal
5506 • Segpy
5508 • simoa
5510 • srt
5512 • tchannel
5514 • vdirsyncer
5516 • wcag-contrast-ratio
5518 • yacluster
5520 • yturl
5522 • zenml
5524
5526 Hypothesis has been eagerly used and extended by the open source commu‐
5527 nity. This page lists extensions and applications; you can find more
5528 or newer packages by searching PyPI by keyword or filter by classifier,
5529 or search libraries.io.
5530 If there's something missing which you think should be here, let us
5532 know!
5533 NOTE:
5535 Being listed on this page does not imply that the Hypothesis main‐
5536 tainers endorse a package.
5537 External strategies
5539 Some packages provide strategies directly:
5540 • hypothesis-fspaths - strategy to generate filesystem paths.
5542 • hypothesis-geojson - strategy to generate GeoJson.
5544 • hypothesis-geometry - strategies to generate geometric objects.
5546 • hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
5548 • hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
5550 • hypothesis-ros - strategies to generate messages and parameters for
5552 the Robot Operating System.
5553 • hypothesis-csv - strategy to generate CSV files.
5555 • hypothesis-networkx - strategy to generate networkx graphs.
5557 • hypothesis-bio - strategies for bioinformatics data, such as DNA,
5559 codons, FASTA, and FASTQ formats.
5560 • hypothesmith - strategy to generate syntatically-valid Python code.
5562 Others provide a function to infer a strategy from some other schema:
5564 • hypothesis-jsonschema - infer strategies from JSON schemas.
5566 • lollipop-hypothesis - infer strategies from lollipop schemas.
5568 • hypothesis-drf - infer strategies from a djangorestframework seri‐
5570 aliser.
5571 • hypothesis-graphql - infer strategies from GraphQL schemas.
5573 • hypothesis-mongoengine - infer strategies from a mongoengine model.
5575 • hypothesis-pb - infer strategies from Protocol Buffer schemas.
5577 Or some other custom integration, such as a "hypothesis" entry point:
5579 • deal is a design-by-contract library with built-in Hypothesis sup‐
5581 port.
5582 • icontract-hypothesis infers strategies from icontract code contracts.
5584 • Pandera schemas all have a .strategy() method, which returns a strat‐
5586 egy for matching DataFrames.
5587 • Pydantic automatically registers constrained types - so builds() and
5589 from_type() "just work" regardless of the underlying implementation.
5590 Other cool things
5592 schemathesis is a tool for testing web applications built with Open API
5593 / Swagger specifications. It reads the schema and generates test cases
5594 which will ensure that the application is compliant with its schema.
5595 The application under test could be written in any language, the only
5596 thing you need is a valid API schema in a supported format. Includes
5597 CLI and convenient pytest integration. Powered by Hypothesis and
5598 hypothesis-jsonschema, inspired by the earlier swagger-conformance li‐
5599 brary.
5600 Trio is an async framework with "an obsessive focus on usability and
5602 correctness", so naturally it works with Hypothesis! pytest-trio in‐
5603 cludes a custom hook that allows @given(...) to work with Trio-style
5604 async test functions, and hypothesis-trio includes stateful testing ex‐
5605 tensions to support concurrent programs.
5606 pymtl3 is "an open-source Python-based hardware generation, simulation,
5608 and verification framework with multi-level hardware modeling support",
5609 which ships with Hypothesis integrations to check that all of those
5610 levels are equivalent, from function-level to register-transfer level
5611 and even to hardware.
5612 libarchimedes makes it easy to use Hypothesis in the Hy language, a
5614 Lisp embedded in Python.
5615 battle_tested is a fuzzing tool that will show you how your code can
5617 fail - by trying all kinds of inputs and reporting whatever happens.
5618 pytest-subtesthack functions as a workaround for issue #377.
5620 returns uses Hypothesis to verify that Higher Kinded Types correctly
5622 implement functor, applicative, monad, and other laws; allowing a
5623 declarative approach to be combined with traditional pythonic code.
5624 icontract-hypothesis includes a ghostwriter for test files and IDE in‐
5626 tegrations such as icontract-hypothesis-vim,
5627 icontract-hypothesis-pycharm, and icontract-hypothesis-vscode - you can
5628 run a quick 'smoke test' with only a few keystrokes for any type-anno‐
5629 tated function, even if it doesn't have any contracts!
5630 Writing an extension
5632 See CONTRIBUTING.rst for more information.
5633 New strategies can be added to Hypothesis, or published as an external
5635 package on PyPI - either is fine for most strategies. If in doubt, ask!
5636 It's generally much easier to get things working outside, because
5638 there's more freedom to experiment and fewer requirements in stability
5639 and API style. We're happy to review and help with external packages as
5640 well as pull requests!
5641 If you're thinking about writing an extension, please name it hypothe‐
5643 sis-{something} - a standard prefix makes the community more visible
5644 and searching for extensions easier. And make sure you use the Frame‐
5645 work :: Hypothesis trove classifier!
5646 On the other hand, being inside gets you access to some deeper imple‐
5648 mentation features (if you need them) and better long-term guarantees
5649 about maintenance. We particularly encourage pull requests for new
5650 composable primitives that make implementing other strategies easier,
5651 or for widely used types in the standard library. Strategies for other
5652 things are also welcome; anything with external dependencies just goes
5653 in hypothesis.extra.
5654 Tools such as assertion helpers may also need to check whether the cur‐
5656 rent test is using Hypothesis:
5657 hypothesis.currently_in_test_context()
5659 Return True if the calling code is currently running inside an
5660 @given or stateful test, False otherwise.
5661 This is useful for third-party integrations and assertion
5663 helpers which may be called from traditional or property-based
5664 tests, but can only use assume() or target() in the latter case.
5665 Hypothesis integration via setuptools entry points
5667 If you would like to ship Hypothesis strategies for a custom type - ei‐
5668 ther as part of the upstream library, or as a third-party extension,
5669 there's a catch: from_type() only works after the corresponding call to
5670 register_type_strategy(), and you'll have the same problem with
5671 register_random(). This means that either
5672 • you have to try importing Hypothesis to register the strategy when
5674 your library is imported, though that's only useful at test time, or
5675 • the user has to call a 'register the strategies' helper that you pro‐
5677 vide before running their tests
5678 Entry points are Python's standard way of automating the latter: when
5680 you register a "hypothesis" entry point in your setup.py, we'll import
5681 and run it automatically when hypothesis is imported. Nothing happens
5682 unless Hypothesis is already in use, and it's totally seamless for
5683 downstream users!
5684 Let's look at an example. You start by adding a function somewhere in
5686 your package that does all the Hypothesis-related setup work:
5687 # mymodule.py
5689 class MyCustomType:
5692 def __init__(self, x: int):
5693 assert x >= 0, f"got {x}, but only positive numbers are allowed"
5694 self.x = x
5695 def _hypothesis_setup_hook():
5698 import hypothesis.strategies as st
5699 st.register_type_strategy(MyCustomType, st.integers(min_value=0))
5701 and then tell setuptools that this is your "hypothesis" entry point:
5703 # setup.py
5705 # You can list a module to import by dotted name
5707 entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}
5708 # Or name a specific function too, and Hypothesis will call it for you
5710 entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}
5711 And that's all it takes!
5713 NOTE:
5715 On Python 3.7, where the importlib.metadata module is not in the
5716 standard library, loading entry points requires either the
5717 importlib_metadata (preferred) or setuptools (fallback) package to
5718 be installed.
5719 Interaction with pytest-cov
5721 Because pytest does not load plugins from entrypoints in any particular
5722 order, using the Hypothesis entrypoint may import your module before
5723 pytest-cov starts. This is a known issue, but there are workarounds.
5724 You can use coverage run pytest ... instead of pytest --cov ..., opting
5726 out of the pytest plugin entirely. Alternatively, you can ensure that
5727 Hypothesis is loaded after coverage measurement is started by disabling
5728 the entrypoint, and loading our pytest plugin from your conftest.py in‐
5729 stead:
5730 echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
5732 pytest -p "no:hypothesispytest" ...
5733
5735 This is a record of all past Hypothesis releases and what went into
5736 them, in reverse chronological order. All previous releases should
5737 still be available on PyPI.
5738 Hypothesis 6.x
5740 Current pull request
5741 RELEASE_TYPE: patch
5742 This patch fixes from_type() on Python 3.11, following
5744 python/cpython#93754.
5745 6.45.0 - 2022-04-22
5747 This release updates xps.indices() by introducing an allow_newaxis ar‐
5748 gument, defaulting to False. If allow_newaxis=True, indices can be gen‐
5749 erated that add dimensions to arrays, which is achieved by the indexer
5750 containing None. This change is to support a specification change that
5751 expand dimensions via indexing (data-apis/array-api#408).
5752 6.44.0 - 2022-04-21
5754 This release adds the option to pass strategies for generating names
5755 for:
5756 • pandas.Index created via indexes()
5758 • pandas.Series created via series()
5760 Hacked together by Sam Watts :)
5762 6.43.3 - 2022-04-18
5764 This patch updates the type annotations for @given so that type-check‐
5765 ers will warn on mixed positional and keyword arguments, as well as
5766 fixing issue #3296.
5767 6.43.2 - 2022-04-16
5769 Fixed a type annotation for pyright --strict (issue #3287).
5770 6.43.1 - 2022-04-13
5772 This patch makes it an explicit error to call register_type_strategy()
5773 with a Pydantic GenericModel and a callable, because GenericModel isn't
5774 actually a generic type at runtime and so you have to register each of
5775 the "parametrized versions" (actually subclasses!) manually. See issue
5776 #2940 for more details.
5777 6.43.0 - 2022-04-12
5779 This release makes it an explicit error to apply @pytest.fixture to a
5780 function which has already been decorated with @given(). Previously,
5781 pytest would convert your test to a fixture, and then never run it.
5782 6.42.3 - 2022-04-10
5784 This patch fixes from_type() on a TypedDict with complex annotations,
5785 defined in a file using from __future__ import annotations. Thanks to
5786 Katelyn Gigante for identifying and fixing this bug!
5787 6.42.2 - 2022-04-10
5789 The Hypothesis pytest plugin was not outputting valid xunit2 nodes when
5790 --junit-xml was specified. This has been broken since Pytest 5.4, which
5791 changed the internal API for adding nodes to the junit report.
5792 This also fixes the issue when using hypothesis with --junit-xml and
5794 pytest-xdist where the junit xml report would not be xunit2 compatible.
5795 Now, when using with pytest-xdist, the junit report will just omit the
5796 <properties> node.
5797 For more details, see this pytest issue, this pytest issue, and issue
5799 #1935
5800 Thanks to Brandon Chinn for this bug fix!
5802 6.42.1 - 2022-04-10
5804 This patch fixes pretty-printing of regular expressions in Python
5805 3.11.0a7, and updates our vendored list of top-level domains,.
5806 6.42.0 - 2022-04-09
5808 This release makes st.functions(pure=True) less noisy (issue #3253),
5809 and generally improves pretty-printing of functions.
5810 6.41.0 - 2022-04-01
5812 This release changes the implementation of infer to be an alias for
5813 python:Ellipsis. E.g. @given(a=infer) is now equivalent to
5814 @given(a=...). Furthermore, @given(...) can now be specified so that
5815 @given will infer the strategies for all arguments of the decorated
5816 function based on its annotations.
5817 6.40.3 - 2022-04-01
5819 This patch simplifies the repr of the strategies namespace returned in
5820 make_strategies_namespace(), e.g.
5821 >>> from hypothesis.extra.array_api import make_strategies_namespace
5823 >>> from numpy import array_api as xp
5824 >>> xps = make_strategies_namespace(xp)
5825 >>> xps
5826 make_strategies_namespace(numpy.array_api)
5827 6.40.2 - 2022-04-01
5829 Fixed from_type() support for PEP 604 union types, like int | None (‐
5830 issue #3255).
5831 6.40.1 - 2022-04-01
5833 Fixed an internal error when given() was passed a lambda.
5834 6.40.0 - 2022-03-29
5836 The Ghostwriter can now write tests which check that two or more func‐
5837 tions are equivalent on valid inputs, or raise the same type of excep‐
5838 tion for invalid inputs (issue #3267).
5839 6.39.6 - 2022-03-27
5841 This patch makes some quality-of-life improvements to the Ghostwriter:
5842 we guess the text() strategy for arguments named text (...obvious in
5843 hindsight, eh?); and improved the error message if you accidentally
5844 left in a nothing() or broke your rich install.
5845 6.39.5 - 2022-03-26
5847 This patch improves our error detection and message when Hypothesis is
5848 run on a Python implementation without support for -0.0, which is re‐
5849 quired for the floats() strategy but can be disabled by unsafe compiler
5850 options (issue #3265).
5851 6.39.4 - 2022-03-17
5853 This patch tweaks some internal formatting. There is no user-visible
5854 change.
5855 6.39.3 - 2022-03-07
5857 If the shrink phase is disabled, we now stop the generate phase as soon
5858 as an error is found regardless of the value of the report_multiple_ex‐
5859 amples setting, since that's probably what you wanted (issue #3244).
5860 6.39.2 - 2022-03-07
5862 This patch clarifies rare error messages in builds() (issue #3225) and
5863 floats() (issue #3207).
5864 6.39.1 - 2022-03-03
5866 This patch fixes a regression where the bound inner function
5867 (your_test.hypothesis.inner_test) would be invoked with positional ar‐
5868 guments rather than passing them by name, which broke pytest-asyncio (‐
5869 issue #3245).
5870 6.39.0 - 2022-03-01
5872 This release improves Hypothesis' handling of positional-only argu‐
5873 ments, which are now allowed @st.composite strategies.
5874 On Python 3.8 and later, the first arguments to builds() and
5876 from_model() are now natively positional-only. In cases which were al‐
5877 ready errors, the TypeError from incorrect usage will therefore be
5878 raises immediately when the function is called, rather than when the
5879 strategy object is used.
5880 6.38.0 - 2022-02-26
5882 This release makes floats() error consistently when your floating-point
5883 hardware has been configured to violate IEEE-754 for subnormal numbers,
5884 instead of only when an internal assertion was tripped (issue #3092).
5885 If this happens to you, passing allow_subnormal=False will suppress the
5887 explicit error. However, we strongly recommend fixing the root cause
5888 by disabling global-effect unsafe-math compiler options instead, or at
5889 least consulting e.g. Simon Byrne's Beware of fast-math explainer
5890 first.
5891 6.37.2 - 2022-02-21
5893 This patch fixes a bug in stateful testing, where returning a single
5894 value wrapped in multiple() would be printed such that the assigned
5895 variable was a tuple rather than the single element (issue #3236).
5896 6.37.1 - 2022-02-21
5898 This patch fixes a warning under pytest 7 relating to our rich trace‐
5899 back display logic (issue #3223).
5900 6.37.0 - 2022-02-18
5902 When distinguishing multiple errors, Hypothesis now looks at the inner
5903 exceptions of PEP 654 ExceptionGroups.
5904 6.36.2 - 2022-02-13
5906 This patch updates our vendored list of top-level domains, which is
5907 used by the provisional domains() strategy.
5908 6.36.1 - 2022-01-31
5910 This patch fixes some deprecation warnings from pytest 7.0, along with
5911 some code formatting and docs updates.
5912 6.36.0 - 2022-01-19
5914 This release disallows using python:typing.Final with from_type() and
5915 register_type_strategy().
5916 Why? Because Final can only be used during class definition. We don't
5918 generate class attributes.
5919 It also does not make sense as a runtime type on its own.
5921 6.35.1 - 2022-01-17
5923 This patch fixes hypothesis write output highlighting with rich version
5924 12.0 and later.
5925 6.35.0 - 2022-01-08
5927 This release disallows using python:typing.ClassVar with from_type()
5928 and register_type_strategy().
5929 Why? Because ClassVar can only be used during class definition. We
5931 don't generate class attributes.
5932 It also does not make sense as a runtime type on its own.
5934 6.34.2 - 2022-01-05
5936 This patch updates our vendored list of top-level domains, which is
5937 used by the provisional domains() strategy.
5938 6.34.1 - 2021-12-31
5940 This patch fixes issue #3169, an extremely rare bug which would trigger
5941 if an internal least-recently-reused cache dropped a newly added entry
5942 immediately after it was added.
5943 6.34.0 - 2021-12-31
5945 This release fixes issue #3133 and issue #3144, where attempting to
5946 generate Pandas series of lists or sets would fail with confusing er‐
5947 rors if you did not specify dtype=object.
5948 6.33.0 - 2021-12-30
5950 This release disallows using python:typing.TypeAlias with from_type()
5951 and register_type_strategy().
5952 Why? Because TypeAlias is not really a type, it is a tag for type
5954 checkers that some expression is a type alias, not something else.
5955 It does not make sense for Hypothesis to resolve it as a strategy.
5957 References issue #2978.
5958 6.32.1 - 2021-12-23
5960 This patch updates our autoformatting tools, improving our code style
5961 without any API changes.
5962 6.32.0 - 2021-12-23
5964 This release drops support for Python 3.6, which reached end of life
5965 upstream on 2021-12-23.
5966 6.31.6 - 2021-12-15
5968 This patch adds a temporary hook for a downstream tool, which is not
5969 part of the public API.
5970 6.31.5 - 2021-12-14
5972 This release updates our copyright headers to use a general authorship
5973 statement and omit the year.
5974 6.31.4 - 2021-12-11
5976 This patch makes the .example() method more representative of test-time
5977 data generation, albeit often at a substantial cost to readability (‐
5978 issue #3182).
5979 6.31.3 - 2021-12-10
5981 This patch improves annotations on some of Hypothesis' internal func‐
5982 tions, in order to deobfuscate the signatures of some strategies. In
5983 particular, strategies shared between hypothesis.extra.numpy and the
5984 hypothesis.extra.array_api extra will benefit from this patch.
5985 6.31.2 - 2021-12-10
5987 This patch fix invariants display in stateful falsifying examples (‐
5988 issue #3185).
5989 6.31.1 - 2021-12-10
5991 This patch updates xps.indices() so no flat indices are generated, i.e.
5992 generated indices will now always explicitly cover each axes of an ar‐
5993 ray if no ellipsis is present. This is to be consistent with a specifi‐
5994 cation change that dropped support for flat indexing (#272).
5995 6.31.0 - 2021-12-09
5997 This release makes us compatible with Django 4.0, in particular by
5998 adding support for use of zoneinfo timezones (though we respect the new
5999 USE_DEPRECATED_PYTZ setting if you need it).
6000 6.30.1 - 2021-12-05
6002 This patch updates our vendored list of top-level domains, which is
6003 used by the provisional domains() strategy.
6004 6.30.0 - 2021-12-03
6006 This release adds an allow_subnormal argument to the floats() strategy,
6007 which can explicitly toggle the generation of subnormal floats (issue
6008 #3155). Disabling such generation is useful when testing flush-to-zero
6009 builds of libraries.
6010 nps.from_dtype() and xps.from_dtype() can also accept the allow_subnor‐
6012 mal argument, and xps.from_dtype() or xps.arrays() will disable subnor‐
6013 mals by default if the array module xp is detected to flush-to-zero
6014 (like is typical with CuPy).
6015 6.29.3 - 2021-12-02
6017 This patch fixes a bug in mutually_broadcastable_shapes(), which re‐
6018 stricted the patterns of singleton dimensions that could be generated
6019 for dimensions that extended beyond base_shape (issue #3170).
6020 6.29.2 - 2021-12-02
6022 This patch clarifies our pretty-printing of DataFrames (issue #3114).
6023 6.29.1 - 2021-12-02
6025 This patch documents timezones() Windows-only requirement for the
6026 tzdata package, and ensures that pip install hypothesis[zoneinfo] will
6027 install the latest version.
6028 6.29.0 - 2021-11-29
6030 This release teaches builds() to use deferred() when resolving unrecog‐
6031 nised type hints, so that you can conveniently register strategies for
6032 recursive types with constraints on some arguments (issue #3026):
6033 class RecursiveClass:
6035 def __init__(self, value: int, next_node: typing.Optional["SomeClass"]):
6036 assert value > 0
6037 self.value = value
6038 self.next_node = next_node
6039 st.register_type_strategy(
6042 RecursiveClass, st.builds(RecursiveClass, value=st.integers(min_value=1))
6043 )
6044 6.28.1 - 2021-11-28
6046 This release fixes some internal calculations related to collection
6047 sizes (issue #3143).
6048 6.28.0 - 2021-11-28
6050 This release modifies our pytest plugin, to avoid importing Hypothesis
6051 and therefore triggering Hypothesis' entry points for test suites where
6052 Hypothesis is installed but not actually used (issue #3140).
6053 6.27.3 - 2021-11-28
6055 This release fixes issue #3080, where from_type() failed on unions con‐
6056 taining PEP 585 builtin generic types (like list[int]) in Python 3.9
6057 and later.
6058 6.27.2 - 2021-11-26
6060 This patch makes the hypothesis codemod command somewhat faster.
6061 6.27.1 - 2021-11-22
6063 This patch changes the backing datastructures of register_random() and
6064 a few internal caches to use weakref.WeakKeyDictionary. This reduces
6065 memory usage and may improve performance when registered Random in‐
6066 stances are only used for a subset of your tests (issue #3131).
6067 6.27.0 - 2021-11-22
6069 This release teaches Hypothesis' multiple-error reporting to format
6070 tracebacks using pytest or better-exceptions, if they are installed and
6071 enabled (issue #3116).
6072 6.26.0 - 2021-11-21
6074 Did you know that of the 264 possible floating-point numbers, 253 of
6075 them are nan - and Python prints them all the same way?
6076 While nans usually have all zeros in the sign bit and mantissa, this
6078 isn't always true, and 'signaling' nans might trap or error. To help
6079 distinguish such errors in e.g. CI logs, Hypothesis now prints -nan for
6080 negative nans, and adds a comment like # Saw 3 signaling NaNs if appli‐
6081 cable.
6082 6.25.0 - 2021-11-19
6084 This release adds special filtering logic to make a few special cases
6085 like s.map(lambda x: x) and lists().filter(len) more efficient (issue
6086 #2701).
6087 6.24.6 - 2021-11-18
6089 This patch makes floats() generate "subnormal" floating point numbers
6090 more often, as these rare values can have strange interactions with
6091 unsafe compiler optimisations like -ffast-math (issue #2976).
6092 6.24.5 - 2021-11-16
6094 This patch fixes a rare internal error in the datetimes() strategy,
6095 where the implementation of allow_imaginary=False crashed when checking
6096 a time during the skipped hour of a DST transition if the DST offset is
6097 negative - only true of Europe/Dublin, who we presume have their rea‐
6098 sons - and the tzinfo object is a pytz timezone (which predates PEP
6099 495).
6100 6.24.4 - 2021-11-15
6102 This patch gives Hypothesis it's own internal Random instance, ensuring
6103 that test suites which reset the global random state don't induce weird
6104 correlations between property-based tests (issue #2135).
6105 6.24.3 - 2021-11-13
6107 This patch updates documentation of note() (issue #3147).
6108 6.24.2 - 2021-11-05
6110 This patch updates internal testing for the Array API extra to be con‐
6111 sistent with new specification changes: sum() not accepting boolean ar‐
6112 rays (#234), unique() split into separate functions (#275), and treat‐
6113 ing NaNs as distinct (#310). It has no user visible impact.
6114 6.24.1 - 2021-11-01
6116 This patch updates our vendored list of top-level domains, which is
6117 used by the provisional domains() strategy.
6118 6.24.0 - 2021-10-23
6120 This patch updates our vendored list of top-level domains, which is
6121 used by the provisional domains() strategy.
6122 (did you know that gTLDs can be both added and removed?)
6124 6.23.4 - 2021-10-20
6126 This patch adds an error for when shapes in xps.arrays() is not passed
6127 as either a valid shape or strategy.
6128 6.23.3 - 2021-10-18
6130 This patch updates our formatting with shed.
6131 6.23.2 - 2021-10-08
6133 This patch replaces external links to NumPy API docs with
6134 sphinx.ext.intersphinx cross-references. It is purely a documentation
6135 improvement.
6136 6.23.1 - 2021-09-29
6138 This patch cleans up internal logic for xps.arrays(). There is no
6139 user-visible change.
6140 6.23.0 - 2021-09-26
6142 This release follows pytest in considering SystemExit and GeneratorExit
6143 exceptions to be test failures, meaning that we will shink to minimal
6144 examples and check for flakiness even though they subclass BaseExcep‐
6145 tion directly (issue #2223).
6146 KeyboardInterrupt continues to interrupt everything, and will be
6148 re-raised immediately.
6149 6.22.0 - 2021-09-24
6151 This release adds LiveServerTestCase and StaticLiveServerTestCase for
6152 django test. Thanks to Ivan Tham for this feature!
6153 6.21.6 - 2021-09-19
6155 This patch fixes some new linter warnings such as flake8-bugbear's B904
6156 for explicit exception chaining, so tracebacks might be a bit nicer.
6157 6.21.5 - 2021-09-16
6159 This release fixes None being inferred as the float64 dtype in
6160 from_dtype() and arrays() from the Array API extra.
6161 6.21.4 - 2021-09-16
6163 This release fixes the type hint for the @given() decorator when deco‐
6164 rating an async function (issue #3099).
6165 6.21.3 - 2021-09-15
6167 This release improves Ghostwritten tests for builtins (issue #2977).
6168 6.21.2 - 2021-09-15
6170 This release deprecates use of both min_dims > len(shape) and max_dims
6171 > len(shape) when allow_newaxis == False in basic_indices() (issue
6172 #3091).
6173 6.21.1 - 2021-09-13
6175 This release improves the behaviour of builds() and from_type() in cer‐
6176 tain situations involving decorators (issue #2495 and issue #3029).
6177 6.21.0 - 2021-09-11
6179 This release introduces strategies for array/tensor libraries adopting
6180 the Array API standard (issue #3037). They are available in the hy‐
6181 pothesis.extra.array_api extra, and work much like the existing
6182 strategies for NumPy.
6183 6.20.1 - 2021-09-10
6185 This patch fixes issue #961, where calling given() inline on a bound
6186 method would fail to handle the self argument correctly.
6187 6.20.0 - 2021-09-09
6189 This release allows slices() to generate step=None, and fixes an
6190 off-by-one error where the start index could be equal to size. This
6191 works fine for all Python sequences and Numpy arrays, but is undefined
6192 behaviour in the Array API standard (see pull request #3065).
6193 6.19.0 - 2021-09-08
6195 This release makes stateful testing more likely to tell you if you do
6196 something unexpected and unsupported:
6197 • The return_value health check now applies to rule() and initialize()
6199 rules, if they don't have target bundles, as well as invariant().
6200 • Using a consumes() bundle as a target is deprecated, and will be an
6202 error in a future version.
6203 If existing code triggers these new checks, check for related bugs and
6205 misunderstandings - these patterns never had any effect.
6206 6.18.0 - 2021-09-06
6208 This release teaches from_type() a neat trick: when resolving an
6209 python:typing.Annotated type, if one of the annotations is a strategy
6210 object we use that as the inferred strategy. For example:
6211 PositiveInt = Annotated[int, st.integers(min_value=1)]
6213 If there are multiple strategies, we use the last outer-most annota‐
6215 tion. See issue #2978 and pull request #3082 for discussion.
6216 Requires Python 3.9 or later for get_type_hints(..., include_ex‐
6218 tras=False).
6219 6.17.4 - 2021-08-31
6221 This patch makes unique arrays() much more efficient, especially when
6222 there are only a few valid elements - such as for eight-bit integers (‐
6223 issue #3066).
6224 6.17.3 - 2021-08-30
6226 This patch fixes the repr of array_shapes().
6227 6.17.2 - 2021-08-30
6229 This patch wraps some internal helper code in our proxies decorator to
6230 prevent mutations of method docstrings carrying over to other instances
6231 of the respective methods.
6232 6.17.1 - 2021-08-29
6234 This patch moves some internal helper code in preparation for issue
6235 #3065. There is no user-visible change, unless you depended on undocu‐
6236 mented internals.
6237 6.17.0 - 2021-08-27
6239 This release adds type annotations to the stateful testing API.
6240 Thanks to Ruben Opdebeeck for this contribution!
6242 6.16.0 - 2021-08-27
6244 This release adds the DrawFn type as a reusable type hint for the draw
6245 argument of @composite functions.
6246 Thanks to Ruben Opdebeeck for this contribution!
6248 6.15.0 - 2021-08-22
6250 This release emits a more useful error message when @given() is applied
6251 to a coroutine function, i.e. one defined using async def (issue
6252 #3054).
6253 This was previously only handled by the generic return_value health
6255 check, which doesn't direct you to use either a custom executor or a
6256 library such as pytest-trio or pytest-asyncio to handle it for you.
6257 6.14.9 - 2021-08-20
6259 This patch fixes a regression in Hypothesis 6.14.8, where from_type()
6260 failed to resolve types which inherit from multiple parametrised
6261 generic types, affecting the returns package (issue #3060).
6262 6.14.8 - 2021-08-16
6264 This patch ensures that registering a strategy for a subclass of a a
6265 parametrised generic type such as class Lines(Sequence[str]): will not
6266 "leak" into unrelated strategies such as st.from_type(Sequence[int]) (‐
6267 issue #2951). Unfortunately this fix requires PEP 560, meaning Python
6268 3.7 or later.
6269 6.14.7 - 2021-08-14
6271 This patch fixes issue #3050, where attrs classes could cause an inter‐
6272 nal error in the ghostwriter.
6273 6.14.6 - 2021-08-07
6275 This patch improves the error message for issue #3016, where PEP 585
6276 builtin generics with self-referential forward-reference strings cannot
6277 be resolved to a strategy by from_type().
6278 6.14.5 - 2021-07-27
6280 This patch fixes hypothesis.strategies._internal.types.is_a_new_type.
6281 It was failing on Python 3.10.0b4, where NewType is a function.
6282 6.14.4 - 2021-07-26
6284 This patch fixes from_type() and register_type_strategy() for
6285 python:typing.NewType on Python 3.10, which changed the underlying im‐
6286 plementation (see bpo-44353 for details).
6287 6.14.3 - 2021-07-18
6289 This patch updates our autoformatting tools, improving our code style
6290 without any API changes.
6291 6.14.2 - 2021-07-12
6293 This patch ensures that we shorten tracebacks for tests which fail due
6294 to inconsistent data generation between runs (i.e. raise Flaky).
6295 6.14.1 - 2021-07-02
6297 This patch updates some internal type annotations. There is no
6298 user-visible change.
6299 6.14.0 - 2021-06-09
6301 The explain phase now requires shrinking to be enabled, and will be au‐
6302 tomatically skipped for deadline-exceeded errors.
6303 6.13.14 - 2021-06-04
6305 This patch improves the tuples() strategy type annotations, to preserve
6306 the element types for up to length-five tuples (issue #3005).
6307 As for one_of(), this is the best we can do before a planned extension
6309 to PEP 646 is released, hopefully in Python 3.11.
6310 6.13.13 - 2021-06-04
6312 This patch teaches the Ghostwriter how to find custom ufuncs from any
6313 module that defines them, and that yaml.unsafe_load() does not undo
6314 yaml.safe_load().
6315 6.13.12 - 2021-06-03
6317 This patch reduces the amount of internal code excluded from our test
6318 suite's code coverage checks.
6319 There is no user-visible change.
6321 6.13.11 - 2021-06-02
6323 This patch removes some old internal helper code that previously ex‐
6324 isted to make Python 2 compatibility easier.
6325 There is no user-visible change.
6327 6.13.10 - 2021-05-30
6329 This release adjusts some internal code to help make our test suite
6330 more reliable.
6331 There is no user-visible change.
6333 6.13.9 - 2021-05-30
6335 This patch cleans up some internal code related to filtering strate‐
6336 gies.
6337 There is no user-visible change.
6339 6.13.8 - 2021-05-28
6341 This patch slightly improves the performance of some internal code for
6342 generating integers.
6343 6.13.7 - 2021-05-27
6345 This patch fixes a bug in from_regex() that caused from_regex("", full‐
6346 match=True) to unintentionally generate non-empty strings (issue
6347 #4982).
6348 The only strings that completely match an empty regex pattern are empty
6350 strings.
6351 6.13.6 - 2021-05-26
6353 This patch fixes a bug that caused integers() to shrink towards nega‐
6354 tive values instead of positive values in some cases.
6355 6.13.5 - 2021-05-24
6357 This patch fixes rare cases where hypothesis write --binary-op could
6358 print reproducing instructions from the internal search for an identity
6359 element.
6360 6.13.4 - 2021-05-24
6362 This patch removes some unnecessary intermediate list-comprehensions,
6363 using the latest versions of pyupgrade and shed.
6364 6.13.3 - 2021-05-23
6366 This patch adds a .hypothesis property to invalid test functions,
6367 bringing them inline with valid tests and fixing a bug where
6368 pytest-asyncio would swallow the real error message and mistakenly
6369 raise a version incompatibility error.
6370 6.13.2 - 2021-05-23
6372 Some of Hypothesis's numpy/pandas strategies use a fill argument to
6373 speed up generating large arrays, by generating a single fill value and
6374 sharing that value among many array slots instead of filling every sin‐
6375 gle slot individually.
6376 When no fill argument is provided, Hypothesis tries to detect whether
6378 it is OK to automatically use the elements argument as a fill strategy,
6379 so that it can still use the faster approach.
6380 This patch fixes a bug that would cause that optimization to trigger in
6382 some cases where it isn't 100% guaranteed to be OK.
6383 If this makes some of your numpy/pandas tests run more slowly, try
6385 adding an explicit fill argument to the relevant strategies to ensure
6386 that Hypothesis always uses the faster approach.
6387 6.13.1 - 2021-05-20
6389 This patch strengthens some internal import-time consistency checks for
6390 the built-in strategies.
6391 There is no user-visible change.
6393 6.13.0 - 2021-05-18
6395 This release adds URL fragment generation to the urls() strategy (issue
6396 #2908). Thanks to Pax (R. Margret) for contributing this patch at the
6397 PyCon US Mentored Sprints!
6398 6.12.1 - 2021-05-17
6400 This patch fixes issue #2964, where .map() and .filter() methods were
6401 omitted from the repr() of just() and sampled_from() strategies, since
6402 version 5.43.7.
6403 6.12.0 - 2021-05-06
6405 This release automatically rewrites some simple filters, such as inte‐
6406 gers().filter(lambda x: x > 9) to the more efficient inte‐
6407 gers(min_value=10), based on the AST of the predicate.
6408 We continue to recommend using the efficient form directly wherever
6410 possible, but this should be useful for e.g. pandera "Checks" where you
6411 already have a simple predicate and translating manually is really an‐
6412 noying. See issue #2701 for ideas about floats and simple text strate‐
6413 gies.
6414 6.11.0 - 2021-05-06
6416 hypothesis.target() now returns the observation value, allowing it to
6417 be conveniently used inline in expressions such as assert target(abs(a
6418 - b)) < 0.1.
6419 6.10.1 - 2021-04-26
6421 This patch fixes a deprecation warning if you're using recent versions
6422 of importlib-metadata (issue #2934), which we use to load third-party
6423 plugins such as Pydantic's integration. On older versions of
6424 importlib-metadata, there is no change and you don't need to upgrade.
6425 6.10.0 - 2021-04-17
6427 This release teaches the Ghostwriter to read parameter types from
6428 Sphinx, Google, or Numpy-style structured docstrings, and improves some
6429 related heuristics about how to test scientific and numerical programs.
6430 6.9.2 - 2021-04-15
6432 This release improves the Ghostwriter's handling of exceptions, by
6433 reading :raises ...: entries in function docstrings and ensuring that
6434 we don't suppresss the error raised by test assertions.
6435 6.9.1 - 2021-04-12
6437 This patch updates our autoformatting tools, improving our code style
6438 without any API changes.
6439 6.9.0 - 2021-04-11
6441 This release teaches from_type() how to see through python:typing.Anno‐
6442 tated. Thanks to Vytautas Strimaitis for reporting and fixing issue
6443 #2919!
6444 6.8.12 - 2021-04-11
6446 If rich is installed, the hypothesis write command will use it to syn‐
6447 tax-highlight the Ghostwritten code.
6448 6.8.11 - 2021-04-11
6450 This patch improves an error message from builds() when from_type()
6451 would be more suitable (issue #2930).
6452 6.8.10 - 2021-04-11
6454 This patch updates the type annotations for arrays() to reflect that
6455 shape: SearchStrategy[int] is supported.
6456 6.8.9 - 2021-04-07
6458 This patch fixes from_type() with abstract types which have either re‐
6459 quired but non-type-annotated arguments to __init__, or where
6460 from_type() can handle some concrete subclasses but not others.
6461 6.8.8 - 2021-04-07
6463 This patch teaches hypothesis write to check for possible roundtrips in
6464 several more cases, such as by looking for an inverse in the module
6465 which defines the function to test.
6466 6.8.7 - 2021-04-07
6468 This patch adds a more helpful error message if you try to call
6469 sampled_from() on an Enum which has no members, but does have data‐
6470 class()-style annotations (issue #2923).
6471 6.8.6 - 2021-04-06
6473 The fixed_dictionaries() strategy now preserves dict iteration order
6474 instead of sorting the keys. This also affects the pretty-printing of
6475 keyword arguments to @given() (issue #2913).
6476 6.8.5 - 2021-04-05
6478 This patch teaches hypothesis write to default to ghostwriting tests
6479 with --style=pytest only if pytest is installed, or --style=unittest
6480 otherwise.
6481 6.8.4 - 2021-04-01
6483 This patch adds type annotations for the settings decorator, to avoid
6484 an error when running mypy in strict mode.
6485 6.8.3 - 2021-03-28
6487 This patch improves the Ghostwriter's handling of strategies to gener‐
6488 ate various fiddly types including frozensets, keysviews, valuesviews,
6489 regex matches and patterns, and so on.
6490 6.8.2 - 2021-03-27
6492 This patch fixes some internal typos. There is no user-visible change.
6493 6.8.1 - 2021-03-14
6495 This patch lays more groundwork for filter rewriting (issue #2701).
6496 There is no user-visible change... yet.
6497 6.8.0 - 2021-03-11
6499 This release registers the remaining builtin types, and teaches
6500 from_type() to try resolving ForwardRef and Type references to built-in
6501 types.
6502 6.7.0 - 2021-03-10
6504 This release teaches RuleBasedStateMachine to avoid checking
6505 invariant()s until all initialize() rules have been run. You can en‐
6506 able checking of specific invariants for incompletely initialized ma‐
6507 chines by using @invariant(check_during_init=True) (issue #2868).
6508 In previous versions, it was possible if awkward to implement this be‐
6510 haviour using precondition() and an auxiliary variable.
6511 6.6.1 - 2021-03-09
6513 This patch improves the error message when from_type() fails to resolve
6514 a forward-reference inside a python:typing.Type such as Type["int"] (‐
6515 issue #2565).
6516 6.6.0 - 2021-03-07
6518 This release makes it an explicit error to apply invariant() to a
6519 rule() or initialize() rule in stateful testing. Such a combination
6520 had unclear semantics, especially in combination with precondition(),
6521 and was never meant to be allowed (issue #2681).
6522 6.5.0 - 2021-03-07
6524 This release adds the explain phase, in which Hypothesis attempts to
6525 explain why your test failed by pointing to suspicious lines of code
6526 (i.e. those which were always, and only, run on failing inputs). We
6527 plan to include "generalising" failing examples in this phase in a fu‐
6528 ture release (issue #2192).
6529 6.4.3 - 2021-03-04
6531 This patch fixes issue #2794, where nesting deferred() strategies
6532 within recursive() strategies could trigger an internal assertion.
6533 While it was always possible to get the same results from a more sensi‐
6534 ble strategy, the convoluted form now works too.
6535 6.4.2 - 2021-03-04
6537 This patch fixes several problems with mypy when --no-implicit-reexport
6538 was activated in user projects.
6539 Thanks to Nikita Sobolev for fixing issue #2884!
6541 6.4.1 - 2021-03-04
6543 This patch fixes an exception that occurs when using type unions of the
6544 typing_extensions Literal backport on Python 3.6.
6545 Thanks to Ben Anhalt for identifying and fixing this bug.
6547 6.4.0 - 2021-03-02
6549 This release fixes stateful testing methods with multiple
6550 precondition() decorators. Previously, only the outer-most precondi‐
6551 tion was checked (issue #2681).
6552 6.3.4 - 2021-02-28
6554 This patch refactors some internals of RuleBasedStateMachine. There is
6555 no change to the public API or behaviour.
6556 6.3.3 - 2021-02-26
6558 This patch moves some internal code, so that future work can avoid cre‐
6559 ating import cycles. There is no user-visible change.
6560 6.3.2 - 2021-02-25
6562 This patch enables register_type_strategy() for subclasses of
6563 python:typing.TypedDict. Previously, from_type() would ignore the reg‐
6564 istered strategy (issue #2872).
6565 Thanks to Ilya Lebedev for identifying and fixing this bug!
6567 6.3.1 - 2021-02-24
6569 This release lays the groundwork for automatic rewriting of simple fil‐
6570 ters, for example converting integers().filter(lambda x: x > 9) to in‐
6571 tegers(min_value=10).
6572 Note that this is not supported yet, and we will continue to recommend
6574 writing the efficient form directly wherever possible - predicate
6575 rewriting is provided mainly for the benefit of downstream libraries
6576 which would otherwise have to implement it for themselves (e.g. pandera
6577 and icontract-hypothesis). See issue #2701 for details.
6578 6.3.0 - 2021-02-20
6580 The Hypothesis pytest plugin now requires pytest version 4.6 or later.
6581 If the plugin detects an earlier version of pytest, it will automati‐
6582 cally deactivate itself.
6583 (4.6.x is the earliest pytest branch that still accepts community bug‐
6585 fixes.)
6586 Hypothesis-based tests should continue to work in earlier versions of
6588 pytest, but enhanced integrations provided by the plugin (such as --hy‐
6589 pothesis-show-statistics and other command-line flags) will no longer
6590 be available in obsolete pytest versions.
6591 6.2.0 - 2021-02-12
6593 If you use pytest-html, Hypothesis now includes the summary statistics
6594 for each test in the HTML report, whether or not the --hypothe‐
6595 sis-show-statistics argument was passed to show them in the com‐
6596 mand-line output.
6597 6.1.1 - 2021-01-31
6599 This patch updates our automatic code formatting to use shed, which in‐
6600 cludes autoflake, black, isort, and pyupgrade (issue #2780).
6601 6.1.0 - 2021-01-29
6603 This release teaches Hypothesis to distinguish between errors based on
6604 the __cause__ or __context__ of otherwise identical exceptions, which
6605 is particularly useful when internal errors can be wrapped by a li‐
6606 brary-specific or semantically appropriate exception such as:
6607 try:
6609 do_the_thing(foo, timeout=10)
6610 except Exception as err:
6611 raise FooError("Failed to do the thing") from err
6612 Earlier versions of Hypothesis only see the FooError, while we can now
6614 distinguish a FooError raised because of e.g. an internal assertion
6615 from one raised because of a TimeoutExceeded exception.
6616 6.0.4 - 2021-01-27
6618 This release prevents a race condition inside recursive() strategies.
6619 The race condition occurs when the same recursive() strategy is shared
6620 among tests that are running in multiple threads (issue #2717).
6621 6.0.3 - 2021-01-23
6623 This patch improves the type annotations for one_of(), by adding over‐
6624 loads to handle up to five distinct arguments as Union before falling
6625 back to Any, as well as annotating the | (__or__) operator for strate‐
6626 gies (issue #2765).
6627 6.0.2 - 2021-01-14
6629 This release makes some small improvements to how filtered strategies
6630 work. It should improve the performance of shrinking filtered strate‐
6631 gies, and may under some (probably rare) circumstances improve the di‐
6632 versity of generated examples.
6633 6.0.1 - 2021-01-13
6635 This patch fixes an interaction where our test statistics handling made
6636 Pytest's --junit-xml output fail to validate against the strict xunit2
6637 schema (issue #1975).
6638 6.0.0 - 2021-01-08
6640 Welcome to the next major version of Hypothesis!
6641 There are no new features here, as we release those in minor versions.
6643 Instead, 6.0 is a chance for us to remove deprecated features (many al‐
6644 ready converted into no-ops), and turn a variety of warnings into er‐
6645 rors.
6646 If you were running on the last version of Hypothesis 5.x without any
6648 Hypothesis deprecation warnings, this will be a very boring upgrade.
6649 In fact, nothing will change for you at all.
6650 Changes
6652 • Many functions now use PEP 3102 keyword-only arguments where passing
6653 positional arguments was deprecated since 5.5.
6654 • hypothesis.extra.django.from_model() no longer accepts model as a
6656 keyword argument, where it could conflict with fields named "model".
6657 • randoms() now defaults to use_true_random=False.
6659 • complex_numbers() no longer accepts min_magnitude=None; either use
6661 min_magnitude=0 or just omit the argument.
6662 • hypothesis.provisional.ip4_addr_strings and ip6_addr_strings are re‐
6664 moved in favor of ip_addresses(v=...).map(str).
6665 • register_type_strategy() no longer accepts generic types with type
6667 arguments, which were always pretty badly broken.
6668 • Using function-scoped pytest fixtures is now a health-check error,
6670 instead of a warning.
6671 TIP:
6673 The hypothesis codemod command can automatically refactor your code,
6674 particularly to convert positional to keyword arguments where those
6675 are now required.
6676 Hypothesis 5.x
6678 5.49.0 - 2021-01-07
6679 This release adds the function_scoped_fixture health check value, which
6680 can be used to suppress the existing warning that appears when @given
6681 is applied to a test that uses pytest function-scoped fixtures.
6682 (This warning exists because function-scoped fixtures only run once per
6684 function, not once per example, which is usually unexpected and can
6685 cause subtle problems.)
6686 When this warning becomes a health check error in a future release,
6688 suppressing it via Python warning settings will no longer be possible.
6689 In the rare case that once-per-function behaviour is intended, it will
6690 still be possible to use function_scoped_fixture to opt out of the
6691 health check error for specific tests.
6692 5.48.0 - 2021-01-06
6694 This release adds hypothesis.currently_in_test_context(), which can be
6695 used to check whether the calling code is currently running inside an
6696 @given or stateful test.
6697 This is most useful for third-party integrations and assertion helpers
6699 which may wish to use assume() or target(), without also requiring that
6700 the helper only be used from property-based tests (issue #2581).
6701 5.47.0 - 2021-01-05
6703 This release upgrades the import logic for ghostwritten tests, handling
6704 many cases where imports would previously be missing or from unexpected
6705 locations.
6706 5.46.0 - 2021-01-04
6708 This release upgrades from_type(), to infer strategies for type-anno‐
6709 tated arguments even if they have defaults when it otherwise falls back
6710 to builds() (issue #2708).
6711 5.45.0 - 2021-01-04
6713 This release adds the codemods extra, which you can use to check for
6714 and automatically fix issues such as use of deprecated Hypothesis APIs
6715 (issue #2705).
6716 5.44.0 - 2021-01-03
6718 This patch fixes from_type() with the typing_extensions Literal back‐
6719 port on Python 3.6.
6720 5.43.9 - 2021-01-02
6722 This patch fixes issue #2722, where certain orderings of
6723 register_type_strategy(), ForwardRef, and from_type() could trigger an
6724 internal error.
6725 5.43.8 - 2021-01-02
6727 This patch makes some strategies for collections with a uniqueness con‐
6728 straint much more efficient, including dictionaries(keys=sam‐
6729 pled_from(...), values=..) and lists(tuples(sampled_from(...), ...),
6730 unique_by=lambda x: x[0]). (related to issue #2036)
6731 5.43.7 - 2021-01-02
6733 This patch extends our faster special case for sampled_from() elements
6734 in unique lists() to account for chains of .map(...) and .filter(...)
6735 calls (issue #2036).
6736 5.43.6 - 2021-01-02
6738 This patch improves the type annotations on assume() and
6739 @reproduce_failure().
6740 5.43.5 - 2021-01-01
6742 This patch updates our copyright headers to include 2021. Happy new
6743 year!
6744 5.43.4 - 2020-12-24
6746 This change fixes a documentation error in the database setting.
6747 The previous documentation suggested that callers could specify a data‐
6749 base path string, or the special string ":memory:", but this setting
6750 has never actually allowed string arguments.
6751 Permitted values are None, and instances of ExampleDatabase.
6753 5.43.3 - 2020-12-11
6755 This patch fixes issue #2696, an internal error triggered when the
6756 @example decorator was used and the verbosity setting was quiet.
6757 5.43.2 - 2020-12-10
6759 This patch improves the error message from the data_frames() strategy
6760 when both the rows and columns arguments are given, but there is a
6761 missing entry in rows and the corresponding column has no fill value (‐
6762 issue #2678).
6763 5.43.1 - 2020-12-10
6765 This patch improves the error message if builds() is passed an Enum
6766 which cannot be called without arguments, to suggest using
6767 sampled_from() (issue #2693).
6768 5.43.0 - 2020-12-09
6770 This release adds new timezones() and timezone_keys() strategies (issue
6771 #2630) based on the new python:zoneinfo module in Python 3.9.
6772 pip install hypothesis[zoneinfo] will ensure that you have the appro‐
6774 priate backports installed if you need them.
6775 5.42.3 - 2020-12-09
6777 This patch fixes an internal error in datetimes() with allow_imagi‐
6778 nary=False where the timezones argument can generate tzinfo=None (issue
6779 #2662).
6780 5.42.2 - 2020-12-09
6782 This patch teaches hypothesis.extra.django.from_field() to infer more
6783 efficient strategies by inspecting (not just filtering by) field val‐
6784 idators for numeric and string fields (issue #1116).
6785 5.42.1 - 2020-12-09
6787 This patch refactors hypothesis.settings to use type-annotated keyword
6788 arguments instead of **kwargs, which makes tab-completion much more
6789 useful - as well as type-checkers like mypy.
6790 5.42.0 - 2020-12-09
6792 This patch teaches the magic() ghostwriter to recognise "en/de" func‐
6793 tion roundtrips other than the common encode/decode pattern, such as
6794 encrypt/decrypt or, encipher/decipher.
6795 5.41.5 - 2020-12-05
6797 This patch adds a performance optimisation to avoid saving redundant
6798 seeds when using the .fuzz_one_input hook.
6799 5.41.4 - 2020-11-28
6801 This patch fixes issue #2657, where passing unicode patterns compiled
6802 with python:re.IGNORECASE to from_regex() could trigger an internal er‐
6803 ror when casefolding a character creates a longer string (e.g.
6804 "\u0130".lower() -> "i\u0370").
6805 5.41.3 - 2020-11-18
6807 This patch adds a final fallback clause to our plugin logic to fail
6808 with a warning rather than error on Python < 3.8 when neither the
6809 importlib_metadata (preferred) or setuptools (fallback) packages are
6810 available.
6811 5.41.2 - 2020-11-08
6813 This patch fixes urls() strategy ensuring that ~ (tilde) is treated as
6814 one of the url-safe characters (issue #2658).
6815 5.41.1 - 2020-11-03
6817 This patch improves our CLI help and documentation.
6818 5.41.0 - 2020-10-30
6820 Hypothesis now shrinks examples where the error is raised while drawing
6821 from a strategy. This makes complicated custom strategies much easier
6822 to debug, at the cost of a slowdown for use-cases where you catch and
6823 ignore such errors.
6824 5.40.0 - 2020-10-30
6826 This release teaches from_type() how to handle ChainMap, Counter,
6827 Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).
6828 5.39.0 - 2020-10-30
6830 from_type() now knows how to resolve PEP 585 parameterized standard
6831 collection types, which are new in Python 3.9 (issue #2629).
6832 5.38.1 - 2020-10-26
6834 This patch fixes builds(), so that when passed infer for an argument
6835 with a non-Optional type annotation and a default value of None to
6836 build a class which defines an explicit __signature__ attribute, either
6837 None or that type may be generated.
6838 This is unlikely to happen unless you are using pydantic (issue #2648).
6840 5.38.0 - 2020-10-24
6842 This release improves our support for @st.composite on a python:class‐
6843 method or python:staticmethod (issue #2578).
6844 5.37.5 - 2020-10-24
6846 This patch fixes from_type() with Iterable[T] (issue #2645).
6847 5.37.4 - 2020-10-20
6849 This patch teaches the magic() ghostwriter to recognise that pairs of
6850 functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().
6851 5.37.3 - 2020-10-15
6853 This patch improves builds() and from_type() support for explicitly de‐
6854 fined __signature__ attributes, from version 5.8.3, to support generic
6855 types from the python:typing module.
6856 Thanks to Rónán Carrigan for identifying and fixing this problem!
6858 5.37.2 - 2020-10-14
6860 This patch fixes from_lark() with version 0.10.1+ of the lark-parser
6861 package.
6862 5.37.1 - 2020-10-07
6864 This patch fixes some broken links in the lark extra documentation.
6865 5.37.0 - 2020-10-03
6867 This release adds a new RedisExampleDatabase, along with the
6868 ReadOnlyDatabase and MultiplexedDatabase helpers, to support team work‐
6869 flows where failing examples can be seamlessly shared between everyone
6870 on the team - and your CI servers or buildbots.
6871 5.36.2 - 2020-10-02
6873 This patch ensures that if the "hypothesis" entry point is callable, we
6874 call it after importing it. You can still use non-callable entry
6875 points (like modules), which are only imported.
6876 We also prefer importlib.metadata or the backport over pkg_resources,
6878 which makes import hypothesis around 200 milliseconds faster (issue
6879 #2571).
6880 5.36.1 - 2020-09-25
6882 This patch adds some helpful suggestions to error messages you might
6883 see while learning to use the @example() decorator (issue #2611) or the
6884 one_of() strategy.
6885 5.36.0 - 2020-09-24
6887 This release upgrades the from_dtype() strategy to pass optional
6888 **kwargs to the inferred strategy, and upgrades the arrays() strategy
6889 to accept an elements=kwargs dict to pass through to from_dtype().
6890 arrays(floating_dtypes(), shape, elements={"min_value": -10,
6892 "max_value": 10}) is a particularly useful pattern, as it allows for
6893 any floating dtype without triggering the roundoff warning for smaller
6894 types or sacrificing variety for larger types (issue #2552).
6895 5.35.4 - 2020-09-21
6897 This patch reformats our code with the latest black to take advantage
6898 of the support for magic trailing commas.
6899 5.35.3 - 2020-09-15
6901 This release significantly improves the performance of Hypothesis's in‐
6902 ternal implementation of automaton learning. However this code does not
6903 run as part of the user-accessible API so this has no user-visible im‐
6904 pact.
6905 5.35.2 - 2020-09-14
6907 This patch ensures that, when the generate phases is disabled, we can
6908 replay up to max_examples examples from the database - which is very
6909 useful when using Hypothesis with a fuzzer.
6910 Thanks to Afrida Tabassum for fixing issue #2585!
6912 5.35.1 - 2020-09-14
6914 This patch changes some internal python:struct.Struct.format strings
6915 from bytes to str, to avoid python:BytesWarning when running python
6916 -bb.
6917 Thanks to everyone involved in pytest-xdist issue 596, bpo-16349,
6919 bpo-21071, and bpo-41777 for their work on this - it was a remarkably
6920 subtle issue!
6921 5.35.0 - 2020-09-11
6923 The target() function now accepts integers as well as floats.
6924 5.34.1 - 2020-09-11
6926 This patch adds explicit Optional annotations to our public API, to
6927 better support users who run mypy with --strict or no_implicit_op‐
6928 tional=True.
6929 Thanks to Krzysztof Przybyła for bringing this to our attention and
6931 writing the patch!
6932 5.34.0 - 2020-09-11
6934 This release drops support for Python 3.5, which reached end of life
6935 upstream on 2020-09-13.
6936 5.33.2 - 2020-09-09
6938 This patch fixes a problem with builds() that was not able to generate
6939 valid data for annotated classes with constructors.
6940 Thanks to Nikita Sobolev for fixing issue #2603!
6942 5.33.1 - 2020-09-07
6944 This patch improves the error message from the hypothesis write command
6945 if black (required for the ghostwriter) is not installed.
6946 Thanks to Nikita Sobolev for fixing issue #2604!
6948 5.33.0 - 2020-09-06
6950 When reporting failing examples, or tried examples in verbose mode, Hy‐
6951 pothesis now identifies which were from @example(...) explicit exam‐
6952 ples.
6953 5.32.1 - 2020-09-06
6955 This patch contains some internal refactoring. Thanks to Felix Sheldon
6956 for fixing issue #2516!
6957 5.32.0 - 2020-09-04
6959 An array drawn from arrays() will own its own memory; previously most
6960 arrays returned by this strategy were views.
6961 5.31.0 - 2020-09-04
6963 builds() will use the __signature__ attribute of the target, if it ex‐
6964 ists, to retrieve type hints. Previously python:typ‐
6965 ing.get_type_hints(), was used by default. If argument names varied
6966 between the __annotations__ and __signature__, they would not be sup‐
6967 plied to the target.
6968 This was particularly an issue for pydantic models which use an alias
6970 generator.
6971 5.30.1 - 2020-09-04
6973 This patch makes the ghostwriter much more robust when passed unusual
6974 modules.
6975 • improved support for non-resolvable type annotations
6977 • magic() can now write equivalent() tests
6979 • running magic() on modules where some names in __all__ are undefined
6981 skips such names, instead of raising an error
6982 • magic() now knows to skip mocks
6984 • improved handling of import-time errors found by the ghostwriter CLI
6986 5.30.0 - 2020-08-30
6988 register_type_strategy() now supports python:typing.TypeVar, which was
6989 previously hard-coded, and allows a variety of types to be generated
6990 for an unconstrained TypeVar instead of just text().
6991 Thanks again to Nikita Sobolev for all your work on advanced types!
6993 5.29.4 - 2020-08-28
6995 This release fixes some hard to trigger bugs in Hypothesis's automata
6996 learning code. This code is only run as part of the Hypothesis build
6997 process, and not for user code, so this release has no user visible im‐
6998 pact.
6999 5.29.3 - 2020-08-27
7001 This patch adds type annotations to the hypothesis.database module.
7002 There is no runtime change, but your typechecker might notice.
7003 5.29.2 - 2020-08-27
7005 This patch tracks some additional information in Hypothesis internals,
7006 and has no user-visible impact.
7007 5.29.1 - 2020-08-27
7009 This release fixes a bug in some Hypothesis internal support code for
7010 learning automata. This mostly doesn't have any user visible impact,
7011 although it slightly affects the learned shrink passes so shrinking may
7012 be subtly different.
7013 5.29.0 - 2020-08-24
7015 This release adds support for Hypothesis integration via setuptools en‐
7016 try points, which allows for smoother integration of third-party Hy‐
7017 pothesis extensions and external libraries. Unless you're publishing a
7018 library with Hypothesis integration, you'll probably only ever use this
7019 indirectly!
7020 5.28.0 - 2020-08-24
7022 from_type() can now resolve TypeVar instances when the bound is a For‐
7023 wardRef, so long as that name is in fact defined in the same module as
7024 the typevar (no TYPE_CHECKING tricks, sorry). This feature requires
7025 Python 3.7 or later.
7026 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
7028 5.27.0 - 2020-08-20
7030 This patch adds two new ghostwriters to test binary operations, like
7031 python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().
7032 5.26.1 - 2020-08-19
7034 This release improves the performance of some methods in Hypothesis's
7035 internal automaton library. These are currently only lightly used by
7036 user code, but this may result in slightly faster shrinking.
7037 5.26.0 - 2020-08-17
7039 register_type_strategy() no longer accepts parametrised user-defined
7040 generic types, because the resolution logic was quite badly broken (‐
7041 issue #2537).
7042 Instead of registering a strategy for e.g. MyCollection[int], you
7044 should register a function for MyCollection and inspect the type param‐
7045 eters within that function.
7046 Thanks to Nikita Sobolev for the bug report, design assistance, and
7048 pull request to implement this feature!
7049 5.25.0 - 2020-08-16
7051 Tired of writing tests? Or new to Hypothesis and not sure where to
7052 start?
7053 This release is for you! With our new Ghostwriter functions and hy‐
7055 pothesis write ... command-line interface, you can stop writing tests
7056 entirely... or take the source code Hypothesis writes for you as a
7057 starting point.
7058 This has been in the works for months, from issue #2118 to versions
7060 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who
7061 reviewed pull requests or commented on demos, and to Timothy Crosley's
7062 hypothesis-auto project for inspiration.
7063 5.24.4 - 2020-08-14
7065 This patch adds yet more internal functions to support a new feature
7066 we're working on, like version 5.18.3 and version 5.23.6. We promise
7067 it's worth the wait!
7068 5.24.3 - 2020-08-13
7070 This release fixes a small internal bug in Hypothesis's internal autom‐
7071 aton library. Fortunately this bug was currently impossible to hit in
7072 user facing code, so this has no user visible impact.
7073 5.24.2 - 2020-08-12
7075 This release improves shrink quality by allowing Hypothesis to automat‐
7076 ically learn new shrink passes for difficult to shrink tests.
7077 The automatic learning is not currently accessible in user code (it
7079 still needs significant work on robustness and performance before it is
7080 ready for that), but this release includes learned passes that should
7081 improve shrinking quality for tests which use any of the text(),
7082 floats(), datetimes(), emails(), and complex_numbers() strategies.
7083 5.24.1 - 2020-08-12
7085 This patch updates some docstrings, without changing runtime behaviour.
7086 5.24.0 - 2020-08-10
7088 The functions() strategy has a new argument pure=True, which ensures
7089 that the same return value is generated for identical calls to the gen‐
7090 erated function (issue #2538).
7091 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
7093 5.23.12 - 2020-08-10
7095 This release removes a number of Hypothesis's internal "shrink passes"
7096 - transformations it makes to a generated test case during shrinking -
7097 which appeared to be redundant with other transformations.
7098 It is unlikely that you will see much impact from this. If you do, it
7100 will likely show up as a change in shrinking performance (probably
7101 slower, maybe faster), or possibly in worse shrunk results. If you en‐
7102 counter the latter, please let us know.
7103 5.23.11 - 2020-08-04
7105 This release fixes a bug in some internal Hypothesis support code. It
7106 has no user visible impact.
7107 5.23.10 - 2020-08-04
7109 This release improves the quality of shrunk test cases in some special
7110 cases. Specifically, it should get shrinking unstuck in some scenarios
7111 which require simultaneously changing two parts of the generated test
7112 case.
7113 5.23.9 - 2020-08-03
7115 This release improves the performance of some internal support code. It
7116 has no user visible impact, as that code is not currently run during
7117 normal Hypothesis operation.
7118 5.23.8 - 2020-07-31
7120 This release adds a heuristic to detect when shrinking has finished de‐
7121 spite the fact that there are many more possible transformations to
7122 try. This will be particularly useful for tests where the minimum fail‐
7123 ing test case is very large despite there being many smaller test cases
7124 possible, where it is likely to speed up shrinking dramatically.
7125 In some cases it is likely that this will result in worse shrunk test
7127 cases. In those cases rerunning the test will result in further shrink‐
7128 ing.
7129 5.23.7 - 2020-07-29
7131 This release makes some performance improvements to shrinking. They
7132 should only be noticeable for tests that are currently particularly
7133 slow to shrink.
7134 5.23.6 - 2020-07-29
7136 This patch adds some more internal functions to support a new feature
7137 we're working on, like version 5.18.3. There is still no user-visible
7138 change... yet.
7139 5.23.5 - 2020-07-29
7141 This release makes some changes to internal support code that is not
7142 currently used in production Hypothesis. It has no user visible effect
7143 at present.
7144 5.23.4 - 2020-07-29
7146 This release improves shrinking quality in some special cases.
7147 5.23.3 - 2020-07-27
7149 This release fixes issue #2507, where lazy evaluation meant that the
7150 values drawn from a sampled_from() strategy could depend on mutations
7151 of the sampled sequence that happened after the strategy was construc‐
7152 ted.
7153 5.23.2 - 2020-07-27
7155 This patch fixes issue #2462, a bug in our handling of unittest.Test‐
7156 Case.subTest(). Thanks to Israel Fruchter for fixing this at the Eu‐
7157 roPython sprints!
7158 5.23.1 - 2020-07-26
7160 This release improves the behaviour of the characters() strategy when
7161 shrinking, by changing which characters are considered smallest to pre‐
7162 fer more "normal" ascii characters where available.
7163 5.23.0 - 2020-07-26
7165 The default print_blob setting is now smarter. It defaults to True in
7166 CI and False for local development.
7167 Thanks to Hugo van Kemenade for implementing this feature at the Eu‐
7169 roPython sprints!
7170 5.22.0 - 2020-07-25
7172 The slices() strategy can now generate slices for empty sequences,
7173 slices with negative start and stop indices (from the end of the se‐
7174 quence), and step=None in place of step=1.
7175 Thanks to Sangarshanan for implementing this feature at the EuroPython
7177 sprints!
7178 5.21.0 - 2020-07-23
7180 This release ensures that tests which raise RecursionError are not re‐
7181 ported as flaky simply because we run them from different initial stack
7182 depths (issue #2494).
7183 5.20.4 - 2020-07-23
7185 This release improves the performance of the sample method on objects
7186 obtained from randoms() when use_true_random=False. This should mostly
7187 only be noticeable when the sample size is a large fraction of the pop‐
7188 ulation size, but may also help avoid health check failures in some
7189 other cases.
7190 5.20.3 - 2020-07-21
7192 This release makes some internal changes for testing purposes and
7193 should have no user visible effect.
7194 5.20.2 - 2020-07-18
7196 This release fixes a small caching bug in Hypothesis internals that may
7197 under some circumstances have resulted in a less diverse set of test
7198 cases being generated than was intended.
7199 Fixing this problem revealed some performance problems that could occur
7201 during targeted property based testing, so this release also fixes
7202 those. Targeted property-based testing should now be significantly
7203 faster in some cases, but this may be at the cost of reduced effective‐
7204 ness.
7205 5.20.1 - 2020-07-17
7207 This patch updates our formatting to use isort 5. There is no
7208 user-visible change.
7209 5.20.0 - 2020-07-17
7211 The basic_indices() strategy can now generate bare indexers in place of
7212 length-one tuples. Thanks to Andrea for this patch!
7213 5.19.3 - 2020-07-15
7215 This patch removes an internal use of distutils in order to avoid this
7216 setuptools warning for some users.
7217 5.19.2 - 2020-07-13
7219 This patch contains a small internal refactoring with no user-visible
7220 impact.
7221 Thanks to Andrea for writing this at the SciPy 2020 Sprints!
7223 5.19.1 - 2020-07-12
7225 This release slightly improves shrinking behaviour. This should mainly
7226 only impact stateful tests, but may have some minor positive impact on
7227 shrinking collections (lists, sets, etc).
7228 5.19.0 - 2020-06-30
7230 This release improves the randoms() strategy by adding support for Ran‐
7231 dom instances where Hypothesis generates the random values rather than
7232 having them be "truly" random.
7233 5.18.3 - 2020-06-27
7235 This patch adds some internal functions to support a new feature we're
7236 working on. There is no user-visible change... yet.
7237 5.18.2 - 2020-06-26
7239 This patch improves our docs for the derandomize setting.
7240 5.18.1 - 2020-06-25
7242 This release consists of some internal refactoring to the shrinker in
7243 preparation for future work. It has no user visible impact.
7244 5.18.0 - 2020-06-22
7246 This release teaches Hypothesis to shorten tracebacks for explicit ex‐
7247 amples, as we already do for generated examples, so that you can focus
7248 on your code rather than ours.
7249 If you have multiple failing explicit examples, they will now all be
7251 reported. To report only the first failure, you can use the
7252 report_multiple_bugs=False setting as for generated examples.
7253 5.17.0 - 2020-06-22
7255 This patch adds strategy inference for the Literal, NewType, Type, De‐
7256 faultDict, and TypedDict types from the typing_extensions backport on
7257 PyPI.
7258 5.16.3 - 2020-06-21
7260 This patch precomputes some of the setup logic for our external fuzzer
7261 integration and sets deadline=None in fuzzing mode, saving around 150us
7262 on each iteration.
7263 This is around two-thirds the runtime to fuzz an empty test with
7265 @given(st.none()), and nice to have even as a much smaller fraction of
7266 the runtime for non-trivial tests.
7267 5.16.2 - 2020-06-19
7269 This patch fixes an internal error when warning about the use of func‐
7270 tion-scoped fixtures for parametrised tests where the parametrised
7271 value contained a % character. Thanks to Bryant for reporting and fix‐
7272 ing this bug!
7273 5.16.1 - 2020-06-10
7275 If you pass a python:list or python:tuple where a strategy was ex‐
7276 pected, the error message now mentions sampled_from() as an example
7277 strategy.
7278 Thanks to the enthusiastic participants in the PyCon Mentored Sprints
7280 who suggested adding this hint.
7281 5.16.0 - 2020-05-27
7283 functions() can now infer the appropriate returns strategy if you pass
7284 a like function with a return-type annotation. Before, omitting the
7285 returns argument would generate functions that always returned None.
7286 5.15.1 - 2020-05-21
7288 Fix from_type() with generic types under Python 3.9.
7289 5.15.0 - 2020-05-19
7291 This patch fixes an error that happens when multiple threads create new
7292 strategies.
7293 5.14.0 - 2020-05-13
7295 Passing min_magnitude=None to complex_numbers() is now deprecated - you
7296 can explicitly pass min_magnitude=0, or omit the argument entirely.
7297 5.13.1 - 2020-05-13
7299 This patch fixes an internal error in from_type() for python:typ‐
7300 ing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and
7301 fixing issue #2427!
7302 5.13.0 - 2020-05-12
7304 This release upgrades the test statistics available via the
7305 --hypothesis-show-statistics option to include separate information on
7306 each of the phases (issue #1555).
7307 5.12.2 - 2020-05-12
7309 This patch teaches the from_type() internals to return slightly more
7310 efficient strategies for some generic sets and mappings.
7311 5.12.1 - 2020-05-12
7313 This patch adds a # noqa comment for flake8 3.8.0, which disagrees with
7314 mypy about how to write the type of ....
7315 5.12.0 - 2020-05-10
7317 This release limits the maximum duration of the shrinking phase to five
7318 minutes, so that Hypothesis does not appear to hang when making very
7319 slow progress shrinking a failing example (issue #2340).
7320 If one of your tests triggers this logic, we would really appreciate a
7322 bug report to help us improve the shrinker for difficult but realistic
7323 workloads.
7324 5.11.0 - 2020-05-07
7326 This release improves the interaction between assume() and the
7327 @example() decorator, so that the following test no longer fails with
7328 UnsatisfiedAssumption (issue #2125):
7329 @given(value=floats(0, 1))
7331 @example(value=0.56789) # used to make the test fail!
7332 @pytest.mark.parametrize("threshold", [0.5, 1])
7333 def test_foo(threshold, value):
7334 assume(value < threshold)
7335 ...
7336 5.10.5 - 2020-05-04
7338 If you have django installed but don't use it, this patch will make im‐
7339 port hypothesis a few hundred milliseconds faster (e.g. 0.704s ->
7340 0.271s).
7341 Thanks to importtime-waterfall for highlighting this problem and Jake
7343 Vanderplas for the solution - it's impossible to misuse code from a
7344 module you haven't imported!
7345 5.10.4 - 2020-04-24
7347 This patch improves the internals of builds() type inference, to handle
7348 recursive forward references in certain dataclasses. This is useful
7349 for e.g. hypothesmith's forthcoming LibCST mode.
7350 5.10.3 - 2020-04-22
7352 This release reverses the order in which some operations are tried dur‐
7353 ing shrinking. This should generally be a slight performance improve‐
7354 ment, but most tests are unlikely to notice much difference.
7355 5.10.2 - 2020-04-22
7357 This patch fixes issue #2406, where use of pandas:pandas.Timestamp ob‐
7358 jects as bounds for the datetimes() strategy caused an internal error.
7359 This bug was introduced in version 5.8.1.
7360 5.10.1 - 2020-04-19
7362 This release is a small internal refactoring to how shrinking interacts
7363 with targeted property-based testing that should have no user user vis‐
7364 ible impact.
7365 5.10.0 - 2020-04-18
7367 This release improves our support for datetimes and times around DST
7368 transitions.
7369 times() and datetimes() are now sometimes generated with fold=1, indi‐
7371 cating that they represent the second occurrence of a given wall-time
7372 when clocks are set backwards. This may be set even when there is no
7373 transition, in which case the fold value should be ignored.
7374 For consistency, timezones provided by the pytz package can now gener‐
7376 ate imaginary times (such as the hour skipped over when clocks 'spring
7377 forward' to daylight saving time, or during some historical timezone
7378 transitions). All other timezones have always supported generation of
7379 imaginary times.
7380 If you prefer the previous behaviour, datetimes() now takes an argument
7382 allow_imaginary which defaults to True but can be set to False for any
7383 timezones strategy.
7384 5.9.1 - 2020-04-16
7386 This patch fixes the rendering of binary() docstring by using the
7387 proper backticks syntax.
7388 5.9.0 - 2020-04-15
7390 Failing tests which use target() now report the highest score observed
7391 for each target alongside the failing example(s), even without
7392 explicitly showing test statistics.
7393 This improves the debugging workflow for tests of accuracy, which as‐
7395 sert that the total imprecision is within some error budget - for exam‐
7396 ple, abs(a - b) < 0.5. Previously, shrinking to a minimal failing ex‐
7397 ample could often make errors seem smaller or more subtle than they re‐
7398 ally are (see the threshold problem, and issue #2180).
7399 5.8.6 - 2020-04-15
7401 This patch improves the docstring of binary(), the python:repr() of
7402 sampled_from() on an python:enum.Enum subclass, and a warning in our
7403 pytest plugin. There is no change in runtime behaviour.
7404 5.8.5 - 2020-04-15
7406 This release (potentially very significantly) improves the performance
7407 of failing tests in some rare cases, mostly only relevant when using
7408 targeted property-based testing, by stopping further optimisation of
7409 unrelated test cases once a failing example is found.
7410 5.8.4 - 2020-04-14
7412 This release fixes issue #2395, where under some circumstances targeted
7413 property-based testing could cause Hypothesis to get caught in an infi‐
7414 nite loop.
7415 5.8.3 - 2020-04-12
7417 This patch teaches builds() and from_type() to use the __signature__
7418 attribute of classes where it has been set, improving our support for
7419 Pydantic models (in pydantic >= 1.5).
7420 5.8.2 - 2020-04-12
7422 This release improves the performance of the part of the core engine
7423 that deliberately generates duplicate values.
7424 5.8.1 - 2020-04-12
7426 This patch improves dates() shrinking, to simplify year, month, and day
7427 like datetimes() rather than minimizing the number of days since
7428 2000-01-01.
7429 5.8.0 - 2020-03-24
7431 This release adds a .hypothesis.fuzz_one_input attribute to @given
7432 tests, for easy integration with external fuzzers such as python-afl
7433 (supporting issue #171).
7434 5.7.2 - 2020-03-24
7436 This patch fixes issue #2341, ensuring that the printed output from a
7437 stateful test cannot use variable names before they are defined.
7438 5.7.1 - 2020-03-23
7440 This patch fixes issue #2375, preventing incorrect failure when a func‐
7441 tion scoped fixture is overridden with a higher scoped fixture.
7442 5.7.0 - 2020-03-19
7444 This release allows the array_dtypes() strategy to generate Numpy
7445 dtypes which have field titles in addition to field names. We expect
7446 this to expose latent bugs where code expects that set(dtype.names) ==
7447 set(dtype.fields), though the latter may include titles.
7448 5.6.1 - 2020-03-18
7450 This makes model a positional-only argument to from_model(), to support
7451 models with a field literally named "model" (issue #2369).
7452 5.6.0 - 2020-02-29
7454 This release adds an explicit warning for tests that are both decorated
7455 with @given(...) and request a function-scoped pytest fixture, because
7456 such fixtures are only executed once for all Hypothesis test cases and
7457 that often causes trouble (issue #377).
7458 It's very difficult to fix this on the pytest side, so since 2015 our
7460 advice has been "just don't use function-scoped fixtures with Hypothe‐
7461 sis". Now we detect and warn about the issue at runtime!
7462 5.5.5 - 2020-02-29
7464 This release cleans up the internal machinery for Stateful testing, af‐
7465 ter we dropped the legacy APIs in Hypothesis 5.0 (issue #2218). There
7466 is no user-visible change.
7467 5.5.4 - 2020-02-16
7469 This patch fixes issue #2351, arrays() would raise a confusing error if
7470 we inferred a strategy for datetime64 or timedelta64 values with vary‐
7471 ing time units.
7472 We now infer an internally-consistent strategy for such arrays, and
7474 have a more helpful error message if an inconsistent strategy is ex‐
7475 plicitly specified.
7476 5.5.3 - 2020-02-14
7478 This patch improves the signature of builds() by specifying target as a
7479 positional-only argument on Python 3.8 (see PEP 570). The semantics of
7480 builds() have not changed at all - this just clarifies the documenta‐
7481 tion.
7482 5.5.2 - 2020-02-13
7484 This release makes Hypothesis faster at generating test cases that con‐
7485 tain duplicated values in their inputs.
7486 5.5.1 - 2020-02-07
7488 This patch has some tiny internal code clean-ups, with no user-visible
7489 change.
7490 5.5.0 - 2020-02-07
7492 Our style guide suggests that optional parameters should usually be
7493 keyword-only arguments (see PEP 3102) to prevent confusion based on po‐
7494 sitional arguments - for example, hypothesis.strategies.floats() takes
7495 up to four boolean flags and many of the Numpy strategies have both
7496 dims and side bounds.
7497 This release converts most optional parameters in our API to use key‐
7499 word-only arguments - and adds a compatibility shim so you get warnings
7500 rather than errors everywhere (issue #2130).
7501 5.4.2 - 2020-02-06
7503 This patch fixes compatibility with Python 3.5.2 (issue #2334). Note
7504 that we only test the latest patch of each minor version, though as in
7505 this case we usually accept pull requests for older patch versions.
7506 5.4.1 - 2020-02-01
7508 This patch improves the repr of from_type(), so that in most cases it
7509 will display the strategy it resolves to rather than from_type(...).
7510 The latter form will continue to be used where resolution is not imme‐
7511 diately successful, e.g. invalid arguments or recursive type defini‐
7512 tions involving forward references.
7513 5.4.0 - 2020-01-30
7515 This release removes support for Python 3.5.0 and 3.5.1, where the
7516 python:typing module was quite immature (e.g. missing overload() and
7517 Type).
7518 Note that Python 3.5 will reach its end-of-life in September 2020, and
7520 new releases of Hypothesis may drop support somewhat earlier.
7521 NOTE:
7523 pip install hypothesis should continue to give you the latest com‐
7524 patible version. If you have somehow ended up with an incompatible
7525 version, you need to update your packaging stack to pip >= 9.0 and
7526 setuptools >= 24.2 - see here for details. Then pip uninstall hy‐
7527 pothesis && pip install hypothesis will get you back to a compatible
7528 version.
7529 5.3.1 - 2020-01-26
7531 This patch does some minor internal cleanup; there is no user-visible
7532 change.
7533 5.3.0 - 2020-01-21
7535 The standard library ipaddress module is new in Python 3, and this re‐
7536 lease adds the new ip_addresses() strategy to generate IPv4Addresses
7537 and/or IPv6Addresses (depending on the v and network arguments).
7538 If you use them in type annotations, from_type() now has strategies
7540 registered for ipaddress address, network, and interface types.
7541 The provisional strategies for IP address strings are therefore depre‐
7543 cated.
7544 5.2.1 - 2020-01-21
7546 This patch reverts version 5.2, due to a strange issue where indexing
7547 an array of strings can raise an error instead of returning an item
7548 which contains certain surrogate characters.
7549 5.2.0 - 2020-01-19
7551 This release allows from_dtype() to generate Unicode strings which can‐
7552 not be encoded in UTF-8, but are valid in Numpy arrays (which use
7553 UTF-32).
7554 5.1.6 - 2020-01-19
7556 This patch fixes issue #2320, where from_type(Set[Hashable]) could
7557 raise an internal error because Decimal("snan") is of a hashable type,
7558 but raises an error when hashed. We now ensure that set elements and
7559 dict keys in generic types can actually be hashed.
7560 5.1.5 - 2020-01-12
7562 This patch fixes an internal error when running in an IPython repl or
7563 Jupyter notebook on Windows (issue #2319), and an internal error on
7564 Python 3.5.1 (issue #2318).
7565 5.1.4 - 2020-01-11
7567 This patch fixes a bug where errors in third-party extensions such as
7568 hypothesis-trio or hypothesis-jsonschema were incorrectly considered to
7569 be Hypothesis internal errors, which could result in confusing error
7570 messages.
7571 Thanks to Vincent Michel for reporting and fixing the bug!
7573 5.1.3 - 2020-01-11
7575 This release converts the type hint comments on our public API to PEP
7576 484 type annotations.
7577 Thanks to Ivan Levkivskyi for com2ann - with the refactoring tools from
7579 5.0.1 it made this process remarkably easy!
7580 5.1.2 - 2020-01-09
7582 This patch makes multiple() iterable, so that output like a, b =
7583 state.some_rule() is actually executable and can be used to reproduce
7584 failing examples.
7585 Thanks to Vincent Michel for reporting and fixing issue #2311!
7587 5.1.1 - 2020-01-06
7589 This patch contains many small refactorings to replace our Python 2
7590 compatibility functions with their native Python 3 equivalents. Since
7591 Hypothesis is now Python 3 only, there is no user-visible change.
7592 5.1.0 - 2020-01-03
7594 This release teaches from_type() how to generate python:datetime.time‐
7595 zone. As a result, you can now generate python:datetime.tzinfo objects
7596 without having pytz installed.
7597 If your tests specifically require pytz timezones, you should be using
7599 hypothesis.extra.pytz.timezones() instead of st.from_type(tzinfo).
7600 5.0.1 - 2020-01-01
7602 This patch contains mostly-automated refactorings to remove code that
7603 we only needed to support Python 2. Since Hypothesis is now Python 3
7604 only (hurray!), there is no user-visible change.
7605 Our sincere thanks to the authors of autoflake, black, isort, and
7607 pyupgrade, who have each and collectively made this kind of update
7608 enormously easier.
7609 5.0.0 - 2020-01-01
7611 Welcome to the next major version of Hypothesis!
7612 There are no new features here, as we release those in minor versions.
7614 Instead, 5.0 is a chance for us to remove deprecated features (many al‐
7615 ready converted into no-ops), and turn a variety of warnings into er‐
7616 rors.
7617 If you were running on the last version of Hypothesis 4.x without any
7619 Hypothesis deprecation warnings, this will be a very boring upgrade.
7620 In fact, nothing will change for you at all.
7621 NOTE:
7623 This release drops support for Python 2, which has passed its end of
7624 life date. The Python 3 Statement outlines our reasons, and lists
7625 many other packages that have made the same decision.
7626 pip install hypothesis should continue to give you the latest com‐
7628 patible version. If you have somehow ended up with Hypothesis 5.0
7629 on Python 2, you need to update your packaging stack to pip >= 9.0
7630 and setuptools >= 24.2 - see here for details. Then pip uninstall
7631 hypothesis && pip install hypothesis will get you back to a compati‐
7632 ble version.
7633 Strategies
7635 • integers() bounds must be equal to an integer, though they can still
7636 be other types.
7637 • If fractions() is passed a max_denominator, the bounds must have at
7639 most that denominator.
7640 • floats() bounds must be exactly representable as a floating-point
7642 number with the given width. If not, the error message includes the
7643 nearest such number.
7644 • sampled_from([]) is now an error.
7646 • The values from the elements and fill strategies for
7648 hypothesis.extra.numpy.arrays() must be losslessly representable in
7649 an array of the given dtype.
7650 • The min_size and max_size arguments to all collection strategies must
7652 be of type python:int (or max_size may be None).
7653 Miscellaneous
7655 • The .example() method of strategies (intended for interactive explo‐
7656 ration) no longer takes a random argument.
7657 • It is now an error to apply @example, @seed, or @reproduce_failure
7659 without also applying @given.
7660 • You may pass either the target or targets argument to stateful rules,
7662 but not both.
7663 • deadline must be None (to disable), a timedelta, or an integer or
7665 float number of milliseconds.
7666 • Both of derandomize and print_blob must be either True or False,
7668 where they previously accepted other values.
7669 • stateful_step_count must be at least one.
7671 • max_examples must be at least one. To disable example generation,
7673 use the phases setting.
7674 Removals
7676 • hypothesis.stateful.GenericStateMachine in favor of
7677 hypothesis.stateful.RuleBasedStateMachine
7678 • hypothesis.extra.django.models.models in favor of
7680 hypothesis.extra.django.from_model() and hypothesis.extra.django.mod‐
7681 els.add_default_field_mapping in favor of
7682 hypothesis.extra.django.register_field_strategy()
7683 • hypothesis.HealthCheck.hung_test, without replacement
7685 • hypothesis.settings.buffer, without replacement
7687 • hypothesis.PrintSettings, because hypothesis.settings.print_blob
7689 takes True or False
7690 • hypothesis.settings.timeout, in favor of hypothesis.settings.deadline
7692 • hypothesis.unlimited without replacement (only only useful as argu‐
7694 ment to timeout)
7695 Hypothesis 4.x
7697 4.57.1 - 2019-12-29
7698 This patch improves the type hints and documentation for the django ex‐
7699 tra. There is no runtime change.
7700 4.57.0 - 2019-12-28
7702 This release improves support for the SupportsOp protocols from the
7703 python:typing module when using on from_type() as outlined in issue
7704 #2292. The following types now generate much more varied strategies
7705 when called with from_type():
7706 • python:typing.SupportsAbs
7708 • python:typing.SupportsBytes
7710 • python:typing.SupportsComplex
7712 • python:typing.SupportsInt
7714 • python:typing.SupportsFloat
7716 • python:typing.SupportsRound
7718 Note that using from_type() with one of the above strategies will not
7720 ensure that the the specified function will execute successfully (ie :
7721 the strategy returned for from_type(typing.SupportsAbs) may include
7722 NaNs or things which cause the python:abs() function to error. )
7723 Thanks to Lea Provenzano for this patch.
7725 4.56.3 - 2019-12-22
7727 This release fixes a small internal bug in shrinking which could have
7728 caused it to perform slightly more tests than were necessary. Fixing
7729 this shouldn't have much effect but it will make shrinking slightly
7730 faster.
7731 4.56.2 - 2019-12-21
7733 This release removes an internal heuristic that was no longer providing
7734 much benefit. It is unlikely that there will be any user visible ef‐
7735 fect.
7736 4.56.1 - 2019-12-19
7738 This release further improves the optimisation algorithm for targeted
7739 property-based testing.
7740 4.56.0 - 2019-12-18
7742 This release enables deprecation warnings even when the verbosity set‐
7743 ting is quiet, in preparation for Hypothesis 5.0 (issue #2218).
7744 Warnings can still be filtered by the standard mechanisms provided in
7746 the standard-library python:warnings module.
7747 4.55.4 - 2019-12-18
7749 This release improves Hypothesis's management of the set of test cases
7750 it tracks between runs. It will only do anything if you have the target
7751 phase enabled and an example database set. In those circumstances it
7752 should result in a more thorough and faster set of examples that are
7753 tried on each run.
7754 4.55.3 - 2019-12-18
7756 This release makes Hypothesis better at generating test cases where
7757 generated values are duplicated in different parts of the test case.
7758 This will be especially noticeable with reasonably complex values, as
7759 it was already able to do this for simpler ones such as integers or
7760 floats.
7761 4.55.2 - 2019-12-17
7763 This release expands the set of test cases that Hypothesis saves in its
7764 database for future runs to include a representative set of "struc‐
7765 turally different" test cases - e.g. it might try to save test cases
7766 where a given list is empty or not.
7767 Currently this is unlikely to have much user visible impact except to
7769 produce slightly more consistent behaviour between consecutive runs of
7770 a test suite. It is mostly groundwork for future improvements which
7771 will exploit this functionality more effectively.
7772 4.55.1 - 2019-12-16
7774 This patch fixes issue #2257, where from_type() could incorrectly gen‐
7775 erate bytestrings when passed a generic python:typing.Sequence such as
7776 Sequence[set].
7777 4.55.0 - 2019-12-16
7779 This release adds database support for targeted property-based testing,
7780 so the best examples based on the targeting will be saved and reused
7781 between runs. This is mostly laying groundwork for future features in
7782 this area, but will also make targeted property-based tests more useful
7783 during development, where the same tests tend to get run over and over
7784 again.
7785 If max_examples is large, this may increase memory usage significantly
7787 under some circumstances, but these should be relatively rare.
7788 This release also adds a dependency on the sortedcontainers package.
7790 4.54.2 - 2019-12-16
7792 This release improves the optimisation algorithm for targeted prop‐
7793 erty-based testing, so that it will find higher quality results more
7794 reliably. Specifically, in cases where it would previously have got
7795 near a local optimum, it will now tend to achieve the locally optimal
7796 value.
7797 4.54.1 - 2019-12-16
7799 This release is mostly internal changes in support of better testing of
7800 the core engine. You are unlikely to see much effect, although some in‐
7801 ternal heuristics have changed slightly.
7802 4.54.0 - 2019-12-15
7804 This release adds a dedicated phase for targeted property-based test‐
7805 ing, and (somewhat) improves the targeting algorithm so that it will
7806 find higher quality results more reliably. This comes at a cost of
7807 making it more likely to get stuck in a local optimum.
7808 4.53.3 - 2019-12-15
7810 This patch fixes from_type() with python:typing.Hashable and
7811 python:typing.Sized, which previously failed with an internal error on
7812 Python 3.7 or later.
7813 Thanks to Lea Provenzano for both reporting issue #2272 and writing the
7815 patch!
7816 4.53.2 - 2019-12-11
7818 This release reorganises a number of the Hypothesis internal modules
7819 into a package structure. If you are only depending on the public API
7820 it should have no effect. If you are depending on the internal API
7821 (which you shouldn't be, and which we don't guarantee compatibility on)
7822 you may have to rename some imports.
7823 4.53.1 - 2019-12-09
7825 This release changes the size distribution of the number of steps run
7826 in stateful testing: It will now almost always run the maximum number
7827 of steps permitted.
7828 4.53.0 - 2019-12-09
7830 Test statistics now include the best score seen for each label, which
7831 can help avoid the threshold problem when the minimal example shrinks
7832 right down to the threshold of failure (issue #2180).
7833 4.52.0 - 2019-12-09
7835 This release changes the stateful_step_count setting to raise an error
7836 if set to 0. This is a backwards compatible change because a value of 0
7837 would never have worked and attempting to run it would have resulted in
7838 an internal assertion error.
7839 4.51.1 - 2019-12-09
7841 This release makes a small internal change to the distribution of test
7842 cases. It is unlikely to have much user visible impact.
7843 4.51.0 - 2019-12-07
7845 This release deprecates use of @example, @seed, or @reproduce_failure
7846 without @given.
7847 Thanks to Nick Anyos for the patch!
7849 4.50.8 - 2019-12-05
7851 This patch makes certain uses of Bundles more efficient in stateful
7852 testing (issue #2078).
7853 4.50.7 - 2019-12-05
7855 This release refactors some of Hypothesis's internal interfaces for
7856 representing data generation. It should have no user visible effect.
7857 4.50.6 - 2019-12-02
7859 This patch removes some old debugging helpers in our Numpy extra which
7860 have not been needed since issue #1963 and issue #2245.
7861 4.50.5 - 2019-12-01
7863 This patch fixes issue #2229, where Numpy arrays of unsized strings
7864 would only ever have strings of size one due to an interaction between
7865 our generation logic and Numpy's allocation strategy.
7866 4.50.4 - 2019-12-01
7868 This patch fixes a rare internal error in strategies for a list of
7869 unique items sampled from a short non-unique sequence (issue #2247).
7870 The bug was discovered via hypothesis-jsonschema.
7871 4.50.3 - 2019-12-01
7873 This release improves the error message when @settings tries to inherit
7874 settings from a parent argument that isn't a settings instance.
7875 4.50.2 - 2019-11-29
7877 This release improves Hypothesis's "Falsifying example" output, by
7878 breaking output across multiple lines where necessary, and by removing
7879 irrelevant information from the stateful testing output.
7880 4.50.1 - 2019-11-29
7882 This patch adds flake8-comprehensions to our linter suite. There is no
7883 user-visible change - expect perhaps via some strange microbenchmarks -
7884 but certain parts of the code now have a clear and more consistent
7885 style.
7886 4.50.0 - 2019-11-28
7888 This release fixes some cases where we might previously have failed to
7889 run the validation logic for some strategies. As a result tests which
7890 would previously have been silently testing significantly less than
7891 they should may now start to raise InvalidArgument now that these er‐
7892 rors are caught.
7893 4.49.0 - 2019-11-28
7895 This release significantly improves the data distribution in rule based
7896 stateful testing, by using a technique called Swarm Testing (Groce,
7897 Alex, et al. "Swarm testing." Proceedings of the 2012 International
7898 Symposium on Software Testing and Analysis. ACM, 2012.) to select
7899 which rules are run in any given test case. This should allow it to
7900 find many issues that it would previously have missed.
7901 This change is likely to be especially beneficial for stateful tests
7903 with large numbers of rules.
7904 4.48.1 - 2019-11-28
7906 This release adds some heuristics to test case generation that try to
7907 ensure that test cases generated early on will be relatively small.
7908 This fixes a bug introduced in Hypothesis 4.42.0 which would cause oc‐
7910 casional too_slow failures on some tests.
7911 4.48.0 - 2019-11-28
7913 This release revokes the deprecation of find, as we've now rebuilt it
7914 on top of @given, which means it has minimal maintenance burden and
7915 we're happy to support it.
7916 4.47.5 - 2019-11-28
7918 This release rebuilds find() on top of @given in order to have more
7919 code in common. It should have minimal user visible effect.
7920 4.47.4 - 2019-11-27
7922 This patch removes an internal compatibility shim that we no longer
7923 need.
7924 4.47.3 - 2019-11-26
7926 This patch fixes several typos in our docstrings and comments, with no
7927 change in behaviour. Thanks to Dmitry Dygalo for identifying and fix‐
7928 ing them!
7929 4.47.2 - 2019-11-25
7931 This release fixes an internal issue where Hypothesis would sometimes
7932 generate test cases that were above its intended maximum size. This
7933 would only have happened rarely and probably would not have caused ma‐
7934 jor problems when it did.
7935 Users of the new targeted property-based testing might see minor im‐
7937 pact (possibly slightly faster tests and slightly worse target scores),
7938 but only in the unlikely event that they were hitting this problem.
7939 Other users should not see any effect at all.
7940 4.47.1 - 2019-11-24
7942 This release removes some unused code from the core engine. There is
7943 no user-visible change.
7944 4.47.0 - 2019-11-24
7946 This release commonizes some code between running explicit examples and
7947 normal test execution. The main user visible impact of this is that
7948 deadlines are now enforced when running explicit examples.
7949 4.46.1 - 2019-11-23
7951 This patch ensures that a KeyboardInterrupt received during example
7952 generation is not treated as a mystery test failure but instead propa‐
7953 gates to the top level, not recording the interrupted generation in the
7954 conjecture data tree. Thanks to Anne Archibald for identifying and
7955 fixing the problem.
7956 4.46.0 - 2019-11-22
7958 This release changes the behaviour of floats() when excluding signed
7959 zeros - floats(max_value=0.0, exclude_max=True) can no longer generate
7960 -0.0 nor the much rarer floats(min_value=-0.0, exclude_min=True) gener‐
7961 ate +0.0.
7962 The correct interaction between signed zeros and exclusive endpoints
7964 was unclear; we now enforce the invariant that floats() will never gen‐
7965 erate a value equal to an excluded endpoint (issue #2201).
7966 If you prefer the old behaviour, you can pass floats(max_value=-0.0) or
7968 floats(min_value=0.0) which is exactly equivalent and has not changed.
7969 If you had two endpoints equal to zero, we recommend clarifying your
7970 tests by using just() or sampled_from() instead of floats().
7971 4.45.1 - 2019-11-20
7973 This patch improves the error message when invalid arguments are passed
7974 to rule() or invariant() (issue #2149).
7975 Thanks to Benjamin Palmer for this bugfix!
7977 4.45.0 - 2019-11-20
7979 This release supports python:typing.Final and python:typing.TypedDict
7980 in from_type().
7981 4.44.5 - 2019-11-20
7983 This patch disables our pytest plugin when running on versions of
7984 pytest before 4.3, the oldest our plugin supports. Note that at time
7985 of writing the Pytest developers only support 4.6 and later!
7986 Hypothesis tests using @given() work on any test runner, but our inte‐
7988 grations to e.g. avoid example database collisions when combined with
7989 @pytest.mark.parametrize eventually drop support for obsolete versions.
7990 4.44.4 - 2019-11-20
7992 This patch adds some internal comments and clarifications to the Hy‐
7993 pothesis implementation. There is no user-visible change.
7994 4.44.3 - 2019-11-20
7996 This patch avoids importing test runners such as pytest, unittest2, or
7997 nose solely to access their special "skip test" exception types - if
7998 the module is not in sys.modules, the exception can't be raised anyway.
7999 This fixes a problem where importing an otherwise unused module could
8001 cause spurious errors due to import-time side effects (and possibly
8002 -Werror).
8003 4.44.2 - 2019-11-12
8005 This release fixes @given to only complain about missing keyword-only
8006 arguments if the associated test function is actually called.
8007 This matches the behaviour of other InvalidArgument errors produced by
8009 @given.
8010 4.44.1 - 2019-11-11
8012 This patch allows Hypothesis to run in environments that do not specify
8013 a __file__, such as a python:zipapp (issue #2196).
8014 4.44.0 - 2019-11-11
8016 This release adds a signature argument to
8017 mutually_broadcastable_shapes() (issue #2174), which allows us to gen‐
8018 erate shapes which are valid for functions like np.matmul() that re‐
8019 quire shapes which are not simply broadcastable.
8020 Thanks to everyone who has contributed to this feature over the last
8022 year, and a particular shout-out to Zac Hatfield-Dodds and Ryan Sok‐
8023 laski for mutually_broadcastable_shapes() and to Ryan Turner for the
8024 downstream hypothesis-gufunc project.
8025 4.43.9 - 2019-11-11
8027 This patch fixes issue #2108, where the first test using data() to draw
8028 from characters() or text() would be flaky due to unreliable test tim‐
8029 ings.
8030 Time taken by lazy instantiation of strategies is now counted towards
8032 drawing from the strategy, rather than towards the deadline for the
8033 test function.
8034 4.43.8 - 2019-11-08
8036 This release ensures that the strategies passed to @given are properly
8037 validated when applied to a test method inside a test class.
8038 This should result in clearer error messages when some of those strate‐
8040 gies are invalid.
8041 4.43.7 - 2019-11-08
8043 This release changes how Hypothesis manages its search space in cases
8044 where it generates redundant data. This should cause it to generate
8045 significantly fewer duplicated examples (especially with short integer
8046 ranges), and may cause it to produce more useful examples in some cases
8047 (especially ones where there is a significant amount of filtering).
8048 4.43.6 - 2019-11-07
8050 This patch refactors width handling in floats(); you may notice small
8051 performance improvements but the main purpose is to enable work on
8052 issue #1704 (improving shrinking of bounded floats).
8053 4.43.5 - 2019-11-06
8055 This patch removes an unused internal flag. There is no user-visible
8056 change.
8057 4.43.4 - 2019-11-05
8059 This patch corrects the exception type and error message you get if you
8060 attempt to use data() to draw from something which is not a strategy.
8061 This never worked, but the error is more helpful now.
8062 4.43.3 - 2019-11-05
8064 We've adopted flake8-bugbear to check for a few more style issues, and
8065 this patch implements the minor internal cleanups it suggested. There
8066 is no user-visible change.
8067 4.43.2 - 2019-11-05
8069 This patch fixes the formatting of some documentation, but there is no
8070 change to any executed code.
8071 4.43.1 - 2019-11-04
8073 Python 3.8's new python:typing.Literal type - see PEP 586 for details -
8074 is now supported in from_type().
8075 4.43.0 - 2019-11-04
8077 This release adds the strategy mutually_broadcastable_shapes(), which
8078 generates multiple array shapes that are mutually broadcast-compatible
8079 with an optional user-specified base-shape.
8080 This is a generalisation of broadcastable_shapes(). It relies heavily
8082 on non-public internals for performance when generating and shrinking
8083 examples. We intend to support generating shapes matching a ufunc sig‐
8084 nature in a future version (issue #2174).
8085 Thanks to Ryan Soklaski, Zac Hatfield-Dodds, and @rdturnermtl who con‐
8087 tributed to this new feature.
8088 4.42.10 - 2019-11-03
8090 This release fixes from_type() when used with bounded or constrained
8091 python:typing.TypeVar objects (issue #2094).
8092 Previously, distinct typevars with the same constraints would be
8094 treated as all single typevar, and in cases where a typevar bound was
8095 resolved to a union of subclasses this could result in mixed types be‐
8096 ing generated for that typevar.
8097 4.42.9 - 2019-11-03
8099 This patch ensures that the default value broadcastable_shapes()
8100 chooses for max_dims is always valid (at most 32), even if you pass
8101 min_dims=32.
8102 4.42.8 - 2019-11-02
8104 This patch ensures that we only add profile information to the pytest
8105 header if running either pytest or Hypothesis in verbose mode, matching
8106 the builtin cache plugin (issue #2155).
8107 4.42.7 - 2019-11-02
8109 This patch makes stateful step printing expand the result of a step
8110 into multiple variables when you return multiple() (issue #2139).
8111 Thanks to Joseph Weston for reporting and fixing this bug!
8112 4.42.6 - 2019-11-02
8114 This release fixes a bug (issue #2166) where a Unicode character info
8115 cache file was generated but never used on subsequent test runs, caus‐
8116 ing tests to run more slowly than they should have.
8117 Thanks to Robert Knight for this bugfix!
8119 4.42.5 - 2019-11-01
8121 This patch corrects some internal documentation. There is no user-vis‐
8122 ible change.
8123 4.42.4 - 2019-11-01
8125 This release fixes a bug (issue #2160) where decorators applied after
8126 @settings and before @given were ignored.
8127 Thanks to Tom Milligan for this bugfix!
8129 4.42.3 - 2019-10-30
8131 This release updates Hypothesis's formatting to the new version of
8132 black, and has absolutely no user visible effect.
8133 4.42.2 - 2019-10-30
8135 This release fixes a bug in recursive() which would have meant that in
8136 practice max_leaves was treated as if it was lower than it actually is
8137 - specifically it would be capped at the largest power of two smaller
8138 than it. It is now handled correctly.
8139 4.42.1 - 2019-10-30
8141 Python 3.8's new python:typing.SupportsIndex type - see PEP 357 for de‐
8142 tails - is now supported in from_type().
8143 Thanks to Grigorios Giannakopoulos for the patch!
8145 4.42.0 - 2019-10-27
8147 This release significantly simplifies Hypothesis's internal logic for
8148 data generation, by removing a number of heuristics of questionable or
8149 unproven value.
8150 The results of this change will vary significantly from test to test.
8152 Most test suites will see significantly faster data generation and
8153 lower memory usage. The "quality" of the generated data may go up or
8154 down depending on your particular test suites.
8155 If you see any significant regressions in Hypothesis's ability to find
8157 bugs in your code as a result of this release, please file an issue to
8158 let us know.
8159 Users of the new targeted property-based testing functionality are
8161 reasonably likely to see improvements in data generation, as this re‐
8162 lease changes the search algorithm for targeted property based testing
8163 to one that is more likely to be productive than the existing approach.
8164 4.41.3 - 2019-10-21
8166 This patch is to ensure that our internals remain comprehensible to
8167 mypy 0.740 - there is no user-visible change.
8168 4.41.2 - 2019-10-17
8170 This patch changes some internal hashes to SHA384, to better support
8171 users subject to FIPS-140. There is no user-visible API change.
8172 Thanks to Paul Kehrer for this contribution!
8174 4.41.1 - 2019-10-16
8176 This release makes --hypothesis-show-statistics much more useful for
8177 tests using a RuleBasedStateMachine, by simplifying the reprs so that
8178 events are aggregated correctly.
8179 4.41.0 - 2019-10-16
8181 This release upgrades the fixed_dictionaries() strategy to support op‐
8182 tional keys (issue #1913).
8183 4.40.2 - 2019-10-16
8185 This release makes some minor internal changes in support of improving
8186 the Hypothesis test suite. It should not have any user visible impact.
8187 4.40.1 - 2019-10-14
8189 This release changes how Hypothesis checks if a parameter to a test
8190 function is a mock object. It is unlikely to have any noticeable ef‐
8191 fect, but may result in a small performance improvement, especially for
8192 test functions where a mock object is being passed as the first argu‐
8193 ment.
8194 4.40.0 - 2019-10-09
8196 This release fixes a bug where our example database logic did not dis‐
8197 tinguish between failing examples based on arguments from a
8198 @pytest.mark.parametrize(...). This could in theory cause data loss if
8199 a common failure overwrote a rare one, and in practice caused occa‐
8200 sional file-access collisions in highly concurrent workloads (e.g. dur‐
8201 ing a 300-way parametrize on 16 cores).
8202 For internal reasons this also involves bumping the minimum supported
8204 version of pytest to 4.3
8205 Thanks to Peter C Kroon for the Hacktoberfest patch!
8207 4.39.3 - 2019-10-09
8209 This patch improves our type hints on the emails(), functions(),
8210 integers(), iterables(), and slices() strategies, as well as the .fil‐
8211 ter() method.
8212 There is no runtime change, but if you use mypy or a similar
8214 type-checker on your tests the results will be a bit more precise.
8215 4.39.2 - 2019-10-09
8217 This patch improves the performance of unique collections such as
8218 sets() of just() or booleans() strategies. They were already pretty
8219 good though, so you're unlikely to notice much!
8220 4.39.1 - 2019-10-09
8222 If a value in a dict passed to fixed_dictionaries() is not a strategy,
8223 Hypothesis now tells you which one.
8224 4.39.0 - 2019-10-07
8226 This release adds the basic_indices() strategy, to generate basic in‐
8227 dexes for arrays of the specified shape (issue #1930).
8228 It generates tuples containing some mix of integers, python:slice ob‐
8230 jects, ... (Ellipsis), and numpy:numpy.newaxis; which when used to in‐
8231 dex an array of the specified shape produce either a scalar or a
8232 shared-memory view of the array. Note that the index tuple may be
8233 longer or shorter than the array shape, and may produce a view with an‐
8234 other dimensionality again!
8235 Thanks to Lampros Mountrakis, Ryan Soklaski, and Zac Hatfield-Dodds for
8237 their collaboration on this surprisingly subtle strategy!
8238 4.38.3 - 2019-10-04
8240 This patch defers creation of the .hypothesis directory until we have
8241 something to store in it, meaning that it will appear when Hypothesis
8242 is used rather than simply installed.
8243 Thanks to Peter C Kroon for the Hacktoberfest patch!
8245 4.38.2 - 2019-10-02
8247 This patch bumps our dependency on attrs to >=19.2.0; but there are no
8248 user-visible changes to Hypothesis.
8249 4.38.1 - 2019-10-01
8251 This is a comment-only patch which tells mypy 0.730 to ignore some in‐
8252 ternal compatibility shims we use to support older Pythons.
8253 4.38.0 - 2019-10-01
8255 This release adds the hypothesis.target() function, which implements
8256 targeted property-based testing (issue #1779).
8257 By calling target() in your test function, Hypothesis can do a
8259 hill-climbing search for bugs. If you can calculate a suitable metric
8260 such as the load factor or length of a queue, this can help you find
8261 bugs with inputs that are highly improbably from unguided generation -
8262 however good our heuristics, example diversity, and deduplication logic
8263 might be. After all, those features are at work in targeted PBT too!
8264 4.37.0 - 2019-09-28
8266 This release emits a warning if you use the .example() method of a
8267 strategy in a non-interactive context.
8268 given() is a much better choice for writing tests, whether you care
8270 about performance, minimal examples, reproducing failures, or even just
8271 the variety of inputs that will be tested!
8272 4.36.2 - 2019-09-20
8274 This patch disables part of the typing-based inference for the attrs
8275 package under Python 3.5.0, which has some incompatible internal de‐
8276 tails (issue #2095).
8277 4.36.1 - 2019-09-17
8279 This patch fixes a bug in strategy inference for attrs classes where
8280 Hypothesis would fail to infer a strategy for attributes of a generic
8281 type such as Union[int, str] or List[bool] (issue #2091).
8282 Thanks to Jonathan Gayvallet for the bug report and this patch!
8284 4.36.0 - 2019-09-09
8286 This patch deprecates min_len or max_len of 0 in byte_string_dtypes()
8287 and unicode_string_dtypes(). The lower limit is now 1.
8288 Numpy uses a length of 0 in these dtypes to indicate an undetermined
8290 size, chosen from the data at array creation. However, as the arrays()
8291 strategy creates arrays before filling them, strings were truncated to
8292 1 byte.
8293 4.35.1 - 2019-09-09
8295 This patch improves the messaging that comes from invalid size argu‐
8296 ments to collection strategies such as lists().
8297 4.35.0 - 2019-09-04
8299 This release improves the from_lark() strategy, tightening argument
8300 validation and adding the explicit argument to allow use with terminals
8301 that use @declare instead of a string or regular expression.
8302 This feature is required to handle features such as indent and dedent
8304 tokens in Python code, which can be generated with the hypothesmith
8305 package.
8306 4.34.0 - 2019-08-23
8308 The from_type() strategy now knows to look up the subclasses of ab‐
8309 stract types, which cannot be instantiated directly.
8310 This is very useful for hypothesmith to support libCST.
8312 4.33.1 - 2019-08-21
8314 This patch works around a crash when an incompatible version of Numpy
8315 is installed under PyPy 5.10 (Python 2.7).
8316 If you are still using Python 2, please upgrade to Python 3 as soon as
8318 possible - it will be unsupported at the end of this year.
8319 4.33.0 - 2019-08-20
8321 This release improves the domains() strategy, as well as the urls() and
8322 the emails() strategies which use it. These strategies now use the
8323 full IANA list of Top Level Domains and are correct as per RFC 1035.
8324 Passing tests using these strategies may now fail.
8326 Thanks to TechDragon for this improvement.
8328 4.32.3 - 2019-08-05
8330 This patch tidies up the repr of several settings-related objects, at
8331 runtime and in the documentation, and deprecates the undocumented edge
8332 case that phases=None was treated like phases=tuple(Phase).
8333 It also fixes from_lark() with lark 0.7.2 and later.
8335 4.32.2 - 2019-07-30
8337 This patch updates some internal comments for mypy 0.720. There is no
8338 user-visible impact.
8339 4.32.1 - 2019-07-29
8341 This release changes how the shrinker represents its progress inter‐
8342 nally. For large generated test cases this should result in signifi‐
8343 cantly less memory usage and possibly faster shrinking. Small generated
8344 test cases may be slightly slower to shrink but this shouldn't be very
8345 noticeable.
8346 4.32.0 - 2019-07-28
8348 This release makes arrays() more pedantic about elements strategies
8349 that cannot be exactly represented as array elements.
8350 In practice, you will see new warnings if you were using a float16 or
8352 float32 dtype without passing floats() the width=16 or width=32 argu‐
8353 ments respectively.
8354 The previous behaviour could lead to silent truncation, and thus some
8356 elements being equal to an explicitly excluded bound (issue #1899).
8357 4.31.1 - 2019-07-28
8359 This patch changes an internal use of MD5 to SHA hashes, to better sup‐
8360 port users subject to FIPS-140. There is no user-visible or API
8361 change.
8362 Thanks to Alex Gaynor for this patch.
8364 4.31.0 - 2019-07-24
8366 This release simplifies the logic of the print_blob setting by removing
8367 the option to set it to PrintSettings.INFER. As a result the
8368 print_blob setting now takes a single boolean value, and the use of
8369 PrintSettings is deprecated.
8370 4.28.2 - 2019-07-14
8372 This patch improves the docstrings of several Hypothesis strategies, by
8373 clarifying markup and adding cross-references. There is no runtime
8374 change.
8375 Thanks to Elizabeth Williams and Serah Njambi Rono for their contribu‐
8377 tions at the SciPy 2019 sprints!
8378 4.28.1 - 2019-07-12
8380 This patch improves the behaviour of the text() strategy when passed an
8381 alphabet which is not a strategy. The value is now interpreted as
8382 whitelist_characters to characters() instead of a sequence for
8383 sampled_from(), which standardises the distribution of examples and the
8384 shrinking behaviour.
8385 You can get the previous behaviour by using lists(sampled_from(alpha‐
8387 bet)).map("".map) instead.
8388 4.28.0 - 2019-07-11
8390 This release deprecates find(). The .example() method is a better re‐
8391 placement if you want an example, and for the rare occasions where you
8392 want the minimal example you can get it from @given.
8393 @given has steadily outstripped find() in both features and performance
8395 over recent years, and as we do not have the resources to maintain and
8396 test both we think it is better to focus on just one.
8397 4.27.0 - 2019-07-08
8399 This release refactors the implementation of the .example() method, to
8400 more accurately represent the data which will be generated by @given.
8401 As a result, calling s.example() on an empty strategy s (such as
8403 nothing()) now raises Unsatisfiable instead of the old NoExamples ex‐
8404 ception.
8405 4.26.4 - 2019-07-07
8407 This patch ensures that the Pandas extra will keep working when Python
8408 3.8 removes abstract base classes from the top-level python:collections
8409 namespace. This also fixes the relevant warning in Python 3.7, but
8410 there is no other difference in behaviour and you do not need to do
8411 anything.
8412 4.26.3 - 2019-07-05
8414 This release fixes issue #2027, by changing the way Hypothesis tries
8415 to generate distinct examples to be more efficient.
8416 This may result in slightly different data distribution, and should im‐
8418 prove generation performance in general, but should otherwise have min‐
8419 imal user impact.
8420 4.26.2 - 2019-07-04
8422 This release fixes issue #1864, where some simple tests would perform
8423 very slowly, because they would run many times with each subsequent run
8424 being progressively slower. They will now stop after a more reasonable
8425 number of runs without hitting this problem.
8426 Unless you are hitting exactly this issue, it is unlikely that this re‐
8428 lease will have any effect, but certain classes of custom generators
8429 that are currently very slow may become a bit faster, or start to trig‐
8430 ger health check failures.
8431 4.26.1 - 2019-07-04
8433 This release adds the strategy integer_array_indices(), which generates
8434 tuples of Numpy arrays that can be used for advanced indexing to select
8435 an array of a specified shape.
8436 4.26.0 - 2019-07-04
8438 This release significantly improves the performance of drawing unique
8439 collections whose elements are drawn from sampled_from() strategies.
8440 As a side effect, this detects an error condition that would previously
8442 have passed silently: When the min_size argument on a collection with
8443 distinct elements is greater than the number of elements being sampled,
8444 this will now raise an error.
8445 4.25.1 - 2019-07-03
8447 This release removes some defunct internal functionality that was only
8448 being used for testing. It should have no user visible impact.
8449 4.25.0 - 2019-07-03
8451 This release deprecates and disables the buffer_size setting, which
8452 should have been treated as a private implementation detail all along.
8453 We recommend simply deleting this settings argument.
8454 4.24.6 - 2019-06-26
8456 This patch makes datetimes() more efficient, as it now handles short
8457 months correctly by construction instead of filtering.
8458 4.24.5 - 2019-06-23
8460 This patch improves the development experience by simplifying the
8461 tracebacks you will see when e.g. you have used the .map(...) method of
8462 a strategy and the mapped function raises an exception.
8463 No new exceptions can be raised, nor existing exceptions change any‐
8465 thing but their traceback. We're simply using if-statements rather
8466 than exceptions for control flow in a certain part of the internals!
8467 4.24.4 - 2019-06-21
8469 This patch fixes issue #2014, where our compatibility layer broke with
8470 version 3.7.4 of the typing module backport on PyPI.
8471 This issue only affects Python 2. We remind users that Hypothesis,
8473 like many other packages, will drop Python 2 support on 2020-01-01 and
8474 already has several features that are only available on Python 3.
8475 4.24.3 - 2019-06-07
8477 This patch improves the implementation of an internal wrapper on Python
8478 3.8 beta1 (and will break on the alphas; but they're not meant to be
8479 stable). On other versions, there is no change at all.
8480 Thanks to Daniel Hahler for the patch, and Victor Stinner for his work
8482 on bpo-37032 that made it possible.
8483 4.24.2 - 2019-06-06
8485 Deprecation messages for functions in hypothesis.extra.django.models
8486 now explicitly name the deprecated function to make it easier to track
8487 down usages. Thanks to Kristian Glass for this contribution!
8488 4.24.1 - 2019-06-04
8490 This patch fixes issue #1999, a spurious bug raised when a
8491 @st.composite function was passed a keyword-only argument.
8492 Thanks to Jim Nicholls for his fantastic bug report.
8494 4.24.0 - 2019-05-29
8496 This release deprecates GenericStateMachine, in favor of
8497 RuleBasedStateMachine. Rule-based stateful testing is significantly
8498 faster, especially during shrinking.
8499 If your use-case truly does not fit rule-based stateful testing, we
8501 recommend writing a custom test function which drives your specific
8502 control-flow using data().
8503 4.23.9 - 2019-05-28
8505 This patch fixes a very rare example database issue with file permis‐
8506 sions.
8507 When running a test that uses both @given and pytest.mark.parametrize,
8509 using pytest-xdist on Windows, with failing examples in the database,
8510 two attempts to read a file could overlap and we caught FileNotFound
8511 but not other OSErrors.
8512 4.23.8 - 2019-05-26
8514 This patch has a minor cleanup of the internal engine. There is no
8515 user-visible impact.
8516 4.23.7 - 2019-05-26
8518 This patch clarifies some error messages when the test function signa‐
8519 ture is incompatible with the arguments to @given, especially when the
8520 @settings() decorator is also used (issue #1978).
8521 4.23.6 - 2019-05-19
8523 This release adds the pyupgrade fixer to our code style, for consistent
8524 use of dict and set literals and comprehensions.
8525 4.23.5 - 2019-05-16
8527 This release slightly simplifies a small part of the core engine.
8528 There is no user-visible change.
8529 4.23.4 - 2019-05-09
8531 Fixes a minor formatting issue the docstring of from_type()
8532 4.23.3 - 2019-05-09
8534 Adds a recipe to the docstring of from_type() that describes a means
8535 for drawing values for "everything except" a specified type. This
8536 recipe is especially useful for writing tests that perform input-type
8537 validation.
8538 4.23.2 - 2019-05-08
8540 This patch uses autoflake to remove some pointless pass statements,
8541 which improves our workflow but has no user-visible impact.
8542 4.23.1 - 2019-05-08
8544 This patch fixes an OverflowError in from_type(xrange) on Python 2.
8545 It turns out that not only do the start and stop values have to fit in
8547 a C long, but so does stop - start. We now handle this even on 32bit
8548 platforms, but remind users that Python2 will not be supported after
8549 2019 without specific funding.
8550 4.23.0 - 2019-05-08
8552 This release implements the slices() strategy, to generate slices of a
8553 length-size sequence.
8554 Thanks to Daniel J. West for writing this patch at the PyCon 2019
8556 sprints!
8557 4.22.3 - 2019-05-07
8559 This patch exposes DataObject, solely to support more precise type
8560 hints. Objects of this type are provided by data(), and can be used to
8561 draw examples from strategies intermixed with your test code.
8562 4.22.2 - 2019-05-07
8564 This patch fixes the very rare issue #1798 in array_dtypes(), which
8565 caused an internal error in our tests.
8566 4.22.1 - 2019-05-07
8568 This patch fixes a rare bug in from_type(range).
8569 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
8571 Sprints.
8572 4.22.0 - 2019-05-07
8574 The unique_by argument to lists now accepts a tuple of callables such
8575 that every element of the generated list will be unique with respect to
8576 each callable in the tuple (issue #1916).
8577 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
8579 4.21.1 - 2019-05-06
8581 This patch cleans up the internals of one_of(). You may see a slight
8582 change to the distribution of examples from this strategy but there is
8583 no change to the public API.
8584 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
8586 sprints!
8587 4.21.0 - 2019-05-05
8589 The from_type() strategy now supports python:slice objects.
8590 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
8592 Mentored Sprints.
8593 4.20.0 - 2019-05-05
8595 This release improves the array_shapes() strategy, to choose an appro‐
8596 priate default for max_side based on the min_side, and max_dims based
8597 on the min_dims. An explicit error is raised for dimensions greater
8598 than 32, which are not supported by Numpy, as for other invalid combi‐
8599 nations of arguments.
8600 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
8602 tored Sprints.
8603 4.19.0 - 2019-05-05
8605 The from_type() strategy now supports python:range objects (or xrange
8606 on Python 2).
8607 Thanks to Katrina Durance for writing this feature at the PyCon 2019
8609 Mentored Sprints.
8610 4.18.3 - 2019-04-30
8612 This release fixes a very rare edge case in the test-case mutator,
8613 which could cause an internal error with certain unusual tests.
8614 4.18.2 - 2019-04-30
8616 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
8617 changed the representation of code objects to support positional-only
8618 arguments. Note however that Hypothesis does not (yet) support such
8619 functions as e.g. arguments to builds() or inputs to @given.
8620 Thanks to Paul Ganssle for identifying and fixing this bug.
8622 4.18.1 - 2019-04-29
8624 This patch improves the performance of unique collections such as
8625 sets() when the elements are drawn from a sampled_from() strategy (‐
8626 issue #1115).
8627 4.18.0 - 2019-04-24
8629 This release adds the functions() strategy, which can be used to imi‐
8630 tate your 'real' function for callbacks.
8631 4.17.2 - 2019-04-19
8633 This release refactors stateful rule selection to share the new machin‐
8634 ery with sampled_from() instead of using the original independent im‐
8635 plementation.
8636 4.17.1 - 2019-04-16
8638 This patch allows Hypothesis to try a few more examples after finding
8639 the first bug, in hopes of reporting multiple distinct bugs. The
8640 heuristics described in issue #847 ensure that we avoid wasting time on
8641 fruitless searches, while still surfacing each bug as soon as possible.
8642 4.17.0 - 2019-04-16
8644 This release adds the strategy broadcastable_shapes(), which generates
8645 array shapes that are broadcast-compatible with a provided shape.
8646 4.16.0 - 2019-04-12
8648 This release allows register_type_strategy() to be used with
8649 python:typing.NewType instances. This may be useful to e.g. provide
8650 only positive integers for from_type(UserId) with a UserId = New‐
8651 Type('UserId', int) type.
8652 Thanks to PJCampi for suggesting and writing the patch!
8654 4.15.0 - 2019-04-09
8656 This release supports passing a timedelta as the deadline setting, so
8657 you no longer have to remember that the number is in milliseconds (‐
8658 issue #1900).
8659 Thanks to Damon Francisco for this change!
8661 4.14.7 - 2019-04-09
8663 This patch makes the type annotations on hypothesis.extra.dateutil com‐
8664 patible with mypy 0.700.
8665 4.14.6 - 2019-04-07
8667 This release fixes a bug introduced in Hypothesis 4.14.3 that would
8668 sometimes cause sampled_from(...).filter(...) to encounter an internal
8669 assertion failure when there are three or fewer elements, and every el‐
8670 ement is rejected by the filter.
8671 4.14.5 - 2019-04-05
8673 This patch takes the previous efficiency improvements to
8674 sampled_from(...).filter(...) strategies that reject most elements,
8675 and generalises them to also apply to sampled_from(...).fil‐
8676 ter(...).filter(...) and longer chains of filters.
8677 4.14.4 - 2019-04-05
8679 This release fixes a bug that prevented random_module() from correctly
8680 restoring the previous state of the random module.
8681 The random state was instead being restored to a temporary determinis‐
8683 tic state, which accidentally caused subsequent tests to see the same
8684 random values across multiple test runs.
8685 4.14.3 - 2019-04-03
8687 This patch adds an internal special case to make
8688 sampled_from(...).filter(...) much more efficient when the filter re‐
8689 jects most elements (issue #1885).
8690 4.14.2 - 2019-03-31
8692 This patch improves the error message if the function f in s.flatmap(f)
8693 does not return a strategy.
8694 Thanks to Kai Chen for this change!
8696 4.14.1 - 2019-03-30
8698 This release modifies how Hypothesis selects operations to run during
8699 shrinking, by causing it to deprioritise previously useless classes of
8700 shrink until others have reached a fixed point.
8701 This avoids certain pathological cases where the shrinker gets very
8703 close to finishing and then takes a very long time to finish the last
8704 small changes because it tries many useless shrinks for each useful one
8705 towards the end. It also should cause a more modest improvement (prob‐
8706 ably no more than about 30%) in shrinking performance for most tests.
8707 4.14.0 - 2019-03-19
8709 This release blocks installation of Hypothesis on Python 3.4, which
8710 reached its end of life date on 2019-03-18.
8711 This should not be of interest to anyone but downstream maintainers -
8713 if you are affected, migrate to a secure version of Python as soon as
8714 possible or at least seek commercial support.
8715 4.13.0 - 2019-03-19
8717 This release makes it an explicit error to call floats(min_value=inf,
8718 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
8719 are no possible values that can be generated (issue #1859).
8720 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
8722 -0. and we could thus generate either if comparing by value, violating
8723 the sequence ordering of floats is a special case we don't want or
8724 need.
8725 4.12.1 - 2019-03-18
8727 This release should significantly reduce the amount of memory that Hy‐
8728 pothesis uses for representing large test cases, by storing information
8729 in a more compact representation and only unpacking it lazily when it
8730 is first needed.
8731 4.12.0 - 2019-03-18
8733 This update adds the report_multiple_bugs setting, which you can use to
8734 disable multi-bug reporting and only raise whichever bug had the small‐
8735 est minimal example. This is occasionally useful when using a debugger
8736 or tools that annotate tracebacks via introspection.
8737 4.11.7 - 2019-03-18
8739 This change makes a tiny improvement to the core engine's bookkeeping.
8740 There is no user-visible change.
8741 4.11.6 - 2019-03-15
8743 This release changes some of Hypothesis's internal shrinking behaviour
8744 in order to reduce memory usage and hopefully improve performance.
8745 4.11.5 - 2019-03-13
8747 This release adds a micro-optimisation to how Hypothesis handles debug
8748 reporting internally. Hard to shrink test may see a slight performance
8749 improvement, but in most common scenarios it is unlikely to be notice‐
8750 able.
8751 4.11.4 - 2019-03-13
8753 This release removes some redundant code that was no longer needed but
8754 was still running a significant amount of computation and allocation on
8755 the hot path. This should result in a modest speed improvement for
8756 most tests, especially those with large test cases.
8757 4.11.3 - 2019-03-13
8759 This release adds a micro-optimisation to how Hypothesis caches test
8760 cases. This will cause a small improvement in speed and memory usage
8761 for large test cases, but in most common scenarios it is unlikely to be
8762 noticeable.
8763 4.11.2 - 2019-03-13
8765 This release removes some internal code that populates a field that is
8766 no longer used anywhere. This should result in some modest performance
8767 and speed improvements and no other user visible effects.
8768 4.11.1 - 2019-03-13
8770 This is a formatting-only patch, enabled by a new version of isort.
8771 4.11.0 - 2019-03-12
8773 This release deprecates sampled_from() with empty sequences. This re‐
8774 turns nothing(), which gives a clear error if used directly... but sim‐
8775 ply vanishes if combined with another strategy.
8776 Tests that silently generate less than expected are a serious problem
8778 for anyone relying on them to find bugs, and we think reliability more
8779 important than convenience in this case.
8780 4.10.0 - 2019-03-11
8782 This release improves Hypothesis's to detect flaky tests, by noticing
8783 when the behaviour of the test changes between runs. In particular
8784 this will notice many new cases where data generation depends on exter‐
8785 nal state (e.g. external sources of randomness) and flag those as flaky
8786 sooner and more reliably.
8787 The basis of this feature is a considerable reengineering of how Hy‐
8789 pothesis stores its history of test cases, so on top of this its memory
8790 usage should be considerably reduced.
8791 4.9.0 - 2019-03-09
8793 This release adds the strategy valid_tuple_axes(), which generates tu‐
8794 ples of axis-indices that can be passed to the axis argument in NumPy's
8795 sequential functions (e.g. numpy:numpy.sum()).
8796 Thanks to Ryan Soklaski for this strategy.
8798 4.8.0 - 2019-03-06
8800 This release significantly tightens validation in hypothesis.settings.
8801 max_examples, buffer_size, and stateful_step_count must be positive in‐
8802 tegers; deadline must be a positive number or None; and derandomize
8803 must be either True or False.
8804 As usual, this replaces existing errors with a more helpful error and
8806 starts new validation checks as deprecation warnings.
8807 4.7.19 - 2019-03-04
8809 This release makes some micro-optimisations to certain calculations
8810 performed in the shrinker. These should particularly speed up large
8811 test cases where the shrinker makes many small changes. It will also
8812 reduce the amount allocated, but most of this is garbage that would
8813 have been immediately thrown away, so you probably won't see much ef‐
8814 fect specifically from that.
8815 4.7.18 - 2019-03-03
8817 This patch removes some overhead from arrays() with a constant shape
8818 and dtype. The resulting performance improvement is modest, but worth‐
8819 while for small arrays.
8820 4.7.17 - 2019-03-01
8822 This release makes some micro-optimisations within Hypothesis's inter‐
8823 nal representation of test cases. This should cause heavily nested
8824 test cases to allocate less during generation and shrinking, which
8825 should speed things up slightly.
8826 4.7.16 - 2019-02-28
8828 This changes the order in which Hypothesis runs certain operations dur‐
8829 ing shrinking. This should significantly decrease memory usage and
8830 speed up shrinking of large examples.
8831 4.7.15 - 2019-02-28
8833 This release allows Hypothesis to calculate a number of attributes of
8834 generated test cases lazily. This should significantly reduce memory
8835 usage and modestly improve performance, especially for large test
8836 cases.
8837 4.7.14 - 2019-02-28
8839 This release reduces the number of operations the shrinker will try
8840 when reordering parts of a test case. This should in some circum‐
8841 stances significantly speed up shrinking. It may result in different
8842 final test cases, and if so usually slightly worse ones, but it should
8843 not generally have much impact on the end result as the operations re‐
8844 moved were typically useless.
8845 4.7.13 - 2019-02-27
8847 This release changes how Hypothesis reorders examples within a test
8848 case during shrinking. This should make shrinking considerably faster.
8849 4.7.12 - 2019-02-27
8851 This release slightly improves the shrinker's ability to replace parts
8852 of a test case with their minimal version, by allowing it to do so in
8853 bulk rather than one at a time. Where this is effective, shrinker per‐
8854 formance should be modestly improved.
8855 4.7.11 - 2019-02-25
8857 This release makes some micro-optimisations to common operations per‐
8858 formed during shrinking. Shrinking should now be slightly faster, es‐
8859 pecially for large examples with relatively fast test functions.
8860 4.7.10 - 2019-02-25
8862 This release is a purely internal refactoring of Hypothesis's API for
8863 representing test cases. There should be no user visible effect.
8864 4.7.9 - 2019-02-24
8866 This release changes certain shrink passes to make them more efficient
8867 when they aren't making progress.
8868 4.7.8 - 2019-02-23
8870 This patch removes some unused code, which makes the internals a bit
8871 easier to understand. There is no user-visible impact.
8872 4.7.7 - 2019-02-23
8874 This release reduces the number of operations the shrinker will try
8875 when reordering parts of a test case. This should in some circum‐
8876 stances significantly speed up shrinking. It may result in different
8877 final test cases, and if so usually slightly worse ones, but it should
8878 not generally have much impact on the end result as the operations re‐
8879 moved were typically useless.
8880 4.7.6 - 2019-02-23
8882 This patch removes some unused code from the shrinker. There is no
8883 user-visible change.
8884 4.7.5 - 2019-02-23
8886 This release changes certain shrink passes to make them adaptive - that
8887 is, in cases where they are successfully making progress they may now
8888 do so significantly faster.
8889 4.7.4 - 2019-02-22
8891 This is a docs-only patch, noting that because the lark-parser is under
8892 active development at version 0.x, hypothesis[lark] APIs may break in
8893 minor releases if necessary to keep up with the upstream package.
8894 4.7.3 - 2019-02-22
8896 This changes Hypothesis to no longer import various test frameworks by
8897 default (if they are installed). which will speed up the initial im‐
8898 port hypothesis call.
8899 4.7.2 - 2019-02-22
8901 This release changes Hypothesis's internal representation of a test
8902 case to calculate some expensive structural information on demand
8903 rather than eagerly. This should reduce memory usage a fair bit, and
8904 may make generation somewhat faster.
8905 4.7.1 - 2019-02-21
8907 This release refactors the internal representation of previously run
8908 test cases. The main thing you should see as a result is that Hypothe‐
8909 sis becomes somewhat less memory hungry.
8910 4.7.0 - 2019-02-21
8912 This patch allows array_shapes() to generate shapes with side-length or
8913 even dimension zero, though the minimum still defaults to one. These
8914 shapes are rare and have some odd behavior, but are particularly impor‐
8915 tant to test for just that reason!
8916 In a related bigfix, arrays() now supports generating zero-dimensional
8918 arrays with dtype=object and a strategy for iterable elements. Previ‐
8919 ously, the array element would incorrectly be set to the first item in
8920 the generated iterable.
8921 Thanks to Ryan Turner for continuing to improve our Numpy support.
8923 4.6.1 - 2019-02-19
8925 This release is a trivial micro-optimisation inside Hypothesis which
8926 should result in it using significantly less memory.
8927 4.6.0 - 2019-02-18
8929 This release changes some inconsistent behavior of arrays() from the
8930 Numpy extra when asked for an array of shape=(). arrays() will now al‐
8931 ways return a Numpy ndarray, and the array will always be of the re‐
8932 quested dtype.
8933 Thanks to Ryan Turner for this change.
8935 4.5.12 - 2019-02-18
8937 This release fixes a minor typo in an internal comment. There is no
8938 user-visible change.
8939 4.5.11 - 2019-02-15
8941 This release fixes issue #1813, a bug introduced in 3.59.1, which
8942 caused random_module() to no longer affect the body of the test: Al‐
8943 though Hypothesis would claim to be seeding the random module in fact
8944 tests would always run with a seed of zero.
8945 4.5.10 - 2019-02-14
8947 This patch fixes an off-by-one error in the maximum length of emails().
8948 Thanks to Krzysztof Jurewicz for pull request #1812.
8949 4.5.9 - 2019-02-14
8951 This patch removes some unused code from the shrinker. There is no
8952 user-visible change.
8953 4.5.8 - 2019-02-12
8955 This release fixes an internal IndexError in Hypothesis that could
8956 sometimes be triggered during shrinking.
8957 4.5.7 - 2019-02-11
8959 This release modifies the shrinker to interleave different types of re‐
8960 duction operations, e.g. switching between deleting data and lowering
8961 scalar values rather than trying entirely deletions then entirely low‐
8962 ering.
8963 This may slow things down somewhat in the typical case, but has the ma‐
8965 jor advantage that many previously difficult to shrink examples should
8966 become much faster, because the shrinker will no longer tend to stall
8967 when trying some ineffective changes to the shrink target but will in‐
8968 stead interleave it with other more effective operations.
8969 4.5.6 - 2019-02-11
8971 This release makes a number of internal changes to the implementation
8972 of hypothesis.extra.lark.from_lark(). These are primarily intended as
8973 a refactoring, but you may see some minor improvements to performance
8974 when generating large strings, and possibly to shrink quality.
8975 4.5.5 - 2019-02-10
8977 This patch prints an explanatory note when issue #1798 is triggered,
8978 because the error message from Numpy is too terse to locate the prob‐
8979 lem.
8980 4.5.4 - 2019-02-08
8982 In Python 2, long integers are not allowed in the shape argument to
8983 arrays(). Thanks to Ryan Turner for fixing this.
8984 4.5.3 - 2019-02-08
8986 This release makes a small internal refactoring to clarify how Hypothe‐
8987 sis instructs tests to stop running when appropriate. There is no
8988 user-visible change.
8989 4.5.2 - 2019-02-06
8991 This release standardises all of the shrinker's internal operations on
8992 running in a random order.
8993 The main effect you will see from this that it should now be much less
8995 common for the shrinker to stall for a long time before making further
8996 progress. In some cases this will correspond to shrinking more slowly,
8997 but on average it should result in faster shrinking.
8998 4.5.1 - 2019-02-05
9000 This patch updates some docstrings, but has no runtime changes.
9001 4.5.0 - 2019-02-03
9003 This release adds exclude_min and exclude_max arguments to floats(), so
9004 that you can easily generate values from open or half-open intervals (‐
9005 issue #1622).
9006 4.4.6 - 2019-02-03
9008 This patch fixes a bug where from_regex() could throw an internal error
9009 if the python:re.IGNORECASE flag was used (issue #1786).
9010 4.4.5 - 2019-02-02
9012 This release removes two shrink passes that Hypothesis runs late in the
9013 process. These were very expensive when the test function was slow and
9014 often didn't do anything useful.
9015 Shrinking should get faster for most failing tests. If you see any re‐
9017 gression in example quality as a result of this release, please let us
9018 know.
9019 4.4.4 - 2019-02-02
9021 This release modifies the way that Hypothesis deletes data during
9022 shrinking. It will primarily be noticeable for very large examples,
9023 which should now shrink faster.
9024 The shrinker is now also able to perform some deletions that it could
9026 not previously, but this is unlikely to be very noticeable.
9027 4.4.3 - 2019-01-25
9029 This release fixes an open file leak that used to cause ResourceWarn‐
9030 ings.
9031 4.4.2 - 2019-01-24
9033 This release changes Hypothesis's internal approach to caching the re‐
9034 sults of executing test cases. The result should be that it is now
9035 significantly less memory hungry, especially when shrinking large test
9036 cases.
9037 Some tests may get slower or faster depending on whether the new or old
9039 caching strategy was well suited to them, but any change in speed in
9040 either direction should be minor.
9041 4.4.1 - 2019-01-24
9043 This patch tightens up some of our internal heuristics to deal with
9044 shrinking floating point numbers, which will now run in fewer circum‐
9045 stances.
9046 You are fairly unlikely to see much difference from this, but if you do
9048 you are likely to see shrinking become slightly faster and/or producing
9049 slightly worse results.
9050 4.4.0 - 2019-01-24
9052 This release adds the from_form() function, which allows automatic
9053 testing against Django forms. (issue #35)
9054 Thanks to Paul Stiverson for this feature, which resolves our oldest
9056 open issue!
9057 4.3.0 - 2019-01-24
9059 This release deprecates HealthCheck.hung_test and disables the associ‐
9060 ated runtime check for tests that ran for more than five minutes. Such
9061 a check is redundant now that we enforce the deadline and max_examples
9062 setting, which can be adjusted independently.
9063 4.2.0 - 2019-01-23
9065 This release adds a new module, hypothesis.extra.lark, which you can
9066 use to generate strings matching a context-free grammar.
9067 In this initial version, only lark-parser EBNF grammars are supported,
9069 by the new hypothesis.extra.lark.from_lark() function.
9070 4.1.2 - 2019-01-23
9072 This patch fixes a very rare overflow bug (issue #1748) which could
9073 raise an InvalidArgument error in complex_numbers() even though the ar‐
9074 guments were valid.
9075 4.1.1 - 2019-01-23
9077 This release makes some improvements to internal code organisation and
9078 documentation and has no impact on behaviour.
9079 4.1.0 - 2019-01-22
9081 This release adds register_random(), which registers random.Random in‐
9082 stances or compatible objects to be seeded and reset by Hypothesis to
9083 ensure that test cases are deterministic.
9084 We still recommend explicitly passing a random.Random instance from
9086 randoms() if possible, but registering a framework-global state for Hy‐
9087 pothesis to manage is better than flaky tests!
9088 4.0.2 - 2019-01-22
9090 This patch fixes issue #1387, where bounded integers() with a very
9091 large range would almost always generate very large numbers. Now, we
9092 usually use the same tuned distribution as unbounded integers().
9093 4.0.1 - 2019-01-16
9095 This release randomizes the order in which the shrinker tries some of
9096 its initial normalization operations. You are unlikely to see much
9097 difference as a result unless your generated examples are very large.
9098 In this case you may see some performance improvements in shrinking.
9099 4.0.0 - 2019-01-14
9101 Welcome to the next major version of Hypothesis!
9102 There are no new features here, as we release those in minor versions.
9104 Instead, 4.0 is a chance for us to remove deprecated features (many al‐
9105 ready converted into no-ops), and turn a variety of warnings into er‐
9106 rors.
9107 If you were running on the last version of Hypothesis 3.x without any
9109 Hypothesis deprecation warnings (or using private APIs), this will be a
9110 very boring upgrade. In fact, nothing will change for you at all. Per
9111 our deprecation policy, warnings added in the last six months (after
9112 2018-07-05) have not been converted to errors.
9113 Removals
9115 • hypothesis.extra.datetime has been removed, replaced by the core date
9116 and time strategies.
9117 • hypothesis.extra.fakefactory has been removed, replaced by general
9119 expansion of Hypothesis' strategies and the third-party ecosystem.
9120 • The SQLite example database backend has been removed.
9122 Settings
9124 • The deadline is now enforced by default, rather than just emitting a
9125 warning when the default (200 milliseconds per test case) deadline is
9126 exceeded.
9127 • The database_file setting has been removed; use database.
9129 • The perform_health_check setting has been removed; use
9131 suppress_health_check.
9132 • The max_shrinks setting has been removed; use phases to disable
9134 shrinking.
9135 • The min_satisfying_examples, max_iterations, strict, timeout, and
9137 use_coverage settings have been removed without user-configurable re‐
9138 placements.
9139 Strategies
9141 • The elements argument is now required for collection strategies.
9142 • The average_size argument was a no-op and has been removed.
9144 • Date and time strategies now only accept min_value and max_value for
9146 bounds.
9147 • builds() now requires that the thing to build is passed as the first
9149 positional argument.
9150 • Alphabet validation for text() raises errors, not warnings, as does
9152 category validation for characters().
9153 • The choices() strategy has been removed. Instead, you can use data()
9155 with sampled_from(), so choice(elements) becomes data.draw(sam‐
9156 pled_from(elements)).
9157 • The streaming() strategy has been removed. Instead, you can use
9159 data() and replace iterating over the stream with data.draw() calls.
9160 • sampled_from() and permutations() raise errors instead of warnings if
9162 passed a collection that is not a sequence.
9163 Miscellaneous
9165 • Applying @given to a test function multiple times was really ineffi‐
9166 cient, and now it's also an error.
9167 • Using the .example() method of a strategy (intended for interactive
9169 exploration) within another strategy or a test function always weak‐
9170 ened data generation and broke shrinking, and now it's an error too.
9171 • The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
9173 ported, as the database_file setting has been removed.
9174 • The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
9176 ported. You can use the --hypothesis-verbosity pytest argument in‐
9177 stead, or write your own setup code using the settings profile system
9178 to replace it.
9179 • Using @seed or derandomize=True now forces database=None to ensure
9181 results are in fact reproducible. If database is not None, doing so
9182 also emits a HypothesisWarning.
9183 • Unused exception types have been removed from hypothesis.errors;
9185 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
9186 ple, Timeout, and WrongFormat.
9187 Hypothesis 3.x
9189 3.88.3 - 2019-01-11
9190 This changes the order that the shrinker tries certain operations in
9191 its "emergency" phase which runs late in the process. The new order
9192 should be better at avoiding long stalls where the shrinker is failing
9193 to make progress, which may be helpful if you have difficult to shrink
9194 test cases. However this will not be noticeable in the vast majority
9195 of use cases.
9196 3.88.2 - 2019-01-11
9198 This is a pure refactoring release that extracts some logic from the
9199 core Hypothesis engine into its own class and file. It should have no
9200 user visible impact.
9201 3.88.1 - 2019-01-11
9203 This patch fixes some markup in our documentation.
9204 3.88.0 - 2019-01-10
9206 Introduces hypothesis.stateful.multiple(), which allows rules in rule
9207 based state machines to send multiple results at once to their target
9208 Bundle, or none at all.
9209 3.87.0 - 2019-01-10
9211 This release contains a massive cleanup of the Hypothesis for Django
9212 extra:
9213 • hypothesis.extra.django.models.models() is deprecated in favor of
9215 hypothesis.extra.django.from_model().
9216 • hypothesis.extra.django.models.add_default_field_mapping() is depre‐
9218 cated in favor of hypothesis.extra.django.register_field_strategy().
9219 • from_model() does not infer a strategy for nullable fields or fields
9221 with a default unless passed infer, like builds(). models.models()
9222 would usually but not always infer, and a special default_value
9223 marker object was required to disable inference.
9224 3.86.9 - 2019-01-09
9226 This release improves some internal logic about when a test case in Hy‐
9227 pothesis's internal representation could lead to a valid test case. In
9228 some circumstances this can lead to a significant speed up during
9229 shrinking. It may have some minor negative impact on the quality of
9230 the final result due to certain shrink passes now having access to less
9231 information about test cases in some circumstances, but this should
9232 rarely matter.
9233 3.86.8 - 2019-01-09
9235 This release has no user visible changes but updates our URLs to use
9236 HTTPS.
9237 3.86.7 - 2019-01-08
9239 Hypothesis can now automatically generate values for Django models with
9240 a URLfield, thanks to a new provisional strategy for URLs (issue
9241 #1388).
9242 3.86.6 - 2019-01-07
9244 This release is a pure refactoring that extracts some internal code
9245 into its own file. It should have no user visible effect.
9246 3.86.5 - 2019-01-06
9248 This is a docs-only patch, which fixes some typos and removes a few hy‐
9249 perlinks for deprecated features.
9250 3.86.4 - 2019-01-04
9252 This release changes the order in which the shrinker tries to delete
9253 data. For large and slow tests this may significantly improve the per‐
9254 formance of shrinking.
9255 3.86.3 - 2019-01-04
9257 This release fixes a bug where certain places Hypothesis internal er‐
9258 rors could be raised during shrinking when a user exception occurred
9259 that suppressed an exception Hypothesis uses internally in its genera‐
9260 tion.
9261 The two known ways to trigger this problem were:
9263 • Errors raised in stateful tests' teardown function.
9265 • Errors raised in finally blocks that wrapped a call to data.draw.
9267 These cases will now be handled correctly.
9269 3.86.2 - 2019-01-04
9271 This patch is a docs-only change to fix a broken hyperlink.
9272 3.86.1 - 2019-01-04
9274 This patch fixes issue #1732, where integers() would always return long
9275 values on Python 2.
9276 3.86.0 - 2019-01-03
9278 This release ensures that infinite numbers are never generated by
9279 floats() with allow_infinity=False, which could previously happen in
9280 some cases where one bound was also provided.
9281 The trivially inconsistent min_value=inf, allow_infinity=False now
9283 raises an InvalidArgumentError, as does the inverse with max_value.
9284 You can still use just(inf) to generate inf without violating other
9285 constraints.
9286 3.85.3 - 2019-01-02
9288 Happy new year everyone! This release has no user visible changes but
9289 updates our copyright headers to include 2019.
9290 3.85.2 - 2018-12-31
9292 This release makes a small change to the way the shrinker works. You
9293 may see some improvements to speed of shrinking on especially large and
9294 hard to shrink examples, but most users are unlikely to see much dif‐
9295 ference.
9296 3.85.1 - 2018-12-30
9298 This patch fixes issue #1700, where a line that contained a Unicode
9299 character before a lambda definition would cause an internal exception.
9300 3.85.0 - 2018-12-29
9302 Introduces the hypothesis.stateful.consumes() function. When defining a
9303 rule in stateful testing, it can be used to mark bundles from which
9304 values should be consumed, i. e. removed after use in the rule. This
9305 has been proposed in issue #136.
9306 Thanks to Jochen Müller for this long-awaited feature.
9308 3.84.6 - 2018-12-28
9310 This patch makes a small internal change to fix an issue in Hypothe‐
9311 sis's own coverage tests (issue #1718).
9312 There is no user-visible change.
9314 3.84.5 - 2018-12-21
9316 This patch refactors the hypothesis.strategies module, so that private
9317 names should no longer appear in tab-completion lists. We previously
9318 relied on __all__ for this, but not all editors respect it.
9319 3.84.4 - 2018-12-21
9321 This is a follow-up patch to ensure that the deprecation date is auto‐
9322 matically recorded for any new deprecations. There is no user-visible
9323 effect.
9324 3.84.3 - 2018-12-20
9326 This patch updates the Hypothesis pytest plugin to avoid a recently
9327 deprecated hook interface. There is no user-visible change.
9328 3.84.2 - 2018-12-19
9330 This patch fixes the internals for integers() with one bound. Values
9331 from this strategy now always shrink towards zero instead of towards
9332 the bound, and should shrink much more efficiently too. On Python 2,
9333 providing a bound incorrectly excluded long integers, which can now be
9334 generated.
9335 3.84.1 - 2018-12-18
9337 This patch adds information about when features were deprecated, but
9338 this is only recorded internally and has no user-visible effect.
9339 3.84.0 - 2018-12-18
9341 This release changes the stateful testing backend from find() to use
9342 @given (issue #1300). This doesn't change how you create stateful
9343 tests, but does make them run more like other Hypothesis tests.
9344 @reproduce_failure and @seed now work for stateful tests.
9346 Stateful tests now respect the deadline and suppress_health_check set‐
9348 tings, though they are disabled by default. You can enable them by us‐
9349 ing @settings(...) as a class decorator with whatever arguments you
9350 prefer.
9351 3.83.2 - 2018-12-17
9353 Hypothesis has adopted Black as our code formatter (issue #1686).
9354 There are no functional changes to the source, but it's prettier!
9355 3.83.1 - 2018-12-13
9357 This patch increases the variety of examples generated by from_type().
9358 3.83.0 - 2018-12-12
9360 Our pytest plugin now warns you when strategy functions have been col‐
9361 lected as tests, which may happen when e.g. using the @composite deco‐
9362 rator when you should be using @given(st.data()) for inline draws.
9363 Such functions always pass when treated as tests, because the lazy cre‐
9364 ation of strategies mean that the function body is never actually exe‐
9365 cuted!
9366 3.82.6 - 2018-12-11
9368 Hypothesis can now show statistics when running under pytest-xdist.
9369 Previously, statistics were only reported when all tests were run in a
9370 single process (issue #700).
9371 3.82.5 - 2018-12-08
9373 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
9374 to integers() could cause errors on Python 3 due to internal rounding.
9375 3.82.4 - 2018-12-08
9377 Hypothesis now seeds and resets the global state of np.random for each
9378 test case, to ensure that tests are reproducible.
9379 This matches and complements the existing handling of the python:random
9381 module - Numpy simply maintains an independent PRNG for performance
9382 reasons.
9383 3.82.3 - 2018-12-08
9385 This is a no-op release to add the new Framework :: Hypothesis trove
9386 classifier to hypothesis on PyPI.
9387 You can use it as a filter to find Hypothesis-related packages such as
9389 extensions as they add the tag over the coming weeks, or simply visit
9390 our curated list.
9391 3.82.2 - 2018-12-08
9393 The Hypothesis for Pandas extension is now listed in setup.py, so you
9394 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
9395 tion.
9396 3.82.1 - 2018-10-29
9398 This patch fixes from_type() on Python 2 for classes where cls.__init__
9399 is object.__init__. Thanks to ccxcz for reporting issue #1656.
9400 3.82.0 - 2018-10-29
9402 The alphabet argument for text() now uses its default value of charac‐
9403 ters(blacklist_categories=('Cs',)) directly, instead of hiding that be‐
9404 hind alphabet=None and replacing it within the function. Passing None
9405 is therefore deprecated.
9406 3.81.0 - 2018-10-27
9408 GenericStateMachine and RuleBasedStateMachine now raise an explicit er‐
9409 ror when instances of settings are assigned to the classes' settings
9410 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
9411 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
9412 tor to handle this automatically.
9413 3.80.0 - 2018-10-25
9415 Since version 3.68.0, arrays() checks that values drawn from the ele‐
9416 ments and fill strategies can be safely cast to the dtype of the array,
9417 and emits a warning otherwise.
9418 This release expands the checks to cover overflow for finite complex64
9420 elements and string truncation caused by too-long elements or trailing
9421 null characters (issue #1591).
9422 3.79.4 - 2018-10-25
9424 Tests using @given now shrink errors raised from pytest helper func‐
9425 tions, instead of reporting the first example found.
9426 This was previously fixed in version 3.56.0, but only for stateful
9428 testing.
9429 3.79.3 - 2018-10-23
9431 Traceback elision is now disabled on Python 2, to avoid an import-time
9432 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
9433 3.79.2: issue #1648).
9434 We encourage all users to upgrade to Python 3 before the end of 2019.
9436 3.79.2 - 2018-10-23
9438 This patch shortens tracebacks from Hypothesis, so you can see exactly
9439 happened in your code without having to skip over irrelevant details
9440 about our internals (issue #848).
9441 In the example test (see pull request #1582), this reduces tracebacks
9443 from nine frames to just three - and for a test with multiple errors,
9444 from seven frames per error to just one!
9445 If you do want to see the internal details, you can disable frame eli‐
9447 sion by setting verbosity to debug.
9448 3.79.1 - 2018-10-22
9450 The abstract number classes Number, Complex, Real, Rational, and Inte‐
9451 gral are now supported by the from_type() strategy. Previously, you
9452 would have to use register_type_strategy() before they could be re‐
9453 solved (issue #1636)
9454 3.79.0 - 2018-10-18
9456 This release adds a CLI flag for verbosity --hypothesis-verbosity to
9457 the Hypothesis pytest plugin, applied after loading the profile speci‐
9458 fied by --hypothesis-profile. Valid options are the names of verbosity
9459 settings, quiet, normal, verbose or debug.
9460 Thanks to Bex Dunn for writing this patch at the PyCon Australia
9462 sprints!
9463 The pytest header now correctly reports the current profile if --hy‐
9465 pothesis-profile has been used.
9466 Thanks to Mathieu Paturel for the contribution at the Canberra Python
9468 Hacktoberfest.
9469 3.78.0 - 2018-10-16
9471 This release has deprecated the generation of integers, floats and
9472 fractions when the conversion of the upper and/ or lower bound is not
9473 100% exact, e.g. when an integer gets passed a bound that is not a
9474 whole number. (issue #1625)
9475 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
9477 3.77.0 - 2018-10-16
9479 This minor release adds functionality to settings allowing it to be
9480 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
9481 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
9483 3.76.1 - 2018-10-16
9485 This patch fixes some warnings added by recent releases of pydocstyle
9486 and mypy.
9487 3.76.0 - 2018-10-11
9489 This release deprecates using floats for min_size and max_size.
9490 The type hint for average_size arguments has been changed from Op‐
9492 tional[int] to None, because non-None values are always ignored and
9493 deprecated.
9494 3.75.4 - 2018-10-10
9496 This patch adds more internal comments to the core engine's se‐
9497 quence-length shrinker. There should be no user-visible change.
9498 3.75.3 - 2018-10-09
9500 This patch adds additional comments to some of the core engine's inter‐
9501 nal data structures. There is no user-visible change.
9502 3.75.2 - 2018-10-09
9504 This patch avoids caching a trivial case, fixing issue #493.
9505 3.75.1 - 2018-10-09
9507 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
9508 for this contribution!
9509 3.75.0 - 2018-10-08
9511 This release deprecates the use of min_size=None, setting the default
9512 min_size to 0 (issue #1618).
9513 3.74.3 - 2018-10-08
9515 This patch makes some small internal changes to comply with a new lint
9516 setting in the build. There should be no user-visible change.
9517 3.74.2 - 2018-10-03
9519 This patch fixes issue #1153, where time spent reifying a strategy was
9520 also counted in the time spent generating the first example. Strate‐
9521 gies are now fully constructed and validated before the timer is
9522 started.
9523 3.74.1 - 2018-10-03
9525 This patch fixes some broken formatting and links in the documentation.
9526 3.74.0 - 2018-10-01
9528 This release checks that the value of the print_blob setting is a
9529 PrintSettings instance.
9530 Being able to specify a boolean value was not intended, and is now dep‐
9532 recated. In addition, specifying True will now cause the blob to al‐
9533 ways be printed, instead of causing it to be suppressed.
9534 Specifying any value that is not a PrintSettings or a boolean is now an
9536 error.
9537 3.73.5 - 2018-10-01
9539 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
9540 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
9541 eter names min_value and max_value instead of the deprecated names
9542 3.73.4 - 2018-09-30
9544 This patch ensures that Hypothesis deprecation warnings display the
9545 code that emitted them when you're not running in -Werror mode (issue
9546 #652).
9547 3.73.3 - 2018-09-27
9549 Tracebacks involving @composite are now slightly shorter due to some
9550 internal refactoring.
9551 3.73.2 - 2018-09-26
9553 This patch fixes errors in the internal comments for one of the
9554 shrinker passes. There is no user-visible change.
9555 3.73.1 - 2018-09-25
9557 This patch substantially improves the distribution of data generated
9558 with recursive(), and fixes a rare internal error (issue #1502).
9559 3.73.0 - 2018-09-24
9561 This release adds the fulfill() function, which is designed for testing
9562 code that uses dpcontracts 0.4 or later for input validation. This
9563 provides some syntactic sugar around use of assume(), to automatically
9564 filter out and retry calls that cause a precondition check to fail (‐
9565 issue #1474).
9566 3.72.0 - 2018-09-24
9568 This release makes setting attributes of the hypothesis.settings class
9569 an explicit error. This has never had any effect, but could mislead
9570 users who confused it with the current settings instance hypothe‐
9571 sis.settings.default (which is also immutable). You can change the
9572 global settings with settings profiles.
9573 3.71.11 - 2018-09-24
9575 This patch factors out some common code in the shrinker for iterating
9576 over pairs of data blocks. There should be no user-visible change.
9577 3.71.10 - 2018-09-18
9579 This patch allows from_type() to handle the empty tuple type, typ‐
9580 ing.Tuple[()].
9581 3.71.9 - 2018-09-17
9583 This patch updates some internal comments for mypy. There is no
9584 user-visible effect, even for Mypy users.
9585 3.71.8 - 2018-09-17
9587 This patch fixes a rare bug that would cause a particular shrinker pass
9588 to raise an IndexError, if a shrink improvement changed the underlying
9589 data in an unexpected way.
9590 3.71.7 - 2018-09-17
9592 This release fixes the broken cross-references in our docs, and adds a
9593 CI check so we don't add new ones.
9594 3.71.6 - 2018-09-16
9596 This patch fixes two bugs (issue #944 and issue #1521), where messages
9597 about @seed did not check the current verbosity setting, and the wrong
9598 settings were active while executing explicit examples.
9599 3.71.5 - 2018-09-15
9601 This patch fixes a DeprecationWarning added in Python 3.8 (issue
9602 #1576).
9603 Thanks to tirkarthi for this contribution!
9605 3.71.4 - 2018-09-14
9607 This is a no-op release, which implements automatic DOI minting and
9608 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
9609 zon 2020 programme for providing this service!
9610 Check our CITATION.cff file for details, or head right on over to
9612 doi.org/10.5281/zenodo.1412597
9613 3.71.3 - 2018-09-10
9615 This release adds the test name to some deprecation warnings, for eas‐
9616 ier debugging.
9617 Thanks to Sanyam Khurana for the patch!
9619 3.71.2 - 2018-09-10
9621 This release makes Hypothesis's memory usage substantially smaller for
9622 tests with many examples, by bounding the number of past examples it
9623 keeps around.
9624 You will not see much difference unless you are running tests with
9626 max_examples set to well over 1000, but if you do have such tests then
9627 you should see memory usage mostly plateau where previously it would
9628 have grown linearly with time.
9629 3.71.1 - 2018-09-09
9631 This patch adds internal comments to some tree traversals in the core
9632 engine. There is no user-visible change.
9633 3.71.0 - 2018-09-08
9635 This release deprecates the coverage-guided testing functionality, as
9636 it has proven brittle and does not really pull its weight.
9637 We intend to replace it with something more useful in the future, but
9639 the feature in its current form does not seem to be worth the cost of
9640 using, and whatever replaces it will likely look very different.
9641 3.70.4 - 2018-09-08
9643 This patch changes the behaviour of reproduce_failure() so that blobs
9644 are only printed in quiet mode when the print_blob setting is set to
9645 ALWAYS.
9646 Thanks to Cameron McGill for writing this patch at the PyCon Australia
9648 sprints!
9649 3.70.3 - 2018-09-03
9651 This patch removes some unnecessary code from the internals. There is
9652 no user-visible change.
9653 3.70.2 - 2018-09-03
9655 This patch fixes an internal bug where a corrupted argument to
9656 @reproduce_failure could raise the wrong type of error. Thanks again
9657 to Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
9658 tently provides excellent bug reports including issue #1558.
9659 3.70.1 - 2018-09-03
9661 This patch updates hypothesis to report its version and settings when
9662 run with pytest. (issue #1223).
9663 Thanks to Jack Massey for this feature.
9665 3.70.0 - 2018-09-01
9667 This release adds a fullmatch argument to from_regex(). When full‐
9668 match=True, the whole example will match the regex pattern as for
9669 python:re.fullmatch().
9670 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
9672 sprints!
9673 3.69.12 - 2018-08-30
9675 This release reverts the changes to logging handling in 3.69.11, which
9676 broke test that use the pytest caplog fixture internally because all
9677 logging was disabled (issue #1546).
9678 3.69.11 - 2018-08-29
9680 This patch will hide all logging messages produced by test cases before
9681 the final, minimal, failing test case (issue #356).
9682 Thanks to Gary Donovan for writing this patch at the PyCon Australia
9684 sprints!
9685 3.69.10 - 2018-08-29
9687 This patch fixes a bug that prevents coverage from reporting unexecuted
9688 Python files (issue #1085).
9689 Thanks to Gary Donovan for writing this patch at the PyCon Australia
9691 sprints!
9692 3.69.9 - 2018-08-28
9694 This patch improves the packaging of the Python package by adding LI‐
9695 CENSE.txt to the sdist (issue #1311), clarifying the minimum supported
9696 versions of pytz and dateutil (issue #1383), and adds keywords to the
9697 metadata (issue #1520).
9698 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
9700 tralia sprints!
9701 3.69.8 - 2018-08-28
9703 This is an internal change which replaces pickle with json to prevent
9704 possible security issues.
9705 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
9707 sprints!
9708 3.69.7 - 2018-08-28
9710 This patch ensures that note() prints the note for every test case when
9711 the verbosity setting is Verbosity.verbose. At normal verbosity it
9712 only prints from the final test case.
9713 Thanks to Tom McDermott for writing this patch at the PyCon Australia
9715 sprints!
9716 3.69.6 - 2018-08-27
9718 This patch improves the testing of some internal caching. It should
9719 have no user-visible effect.
9720 3.69.5 - 2018-08-27
9722 This change performs a small rename and refactoring in the core engine.
9723 There is no user-visible change.
9724 3.69.4 - 2018-08-27
9726 This change improves the core engine's ability to avoid unnecessary
9727 work, by consulting its cache of previously-tried inputs in more cases.
9728 3.69.3 - 2018-08-27
9730 This patch handles passing an empty python:enum.Enum to from_type() by
9731 returning nothing(), instead of raising an internal python:Assertion‐
9732 Error.
9733 Thanks to Paul Amazona for writing this patch at the PyCon Australia
9735 sprints!
9736 3.69.2 - 2018-08-23
9738 This patch fixes a small mistake in an internal comment. There is no
9739 user-visible change.
9740 3.69.1 - 2018-08-21
9742 This change fixes a small bug in how the core engine consults its cache
9743 of previously-tried inputs. There is unlikely to be any user-visible
9744 change.
9745 3.69.0 - 2018-08-20
9747 This release improves argument validation for stateful testing.
9748 • If the target or targets of a rule() are invalid, we now raise a use‐
9750 ful validation error rather than an internal exception.
9751 • Passing both the target and targets arguments is deprecated - append
9753 the target bundle to the targets tuple of bundles instead.
9754 • Passing the name of a Bundle rather than the Bundle itself is also
9756 deprecated.
9757 3.68.3 - 2018-08-20
9759 This is a docs-only patch, fixing some typos and formatting issues.
9760 3.68.2 - 2018-08-19
9762 This change fixes a small bug in how the core engine caches the results
9763 of previously-tried inputs. The effect is unlikely to be noticeable,
9764 but it might avoid unnecessary work in some cases.
9765 3.68.1 - 2018-08-18
9767 This patch documents the from_dtype() function, which infers a strategy
9768 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
9769 directly when creating e.g. Pandas objects.
9770 3.68.0 - 2018-08-15
9772 arrays() now checks that integer and float values drawn from elements
9773 and fill strategies can be safely cast to the dtype of the array, and
9774 emits a warning otherwise (issue #1385).
9775 Elements in the resulting array could previously violate constraints on
9777 the elements strategy due to floating-point overflow or truncation of
9778 integers to fit smaller types.
9779 3.67.1 - 2018-08-14
9781 This release contains a tiny refactoring of the internals. There is no
9782 user-visible change.
9783 3.67.0 - 2018-08-10
9785 This release adds a width argument to floats(), to generate lower-pre‐
9786 cision floating point numbers for e.g. Numpy arrays.
9787 The generated examples are always instances of Python's native float
9789 type, which is 64bit, but passing width=32 will ensure that all values
9790 can be exactly represented as 32bit floats. This can be useful to
9791 avoid overflow (to +/- infinity), and for efficiency of generation and
9792 shrinking.
9793 Half-precision floats (width=16) are also supported, but require Numpy
9795 if you are running Python 3.5 or earlier.
9796 3.66.33 - 2018-08-10
9798 This release fixes a bug in floats(), where setting allow_infin‐
9799 ity=False and exactly one of min_value and max_value would allow infi‐
9800 nite values to be generated.
9801 3.66.32 - 2018-08-09
9803 This release adds type hints to the example() and seed() decorators,
9804 and fixes the type hint on register_type_strategy(). The second argu‐
9805 ment to register_type_strategy() must either be a SearchStrategy, or a
9806 callable which takes a type and returns a SearchStrategy.
9807 3.66.31 - 2018-08-08
9809 Another set of changes designed to improve the performance of shrinking
9810 on large examples. In particular the shrinker should now spend consid‐
9811 erably less time running useless shrinks.
9812 3.66.30 - 2018-08-06
9814 "Bug fixes and performance improvements".
9815 This release is a fairly major overhaul of the shrinker designed to im‐
9817 prove its behaviour on large examples, especially around stateful test‐
9818 ing. You should hopefully see shrinking become much faster, with little
9819 to no quality degradation (in some cases quality may even improve).
9820 3.66.29 - 2018-08-05
9822 This release fixes two very minor bugs in the core engine:
9823 • it fixes a corner case that was missing in 3.66.28, which should
9825 cause shrinking to work slightly better.
9826 • it fixes some logic for how shrinking interacts with the database
9828 that was causing Hypothesis to be insufficiently aggressive about
9829 clearing out old keys.
9830 3.66.28 - 2018-08-05
9832 This release improves how Hypothesis handles reducing the size of inte‐
9833 gers' representation. This change should mostly be invisible as it's
9834 purely about the underlying representation and not the generated value,
9835 but it may result in some improvements to shrink performance.
9836 3.66.27 - 2018-08-05
9838 This release changes the order in which Hypothesis chooses parts of the
9839 test case to shrink. For typical usage this should be a significant
9840 performance improvement on large examples. It is unlikely to have a ma‐
9841 jor impact on example quality, but where it does change the result it
9842 should usually be an improvement.
9843 3.66.26 - 2018-08-05
9845 This release improves the debugging information that the shrinker emits
9846 about the operations it performs, giving better summary statistics
9847 about which passes resulted in test executions and whether they were
9848 successful.
9849 3.66.25 - 2018-08-05
9851 This release fixes several bugs that were introduced to the shrinker in
9852 3.66.24 which would have caused it to behave significantly less well
9853 than advertised. With any luck you should actually see the promised
9854 benefits now.
9855 3.66.24 - 2018-08-03
9857 This release changes how Hypothesis deletes data when shrinking in or‐
9858 der to better handle deletion of large numbers of contiguous sequences.
9859 Most tests should see little change, but this will hopefully provide a
9860 significant speed up for stateful testing.
9861 3.66.23 - 2018-08-02
9863 This release makes some internal changes to enable further improvements
9864 to the shrinker. You may see some changes in the final shrunk examples,
9865 but they are unlikely to be significant.
9866 3.66.22 - 2018-08-01
9868 This release adds some more internal caching to the shrinker. This
9869 should cause a significant speed up for shrinking, especially for
9870 stateful testing and large example sizes.
9871 3.66.21 - 2018-08-01
9873 This patch is for downstream packagers - our tests now pass under
9874 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
9875 of Hypothesis itself.
9876 3.66.20 - 2018-08-01
9878 This release removes some functionality from the shrinker that was tak‐
9879 ing a considerable amount of time and does not appear to be useful any
9880 more due to a number of quality improvements in the shrinker.
9881 You may see some degradation in shrink quality as a result of this, but
9883 mostly shrinking should just get much faster.
9884 3.66.19 - 2018-08-01
9886 This release slightly changes the format of some debugging information
9887 emitted during shrinking, and refactors some of the internal interfaces
9888 around that.
9889 3.66.18 - 2018-07-31
9891 This release is a very small internal refactoring which should have no
9892 user visible impact.
9893 3.66.17 - 2018-07-31
9895 This release fixes a bug that could cause an IndexError to be raised
9896 from inside Hypothesis during shrinking. It is likely that it was im‐
9897 possible to trigger this bug in practice - it was only made visible by
9898 some currently unreleased work.
9899 3.66.16 - 2018-07-31
9901 This release is a very small internal refactoring which should have no
9902 user visible impact.
9903 3.66.15 - 2018-07-31
9905 This release makes Hypothesis's shrinking faster by removing some re‐
9906 dundant work that it does when minimizing values in its internal repre‐
9907 sentation.
9908 3.66.14 - 2018-07-30
9910 This release expands the deprecation of timeout from 3.16.0 to also
9911 emit the deprecation warning in find or stateful testing.
9912 3.66.13 - 2018-07-30
9914 This release adds an additional shrink pass that is able to reduce the
9915 size of examples in some cases where the transformation is non-obvious.
9916 In particular this will improve the quality of some examples which
9917 would have regressed in 3.66.12.
9918 3.66.12 - 2018-07-28
9920 This release changes how we group data together for shrinking. It
9921 should result in improved shrinker performance, especially in stateful
9922 testing.
9923 3.66.11 - 2018-07-28
9925 This patch modifies how which rule to run is selected during rule based
9926 stateful testing. This should result in a slight performance increase
9927 during generation and a significant performance and quality improvement
9928 when shrinking.
9929 As a result of this change, some state machines which would previously
9931 have thrown an InvalidDefinition are no longer detected as invalid.
9932 3.66.10 - 2018-07-28
9934 This release weakens some minor functionality in the shrinker that had
9935 only modest benefit and made its behaviour much harder to reason about.
9936 This is unlikely to have much user visible effect, but it is possible
9938 that in some cases shrinking may get slightly slower. It is primarily
9939 to make it easier to work on the shrinker and pave the way for future
9940 work.
9941 3.66.9 - 2018-07-26
9943 This release improves the information that Hypothesis emits about its
9944 shrinking when verbosity is set to debug.
9945 3.66.8 - 2018-07-24
9947 This patch includes some minor fixes in the documentation, and updates
9948 the minimum version of pytest to 3.0 (released August 2016).
9949 3.66.7 - 2018-07-24
9951 This release fixes a bug where difficult to shrink tests could some‐
9952 times trigger an internal assertion error inside the shrinker.
9953 3.66.6 - 2018-07-23
9955 This patch ensures that Hypothesis fully supports Python 3.7, by up‐
9956 grading from_type() (issue #1264) and fixing some minor issues in our
9957 test suite (issue #1148).
9958 3.66.5 - 2018-07-22
9960 This patch fixes the online docs for various extras, by ensuring that
9961 their dependencies are installed on readthedocs.io (issue #1326).
9962 3.66.4 - 2018-07-20
9964 This release improves the shrinker's ability to reorder examples.
9965 For example, consider the following test:
9967 import hypothesis.strategies as st
9969 from hypothesis import given
9970 @given(st.text(), st.text())
9973 def test_non_equal(x, y):
9974 assert x != y
9975 Previously this could have failed with either of x="", y="0" or x="0",
9977 y="". Now it should always fail with x="", y="0".
9978 This will allow the shrinker to produce more consistent results, espe‐
9980 cially in cases where test cases contain some ordered collection whose
9981 actual order does not matter.
9982 3.66.3 - 2018-07-20
9984 This patch fixes inference in the builds() strategy with subtypes of
9985 python:typing.NamedTuple, where the __init__ method is not useful for
9986 introspection. We now use the field types instead - thanks to James
9987 Uther for identifying this bug.
9988 3.66.2 - 2018-07-19
9990 This release improves the shrinker's ability to handle situations where
9991 there is an additive constraint between two values.
9992 For example, consider the following test:
9994 import hypothesis.strategies as st
9996 from hypothesis import given
9997 @given(st.integers(), st.integers())
10000 def test_does_not_exceed_100(m, n):
10001 assert m + n < 100
10002 Previously this could have failed with almost any pair (m, n) with 0 <=
10004 m <= n and m + n == 100. Now it should almost always fail with m=0,
10005 n=100.
10006 This is a relatively niche specialisation, but can be useful in situa‐
10008 tions where e.g. a bug is triggered by an integer overflow.
10009 3.66.1 - 2018-07-09
10011 This patch fixes a rare bug where an incorrect percentage drawtime
10012 could be displayed for a test, when the system clock was changed during
10013 a test running under Python 2 (we use python:time.monotonic() where it
10014 is available to avoid such problems). It also fixes a possible
10015 zero-division error that can occur when the underlying C library dou‐
10016 ble-rounds an intermediate value in python:math.fsum() and gets the
10017 least significant bit wrong.
10018 3.66.0 - 2018-07-05
10020 This release improves validation of the alphabet argument to the text()
10021 strategy. The following misuses are now deprecated, and will be an er‐
10022 ror in a future version:
10023 • passing an unordered collection (such as set('abc')), which violates
10025 invariants about shrinking and reproducibility
10026 • passing an alphabet sequence with elements that are not strings
10028 • passing an alphabet sequence with elements that are not of length
10030 one, which violates any size constraints that may apply
10031 Thanks to Sushobhit for adding these warnings (issue #1329).
10033 3.65.3 - 2018-07-04
10035 This release fixes a mostly theoretical bug where certain usage of the
10036 internal API could trigger an assertion error inside Hypothesis. It is
10037 unlikely that this problem is even possible to trigger through the pub‐
10038 lic API.
10039 3.65.2 - 2018-07-04
10041 This release fixes dependency information for coverage. Previously Hy‐
10042 pothesis would allow installing coverage with any version, but it only
10043 works with coverage 4.0 or later.
10044 We now specify the correct metadata in our setup.py, so Hypothesis will
10046 only allow installation with compatible versions of coverage.
10047 3.65.1 - 2018-07-03
10049 This patch ensures that stateful tests which raise an error from a
10050 pytest helper still print the sequence of steps taken to reach that
10051 point (issue #1372). This reporting was previously broken because the
10052 helpers inherit directly from python:BaseException, and therefore re‐
10053 quire special handling to catch without breaking e.g. the use of ctrl-C
10054 to quit the test.
10055 3.65.0 - 2018-06-30
10057 This release deprecates the max_shrinks setting in favor of an internal
10058 heuristic. If you need to avoid shrinking examples, use the phases
10059 setting instead. (issue #1235)
10060 3.64.2 - 2018-06-27
10062 This release fixes a bug where an internal assertion error could some‐
10063 times be triggered while shrinking a failing test.
10064 3.64.1 - 2018-06-27
10066 This patch fixes type-checking errors in our vendored pretty-printer,
10067 which were ignored by our mypy config but visible for anyone else
10068 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
10069 3.64.0 - 2018-06-26
10071 This release adds an interface which can be used to insert a wrapper
10072 between the original test function and @given (issue #1257). This will
10073 be particularly useful for test runner extensions such as pytest-trio,
10074 but is not recommended for direct use by other users of Hypothesis.
10075 3.63.0 - 2018-06-26
10077 This release adds a new mechanism to infer strategies for classes de‐
10078 fined using attrs, based on the the type, converter, or validator of
10079 each attribute. This inference is now built in to builds() and
10080 from_type().
10081 On Python 2, from_type() no longer generates instances of int when
10083 passed long, or vice-versa.
10084 3.62.0 - 2018-06-26
10086 This release adds PEP 484 type hints to Hypothesis on a provisional ba‐
10087 sis, using the comment-based syntax for Python 2 compatibility. You
10088 can read more about our type hints here.
10089 It also adds the py.typed marker specified in PEP 561. After you pip
10091 install hypothesis, mypy 0.590 or later will therefore type-check your
10092 use of our public interface!
10093 3.61.0 - 2018-06-24
10095 This release deprecates the use of settings as a context manager, the
10096 use of which is somewhat ambiguous.
10097 Users should define settings with global state or with the
10099 @settings(...) decorator.
10100 3.60.1 - 2018-06-20
10102 Fixed a bug in generating an instance of a Django model from a strategy
10103 where the primary key is generated as part of the strategy. See details
10104 here.
10105 Thanks to Tim Martin for this contribution.
10107 3.60.0 - 2018-06-20
10109 This release adds the @initialize decorator for stateful testing (orig‐
10110 inally discussed in issue #1216). All @initialize rules will be called
10111 once each in an arbitrary order before any normal rule is called.
10112 3.59.3 - 2018-06-19
10114 This is a no-op release to take into account some changes to the re‐
10115 lease process. It should have no user visible effect.
10116 3.59.2 - 2018-06-18
10118 This adds support for partially sorting examples which cannot be fully
10119 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
10120 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
10121 0].
10122 Thanks to Luke for contributing.
10124 3.59.1 - 2018-06-16
10126 This patch uses python:random.getstate() and python:random.setstate()
10127 to restore the PRNG state after @given runs deterministic tests. With‐
10128 out restoring state, you might have noticed problems such as issue
10129 #1266. The fix also applies to stateful testing (issue #702).
10130 3.59.0 - 2018-06-14
10132 This release adds the emails() strategy, which generates unicode
10133 strings representing an email address.
10134 Thanks to Sushobhit for moving this to the public API (issue #162).
10136 3.58.1 - 2018-06-13
10138 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
10139 1 2 3.
10140 Thanks to Luke for contributing.
10142 3.58.0 - 2018-06-13
10144 This adds a new extra timezones() strategy that generates dateutil
10145 timezones.
10146 Thanks to Conrad for contributing.
10148 3.57.0 - 2018-05-20
10150 Using an unordered collection with the permutations() strategy has been
10151 deprecated because the order in which e.g. a set shrinks is arbitrary.
10152 This may cause different results between runs.
10153 3.56.10 - 2018-05-16
10155 This release makes hypothesis.settings.define_setting a private method,
10156 which has the effect of hiding it from the documentation.
10157 3.56.9 - 2018-05-11
10159 This is another release with no functionality changes as part of
10160 changes to Hypothesis's new release tagging scheme.
10161 3.56.8 - 2018-05-10
10163 This is a release with no functionality changes that moves Hypothesis
10164 over to a new release tagging scheme.
10165 3.56.7 - 2018-05-10
10167 This release provides a performance improvement for most tests, but in
10168 particular users of sampled_from() who don't have numpy installed
10169 should see a significant performance improvement.
10170 3.56.6 - 2018-05-09
10172 This patch contains further internal work to support Mypy. There are
10173 no user-visible changes... yet.
10174 3.56.5 - 2018-04-22
10176 This patch contains some internal refactoring to run mypy in CI. There
10177 are no user-visible changes.
10178 3.56.4 - 2018-04-21
10180 This release involves some very minor internal clean up and should have
10181 no user visible effect at all.
10182 3.56.3 - 2018-04-20
10184 This release fixes a problem introduced in 3.56.0 where setting the hy‐
10185 pothesis home directory (through currently undocumented means) would no
10186 longer result in the default database location living in the new home
10187 directory.
10188 3.56.2 - 2018-04-20
10190 This release fixes a problem introduced in 3.56.0 where setting
10191 max_examples to 1 would result in tests failing with Unsatisfiable.
10192 This problem could also occur in other harder to trigger circumstances
10193 (e.g. by setting it to a low value, having a hard to satisfy assump‐
10194 tion, and disabling health checks).
10195 3.56.1 - 2018-04-20
10197 This release fixes a problem that was introduced in 3.56.0: Use of the
10198 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
10199 cated, actually broken due to being read before various setup the dep‐
10200 recation path needed was done. It now works correctly (and emits a dep‐
10201 recation warning).
10202 3.56.0 - 2018-04-17
10204 This release deprecates several redundant or internally oriented
10205 settings, working towards an orthogonal set of configuration options
10206 that are widely useful without requiring any knowledge of our internals
10207 (issue #535).
10208 • Deprecated settings that no longer have any effect are no longer
10210 shown in the __repr__ unless set to a non-default value.
10211 • hypothesis.settings.perform_health_check is deprecated, as it dupli‐
10213 cates suppress_health_check.
10214 • hypothesis.settings.max_iterations is deprecated and disabled, be‐
10216 cause we can usually get better behaviour from an internal heuristic
10217 than a user-controlled setting.
10218 • hypothesis.settings.min_satisfying_examples is deprecated and dis‐
10220 abled, due to overlap with the filter_too_much healthcheck and poor
10221 interaction with max_examples.
10222 • HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
10224 the profile system instead.
10225 • Examples tried by find() are now reported at debug verbosity level
10227 (as well as verbose level).
10228 3.55.6 - 2018-04-14
10230 This release fixes a somewhat obscure condition (issue #1230) under
10231 which you could occasionally see a failing test trigger an assertion
10232 error inside Hypothesis instead of failing normally.
10233 3.55.5 - 2018-04-14
10235 This patch fixes one possible cause of issue #966. When running Python
10236 2 with hash randomisation, passing a python:bytes object to python:ran‐
10237 dom.seed() would use version=1, which broke derandomize (because the
10238 seed depended on a randomised hash). If derandomize is still nondeter‐
10239 ministic for you, please open an issue.
10240 3.55.4 - 2018-04-13
10242 This patch makes a variety of minor improvements to the documentation,
10243 and improves a few validation messages for invalid inputs.
10244 3.55.3 - 2018-04-12
10246 This release updates the URL metadata associated with the PyPI package
10247 (again). It has no other user visible effects.
10248 3.55.2 - 2018-04-11
10250 This release updates the URL metadata associated with the PyPI package.
10251 It has no other user visible effects.
10252 3.55.1 - 2018-04-06
10254 This patch relaxes constraints in our tests on the expected values re‐
10255 turned by the standard library function hypot() and the internal helper
10256 function cathetus, to fix near-exact test failures on some 32-bit sys‐
10257 tems used by downstream packagers.
10258 3.55.0 - 2018-04-05
10260 This release includes several improvements to the handling of the
10261 database setting.
10262 • The database_file setting was a historical artefact, and you should
10264 just use database directly.
10265 • The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
10267 favor of load_profile() and the database setting.
10268 • If you have not configured the example database at all and the de‐
10270 fault location is not usable (due to e.g. permissions issues), Hy‐
10271 pothesis will fall back to an in-memory database. This is not per‐
10272 sisted between sessions, but means that the defaults work on
10273 read-only filesystems.
10274 3.54.0 - 2018-04-04
10276 This release improves the complex_numbers() strategy, which now sup‐
10277 ports min_magnitude and max_magnitude arguments, along with allow_nan
10278 and allow_infinity like for floats().
10279 Thanks to J.J. Green for this feature.
10281 3.53.0 - 2018-04-01
10283 This release removes support for Django 1.8, which reached end of life
10284 on 2018-04-01. You can see Django's release and support schedule on
10285 the Django Project website.
10286 3.52.3 - 2018-04-01
10288 This patch fixes the min_satisfying_examples settings documentation, by
10289 explaining that example shrinking is tracked at the level of the under‐
10290 lying bytestream rather than the output value.
10291 The output from find() in verbose mode has also been adjusted - see the
10293 example session - to avoid duplicating lines when the example repr is
10294 constant, even if the underlying representation has been shrunken.
10295 3.52.2 - 2018-03-30
10297 This release improves the output of failures with rule based stateful
10298 testing in two ways:
10299 • The output from it is now usually valid Python code.
10301 • When the same value has two different names because it belongs to two
10303 different bundles, it will now display with the name associated with
10304 the correct bundle for a rule argument where it is used.
10305 3.52.1 - 2018-03-29
10307 This release improves the behaviour of stateful testing in two ways:
10308 • Previously some runs would run no steps (issue #376). This should no
10310 longer happen.
10311 • RuleBasedStateMachine tests which used bundles extensively would of‐
10313 ten shrink terribly. This should now be significantly improved,
10314 though there is likely a lot more room for improvement.
10315 This release also involves a low level change to how ranges of integers
10317 are handles which may result in other improvements to shrink quality in
10318 some cases.
10319 3.52.0 - 2018-03-24
10321 This release deprecates use of @settings(...) as a decorator, on func‐
10322 tions or methods that are not also decorated with @given. You can
10323 still apply these decorators in any order, though you should only do so
10324 once each.
10325 Applying @given twice was already deprecated, and applying
10327 @settings(...) twice is deprecated in this release and will become an
10328 error in a future version. Neither could ever be used twice to good ef‐
10329 fect.
10330 Using @settings(...) as the sole decorator on a test is completely
10332 pointless, so this common usage error will become an error in a future
10333 version of Hypothesis.
10334 3.51.0 - 2018-03-24
10336 This release deprecates the average_size argument to lists() and other
10337 collection strategies. You should simply delete it wherever it was
10338 used in your tests, as it no longer has any effect.
10339 In early versions of Hypothesis, the average_size argument was treated
10341 as a hint about the distribution of examples from a strategy. Subse‐
10342 quent improvements to the conceptual model and the engine for generat‐
10343 ing and shrinking examples mean it is more effective to simply describe
10344 what constitutes a valid example, and let our internals handle the dis‐
10345 tribution.
10346 3.50.3 - 2018-03-24
10348 This patch contains some internal refactoring so that we can run with
10349 warnings as errors in CI.
10350 3.50.2 - 2018-03-20
10352 This has no user-visible changes except one slight formatting change to
10353 one docstring, to avoid a deprecation warning.
10354 3.50.1 - 2018-03-20
10356 This patch fixes an internal error introduced in 3.48.0, where a check
10357 for the Django test runner would expose import-time errors in Django
10358 configuration (issue #1167).
10359 3.50.0 - 2018-03-19
10361 This release improves validation of numeric bounds for some strategies.
10362 • integers() and floats() now raise InvalidArgument if passed a
10364 min_value or max_value which is not an instance of Real, instead of
10365 various internal errors.
10366 • floats() now converts its bounding values to the nearest float above
10368 or below the min or max bound respectively, instead of just casting
10369 to float. The old behaviour was incorrect in that you could generate
10370 float(min_value), even when this was less than min_value itself (pos‐
10371 sible with eg. fractions).
10372 • When both bounds are provided to floats() but there are no floats in
10374 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
10375 raised.
10376 • decimals() gives a more useful error message if passed a string that
10378 cannot be converted to Decimal in a context where this error is not
10379 trapped.
10380 Code that previously seemed to work may be explicitly broken if there
10382 were no floats between min_value and max_value (only possible with
10383 non-float bounds), or if a bound was not a Real number but still al‐
10384 lowed in python:math.isnan (some custom classes with a __float__
10385 method).
10386 3.49.1 - 2018-03-15
10388 This patch fixes our tests for Numpy dtype strategies on big-endian
10389 platforms, where the strategy behaved correctly but the test assumed
10390 that the native byte order was little-endian.
10391 There is no user impact unless you are running our test suite on
10393 big-endian platforms. Thanks to Graham Inggs for reporting issue
10394 #1164.
10395 3.49.0 - 2018-03-12
10397 This release deprecates passing elements=None to collection strategies,
10398 such as lists().
10399 Requiring lists(nothing()) or builds(list) instead of lists() means
10401 slightly more typing, but also improves the consistency and discover‐
10402 ability of our API - as well as showing how to compose or construct
10403 strategies in ways that still work in more complex situations.
10404 Passing a nonzero max_size to a collection strategy where the elements
10406 strategy contains no values is now deprecated, and will be an error in
10407 a future version. The equivalent with elements=None is already an er‐
10408 ror.
10409 3.48.1 - 2018-03-05
10411 This patch will minimize examples that would come out non-minimal in
10412 previous versions. Thanks to Kyle Reeve for this patch.
10413 3.48.0 - 2018-03-05
10415 This release improves some "unhappy paths" when using Hypothesis with
10416 the standard library python:unittest module:
10417 • Applying @given to a non-test method which is overridden from
10419 python:unittest.TestCase, such as setUp, raises a new health check.
10420 (issue #991)
10421 • Using subTest() within a test decorated with @given would leak inter‐
10423 mediate results when tests were run under the python:unittest test
10424 runner. Individual reporting of failing subtests is now disabled
10425 during a test using @given. (issue #1071)
10426 • @given is still not a class decorator, but the error message if you
10428 try using it on a class has been improved.
10429 As a related improvement, using django:django.test.TestCase with @given
10431 instead of hypothesis.extra.django.TestCase raises an explicit error
10432 instead of running all examples in a single database transaction.
10433 3.47.0 - 2018-03-02
10435 register_profile now accepts keyword arguments for specific settings,
10436 and the parent settings object is now optional. Using a name for a
10437 registered profile which is not a string was never suggested, but it is
10438 now also deprecated and will eventually be an error.
10439 3.46.2 - 2018-03-01
10441 This release removes an unnecessary branch from the code, and has no
10442 user-visible impact.
10443 3.46.1 - 2018-03-01
10445 This changes only the formatting of our docstrings and should have no
10446 user-visible effects.
10447 3.46.0 - 2018-02-26
10449 characters() has improved docs about what arguments are valid, and ad‐
10450 ditional validation logic to raise a clear error early (instead of e.g.
10451 silently ignoring a bad argument). Categories may be specified as the
10452 Unicode 'general category' (eg u'Nd'), or as the 'major category' (eg
10453 [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
10454 In previous versions, general categories were supported and all other
10456 input was silently ignored. Now, major categories are supported in ad‐
10457 dition to general categories (which may change the behaviour of some
10458 existing code), and all other input is deprecated.
10459 3.45.5 - 2018-02-26
10461 This patch improves strategy inference in hypothesis.extra.django to
10462 account for some validators in addition to field type - see issue #1116
10463 for ongoing work in this space.
10464 Specifically, if a CharField or TextField has an attached RegexValida‐
10466 tor, we now use from_regex() instead of text() as the underlying strat‐
10467 egy. This allows us to generate examples of the default User model,
10468 closing issue #1112.
10469 3.45.4 - 2018-02-25
10471 This patch improves some internal debugging information, fixes a typo
10472 in a validation error message, and expands the documentation for new
10473 contributors.
10474 3.45.3 - 2018-02-23
10476 This patch may improve example shrinking slightly for some strategies.
10477 3.45.2 - 2018-02-18
10479 This release makes our docstring style more consistent, thanks to
10480 flake8-docstrings. There are no user-visible changes.
10481 3.45.1 - 2018-02-17
10483 This fixes an indentation issue in docstrings for datetimes(), dates(),
10484 times(), and timedeltas().
10485 3.45.0 - 2018-02-13
10487 This release fixes builds() so that target can be used as a keyword ar‐
10488 gument for passing values to the target. The target itself can still be
10489 specified as a keyword argument, but that behavior is now deprecated.
10490 The target should be provided as the first positional argument.
10491 3.44.26 - 2018-02-06
10493 This release fixes some formatting issues in the Hypothesis source
10494 code. It should have no externally visible effects.
10495 3.44.25 - 2018-02-05
10497 This release changes the way in which Hypothesis tries to shrink the
10498 size of examples. It probably won't have much impact, but might make
10499 shrinking faster in some cases. It is unlikely but not impossible that
10500 it will change the resulting examples.
10501 3.44.24 - 2018-01-27
10503 This release fixes dependency information when installing Hypothesis
10504 from a binary "wheel" distribution.
10505 • The install_requires for enum34 is resolved at install time, rather
10507 than at build time (with potentially different results).
10508 • Django has fixed their python_requires for versions 2.0.0 onward,
10510 simplifying Python2-compatible constraints for downstream projects.
10511 3.44.23 - 2018-01-24
10513 This release improves shrinking in a class of pathological examples
10514 that you are probably never hitting in practice. If you are hitting
10515 them in practice this should be a significant speed up in shrinking. If
10516 you are not, you are very unlikely to notice any difference. You might
10517 see a slight slow down and/or slightly better falsifying examples.
10518 3.44.22 - 2018-01-23
10520 This release fixes a dependency problem. It was possible to install
10521 Hypothesis with an old version of attrs, which would throw a TypeError
10522 as soon as you tried to import hypothesis. Specifically, you need at‐
10523 trs 16.0.0 or newer.
10524 Hypothesis will now require the correct version of attrs when in‐
10526 stalling.
10527 3.44.21 - 2018-01-22
10529 This change adds some additional structural information that Hypothesis
10530 will use to guide its search.
10531 You mostly shouldn't see much difference from this. The two most likely
10533 effects you would notice are:
10534 1. Hypothesis stores slightly more examples in its database for passing
10536 tests.
10537 2. Hypothesis may find new bugs that it was previously missing, but it
10539 probably won't (this is a basic implementation of the feature that
10540 is intended to support future work. Although it is useful on its
10541 own, it's not very useful on its own).
10542 3.44.20 - 2018-01-21
10544 This is a small refactoring release that changes how Hypothesis tracks
10545 some information about the boundary of examples in its internal repre‐
10546 sentation.
10547 You are unlikely to see much difference in behaviour, but memory usage
10549 and run time may both go down slightly during normal test execution,
10550 and when failing Hypothesis might print its failing example slightly
10551 sooner.
10552 3.44.19 - 2018-01-21
10554 This changes how we compute the default average_size for all collection
10555 strategies. Previously setting a max_size without setting an aver‐
10556 age_size would have the seemingly paradoxical effect of making data
10557 generation slower, because it would raise the average size from its de‐
10558 fault. Now setting max_size will either leave the default unchanged or
10559 lower it from its default.
10560 If you are currently experiencing this problem, this may make your
10562 tests substantially faster. If you are not, this will likely have no
10563 effect on you.
10564 3.44.18 - 2018-01-20
10566 This is a small refactoring release that changes how Hypothesis detects
10567 when the structure of data generation depends on earlier values gener‐
10568 ated (e.g. when using flatmap or composite()). It should not have any
10569 observable effect on behaviour.
10570 3.44.17 - 2018-01-15
10572 This release fixes a typo in internal documentation, and has no
10573 user-visible impact.
10574 3.44.16 - 2018-01-13
10576 This release improves test case reduction for recursive data struc‐
10577 tures. Hypothesis now guarantees that whenever a strategy calls itself
10578 recursively (usually this will happen because you are using
10579 deferred()), any recursive call may replace the top level value. e.g.
10580 given a tree structure, Hypothesis will always try replacing it with a
10581 subtree.
10582 Additionally this introduces a new heuristic that may in some circum‐
10584 stances significantly speed up test case reduction - Hypothesis should
10585 be better at immediately replacing elements drawn inside another strat‐
10586 egy with their minimal possible value.
10587 3.44.15 - 2018-01-13
10589 from_type() can now resolve recursive types such as binary trees (issue
10590 #1004). Detection of non-type arguments has also improved, leading to
10591 better error messages in many cases involving forward references.
10592 3.44.14 - 2018-01-08
10594 This release fixes a bug in the shrinker that prevented the optimisa‐
10595 tions in 3.44.6 from working in some cases. It would not have worked
10596 correctly when filtered examples were nested (e.g. with a set of inte‐
10597 gers in some range).
10598 This would not have resulted in any correctness problems, but shrinking
10600 may have been slower than it otherwise could be.
10601 3.44.13 - 2018-01-08
10603 This release changes the average bit length of values drawn from
10604 integers() to be much smaller. Additionally it changes the shrinking
10605 order so that now size is considered before sign - e.g. -1 will be
10606 preferred to +10.
10607 The new internal format for integers required some changes to the mini‐
10609 mizer to make work well, so you may also see some improvements to exam‐
10610 ple quality in unrelated areas.
10611 3.44.12 - 2018-01-07
10613 This changes Hypothesis's internal implementation of weighted sampling.
10614 This will affect example distribution and quality, but you shouldn't
10615 see any other effects.
10616 3.44.11 - 2018-01-06
10618 This is a change to some internals around how Hypothesis handles avoid‐
10619 ing generating duplicate examples and seeking out novel regions of the
10620 search space.
10621 You are unlikely to see much difference as a result of it, but it fixes
10623 a bug where an internal assertion could theoretically be triggered and
10624 has some minor effects on the distribution of examples so could poten‐
10625 tially find bugs that have previously been missed.
10626 3.44.10 - 2018-01-06
10628 This patch avoids creating debug statements when debugging is disabled.
10629 Profiling suggests this is a 5-10% performance improvement (issue
10630 #1040).
10631 3.44.9 - 2018-01-06
10633 This patch blacklists null characters ('\x00') in automatically created
10634 strategies for Django CharField and TextField, due to a database issue
10635 which was recently fixed upstream (Hypothesis issue #1045).
10636 3.44.8 - 2018-01-06
10638 This release makes the Hypothesis shrinker slightly less greedy in or‐
10639 der to avoid local minima - when it gets stuck, it makes a small at‐
10640 tempt to search around the final example it would previously have re‐
10641 turned to find a new starting point to shrink from. This should improve
10642 example quality in some cases, especially ones where the test data has
10643 dependencies among parts of it that make it difficult for Hypothesis to
10644 proceed.
10645 3.44.7 - 2018-01-04
10647 This release adds support for Django 2 in the hypothesis-django extra.
10648 This release drops support for Django 1.10, as it is no longer sup‐
10650 ported by the Django team.
10651 3.44.6 - 2018-01-02
10653 This release speeds up test case reduction in many examples by being
10654 better at detecting large shrinks it can use to discard redundant parts
10655 of its input. This will be particularly noticeable in examples that
10656 make use of filtering and for some integer ranges.
10657 3.44.5 - 2018-01-02
10659 Happy new year!
10660 This is a no-op release that updates the year range on all of the copy‐
10662 right headers in our source to include 2018.
10663 3.44.4 - 2017-12-23
10665 This release fixes issue #1041, which slowed tests by up to 6% due to
10666 broken caching.
10667 3.44.3 - 2017-12-21
10669 This release improves the shrinker in cases where examples drawn ear‐
10670 lier can affect how much data is drawn later (e.g. when you draw a
10671 length parameter in a composite and then draw that many elements). Ex‐
10672 amples found in cases like this should now be much closer to minimal.
10673 3.44.2 - 2017-12-20
10675 This is a pure refactoring release which changes how Hypothesis manages
10676 its set of examples internally. It should have no externally visible
10677 effects.
10678 3.44.1 - 2017-12-18
10680 This release fixes issue #997, in which under some circumstances the
10681 body of tests run under Hypothesis would not show up when run under
10682 coverage even though the tests were run and the code they called out‐
10683 side of the test file would show up normally.
10684 3.44.0 - 2017-12-17
10686 This release adds a new feature: The @reproduce_failure decorator, de‐
10687 signed to make it easy to use Hypothesis's binary format for examples
10688 to reproduce a problem locally without having to share your example
10689 database between machines.
10690 This also changes when seeds are printed:
10692 • They will no longer be printed for normal falsifying examples, as
10694 there are now adequate ways of reproducing those for all cases, so it
10695 just contributes noise.
10696 • They will once again be printed when reusing examples from the data‐
10698 base, as health check failures should now be more reliable in this
10699 scenario so it will almost always work in this case.
10700 This work was funded by Smarkets.
10702 3.43.1 - 2017-12-17
10704 This release fixes a bug with Hypothesis's database management - exam‐
10705 ples that were found in the course of shrinking were saved in a way
10706 that indicated that they had distinct causes, and so they would all be
10707 retried on the start of the next test. The intended behaviour, which is
10708 now what is implemented, is that only a bounded subset of these exam‐
10709 ples would be retried.
10710 3.43.0 - 2017-12-17
10712 HypothesisDeprecationWarning now inherits from python:FutureWarning in‐
10713 stead of python:DeprecationWarning, as recommended by PEP 565 for
10714 user-facing warnings (issue #618). If you have not changed the default
10715 warnings settings, you will now see each distinct
10716 HypothesisDeprecationWarning instead of only the first.
10717 3.42.2 - 2017-12-12
10719 This patch fixes issue #1017, where instances of a list or tuple sub‐
10720 type used as an argument to a strategy would be coerced to tuple.
10721 3.42.1 - 2017-12-10
10723 This release has some internal cleanup, which makes reading the code
10724 more pleasant and may shrink large examples slightly faster.
10725 3.42.0 - 2017-12-09
10727 This release deprecates faker-extra, which was designed as a transition
10728 strategy but does not support example shrinking or coverage-guided dis‐
10729 covery.
10730 3.41.0 - 2017-12-06
10732 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
10733 pling from multi-dimensional ndarrays still results in a deprecation
10734 warning. Thanks to Charlie Tanksley for this patch.
10735 3.40.1 - 2017-12-04
10737 This release makes two changes:
10738 • It makes the calculation of some of the metadata that Hypothesis uses
10740 for shrinking occur lazily. This should speed up performance of test
10741 case generation a bit because it no longer calculates information it
10742 doesn't need.
10743 • It improves the shrinker for certain classes of nested examples. e.g.
10745 when shrinking lists of lists, the shrinker is now able to concate‐
10746 nate two adjacent lists together into a single list. As a result of
10747 this change, shrinking may get somewhat slower when the minimal exam‐
10748 ple found is large.
10749 3.40.0 - 2017-12-02
10751 This release improves how various ways of seeding Hypothesis interact
10752 with the example database:
10753 • Using the example database with seed() is now deprecated. You should
10755 set database=None if you are doing that. This will only warn if you
10756 actually load examples from the database while using @seed.
10757 • The derandomize will behave the same way as @seed.
10759 • Using --hypothesis-seed will disable use of the database.
10761 • If a test used examples from the database, it will not suggest using
10763 a seed to reproduce it, because that won't work.
10764 This work was funded by Smarkets.
10766 3.39.0 - 2017-12-01
10768 This release adds a new health check that checks if the smallest "natu‐
10769 ral" possible example of your test case is very large - this will tend
10770 to cause Hypothesis to generate bad examples and be quite slow.
10771 This work was funded by Smarkets.
10773 3.38.9 - 2017-11-29
10775 This is a documentation release to improve the documentation of shrink‐
10776 ing behaviour for Hypothesis's strategies.
10777 3.38.8 - 2017-11-29
10779 This release improves the performance of characters() when using black‐
10780 list_characters and from_regex() when using negative character classes.
10781 The problems this fixes were found in the course of work funded by
10783 Smarkets.
10784 3.38.7 - 2017-11-29
10786 This is a patch release for from_regex(), which had a bug in handling
10787 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
10788 rectly when parsing regex.
10789 3.38.6 - 2017-11-28
10791 This patch changes a few byte-string literals from double to single
10792 quotes, thanks to an update in unify. There are no user-visible
10793 changes.
10794 3.38.5 - 2017-11-23
10796 This fixes the repr of strategies using lambda that are defined inside
10797 decorators to include the lambda source.
10798 This would mostly have been visible when using the statistics function‐
10800 ality - lambdas used for e.g. filtering would have shown up with a <un‐
10801 known> as their body. This can still happen, but it should happen less
10802 often now.
10803 3.38.4 - 2017-11-22
10805 This release updates the reported statistics so that they show approxi‐
10806 mately what fraction of your test run time is spent in data generation
10807 (as opposed to test execution).
10808 This work was funded by Smarkets.
10810 3.38.3 - 2017-11-21
10812 This is a documentation release, which ensures code examples are up to
10813 date by running them as doctests in CI (issue #711).
10814 3.38.2 - 2017-11-21
10816 This release changes the behaviour of the deadline setting when used
10817 with data(): Time spent inside calls to data.draw will no longer be
10818 counted towards the deadline time.
10819 As a side effect of some refactoring required for this work, the way
10821 flaky tests are handled has changed slightly. You are unlikely to see
10822 much difference from this, but some error messages will have changed.
10823 This work was funded by Smarkets.
10825 3.38.1 - 2017-11-21
10827 This patch has a variety of non-user-visible refactorings, removing
10828 various minor warts ranging from indirect imports to typos in comments.
10829 3.38.0 - 2017-11-18
10831 This release overhauls the health check system in a variety of small
10832 ways. It adds no new features, but is nevertheless a minor release be‐
10833 cause it changes which tests are likely to fail health checks.
10834 The most noticeable effect is that some tests that used to fail health
10836 checks will now pass, and some that used to pass will fail. These
10837 should all be improvements in accuracy. In particular:
10838 • New failures will usually be because they are now taking into account
10840 things like use of data() and assume() inside the test body.
10841 • New failures may also be because for some classes of example the way
10843 data generation performance was measured was artificially faster than
10844 real data generation (for most examples that are hitting performance
10845 health checks the opposite should be the case).
10846 • Tests that used to fail health checks and now pass do so because the
10848 health check system used to run in a way that was subtly different
10849 than the main Hypothesis data generation and lacked some of its sup‐
10850 port for e.g. large examples.
10851 If your data generation is especially slow, you may also see your tests
10853 get somewhat faster, as there is no longer a separate health check
10854 phase. This will be particularly noticeable when rerunning test fail‐
10855 ures.
10856 This work was funded by Smarkets.
10858 3.37.0 - 2017-11-12
10860 This is a deprecation release for some health check related features.
10861 The following are now deprecated:
10863 • Passing HealthCheck.exception_in_generation to suppress_health_check.
10865 This no longer does anything even when passed - All errors that oc‐
10866 cur during data generation will now be immediately reraised rather
10867 than going through the health check mechanism.
10868 • Passing HealthCheck.random_module to suppress_health_check. This
10870 hasn't done anything for a long time, but was never explicitly depre‐
10871 cated. Hypothesis always seeds the random module when running @given
10872 tests, so this is no longer an error and suppressing it doesn't do
10873 anything.
10874 • Passing non-HealthCheck values in suppress_health_check. This was
10876 previously allowed but never did anything useful.
10877 In addition, passing a non-iterable value as suppress_health_check will
10879 now raise an error immediately (it would never have worked correctly,
10880 but it would previously have failed later). Some validation error mes‐
10881 sages have also been updated.
10882 This work was funded by Smarkets.
10884 3.36.1 - 2017-11-10
10886 This is a yak shaving release, mostly concerned with our own tests.
10887 While getfullargspec() was documented as deprecated in Python 3.5, it
10889 never actually emitted a warning. Our code to silence this (nonexis‐
10890 tent) warning has therefore been removed.
10891 We now run our tests with DeprecationWarning as an error, and made some
10893 minor changes to our own tests as a result. This required similar up‐
10894 stream updates to coverage and execnet (a test-time dependency via
10895 pytest-xdist).
10896 There is no user-visible change in Hypothesis itself, but we encourage
10898 you to consider enabling deprecations as errors in your own tests.
10899 3.36.0 - 2017-11-06
10901 This release adds a setting to the public API, and does some internal
10902 cleanup:
10903 • The derandomize setting is now documented (issue #890)
10905 • Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
10907 #953)
10908 • Documented the strict setting as deprecated, and updated the build so
10910 our docs always match deprecations in the code.
10911 3.35.0 - 2017-11-06
10913 This minor release supports constraining uuids() to generate a particu‐
10914 lar version of UUID (issue #721).
10915 Thanks to Dion Misic for this feature.
10917 3.34.1 - 2017-11-02
10919 This patch updates the documentation to suggest builds(callable) in‐
10920 stead of just(callable()).
10921 3.34.0 - 2017-11-02
10923 Hypothesis now emits deprecation warnings if you apply @given more than
10924 once to a target.
10925 Applying @given repeatedly wraps the target multiple times. Each wrap‐
10927 per will search the space of of possible parameters separately. This
10928 is equivalent but will be much more inefficient than doing it with a
10929 single call to @given.
10930 For example, instead of @given(booleans()) @given(integers()), you
10932 could write @given(booleans(), integers())
10933 3.33.1 - 2017-11-02
10935 This is a bugfix release:
10936 • builds() would try to infer a strategy for required positional argu‐
10938 ments of the target from type hints, even if they had been given to
10939 builds() as positional arguments (issue #946). Now it only infers
10940 missing required arguments.
10941 • An internal introspection function wrongly reported self as a re‐
10943 quired argument for bound methods, which might also have affected
10944 builds(). Now it knows better.
10945 3.33.0 - 2017-10-16
10947 This release supports strategy inference for more Django field types -
10948 you can now omit an argument for Date, Time, Duration, Slug, IP Ad‐
10949 dress, and UUID fields. (issue #642)
10950 Strategy generation for fields with grouped choices now selects choices
10952 from each group, instead of selecting from the group names.
10953 3.32.2 - 2017-10-15
10955 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
10956 an experimental basis. It has never actually worked, and the new
10957 Hypothesis example database is designed to make such a tool unneces‐
10958 sary.
10959 3.32.1 - 2017-10-13
10961 This patch has two improvements for strategies based on enumerations.
10962 • from_type() now handles enumerations correctly, delegating to
10964 sampled_from(). Previously it noted that Enum.__init__ has no re‐
10965 quired arguments and therefore delegated to builds(), which would
10966 subsequently fail.
10967 • When sampling from an python:enum.Flag, we also generate combinations
10969 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
10970 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
10971 3.32.0 - 2017-10-09
10973 This changes the default value of the use_coverage setting to True when
10974 running on pypy (it was already True on CPython).
10975 It was previously set to False because we expected it to be too slow,
10977 but recent benchmarking shows that actually performance of the feature
10978 on pypy is fairly acceptable - sometimes it's slower than on CPython,
10979 sometimes it's faster, but it's generally within a factor of two either
10980 way.
10981 3.31.6 - 2017-10-08
10983 This patch improves the quality of strategies inferred from Numpy
10984 dtypes:
10985 • Integer dtypes generated examples with the upper half of their
10987 (non-sign) bits set to zero. The inferred strategies can now produce
10988 any representable integer.
10989 • Fixed-width unicode- and byte-string dtypes now cap the internal ex‐
10991 ample length, which should improve example and shrink quality.
10992 • Numpy arrays can only store fixed-size strings internally, and allow
10994 shorter strings by right-padding them with null bytes. Inferred
10995 string strategies no longer generate such values, as they can never
10996 be retrieved from an array. This improves shrinking performance by
10997 skipping useless values.
10998 This has already been useful in Hypothesis - we found an overflow bug
11000 in our Pandas support, and as a result indexes() and range_indexes()
11001 now check that min_size and max_size are at least zero.
11002 3.31.5 - 2017-10-08
11004 This release fixes a performance problem in tests where the use_cover‐
11005 age setting is True.
11006 Tests experience a slow-down proportionate to the amount of code they
11008 cover. This is still the case, but the factor is now low enough that
11009 it should be unnoticeable. Previously it was large and became much
11010 larger in 3.30.4.
11011 3.31.4 - 2017-10-08
11013 from_type() failed with a very confusing error if passed a NewType (‐
11014 issue #901). These pseudo-types are now unwrapped correctly, and
11015 strategy inference works as expected.
11016 3.31.3 - 2017-10-06
11018 This release makes some small optimisations to our use of coverage that
11019 should reduce constant per-example overhead. This is probably only no‐
11020 ticeable on examples where the test itself is quite fast. On no-op
11021 tests that don't test anything you may see up to a fourfold speed in‐
11022 crease (which is still significantly slower than without coverage). On
11023 more realistic tests the speed up is likely to be less than that.
11024 3.31.2 - 2017-09-30
11026 This release fixes some formatting and small typos/grammar issues in
11027 the documentation, specifically the page docs/settings.rst, and the in‐
11028 line docs for the various settings.
11029 3.31.1 - 2017-09-30
11031 This release improves the handling of deadlines so that they act better
11032 with the shrinking process. This fixes issue #892.
11033 This involves two changes:
11035 1. The deadline is raised during the initial generation and shrinking,
11037 and then lowered to the set value for final replay. This restricts
11038 our attention to examples which exceed the deadline by a more sig‐
11039 nificant margin, which increases their reliability.
11040 2. When despite the above a test still becomes flaky because it is sig‐
11042 nificantly faster on rerun than it was on its first run, the error
11043 message is now more explicit about the nature of this problem, and
11044 includes both the initial test run time and the new test run time.
11045 In addition, this release also clarifies the documentation of the dead‐
11047 line setting slightly to be more explicit about where it applies.
11048 This work was funded by Smarkets.
11050 3.31.0 - 2017-09-29
11052 This release blocks installation of Hypothesis on Python 3.3, which
11053 reached its end of life date on 2017-09-29.
11054 This should not be of interest to anyone but downstream maintainers -
11056 if you are affected, migrate to a secure version of Python as soon as
11057 possible or at least seek commercial support.
11058 3.30.4 - 2017-09-27
11060 This release makes several changes:
11061 1. It significantly improves Hypothesis's ability to use coverage in‐
11063 formation to find interesting examples.
11064 2. It reduces the default max_examples setting from 200 to 100. This
11066 takes advantage of the improved algorithm meaning fewer examples are
11067 typically needed to get the same testing and is sufficiently better
11068 at covering interesting behaviour, and offsets some of the perfor‐
11069 mance problems of running under coverage.
11070 3. Hypothesis will always try to start its testing with an example that
11072 is near minimized.
11073 The new algorithm for 1 also makes some changes to Hypothesis's low
11075 level data generation which apply even with coverage turned off. They
11076 generally reduce the total amount of data generated, which should im‐
11077 prove test performance somewhat. Between this and 3 you should see a
11078 noticeable reduction in test runtime (how much so depends on your tests
11079 and how much example size affects their performance. On our benchmarks,
11080 where data generation dominates, we saw up to a factor of two perfor‐
11081 mance improvement, but it's unlikely to be that large.
11082 3.30.3 - 2017-09-25
11084 This release fixes some formatting and small typos/grammar issues in
11085 the documentation, specifically the page docs/details.rst, and some in‐
11086 line docs linked from there.
11087 3.30.2 - 2017-09-24
11089 This release changes Hypothesis's caching approach for functions in hy‐
11090 pothesis.strategies. Previously it would have cached extremely aggres‐
11091 sively and cache entries would never be evicted. Now it adopts a
11092 least-frequently used, least recently used key invalidation policy, and
11093 is somewhat more conservative about which strategies it caches.
11094 Workloads which create strategies based on dynamic values, e.g. by us‐
11096 ing flatmap or composite(), will use significantly less memory.
11097 3.30.1 - 2017-09-22
11099 This release fixes a bug where when running with the use_coverage=True
11100 setting inside an existing running instance of coverage, Hypothesis
11101 would frequently put files that the coveragerc excluded in the report
11102 for the enclosing coverage.
11103 3.30.0 - 2017-09-20
11105 This release introduces two new features:
11106 • When a test fails, either with a health check failure or a falsifying
11108 example, Hypothesis will print out a seed that led to that failure,
11109 if the test is not already running with a fixed seed. You can then
11110 recreate that failure using either the @seed decorator or (if you are
11111 running pytest) with --hypothesis-seed.
11112 • pytest users can specify a seed to use for @given based tests by
11114 passing the --hypothesis-seed command line argument.
11115 This work was funded by Smarkets.
11117 3.29.0 - 2017-09-19
11119 This release makes Hypothesis coverage aware. Hypothesis now runs all
11120 test bodies under coverage, and uses this information to guide its
11121 testing.
11122 The use_coverage setting can be used to disable this behaviour if you
11124 want to test code that is sensitive to coverage being enabled (either
11125 because of performance or interaction with the trace function).
11126 The main benefits of this feature are:
11128 • Hypothesis now observes when examples it discovers cover particular
11130 lines or branches and stores them in the database for later.
11131 • Hypothesis will make some use of this information to guide its explo‐
11133 ration of the search space and improve the examples it finds (this is
11134 currently used only very lightly and will likely improve signifi‐
11135 cantly in future releases).
11136 This also has the following side-effects:
11138 • Hypothesis now has an install time dependency on the coverage pack‐
11140 age.
11141 • Tests that are already running Hypothesis under coverage will likely
11143 get faster.
11144 • Tests that are not running under coverage now run their test bodies
11146 under coverage by default.
11147 This feature is only partially supported under pypy. It is signifi‐
11149 cantly slower than on CPython and is turned off by default as a result,
11150 but it should still work correctly if you want to use it.
11151 3.28.3 - 2017-09-18
11153 This release is an internal change that affects how Hypothesis handles
11154 calculating certain properties of strategies.
11155 The primary effect of this is that it fixes a bug where use of
11157 deferred() could sometimes trigger an internal assertion error. However
11158 the fix for this bug involved some moderately deep changes to how Hy‐
11159 pothesis handles certain constructs so you may notice some additional
11160 knock-on effects.
11161 In particular the way Hypothesis handles drawing data from strategies
11163 that cannot generate any values has changed to bail out sooner than it
11164 previously did. This may speed up certain tests, but it is unlikely to
11165 make much of a difference in practice for tests that were not already
11166 failing with Unsatisfiable.
11167 3.28.2 - 2017-09-18
11169 This is a patch release that fixes a bug in the hypothesis.extra.pandas
11170 documentation where it incorrectly referred to column() instead of
11171 columns().
11172 3.28.1 - 2017-09-16
11174 This is a refactoring release. It moves a number of internal uses of
11175 namedtuple() over to using attrs based classes, and removes a couple of
11176 internal namedtuple classes that were no longer in use.
11177 It should have no user visible impact.
11179 3.28.0 - 2017-09-15
11181 This release adds support for testing pandas via the
11182 hypothesis.extra.pandas module.
11183 It also adds a dependency on attrs.
11185 This work was funded by Stripe.
11187 3.27.1 - 2017-09-14
11189 This release fixes some formatting and broken cross-references in the
11190 documentation, which includes editing docstrings - and thus a patch re‐
11191 lease.
11192 3.27.0 - 2017-09-13
11194 This release introduces a deadline setting to Hypothesis.
11195 When set this turns slow tests into errors. By default it is unset but
11197 will warn if you exceed 200ms, which will become the default value in a
11198 future release.
11199 This work was funded by Smarkets.
11201 3.26.0 - 2017-09-12
11203 Hypothesis now emits deprecation warnings if you are using the legacy
11204 SQLite example database format, or the tool for merging them. These
11205 were already documented as deprecated, so this doesn't change their
11206 deprecation status, only that we warn about it.
11207 3.25.1 - 2017-09-12
11209 This release fixes a bug with generating numpy datetime and timedelta
11210 types: When inferring the strategy from the dtype, datetime and
11211 timedelta dtypes with sub-second precision would always produce exam‐
11212 ples with one second resolution. Inferring a strategy from a time
11213 dtype will now always produce example with the same precision.
11214 3.25.0 - 2017-09-12
11216 This release changes how Hypothesis shrinks and replays examples to
11217 take into account that it can encounter new bugs while shrinking the
11218 bug it originally found. Previously it would end up replacing the orig‐
11219 inally found bug with the new bug and show you only that one. Now it is
11220 (often) able to recognise when two bugs are distinct and when it finds
11221 more than one will show both.
11222 3.24.2 - 2017-09-11
11224 This release removes the (purely internal and no longer useful) strat‐
11225 egy_test_suite function and the corresponding strategytests module.
11226 3.24.1 - 2017-09-06
11228 This release improves the reduction of examples involving floating
11229 point numbers to produce more human readable examples.
11230 It also has some general purpose changes to the way the minimizer works
11232 internally, which may see some improvement in quality and slow down of
11233 test case reduction in cases that have nothing to do with floating
11234 point numbers.
11235 3.24.0 - 2017-09-05
11237 Hypothesis now emits deprecation warnings if you use some_strategy.ex‐
11238 ample() inside a test function or strategy definition (this was never
11239 intended to be supported, but is sufficiently widespread that it war‐
11240 rants a deprecation path).
11241 3.23.3 - 2017-09-05
11243 This is a bugfix release for decimals() with the places argument.
11244 • No longer fails health checks (issue #725, due to internal filtering)
11246 • Specifying a min_value and max_value without any decimals with places
11248 places between them gives a more useful error message.
11249 • Works for any valid arguments, regardless of the decimal precision
11251 context.
11252 3.23.2 - 2017-09-01
11254 This is a small refactoring release that removes a now-unused parameter
11255 to an internal API. It shouldn't have any user visible effect.
11256 3.23.1 - 2017-09-01
11258 Hypothesis no longer propagates the dynamic scope of settings into
11259 strategy definitions.
11260 This release is a small change to something that was never part of the
11262 public API and you will almost certainly not notice any effect unless
11263 you're doing something surprising, but for example the following code
11264 will now give a different answer in some circumstances:
11265 import hypothesis.strategies as st
11267 from hypothesis import settings
11268 CURRENT_SETTINGS = st.builds(lambda: settings.default)
11270 (We don't actually encourage you writing code like this)
11272 Previously this would have generated the settings that were in effect
11274 at the point of definition of CURRENT_SETTINGS. Now it will generate
11275 the settings that are used for the current test.
11276 It is very unlikely to be significant enough to be visible, but you may
11278 also notice a small performance improvement.
11279 3.23.0 - 2017-08-31
11281 This release adds a unique argument to arrays() which behaves the same
11282 ways as the corresponding one for lists(), requiring all of the ele‐
11283 ments in the generated array to be distinct.
11284 3.22.2 - 2017-08-29
11286 This release fixes an issue where Hypothesis would raise a TypeError
11287 when using the datetime-related strategies if running with PYTHONOPTI‐
11288 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
11289 3.22.1 - 2017-08-28
11291 Hypothesis now transparently handles problems with an internal unicode
11292 cache, including file truncation or read-only filesystems (issue #767).
11293 Thanks to Sam Hames for the patch.
11294 3.22.0 - 2017-08-26
11296 This release provides what should be a substantial performance improve‐
11297 ment to numpy arrays generated using provided numpy support, and adds a
11298 new fill_value argument to arrays() to control this behaviour.
11299 This work was funded by Stripe.
11301 3.21.3 - 2017-08-26
11303 This release fixes some extremely specific circumstances that probably
11304 have never occurred in the wild where users of deferred() might have
11305 seen a python:RuntimeError from too much recursion, usually in cases
11306 where no valid example could have been generated anyway.
11307 3.21.2 - 2017-08-25
11309 This release fixes some minor bugs in argument validation:
11310 • hypothesis.extra.numpy dtype strategies would raise an internal
11312 error instead of an InvalidArgument exception when passed an in‐
11313 valid endianness specification.
11314 • fractions() would raise an internal error instead of an InvalidAr‐
11316 gument if passed float("nan") as one of its bounds.
11317 • The error message for passing float("nan") as a bound to various
11319 strategies has been improved.
11320 • Various bound arguments will now raise InvalidArgument in cases
11322 where they would previously have raised an internal TypeError or
11323 ValueError from the relevant conversion function.
11324 • streaming() would not have emitted a deprecation warning when
11326 called with an invalid argument.
11327 3.21.1 - 2017-08-24
11329 This release fixes a bug where test failures that were the result of an
11330 @example would print an extra stack trace before re-raising the excep‐
11331 tion.
11332 3.21.0 - 2017-08-23
11334 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
11335 sis's deprecation warnings into errors. Similar functionality can be
11336 achieved by using simplefilter('error', HypothesisDeprecationWarning).
11337 3.20.0 - 2017-08-22
11339 This release renames the relevant arguments on the datetimes(),
11340 dates(), times(), and timedeltas() strategies to min_value and
11341 max_value, to make them consistent with the other strategies in the
11342 module.
11343 The old argument names are still supported but will emit a deprecation
11345 warning when used explicitly as keyword arguments. Arguments passed po‐
11346 sitionally will go to the new argument names and are not deprecated.
11347 3.19.3 - 2017-08-22
11349 This release provides a major overhaul to the internals of how Hypothe‐
11350 sis handles shrinking.
11351 This should mostly be visible in terms of getting better examples for
11353 tests which make heavy use of composite(), data() or flatmap where the
11354 data drawn depends a lot on previous choices, especially where size pa‐
11355 rameters are affected. Previously Hypothesis would have struggled to
11356 reliably produce good examples here. Now it should do much better. Per‐
11357 formance should also be better for examples with a non-zero min_size.
11358 You may see slight changes to example generation (e.g. improved example
11360 diversity) as a result of related changes to internals, but they are
11361 unlikely to be significant enough to notice.
11362 3.19.2 - 2017-08-21
11364 This release fixes two bugs in hypothesis.extra.numpy:
11365 • unicode_string_dtypes() didn't work at all due to an incorrect dtype
11367 specifier. Now it does.
11368 • Various impossible conditions would have been accepted but would er‐
11370 ror when they fail to produced any example. Now they raise an ex‐
11371 plicit InvalidArgument error.
11372 3.19.1 - 2017-08-21
11374 This is a bugfix release for issue #739, where bounds for fractions()
11375 or floating-point decimals() were not properly converted to integers
11376 before passing them to the integers strategy. This excluded some val‐
11377 ues that should have been possible, and could trigger internal errors
11378 if the bounds lay between adjacent integers.
11379 You can now bound fractions() with two arbitrarily close fractions.
11381 It is now an explicit error to supply a min_value, max_value, and
11383 max_denominator to fractions() where the value bounds do not include a
11384 fraction with denominator at most max_denominator.
11385 3.19.0 - 2017-08-20
11387 This release adds the from_regex() strategy, which generates strings
11388 that contain a match of a regular expression.
11389 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
11391 then helping to upstream it! (issue #662)
11392 3.18.5 - 2017-08-18
11394 This is a bugfix release for integers(). Previously the strategy would
11395 hit an internal assertion if passed non-integer bounds for min_value
11396 and max_value that had no integers between them. The strategy now
11397 raises InvalidArgument instead.
11398 3.18.4 - 2017-08-18
11400 Release to fix a bug where mocks can be used as test runners under cer‐
11401 tain conditions. Specifically, if a mock is injected into a test via
11402 pytest fixtures or patch decorators, and that mock is the first argu‐
11403 ment in the list, hypothesis will think it represents self and turns
11404 the mock into a test runner. If this happens, the affected test always
11405 passes because the mock is executed instead of the test body. Some‐
11406 times, it will also fail health checks.
11407 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
11409 for this bug fix.
11410 3.18.3 - 2017-08-17
11412 This release should improve the performance of some tests which experi‐
11413 enced a slow down as a result of the 3.13.0 release.
11414 Tests most likely to benefit from this are ones that make extensive use
11416 of min_size parameters, but others may see some improvement as well.
11417 3.18.2 - 2017-08-16
11419 This release fixes a bug introduced in 3.18.0. If the arguments
11420 whitelist_characters and blacklist_characters to characters() contained
11421 overlapping elements, then an InvalidArgument exception would be
11422 raised.
11423 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
11425 3.18.1 - 2017-08-14
11427 This is a bug fix release to fix issue #780, where sets() and similar
11428 would trigger health check errors if their element strategy could only
11429 produce one element (e.g. if it was just()).
11430 3.18.0 - 2017-08-13
11432 This is a feature release:
11433 • characters() now accepts whitelist_characters, particular characters
11435 which will be added to those it produces. (issue #668)
11436 • A bug fix for the internal function _union_interval_lists(), and a
11438 rename to _union_intervals(). It now correctly handles all cases
11439 where intervals overlap, and it always returns the result as a tuple
11440 for tuples.
11441 Thanks to Alex Willmer for these.
11443 3.17.0 - 2017-08-07
11445 This release documents the previously undocumented phases feature, mak‐
11446 ing it part of the public API. It also updates how the example database
11447 is used. Principally:
11448 • The reuse phase will now correctly control whether examples from the
11450 database are run (it previously did exactly the wrong thing and con‐
11451 trolled whether examples would be saved).
11452 • Hypothesis will no longer try to rerun all previously failing exam‐
11454 ples. Instead it will replay the smallest previously failing example
11455 and a selection of other examples that are likely to trigger any
11456 other bugs that will found. This prevents a previous failure from
11457 dominating your tests unnecessarily.
11458 • As a result of the previous change, Hypothesis will be slower about
11460 clearing out old examples from the database that are no longer fail‐
11461 ing (because it can only clear out ones that it actually runs).
11462 3.16.1 - 2017-08-07
11464 This release makes an implementation change to how Hypothesis handles
11465 certain internal constructs.
11466 The main effect you should see is improvement to the behaviour and per‐
11468 formance of collection types, especially ones with a min_size parame‐
11469 ter. Many cases that would previously fail due to being unable to gen‐
11470 erate enough valid examples will now succeed, and other cases should
11471 run slightly faster.
11472 3.16.0 - 2017-08-04
11474 This release introduces a deprecation of the timeout feature. This re‐
11475 sults in the following changes:
11476 • Creating a settings object with an explicit timeout will emit a dep‐
11478 recation warning.
11479 • If your test stops because it hits the timeout (and has not found a
11481 bug) then it will emit a deprecation warning.
11482 • There is a new value unlimited which you can import from hypothesis.
11484 settings(timeout=unlimited) will not cause a deprecation warning.
11485 • There is a new health check, hung_test, which will trigger after a
11487 test has been running for five minutes if it is not suppressed.
11488 3.15.0 - 2017-08-04
11490 This release deprecates two strategies, choices() and streaming().
11491 Both of these are somewhat confusing to use and are entirely redundant
11493 since the introduction of the data() strategy for interactive drawing
11494 in tests, and their use should be replaced with direct use of data()
11495 instead.
11496 3.14.2 - 2017-08-03
11498 This fixes a bug where Hypothesis would not work correctly on Python
11499 2.7 if you had the python:typing module backport installed.
11500 3.14.1 - 2017-08-02
11502 This raises the maximum depth at which Hypothesis starts cutting off
11503 data generation to a more reasonable value which it is harder to hit by
11504 accident.
11505 This resolves (issue #751), in which some examples which previously
11507 worked would start timing out, but it will also likely improve the data
11508 generation quality for complex data types.
11509 3.14.0 - 2017-07-23
11511 Hypothesis now understands inline type annotations (issue #293):
11512 • If the target of builds() has type annotations, a default strategy
11514 for missing required arguments is selected based on the type.
11515 Type-based strategy selection will only override a default if you
11516 pass hypothesis.infer as a keyword argument.
11517 • If @given wraps a function with type annotations, you can pass infer
11519 as a keyword argument and the appropriate strategy will be substi‐
11520 tuted.
11521 • You can check what strategy will be inferred for a type with the new
11523 from_type() function.
11524 • register_type_strategy() teaches Hypothesis which strategy to infer
11526 for custom or unknown types. You can provide a strategy, or for more
11527 complex cases a function which takes the type and returns a strategy.
11528 3.13.1 - 2017-07-20
11530 This is a bug fix release for issue #514 - Hypothesis would continue
11531 running examples after a SkipTest exception was raised, including
11532 printing a falsifying example. Skip exceptions from the standard
11533 python:unittest module, and pytest, nose, or unittest2 modules now
11534 abort the test immediately without printing output.
11535 3.13.0 - 2017-07-16
11537 This release has two major aspects to it: The first is the introduction
11538 of deferred(), which allows more natural definition of recursive (in‐
11539 cluding mutually recursive) strategies.
11540 The second is a number of engine changes designed to support this sort
11542 of strategy better. These should have a knock-on effect of also improv‐
11543 ing the performance of any existing strategies that currently generate
11544 a lot of data or involve heavy nesting by reducing their typical exam‐
11545 ple size.
11546 3.12.0 - 2017-07-07
11548 This release makes some major internal changes to how Hypothesis repre‐
11549 sents data internally, as a prelude to some major engine changes that
11550 should improve data quality. There are no API changes, but it's a sig‐
11551 nificant enough internal change that a minor version bump seemed war‐
11552 ranted.
11553 User facing impact should be fairly mild, but includes:
11555 • All existing examples in the database will probably be invalidated.
11557 Hypothesis handles this automatically, so you don't need to do any‐
11558 thing, but if you see all your examples disappear that's why.
11559 • Almost all data distributions have changed significantly. Possibly
11561 for the better, possibly for the worse. This may result in new bugs
11562 being found, but it may also result in Hypothesis being unable to
11563 find bugs it previously did.
11564 • Data generation may be somewhat faster if your existing bottleneck
11566 was in draw_bytes (which is often the case for large examples).
11567 • Shrinking will probably be slower, possibly significantly.
11569 If you notice any effects you consider to be a significant regression,
11571 please open an issue about them.
11572 3.11.6 - 2017-06-19
11574 This release involves no functionality changes, but is the first to
11575 ship wheels as well as an sdist.
11576 3.11.5 - 2017-06-18
11578 This release provides a performance improvement to shrinking. For cases
11579 where there is some non-trivial "boundary" value (e.g. the bug happens
11580 for all values greater than some other value), shrinking should now be
11581 substantially faster. Other types of bug will likely see improvements
11582 too.
11583 This may also result in some changes to the quality of the final exam‐
11585 ples - it may sometimes be better, but is more likely to get slightly
11586 worse in some edge cases. If you see any examples where this happens in
11587 practice, please report them.
11588 3.11.4 - 2017-06-17
11590 This is a bugfix release: Hypothesis now prints explicit examples when
11591 running in verbose mode. (issue #313)
11592 3.11.3 - 2017-06-11
11594 This is a bugfix release: Hypothesis no longer emits a warning if you
11595 try to use sampled_from() with python:collections.OrderedDict. (issue
11596 #688)
11597 3.11.2 - 2017-06-10
11599 This is a documentation release. Several outdated snippets have been
11600 updated or removed, and many cross-references are now hyperlinks.
11601 3.11.1 - 2017-05-28
11603 This is a minor ergonomics release. Tracebacks shown by pytest no
11604 longer include Hypothesis internals for test functions decorated with
11605 @given.
11606 3.11.0 - 2017-05-23
11608 This is a feature release, adding datetime-related strategies to the
11609 core strategies.
11610 timezones() allows you to sample pytz timezones from the Olsen data‐
11612 base. Use directly in a recipe for tz-aware datetimes, or compose with
11613 none() to allow a mix of aware and naive output.
11614 The new dates(), times(), datetimes(), and timedeltas() strategies are
11616 all constrained by objects of their type. This means that you can gen‐
11617 erate dates bounded by a single day (i.e. a single date), or datetimes
11618 constrained to the microsecond.
11619 times() and datetimes() take an optional timezones= argument, which de‐
11621 faults to none() for naive times. You can use our extra strategy based
11622 on pytz, or roll your own timezones strategy with dateutil or even the
11623 standard library.
11624 The old dates, times, and datetimes strategies in hypothesis.ex‐
11626 tra.datetimes are deprecated in favor of the new core strategies, which
11627 are more flexible and have no dependencies.
11628 3.10.0 - 2017-05-22
11630 Hypothesis now uses python:inspect.getfullargspec() internally. On
11631 Python 2, there are no visible changes.
11632 On Python 3 @given and @composite now preserve PEP 3107 annotations on
11634 the decorated function. Keyword-only arguments are now either handled
11635 correctly (e.g. @composite), or caught in validation instead of
11636 silently discarded or raising an unrelated error later (e.g. @given).
11637 3.9.1 - 2017-05-22
11639 This is a bugfix release: the default field mapping for a DateTimeField
11640 in the Django extra now respects the USE_TZ setting when choosing a
11641 strategy.
11642 3.9.0 - 2017-05-19
11644 This is feature release, expanding the capabilities of the decimals()
11645 strategy.
11646 • The new (optional) places argument allows you to generate decimals
11648 with a certain number of places (e.g. cents, thousandths, satoshis).
11649 • If allow_infinity is None, setting min_bound no longer excludes posi‐
11651 tive infinity and setting max_value no longer excludes negative in‐
11652 finity.
11653 • All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
11655 True, or if allow_nan is None and min_value or max_value is None.
11656 • min_value and max_value may be given as decimal strings, e.g.
11658 "1.234".
11659 3.8.5 - 2017-05-16
11661 Hypothesis now imports python:sqlite3 when a SQLite database is used,
11662 rather than at module load, improving compatibility with Python imple‐
11663 mentations compiled without SQLite support (such as BSD or Jython).
11664 3.8.4 - 2017-05-16
11666 This is a compatibility bugfix release. sampled_from() no longer
11667 raises a deprecation warning when sampling from an python:enum.Enum, as
11668 all enums have a reliable iteration order.
11669 3.8.3 - 2017-05-09
11671 This release removes a version check for older versions of pytest when
11672 using the Hypothesis pytest plugin. The pytest plugin will now run un‐
11673 conditionally on all versions of pytest. This breaks compatibility with
11674 any version of pytest prior to 2.7.0 (which is more than two years
11675 old).
11676 The primary reason for this change is that the version check was a fre‐
11678 quent source of breakage when pytest change their versioning scheme. If
11679 you are not working on pytest itself and are not running a very old
11680 version of it, this release probably doesn't affect you.
11681 3.8.2 - 2017-04-26
11683 This is a code reorganisation release that moves some internal test
11684 helpers out of the main source tree so as to not have changes to them
11685 trigger releases in future.
11686 3.8.1 - 2017-04-26
11688 This is a documentation release. Almost all code examples are now
11689 doctests checked in CI, eliminating stale examples.
11690 3.8.0 - 2017-04-23
11692 This is a feature release, adding the iterables() strategy, equivalent
11693 to lists(...).map(iter) but with a much more useful repr. You can use
11694 this strategy to check that code doesn't accidentally depend on se‐
11695 quence properties such as indexing support or repeated iteration.
11696 3.7.4 - 2017-04-22
11698 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
11699 ture in the same test could cause the test to fail to fill the argu‐
11700 ments, and throw a TypeError.
11701 3.7.3 - 2017-04-21
11703 This release should include no user visible changes and is purely a
11704 refactoring release. This modularises the behaviour of the core given()
11705 function, breaking it up into smaller and more accessible parts, but
11706 its actual behaviour should remain unchanged.
11707 3.7.2 - 2017-04-21
11709 This reverts an undocumented change in 3.7.1 which broke installation
11710 on debian stable: The specifier for the hypothesis[django] extra_re‐
11711 quires had introduced a wild card, which was not supported on the de‐
11712 fault version of pip.
11713 3.7.1 - 2017-04-21
11715 This is a bug fix and internal improvements release.
11716 • In particular Hypothesis now tracks a tree of where it has already
11718 explored. This allows it to avoid some classes of duplicate exam‐
11719 ples, and significantly improves the performance of shrinking failing
11720 examples by allowing it to skip some shrinks that it can determine
11721 can't possibly work.
11722 • Hypothesis will no longer seed the global random arbitrarily unless
11724 you have asked it to using random_module()
11725 • Shrinking would previously have not worked correctly in some special
11727 cases on Python 2, and would have resulted in suboptimal examples.
11728 3.7.0 - 2017-03-20
11730 This is a feature release.
11731 New features:
11733 • Rule based stateful testing now has an @invariant decorator that
11735 specifies methods that are run after init and after every step, al‐
11736 lowing you to encode properties that should be true at all times.
11737 Thanks to Tom Prince for this feature.
11738 • The decimals() strategy now supports allow_nan and allow_infinity
11740 flags.
11741 • There are significantly more strategies available for numpy, includ‐
11743 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
11744 for this feature.
11745 • When using the data() strategy you can now add a label as an argument
11747 to draw(), which will be printed along with the value when an example
11748 fails. Thanks to Peter Inglesby for this feature.
11749 Bug fixes:
11751 • Bug fix: composite() now preserves functions' docstrings.
11753 • The build is now reproducible and doesn't depend on the path you
11755 build it from. Thanks to Chris Lamb for this feature.
11756 • numpy strategies for the void data type did not work correctly.
11758 Thanks to Zac Hatfield Dodds for this fix.
11759 There have also been a number of performance optimizations:
11761 • The permutations() strategy is now significantly faster to use for
11763 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
11764 • Shrinking of failing test cases should have got significantly faster
11766 in some circumstances where it was previously struggling for a long
11767 time.
11768 • Example generation now involves less indirection, which results in a
11770 small speedup in some cases (small enough that you won't really no‐
11771 tice it except in pathological cases).
11772 3.6.1 - 2016-12-20
11774 This release fixes a dependency problem and makes some small behind the
11775 scenes improvements.
11776 • The fake-factory dependency was renamed to faker. If you were depend‐
11778 ing on it through hypothesis[django] or hypothesis[fake-factory]
11779 without pinning it yourself then it would have failed to install
11780 properly. This release changes it so that hypothesis[fakefactory]
11781 (which can now also be installed as hypothesis[faker]) will install
11782 the renamed faker package instead.
11783 • This release also removed the dependency of hypothesis[django] on hy‐
11785 pothesis[fakefactory] - it was only being used for emails. These now
11786 use a custom strategy that isn't from fakefactory. As a result you
11787 should also see performance improvements of tests which generated
11788 User objects or other things with email fields, as well as better
11789 shrinking of email addresses.
11790 • The distribution of code using nested calls to one_of() or the | op‐
11792 erator for combining strategies has been improved, as branches are
11793 now flattened to give a more uniform distribution.
11794 • Examples using composite() or .flatmap should now shrink better. In
11796 particular this will affect things which work by first generating a
11797 length and then generating that many items, which have historically
11798 not shrunk very well.
11799 3.6.0 - 2016-10-31
11801 This release reverts Hypothesis to its old pretty printing of lambda
11802 functions based on attempting to extract the source code rather than
11803 decompile the bytecode. This is unfortunately slightly inferior in
11804 some cases and may result in you occasionally seeing things like lambda
11805 x: <unknown> in statistics reports and strategy reprs.
11806 This removes the dependencies on uncompyle6, xdis and spark-parser.
11808 The reason for this is that the new functionality was based on un‐
11810 compyle6, which turns out to introduce a hidden GPLed dependency - it
11811 in turn depended on xdis, and although the library was licensed under
11812 the MIT license, it contained some GPL licensed source code and thus
11813 should have been released under the GPL.
11814 My interpretation is that Hypothesis itself was never in violation of
11816 the GPL (because the license it is under, the Mozilla Public License
11817 v2, is fully compatible with being included in a GPL licensed work),
11818 but I have not consulted a lawyer on the subject. Regardless of the an‐
11819 swer to this question, adding a GPLed dependency will likely cause a
11820 lot of users of Hypothesis to inadvertently be in violation of the GPL.
11821 As a result, if you are running Hypothesis 3.5.x you really should up‐
11823 grade to this release immediately.
11824 3.5.3 - 2016-10-05
11826 This is a bug fix release.
11827 Bugs fixed:
11829 • If the same test was running concurrently in two processes and there
11831 were examples already in the test database which no longer failed,
11832 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
11833 Python 2) because an example it was trying to read was deleted before
11834 it was read. (issue #372).
11835 • Drawing from an integers() strategy with both a min_value and a
11837 max_value would reject too many examples needlessly. Now it repeat‐
11838 edly redraws until satisfied. (pull request #366. Thanks to Calen
11839 Pennington for the contribution).
11840 3.5.2 - 2016-09-24
11842 This is a bug fix release.
11843 • The Hypothesis pytest plugin broke pytest support for doctests. Now
11845 it doesn't.
11846 3.5.1 - 2016-09-23
11848 This is a bug fix release.
11849 • Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
11851 bytes and unicode.
11852 • python:unittest.TestCase tests would not have shown up in the new
11854 statistics mode. Now they do.
11855 • Similarly, stateful tests would not have shown up in statistics and
11857 now they do.
11858 • Statistics now print with pytest node IDs (the names you'd get in
11860 pytest verbose mode).
11861 3.5.0 - 2016-09-22
11863 This is a feature release.
11864 • fractions() and decimals() strategies now support min_value and
11866 max_value parameters. Thanks go to Anne Mulhern for the development
11867 of this feature.
11868 • The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
11870 tics parameter that gives detailed statistics about the tests that
11871 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
11872 funding the development of this feature.
11873 • There is a new event() function that can be used to add custom sta‐
11875 tistics.
11876 Additionally there have been some minor bug fixes:
11878 • In some cases Hypothesis should produce fewer duplicate examples
11880 (this will mostly only affect cases with a single parameter).
11881 • pytest command line parameters are now under an option group for Hy‐
11883 pothesis (thanks to David Keijser for fixing this)
11884 • Hypothesis would previously error if you used PEP 3107 function anno‐
11886 tations on your tests under Python 3.4.
11887 • The repr of many strategies using lambdas has been improved to in‐
11889 clude the lambda body (this was previously supported in many but not
11890 all cases).
11891 3.4.2 - 2016-07-13
11893 This is a bug fix release, fixing a number of problems with the set‐
11894 tings system:
11895 • Test functions defined using @given can now be called from other
11897 threads (issue #337)
11898 • Attempting to delete a settings property would previously have
11900 silently done the wrong thing. Now it raises an AttributeError.
11901 • Creating a settings object with a custom database_file parameter was
11903 silently getting ignored and the default was being used instead. Now
11904 it's not.
11905 3.4.1 - 2016-07-07
11907 This is a bug fix release for a single bug:
11908 • On Windows when running two Hypothesis processes in parallel (e.g.
11910 using pytest-xdist) they could race with each other and one would
11911 raise an exception due to the non-atomic nature of file renaming on
11912 Windows and the fact that you can't rename over an existing file.
11913 This is now fixed.
11914 3.4.0 - 2016-05-27
11916 This release is entirely provided by Lucas Wiman:
11917 Strategies constructed by the Django extra will now respect much more
11919 of Django's validations out of the box. Wherever possible,
11920 full_clean() should succeed.
11921 In particular:
11923 • The max_length, blank and choices kwargs are now respected.
11925 • Add support for DecimalField.
11927 • If a field includes validators, the list of validators are used to
11929 filter the field strategy.
11930 3.3.0 - 2016-05-27
11932 This release went wrong and is functionally equivalent to 3.2.0. Ignore
11933 it.
11934 3.2.0 - 2016-05-19
11936 This is a small single-feature release:
11937 • All tests using @given now fix the global random seed. This removes
11939 the health check for that. If a non-zero seed is required for the fi‐
11940 nal falsifying example, it will be reported. Otherwise Hypothesis
11941 will assume randomization was not a significant factor for the test
11942 and be silent on the subject. If you use random_module() this will
11943 continue to work and will always display the seed.
11944 3.1.3 - 2016-05-01
11946 Single bug fix release
11947 • Another charmap problem. In 3.1.2 text() and characters() would break
11949 on systems which had /tmp mounted on a different partition than the
11950 Hypothesis storage directory (usually in home). This fixes that.
11951 3.1.2 - 2016-04-30
11953 Single bug fix release:
11954 • Anything which used a text() or characters() strategy was broken on
11956 Windows and I hadn't updated appveyor to use the new repository loca‐
11957 tion so I didn't notice. This is now fixed and windows support should
11958 work correctly.
11959 3.1.1 - 2016-04-29
11961 Minor bug fix release.
11962 • Fix concurrency issue when running tests that use text() from multi‐
11964 ple processes at once (issue #302, thanks to Alex Chan).
11965 • Improve performance of code using lists() with max_size (thanks to
11967 Cristi Cobzarenco).
11968 • Fix install on Python 2 with ancient versions of pip so that it in‐
11970 stalls the enum34 backport (thanks to Donald Stufft for telling me
11971 how to do this).
11972 • Remove duplicated __all__ exports from hypothesis.strategies (thanks
11974 to Piët Delport).
11975 • Update headers to point to new repository location.
11977 • Allow use of strategies that can't be used in find() (e.g. choices())
11979 in stateful testing.
11980 3.1.0 - 2016-03-06
11982 • Add a nothing() strategy that never successfully generates values.
11983 • sampled_from() and one_of() can both now be called with an empty ar‐
11985 gument list, in which case they also never generate any values.
11986 • one_of() may now be called with a single argument that is a collec‐
11988 tion of strategies as well as as varargs.
11989 • Add a runner() strategy which returns the instance of the current
11991 test object if there is one.
11992 • 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
11994 can be used as such.
11995 • Tests using RuleBasedStateMachine should now shrink significantly
11997 better.
11998 • Hypothesis now uses a pretty-printing library internally, compatible
12000 with IPython's pretty printing protocol (actually using the same
12001 code). This may improve the quality of output in some cases.
12002 • Add a 'phases' setting that allows more fine grained control over
12004 which parts of the process Hypothesis runs
12005 • Add a suppress_health_check setting which allows you to turn off spe‐
12007 cific health checks in a fine grained manner.
12008 • Fix a bug where lists of non fixed size would always draw one more
12010 element than they included. This mostly didn't matter, but if would
12011 cause problems with empty strategies or ones with side effects.
12012 • Add a mechanism to the Django model generator to allow you to explic‐
12014 itly request the default value (thanks to Jeremy Thurgood for this
12015 one).
12016 3.0.5 - 2016-02-25
12018 • Fix a bug where Hypothesis would now error on pytest development ver‐
12019 sions.
12020 3.0.4 - 2016-02-24
12022 • Fix a bug where Hypothesis would error when running on Python 2.7.3
12023 or earlier because it was trying to pass a python:bytearray object to
12024 python:struct.unpack() (which is only supported since 2.7.4).
12025 3.0.3 - 2016-02-23
12027 • Fix version parsing of pytest to work with pytest release candidates
12028 • More general handling of the health check problem where things could
12030 fail because of a cache miss - now one "free" example is generated
12031 before the start of the health check run.
12032 3.0.2 - 2016-02-18
12034 • Under certain circumstances, strategies involving text() buried in‐
12035 side some other strategy (e.g. text().filter(...) or recur‐
12036 sive(text(), ...)) would cause a test to fail its health checks the
12037 first time it ran. This was caused by having to compute some related
12038 data and cache it to disk. On travis or anywhere else where the .hy‐
12039 pothesis directory was recreated this would have caused the tests to
12040 fail their health check on every run. This is now fixed for all the
12041 known cases, although there could be others lurking.
12042 3.0.1 - 2016-02-18
12044 • Fix a case where it was possible to trigger an "Unreachable" asser‐
12045 tion when running certain flaky stateful tests.
12046 • Improve shrinking of large stateful tests by eliminating a case where
12048 it was hard to delete early steps.
12049 • Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
12051 cantly by provide a custom implementation for fixed size blocks that
12052 can bypass a lot of machinery.
12053 • Set default home directory based on the current working directory at
12055 the point Hypothesis is imported, not whenever the function first
12056 happens to be called.
12057 3.0.0 - 2016-02-17
12059 Codename: This really should have been 2.1.
12060 Externally this looks like a very small release. It has one small
12062 breaking change that probably doesn't affect anyone at all (some behav‐
12063 iour that never really worked correctly is now outright forbidden) but
12064 necessitated a major version bump and one visible new feature.
12065 Internally this is a complete rewrite. Almost nothing other than the
12067 public API is the same.
12068 New features:
12070 • Addition of data() strategy which allows you to draw arbitrary data
12072 interactively within the test.
12073 • New "exploded" database format which allows you to more easily check
12075 the example database into a source repository while supporting merg‐
12076 ing.
12077 • Better management of how examples are saved in the database.
12079 • Health checks will now raise as errors when they fail. It was too
12081 easy to have the warnings be swallowed entirely.
12082 New limitations:
12084 • choices() and streaming() strategies may no longer be used with
12086 find(). Neither may data() (this is the change that necessitated a
12087 major version bump).
12088 Feature removal:
12090 • The ForkingTestCase executor has gone away. It may return in some
12092 more working form at a later date.
12093 Performance improvements:
12095 • A new model which allows flatmap, composite strategies and stateful
12097 testing to perform much better. They should also be more reliable.
12098 • Filtering may in some circumstances have improved significantly. This
12100 will help especially in cases where you have lots of values with in‐
12101 dividual filters on them, such as lists(x.filter(...)).
12102 • Modest performance improvements to the general test runner by avoid‐
12104 ing expensive operations
12105 In general your tests should have got faster. If they've instead got
12107 significantly slower, I'm interested in hearing about it.
12108 Data distribution:
12110 The data distribution should have changed significantly. This may un‐
12112 cover bugs the previous version missed. It may also miss bugs the pre‐
12113 vious version could have uncovered. Hypothesis is now producing less
12114 strongly correlated data than it used to, but the correlations are ex‐
12115 tended over more of the structure.
12116 Shrinking:
12118 Shrinking quality should have improved. In particular Hypothesis can
12120 now perform simultaneous shrinking of separate examples within a single
12121 test (previously it was only able to do this for elements of a single
12122 collection). In some cases performance will have improved, in some
12123 cases it will have got worse but generally shouldn't have by much.
12124 Older versions
12126 2.0.0 - 2016-01-10
12127 Codename: A new beginning
12128 This release cleans up all of the legacy that accrued in the course of
12130 Hypothesis 1.0. These are mostly things that were emitting deprecation
12131 warnings in 1.19.0, but there were a few additional changes.
12132 In particular:
12134 • non-strategy values will no longer be converted to strategies when
12136 used in given or find.
12137 • FailedHealthCheck is now an error and not a warning.
12139 • Handling of non-ascii reprs in user types have been simplified by us‐
12141 ing raw strings in more places in Python 2.
12142 • given no longer allows mixing positional and keyword arguments.
12144 • given no longer works with functions with defaults.
12146 • given no longer turns provided arguments into defaults - they will
12148 not appear in the argspec at all.
12149 • the basic() strategy no longer exists.
12151 • the n_ary_tree strategy no longer exists.
12153 • the average_list_length setting no longer exists. Note: If you're us‐
12155 ing using recursive() this will cause you a significant slow down.
12156 You should pass explicit average_size parameters to collections in
12157 recursive calls.
12158 • @rule can no longer be applied to the same method twice.
12160 • Python 2.6 and 3.3 are no longer officially supported, although in
12162 practice they still work fine.
12163 This also includes two non-deprecation changes:
12165 • given's keyword arguments no longer have to be the rightmost argu‐
12167 ments and can appear anywhere in the method signature.
12168 • The max_shrinks setting would sometimes not have been respected.
12170 1.19.0 - 2016-01-09
12172 Codename: IT COMES
12173 This release heralds the beginning of a new and terrible age of Hypoth‐
12175 esis 2.0.
12176 It's primary purpose is some final deprecations prior to said release.
12178 The goal is that if your code emits no warnings under this release then
12179 it will probably run unchanged under Hypothesis 2.0 (there are some
12180 caveats to this: 2.0 will drop support for some Python versions, and if
12181 you're using internal APIs then as usual that may break without warn‐
12182 ing).
12183 It does have two new features:
12185 • New @seed() decorator which allows you to manually seed a test. This
12187 may be harmlessly combined with and overrides the derandomize set‐
12188 ting.
12189 • settings objects may now be used as a decorator to fix those settings
12191 to a particular @given test.
12192 API changes (old usage still works but is deprecated):
12194 • Settings has been renamed to settings (lower casing) in order to make
12196 the decorator usage more natural.
12197 • Functions for the storage directory that were in hypothesis.settings
12199 are now in a new hypothesis.configuration module.
12200 Additional deprecations:
12202 • the average_list_length setting has been deprecated in favour of be‐
12204 ing explicit.
12205 • the basic() strategy has been deprecated as it is impossible to sup‐
12207 port it under a Conjecture based model, which will hopefully be im‐
12208 plemented at some point in the 2.x series.
12209 • the n_ary_tree strategy (which was never actually part of the public
12211 API) has been deprecated.
12212 • Passing settings or random as keyword arguments to given is depre‐
12214 cated (use the new functionality instead)
12215 Bug fixes:
12217 • No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
12219 tion in streaming() values.
12220 • When running in health check mode with non strict, don't print quite
12222 so many errors for an exception in reify.
12223 • When an assumption made in a test or a filter is flaky, tests will
12225 now raise Flaky instead of UnsatisfiedAssumption.
12226 1.18.1 - 2015-12-22
12228 Two behind the scenes changes:
12229 • Hypothesis will no longer write generated code to the file system.
12231 This will improve performance on some systems (e.g. if you're using
12232 PythonAnywhere which is running your code from NFS) and prevent some
12233 annoying interactions with auto-restarting systems.
12234 • Hypothesis will cache the creation of some strategies. This can sig‐
12236 nificantly improve performance for code that uses flatmap or compos‐
12237 ite and thus has to instantiate strategies a lot.
12238 1.18.0 - 2015-12-21
12240 Features:
12241 • Tests and find are now explicitly seeded off the global random mod‐
12243 ule. This means that if you nest one inside the other you will now
12244 get a health check error. It also means that you can control global
12245 randomization by seeding random.
12246 • There is a new random_module() strategy which seeds the global random
12248 module for you and handles things so that you don't get a health
12249 check warning if you use it inside your tests.
12250 • floats() now accepts two new arguments: allow_nan and allow_infinity.
12252 These default to the old behaviour, but when set to False will do
12253 what the names suggest.
12254 Bug fixes:
12256 • Fix a bug where tests that used text() on Python 3.4+ would not actu‐
12258 ally be deterministic even when explicitly seeded or using the deran‐
12259 domize mode, because generation depended on dictionary iteration or‐
12260 der which was affected by hash randomization.
12261 • Fix a bug where with complicated strategies the timing of the initial
12263 health check could affect the seeding of the subsequent test, which
12264 would also render supposedly deterministic tests non-deterministic in
12265 some scenarios.
12266 • In some circumstances flatmap() could get confused by two struc‐
12268 turally similar things it could generate and would produce a flaky
12269 test where the first time it produced an error but the second time it
12270 produced the other value, which was not an error. The same bug was
12271 presumably also possible in composite().
12272 • flatmap() and composite() initial generation should now be moderately
12274 faster. This will be particularly noticeable when you have many val‐
12275 ues drawn from the same strategy in a single run, e.g. constructs
12276 like lists(s.flatmap(f)). Shrinking performance may have suffered,
12277 but this didn't actually produce an interestingly worse result in any
12278 of the standard scenarios tested.
12279 1.17.1 - 2015-12-16
12281 A small bug fix release, which fixes the fact that the 'note' function
12282 could not be used on tests which used the @example decorator to provide
12283 explicit examples.
12284 1.17.0 - 2015-12-15
12286 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
12287 because it contains the following additional change that was not in‐
12288 tended to be in a patch release (it's perfectly stable, but is a
12289 larger change that should have required a minor version bump):
12290 • Hypothesis will now perform a series of "health checks" as part of
12292 running your tests. These detect and warn about some common error
12293 conditions that people often run into which wouldn't necessarily have
12294 caused the test to fail but would cause e.g. degraded performance or
12295 confusing results.
12296 1.16.1 - 2015-12-14
12298 Note: This release has been removed.
12299 A small bugfix release that allows bdists for Hypothesis to be built
12301 under 2.7 - the compat3.py file which had Python 3 syntax wasn't in‐
12302 tended to be loaded under Python 2, but when building a bdist it was.
12303 In particular this would break running setup.py test.
12304 1.16.0 - 2015-12-08
12306 There are no public API changes in this release but it includes a be‐
12307 haviour change that I wasn't comfortable putting in a patch release.
12308 • Functions from hypothesis.strategies will no longer raise InvalidAr‐
12310 gument on bad arguments. Instead the same errors will be raised when
12311 a test using such a strategy is run. This may improve startup time in
12312 some cases, but the main reason for it is so that errors in strate‐
12313 gies won't cause errors in loading, and it can interact correctly
12314 with things like pytest.mark.skipif.
12315 • Errors caused by accidentally invoking the legacy API are now much
12317 less confusing, although still throw NotImplementedError.
12318 • hypothesis.extra.django is 1.9 compatible.
12320 • When tests are run with max_shrinks=0 this will now still rerun the
12322 test on failure and will no longer print "Trying example:" before
12323 each run. Additionally note() will now work correctly when used with
12324 max_shrinks=0.
12325 1.15.0 - 2015-11-24
12327 A release with two new features.
12328 • A 'characters' strategy for more flexible generation of text with
12330 particular character ranges and types, kindly contributed by
12331 Alexander Shorin.
12332 • Add support for preconditions to the rule based stateful testing.
12334 Kindly contributed by Christopher Armstrong
12335 1.14.0 - 2015-11-01
12337 New features:
12338 • Add 'note' function which lets you include additional information in
12340 the final test run's output.
12341 • Add 'choices' strategy which gives you a choice function that emu‐
12343 lates random.choice.
12344 • Add 'uuid' strategy that generates UUIDs'
12346 • Add 'shared' strategy that lets you create a strategy that just gen‐
12348 erates a single shared value for each test run
12349 Bugs:
12351 • Using strategies of the form streaming(x.flatmap(f)) with find or in
12353 stateful testing would have caused InvalidArgument errors when the
12354 resulting values were used (because code that expected to only be
12355 called within a test context would be invoked).
12356 1.13.0 - 2015-10-29
12358 This is quite a small release, but deprecates some public API functions
12359 and removes some internal API functionality so gets a minor version
12360 bump.
12361 • All calls to the 'strategy' function are now deprecated, even ones
12363 which pass just a SearchStrategy instance (which is still a no-op).
12364 • Never documented hypothesis.extra entry_points mechanism has now been
12366 removed ( it was previously how hypothesis.extra packages were loaded
12367 and has been deprecated and unused for some time)
12368 • Some corner cases that could previously have produced an OverflowEr‐
12370 ror when simplifying failing cases using hypothesis.extra.datetimes
12371 (or dates or times) have now been fixed.
12372 • Hypothesis load time for first import has been significantly reduced
12374 - it used to be around 250ms (on my SSD laptop) and now is around
12375 100-150ms. This almost never matters but was slightly annoying when
12376 using it in the console.
12377 • hypothesis.strategies.randoms was previously missing from __all__.
12379 1.12.0 - 2015-10-18
12381 • Significantly improved performance of creating strategies using the
12382 functions from the hypothesis.strategies module by deferring the cal‐
12383 culation of their repr until it was needed. This is unlikely to have
12384 been an performance issue for you unless you were using flatmap, com‐
12385 posite or stateful testing, but for some cases it could be quite a
12386 significant impact.
12387 • A number of cases where the repr of strategies build from lambdas is
12389 improved
12390 • Add dates() and times() strategies to hypothesis.extra.datetimes
12392 • Add new 'profiles' mechanism to the settings system
12394 • Deprecates mutability of Settings, both the Settings.default top
12396 level property and individual settings.
12397 • A Settings object may now be directly initialized from a parent Set‐
12399 tings.
12400 • @given should now give a better error message if you attempt to use
12402 it with a function that uses destructuring arguments (it still won't
12403 work, but it will error more clearly),
12404 • A number of spelling corrections in error messages
12406 • pytest should no longer display the intermediate modules Hypothesis
12408 generates when running in verbose mode
12409 • Hypothesis should now correctly handle printing objects with
12411 non-ascii reprs on python 3 when running in a locale that cannot han‐
12412 dle ascii printing to stdout.
12413 • Add a unique=True argument to lists(). This is equivalent to
12415 unique_by=lambda x: x, but offers a more convenient syntax.
12416 1.11.4 - 2015-09-27
12418 • Hide modifications Hypothesis needs to make to sys.path by undoing
12419 them after we've imported the relevant modules. This is a workaround
12420 for issues cryptography experienced on windows.
12421 • Slightly improved performance of drawing from sampled_from on large
12423 lists of alternatives.
12424 • Significantly improved performance of drawing from one_of or strate‐
12426 gies using | (note this includes a lot of strategies internally -
12427 floats() and integers() both fall into this category). There turned
12428 out to be a massive performance regression introduced in 1.10.0 af‐
12429 fecting these which probably would have made tests using Hypothesis
12430 significantly slower than they should have been.
12431 1.11.3 - 2015-09-23
12433 • Better argument validation for datetimes() strategy - previously set‐
12434 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
12435 would not have raised an InvalidArgument error and instead would have
12436 behaved confusingly.
12437 • Compatibility with being run on pytest < 2.7 (achieved by disabling
12439 the plugin).
12440 1.11.2 - 2015-09-23
12442 Bug fixes:
12443 • Settings(database=my_db) would not be correctly inherited when used
12445 as a default setting, so that newly created settings would use the
12446 database_file setting and create an SQLite example database.
12447 • Settings.default.database = my_db would previously have raised an er‐
12449 ror and now works.
12450 • Timeout could sometimes be significantly exceeded if during simplifi‐
12452 cation there were a lot of examples tried that didn't trigger the
12453 bug.
12454 • When loading a heavily simplified example using a basic() strategy
12456 from the database this could cause Python to trigger a recursion er‐
12457 ror.
12458 • Remove use of deprecated API in pytest plugin so as to not emit warn‐
12460 ing
12461 Misc:
12463 • hypothesis-pytest is now part of hypothesis core. This should have no
12465 externally visible consequences, but you should update your dependen‐
12466 cies to remove hypothesis-pytest and depend on only Hypothesis.
12467 • Better repr for hypothesis.extra.datetimes() strategies.
12469 • Add .close() method to abstract base class for Backend (it was al‐
12471 ready present in the main implementation).
12472 1.11.1 - 2015-09-16
12474 Bug fixes:
12475 • When running Hypothesis tests in parallel (e.g. using pytest-xdist)
12477 there was a race condition caused by code generation.
12478 • Example databases are now cached per thread so as to not use sqlite
12480 connections from multiple threads. This should make Hypothesis now
12481 entirely thread safe.
12482 • floats() with only min_value or max_value set would have had a very
12484 bad distribution.
12485 • Running on 3.5, Hypothesis would have emitted deprecation warnings
12487 because of use of inspect.getargspec
12488 1.11.0 - 2015-08-31
12490 • text() with a non-string alphabet would have used the repr() of the
12491 the alphabet instead of its contexts. This is obviously silly. It now
12492 works with any sequence of things convertible to unicode strings.
12493 • @given will now work on methods whose definitions contains no ex‐
12495 plicit positional arguments, only varargs (issue #118). This may
12496 have some knock on effects because it means that @given no longer
12497 changes the argspec of functions other than by adding defaults.
12498 • Introduction of new @composite feature for more natural definition of
12500 strategies you'd previously have used flatmap for.
12501 1.10.6 - 2015-08-26
12503 Fix support for fixtures on Django 1.7.
12504 1.10.4 - 2015-08-21
12506 Tiny bug fix release:
12507 • If the database_file setting is set to None, this would have resulted
12509 in an error when running tests. Now it does the same as setting data‐
12510 base to None.
12511 1.10.3 - 2015-08-19
12513 Another small bug fix release.
12514 • lists(elements, unique_by=some_function, min_size=n) would have
12516 raised a ValidationError if n > Settings.default.average_list_length
12517 because it would have wanted to use an average list length shorter
12518 than the minimum size of the list, which is impossible. Now it in‐
12519 stead defaults to twice the minimum size in these circumstances.
12520 • basic() strategy would have only ever produced at most ten distinct
12522 values per run of the test (which is bad if you e.g. have it inside a
12523 list). This was obviously silly. It will now produce a much better
12524 distribution of data, both duplicated and non duplicated.
12525 1.10.2 - 2015-08-19
12527 This is a small bug fix release:
12528 • star imports from hypothesis should now work correctly.
12530 • example quality for examples using flatmap will be better, as the way
12532 it had previously been implemented was causing problems where Hypoth‐
12533 esis was erroneously labelling some examples as being duplicates.
12534 1.10.0 - 2015-08-04
12536 This is just a bugfix and performance release, but it changes some
12537 semi-public APIs, hence the minor version bump.
12538 • Significant performance improvements for strategies which are
12540 one_of() many branches. In particular this included recursive()
12541 strategies. This should take the case where you use one recursive()
12542 strategy as the base strategy of another from unusably slow (tens of
12543 seconds per generated example) to reasonably fast.
12544 • Better handling of just() and sampled_from() for values which have an
12546 incorrect __repr__ implementation that returns non-ASCII unicode on
12547 Python 2.
12548 • Better performance for flatmap from changing the internal morpher API
12550 to be significantly less general purpose.
12551 • Introduce a new semi-public BuildContext/cleanup API. This allows
12553 strategies to register cleanup activities that should run once the
12554 example is complete. Note that this will interact somewhat weirdly
12555 with find.
12556 • Better simplification behaviour for streaming strategies.
12558 • Don't error on lambdas which use destructuring arguments in Python 2.
12560 • Add some better reprs for a few strategies that were missing good
12562 ones.
12563 • The Random instances provided by randoms() are now copyable.
12565 • Slightly more debugging information about simplify when using a debug
12567 verbosity level.
12568 • Support using given for functions with varargs, but not passing argu‐
12570 ments to it as positional.
12571 1.9.0 - 2015-07-27
12573 Codename: The great bundling.
12574 This release contains two fairly major changes.
12576 The first is the deprecation of the hypothesis-extra mechanism. From
12578 now on all the packages that were previously bundled under it other
12579 than hypothesis-pytest (which is a different beast and will remain sep‐
12580 arate). The functionality remains unchanged and you can still import
12581 them from exactly the same location, they just are no longer separate
12582 packages.
12583 The second is that this introduces a new way of building strategies
12585 which lets you build up strategies recursively from other strategies.
12586 It also contains the minor change that calling .example() on a strategy
12588 object will give you examples that are more representative of the ac‐
12589 tual data you'll get. There used to be some logic in there to make the
12590 examples artificially simple but this proved to be a bad idea.
12591 1.8.5 - 2015-07-24
12593 This contains no functionality changes but fixes a mistake made with
12594 building the previous package that would have broken installation on
12595 Windows.
12596 1.8.4 - 2015-07-20
12598 Bugs fixed:
12599 • When a call to floats() had endpoints which were not floats but
12601 merely convertible to one (e.g. integers), these would be included in
12602 the generated data which would cause it to generate non-floats.
12603 • Splitting lambdas used in the definition of flatmap, map or filter
12605 over multiple lines would break the repr, which would in turn break
12606 their usage.
12607 1.8.3 - 2015-07-20
12609 "Falsifying example" would not have been printed when the failure came
12610 from an explicit example.
12611 1.8.2 - 2015-07-18
12613 Another small bugfix release:
12614 • When using ForkingTestCase you would usually not get the falsifying
12616 example printed if the process exited abnormally (e.g. due to
12617 os._exit).
12618 • Improvements to the distribution of characters when using text() with
12620 a default alphabet. In particular produces a better distribution of
12621 ascii and whitespace in the alphabet.
12622 1.8.1 - 2015-07-17
12624 This is a small release that contains a workaround for people who have
12625 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
12626 for Hypothesis per se because that's not a thing that is actually sup‐
12627 posed to work, but Hypothesis leans more heavily on repr than is typi‐
12628 cal so it's worth having a workaround for.
12629 1.8.0 - 2015-07-16
12631 New features:
12632 • Much more sensible reprs for strategies, especially ones that come
12634 from hypothesis.strategies. These should now have as reprs python
12635 code that would produce the same strategy.
12636 • lists() accepts a unique_by argument which forces the generated lists
12638 to be only contain elements unique according to some function key
12639 (which must return a hashable value).
12640 • Better error messages from flaky tests to help you debug things.
12642 Mostly invisible implementation details that may result in finding new
12644 bugs in your code:
12645 • Sets and dictionary generation should now produce a better range of
12647 results.
12648 • floats with bounds now focus more on 'critical values', trying to
12650 produce values at edge cases.
12651 • flatmap should now have better simplification for complicated cases,
12653 as well as generally being (I hope) more reliable.
12654 Bug fixes:
12656 • You could not previously use assume() if you were using the forking
12658 executor.
12659 1.7.2 - 2015-07-10
12661 This is purely a bug fix release:
12662 • When using floats() with stale data in the database you could some‐
12664 times get values in your tests that did not respect min_value or
12665 max_value.
12666 • When getting a Flaky error from an unreliable test it would have in‐
12668 correctly displayed the example that caused it.
12669 • 2.6 dependency on backports was incorrectly specified. This would
12671 only have caused you problems if you were building a universal wheel
12672 from Hypothesis, which is not how Hypothesis ships, so unless you're
12673 explicitly building wheels for your dependencies and support Python
12674 2.6 plus a later version of Python this probably would never have af‐
12675 fected you.
12676 • If you use flatmap in a way that the strategy on the right hand side
12678 depends sensitively on the left hand side you may have occasionally
12679 seen Flaky errors caused by producing unreliable examples when mini‐
12680 mizing a bug. This use case may still be somewhat fraught to be hon‐
12681 est. This code is due a major rearchitecture for 1.8, but in the
12682 meantime this release fixes the only source of this error that I'm
12683 aware of.
12684 1.7.1 - 2015-06-29
12686 Codename: There is no 1.7.0.
12687 A slight technical hitch with a premature upload means there's was a
12689 yanked 1.7.0 release. Oops.
12690 The major feature of this release is Python 2.6 support. Thanks to Jeff
12692 Meadows for doing most of the work there.
12693 Other minor features
12695 • strategies now has a permutations() function which returns a strategy
12697 yielding permutations of values from a given collection.
12698 • if you have a flaky test it will print the exception that it last saw
12700 before failing with Flaky, even if you do not have verbose reporting
12701 on.
12702 • Slightly experimental git merge script available as "python -m hy‐
12704 pothesis.tools.mergedbs". Instructions on how to use it in the doc‐
12705 string of that file.
12706 Bug fixes:
12708 • Better performance from use of filter. In particular tests which in‐
12710 volve large numbers of heavily filtered strategies should perform a
12711 lot better.
12712 • floats() with a negative min_value would not have worked correctly
12714 (worryingly, it would have just silently failed to run any examples).
12715 This is now fixed.
12716 • tests using sampled_from would error if the number of sampled ele‐
12718 ments was smaller than min_satisfying_examples.
12719 1.6.2 - 2015-06-08
12721 This is just a few small bug fixes:
12722 • Size bounds were not validated for values for a binary() strategy
12724 when reading examples from the database.
12725 • sampled_from is now in __all__ in hypothesis.strategies
12727 • floats no longer consider negative integers to be simpler than posi‐
12729 tive non-integers
12730 • Small floating point intervals now correctly count members, so if you
12732 have a floating point interval so narrow there are only a handful of
12733 values in it, this will no longer cause an error when Hypothesis runs
12734 out of values.
12735 1.6.1 - 2015-05-21
12737 This is a small patch release that fixes a bug where 1.6.0 broke the
12738 use of flatmap with the deprecated API and assumed the passed in func‐
12739 tion returned a SearchStrategy instance rather than converting it to a
12740 strategy.
12741 1.6.0 - 2015-05-21
12743 This is a smallish release designed to fix a number of bugs and smooth
12744 out some weird behaviours.
12745 • Fix a critical bug in flatmap where it would reuse old strategies. If
12747 all your flatmap code was pure you're fine. If it's not, I'm sur‐
12748 prised it's working at all. In particular if you want to use flatmap
12749 with django models, you desperately need to upgrade to this version.
12750 • flatmap simplification performance should now be better in some cases
12752 where it previously had to redo work.
12753 • Fix for a bug where invalid unicode data with surrogates could be
12755 generated during simplification (it was already filtered out during
12756 actual generation).
12757 • The Hypothesis database is now keyed off the name of the test instead
12759 of the type of data. This makes much more sense now with the new
12760 strategies API and is generally more robust. This means you will lose
12761 old examples on upgrade.
12762 • The database will now not delete values which fail to deserialize
12764 correctly, just skip them. This is to handle cases where multiple in‐
12765 compatible strategies share the same key.
12766 • find now also saves and loads values from the database, keyed off a
12768 hash of the function you're finding from.
12769 • Stateful tests now serialize and load values from the database. They
12771 should have before, really. This was a bug.
12772 • Passing a different verbosity level into a test would not have worked
12774 entirely correctly, leaving off some messages. This is now fixed.
12775 • Fix a bug where derandomized tests with unicode characters in the
12777 function body would error on Python 2.7.
12778 1.5.0 - 2015-05-14
12780 Codename: Strategic withdrawal.
12781 The purpose of this release is a radical simplification of the API for
12783 building strategies. Instead of the old approach of @strategy.extend
12784 and things that get converted to strategies, you just build strategies
12785 directly.
12786 The old method of defining strategies will still work until Hypothesis
12788 2.0, because it's a major breaking change, but will now emit depreca‐
12789 tion warnings.
12790 The new API is also a lot more powerful as the functions for defining
12792 strategies give you a lot of dials to turn. See the updated data sec‐
12793 tion for details.
12794 Other changes:
12796 • Mixing keyword and positional arguments in a call to @given is
12798 deprecated as well.
12799 • There is a new setting called 'strict'. When set to True, Hypothe‐
12801 sis will raise warnings instead of merely printing them. Turning
12802 it on by default is inadvisable because it means that Hypothesis
12803 minor releases can break your code, but it may be useful for mak‐
12804 ing sure you catch all uses of deprecated APIs.
12805 • max_examples in settings is now interpreted as meaning the maximum
12807 number of unique (ish) examples satisfying assumptions. A new set‐
12808 ting max_iterations which defaults to a larger value has the old
12809 interpretation.
12810 • Example generation should be significantly faster due to a new
12812 faster parameter selection algorithm. This will mostly show up for
12813 simple data types - for complex ones the parameter selection is
12814 almost certainly dominated.
12815 • Simplification has some new heuristics that will tend to cut down
12817 on cases where it could previously take a very long time.
12818 • timeout would previously not have been respected in cases where
12820 there were a lot of duplicate examples. You probably wouldn't have
12821 previously noticed this because max_examples counted duplicates,
12822 so this was very hard to hit in a way that mattered.
12823 • A number of internal simplifications to the SearchStrategy API.
12825 • You can now access the current Hypothesis version as hypothe‐
12827 sis.__version__.
12828 • A top level function is provided for running the stateful tests
12830 without the TestCase infrastructure.
12831 1.4.0 - 2015-05-04
12833 Codename: What a state.
12834 The big feature of this release is the new and slightly experimental
12836 stateful testing API. You can read more about that in the appropriate
12837 section.
12838 Two minor features the were driven out in the course of developing
12840 this:
12841 • You can now set settings.max_shrinks to limit the number of times Hy‐
12843 pothesis will try to shrink arguments to your test. If this is set to
12844 <= 0 then Hypothesis will not rerun your test and will just raise the
12845 failure directly. Note that due to technical limitations if
12846 max_shrinks is <= 0 then Hypothesis will print every example it calls
12847 your test with rather than just the failing one. Note also that I
12848 don't consider settings max_shrinks to zero a sensible way to run
12849 your tests and it should really be considered a debug feature.
12850 • There is a new debug level of verbosity which is even more verbose
12852 than verbose. You probably don't want this.
12853 Breakage of semi-public SearchStrategy API:
12855 • It is now a required invariant of SearchStrategy that if u simplifies
12857 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
12858 plifying should not increase the complexity even though it is not re‐
12859 quired to decrease it. Enforcing this invariant lead to finding some
12860 bugs where simplifying of integers, floats and sets was suboptimal.
12861 • Integers in basic data are now required to fit into 64 bits. As a re‐
12863 sult python integer types are now serialized as strings, and some
12864 types have stopped using quite so needlessly large random seeds.
12865 Hypothesis Stateful testing was then turned upon Hypothesis itself,
12867 which lead to an amazing number of minor bugs being found in Hypothesis
12868 itself.
12869 Bugs fixed (most but not all from the result of stateful testing) in‐
12871 clude:
12872 • Serialization of streaming examples was flaky in a way that you would
12874 probably never notice: If you generate a template, simplify it, seri‐
12875 alize it, deserialize it, serialize it again and then deserialize it
12876 you would get the original stream instead of the simplified one.
12877 • If you reduced max_examples below the number of examples already
12879 saved in the database, you would have got a ValueError. Additionally,
12880 if you had more than max_examples in the database all of them would
12881 have been considered.
12882 • @given will no longer count duplicate examples (which it never called
12884 your function with) towards max_examples. This may result in your
12885 tests running slower, but that's probably just because they're trying
12886 more examples.
12887 • General improvements to example search which should result in better
12889 performance and higher quality examples. In particular parameters
12890 which have a history of producing useless results will be more ag‐
12891 gressively culled. This is useful both because it decreases the
12892 chance of useless examples and also because it's much faster to not
12893 check parameters which we were unlikely to ever pick!
12894 • integers_from and lists of types with only one value (e.g. [None])
12896 would previously have had a very high duplication rate so you were
12897 probably only getting a handful of examples. They now have a much
12898 lower duplication rate, as well as the improvements to search making
12899 this less of a problem in the first place.
12900 • You would sometimes see simplification taking significantly longer
12902 than your defined timeout. This would happen because timeout was only
12903 being checked after each successful simplification, so if Hypothesis
12904 was spending a lot of time unsuccessfully simplifying things it
12905 wouldn't stop in time. The timeout is now applied for unsuccessful
12906 simplifications too.
12907 • In Python 2.7, integers_from strategies would have failed during sim‐
12909 plification with an OverflowError if their starting point was at or
12910 near to the maximum size of a 64-bit integer.
12911 • flatmap and map would have failed if called with a function without a
12913 __name__ attribute.
12914 • If max_examples was less than min_satisfying_examples this would al‐
12916 ways error. Now min_satisfying_examples is capped to max_examples.
12917 Note that if you have assumptions to satisfy here this will still
12918 cause an error.
12919 Some minor quality improvements:
12921 • Lists of streams, flatmapped strategies and basic strategies should
12923 now now have slightly better simplification.
12924 1.3.0 - 2015-05-22
12926 New features:
12927 • New verbosity level API for printing intermediate results and excep‐
12929 tions.
12930 • New specifier for strings generated from a specified alphabet.
12932 • Better error messages for tests that are failing because of a lack of
12934 enough examples.
12935 Bug fixes:
12937 • Fix error where use of ForkingTestCase would sometimes result in too
12939 many open files.
12940 • Fix error where saving a failing example that used flatmap could er‐
12942 ror.
12943 • Implement simplification for sampled_from, which apparently never
12945 supported it previously. Oops.
12946 General improvements:
12948 • Better range of examples when using one_of or sampled_from.
12950 • Fix some pathological performance issues when simplifying lists of
12952 complex values.
12953 • Fix some pathological performance issues when simplifying examples
12955 that require unicode strings with high codepoints.
12956 • Random will now simplify to more readable examples.
12958 1.2.1 - 2015-04-16
12960 A small patch release for a bug in the new executors feature. Tests
12961 which require doing something to their result in order to fail would
12962 have instead reported as flaky.
12963 1.2.0 - 2015-04-15
12965 Codename: Finders keepers.
12966 A bunch of new features and improvements.
12968 • Provide a mechanism for customizing how your tests are executed.
12970 • Provide a test runner that forks before running each example. This
12972 allows better support for testing native code which might trigger a
12973 segfault or a C level assertion failure.
12974 • Support for using Hypothesis to find examples directly rather than as
12976 just as a test runner.
12977 • New streaming type which lets you generate infinite lazily loaded
12979 streams of data - perfect for if you need a number of examples but
12980 don't know how many.
12981 • Better support for large integer ranges. You can now use inte‐
12983 gers_in_range with ranges of basically any size. Previously large
12984 ranges would have eaten up all your memory and taken forever.
12985 • Integers produce a wider range of data than before - previously they
12987 would only rarely produce integers which didn't fit into a machine
12988 word. Now it's much more common. This percolates to other numeric
12989 types which build on integers.
12990 • Better validation of arguments to @given. Some situations that would
12992 previously have caused silently wrong behaviour will now raise an er‐
12993 ror.
12994 • Include +/- sys.float_info.max in the set of floating point edge
12996 cases that Hypothesis specifically tries.
12997 • Fix some bugs in floating point ranges which happen when given +/-
12999 sys.float_info.max as one of the endpoints... (really any two floats
13000 that are sufficiently far apart so that x, y are finite but y - x is
13001 infinite). This would have resulted in generating infinite values
13002 instead of ones inside the range.
13003 1.1.1 - 2015-04-07
13005 Codename: Nothing to see here
13006 This is just a patch release put out because it fixed some internal
13008 bugs that would block the Django integration release but did not actu‐
13009 ally affect anything anyone could previously have been using. It also
13010 contained a minor quality fix for floats that I'd happened to have fin‐
13011 ished in time.
13012 • Fix some internal bugs with object lifecycle management that were im‐
13014 possible to hit with the previously released versions but broke hy‐
13015 pothesis-django.
13016 • Bias floating point numbers somewhat less aggressively towards very
13018 small numbers
13019 1.1.0 - 2015-04-06
13021 Codename: No-one mention the M word.
13022 • Unicode strings are more strongly biased towards ascii characters.
13024 Previously they would generate all over the space. This is mostly so
13025 that people who try to shape their unicode strings with assume() have
13026 less of a bad time.
13027 • A number of fixes to data deserialization code that could theoreti‐
13029 cally have caused mysterious bugs when using an old version of a Hy‐
13030 pothesis example database with a newer version. To the best of my
13031 knowledge a change that could have triggered this bug has never actu‐
13032 ally been seen in the wild. Certainly no-one ever reported a bug of
13033 this nature.
13034 • Out of the box support for Decimal and Fraction.
13036 • new dictionary specifier for dictionaries with variable keys.
13038 • Significantly faster and higher quality simplification, especially
13040 for collections of data.
13041 • New filter() and flatmap() methods on Strategy for better ways of
13043 building strategies out of other strategies.
13044 • New BasicStrategy class which allows you to define your own strate‐
13046 gies from scratch without needing an existing matching strategy or
13047 being exposed to the full horror or non-public nature of the Search‐
13048 Strategy interface.
13049 1.0.0 - 2015-03-27
13051 Codename: Blast-off!
13052 There are no code changes in this release. This is precisely the 0.9.2
13054 release with some updated documentation.
13055 0.9.2 - 2015-03-26
13057 Codename: T-1 days.
13058 • floats_in_range would not actually have produced floats_in_range un‐
13060 less that range happened to be (0, 1). Fix this.
13061 0.9.1 - 2015-03-25
13063 Codename: T-2 days.
13064 • Fix a bug where if you defined a strategy using map on a lambda then
13066 the results would not be saved in the database.
13067 • Significant performance improvements when simplifying examples using
13069 lists, strings or bounded integer ranges.
13070 0.9.0 - 2015-03-23
13072 Codename: The final countdown
13073 This release could also be called 1.0-RC1.
13075 It contains a teeny tiny bugfix, but the real point of this release is
13077 to declare feature freeze. There will be zero functionality changes be‐
13078 tween 0.9.0 and 1.0 unless something goes really really wrong. No new
13079 features will be added, no breaking API changes will occur, etc. This
13080 is the final shakedown before I declare Hypothesis stable and ready to
13081 use and throw a party to celebrate.
13082 Bug bounty for any bugs found between now and 1.0: I will buy you a
13084 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
13085 we ever find ourselves in the same city at the same time.
13086 The one tiny bugfix:
13088 • Under pypy, databases would fail to close correctly when garbage col‐
13090 lected, leading to a memory leak and a confusing error message if you
13091 were repeatedly creating databases and not closing them. It is very
13092 unlikely you were doing this and the chances of you ever having no‐
13093 ticed this bug are very low.
13094 0.7.2 - 2015-03-22
13096 Codename: Hygienic macros or bust
13097 • You can now name an argument to @given 'f' and it won't break (issue
13099 #38)
13100 • strategy_test_suite is now named strategy_test_suite as the documen‐
13102 tation claims and not in fact strategy_test_suitee
13103 • Settings objects can now be used as a context manager to temporarily
13105 override the default values inside their context.
13106 0.7.1 - 2015-03-21
13108 Codename: Point releases go faster
13109 • Better string generation by parametrizing by a limited alphabet
13111 • Faster string simplification - previously if simplifying a string
13113 with high range unicode characters it would try every unicode charac‐
13114 ter smaller than that. This was pretty pointless. Now it stops after
13115 it's a short range (it can still reach smaller ones through recursive
13116 calls because of other simplifying operations).
13117 • Faster list simplification by first trying a binary chop down the
13119 middle
13120 • Simultaneous simplification of identical elements in a list. So if a
13122 bug only triggers when you have duplicates but you drew e.g. [-17,
13123 -17], this will now simplify to [0, 0].
13124 0.7.0, - 2015-03-20
13126 Codename: Starting to look suspiciously real
13127 This is probably the last minor release prior to 1.0. It consists of
13129 stability improvements, a few usability things designed to make Hypoth‐
13130 esis easier to try out, and filing off some final rough edges from the
13131 API.
13132 • Significant speed and memory usage improvements
13134 • Add an example() method to strategy objects to give an example of the
13136 sort of data that the strategy generates.
13137 • Remove .descriptor attribute of strategies
13139 • Rename descriptor_test_suite to strategy_test_suite
13141 • Rename the few remaining uses of descriptor to specifier (descriptor
13143 already has a defined meaning in Python)
13144 0.6.0 - 2015-03-13
13146 Codename: I'm sorry, were you using that API?
13147 This is primarily a "simplify all the weird bits of the API" release.
13149 As a result there are a lot of breaking changes. If you just use @given
13150 with core types then you're probably fine.
13151 In particular:
13153 • Stateful testing has been removed from the API
13155 • The way the database is used has been rendered less useful (sorry).
13157 The feature for reassembling values saved from other tests doesn't
13158 currently work. This will probably be brought back in post 1.0.
13159 • SpecificationMapper is no longer a thing. Instead there is an
13161 ExtMethod called strategy which you extend to specify how to convert
13162 other types to strategies.
13163 • Settings are now extensible so you can add your own for configuring a
13165 strategy
13166 • MappedSearchStrategy no longer needs an unpack method
13168 • Basically all the SearchStrategy internals have changed massively. If
13170 you implemented SearchStrategy directly rather than using Mapped‐
13171 SearchStrategy talk to me about fixing it.
13172 • Change to the way extra packages work. You now specify the package.
13174 This must have a load() method. Additionally any modules in the pack‐
13175 age will be loaded in under hypothesis.extra
13176 Bug fixes:
13178 • Fix for a bug where calling falsify on a lambda with a non-ascii
13180 character in its body would error.
13181 Hypothesis Extra:
13183 hypothesis-fakefactory: An extension for using faker data in hypothe‐
13185 sis. Depends
13186 on fake-factory.
13187 0.5.0 - 2015-02-10
13189 Codename: Read all about it.
13190 Core hypothesis:
13192 • Add support back in for pypy and python 3.2
13194 • @given functions can now be invoked with some arguments explicitly
13196 provided. If all arguments that hypothesis would have provided are
13197 passed in then no falsification is run.
13198 • Related to the above, this means that you can now use pytest fixtures
13200 and mark.parametrize with Hypothesis without either interfering with
13201 the other.
13202 • Breaking change: @given no longer works for functions with varargs
13204 (varkwargs are fine). This might be added back in at a later date.
13205 • Windows is now fully supported. A limited version (just the tests
13207 with none of the extras) of the test suite is run on windows with
13208 each commit so it is now a first class citizen of the Hypothesis
13209 world.
13210 • Fix a bug for fuzzy equality of equal complex numbers with different
13212 reprs (this can happen when one coordinate is zero). This shouldn't
13213 affect users - that feature isn't used anywhere public facing.
13214 • Fix generation of floats on windows and 32-bit builds of python. I
13216 was using some struct.pack logic that only worked on certain word
13217 sizes.
13218 • When a test times out and hasn't produced enough examples this now
13220 raises a Timeout subclass of Unfalsifiable.
13221 • Small search spaces are better supported. Previously something like a
13223 @given(bool, bool) would have failed because it couldn't find enough
13224 examples. Hypothesis is now aware of the fact that these are small
13225 search spaces and will not error in this case.
13226 • Improvements to parameter search in the case of hard to satisfy as‐
13228 sume. Hypothesis will now spend less time exploring parameters that
13229 are unlikely to provide anything useful.
13230 • Increase chance of generating "nasty" floats
13232 • Fix a bug that would have caused unicode warnings if you had a sam‐
13234 pled_from that was mixing unicode and byte strings.
13235 • Added a standard test suite that you can use to validate a custom
13237 strategy you've defined is working correctly.
13238 Hypothesis extra:
13240 First off, introducing Hypothesis extra packages!
13242 These are packages that are separated out from core Hypothesis because
13244 they have one or more dependencies. Every hypothesis-extra package is
13245 pinned to a specific point release of Hypothesis and will have some
13246 version requirements on its dependency. They use entry_points so you
13247 will usually not need to explicitly import them, just have them in‐
13248 stalled on the path.
13249 This release introduces two of them:
13251 hypothesis-datetime:
13253 Does what it says on the tin: Generates datetimes for Hypothesis. Just
13255 install the package and datetime support will start working.
13256 Depends on pytz for timezone support
13258 hypothesis-pytest:
13260 A very rudimentary pytest plugin. All it does right now is hook the
13262 display of falsifying examples into pytest reporting.
13263 Depends on pytest.
13265 0.4.3 - 2015-02-05
13267 Codename: TIL narrow Python builds are a thing
13268 This just fixes the one bug.
13270 • Apparently there is such a thing as a "narrow python build" and OS X
13272 ships with these by default for python 2.7. These are builds where
13273 you only have two bytes worth of unicode. As a result, generating
13274 unicode was completely broken on OS X. Fix this by only generating
13275 unicode codepoints in the range supported by the system.
13276 0.4.2 - 2015-02-04
13278 Codename: O(dear)
13279 This is purely a bugfix release:
13281 • Provide sensible external hashing for all core types. This will sig‐
13283 nificantly improve performance of tracking seen examples which hap‐
13284 pens in literally every falsification run. For Hypothesis fixing this
13285 cut 40% off the runtime of the test suite. The behaviour is quadratic
13286 in the number of examples so if you're running the default configura‐
13287 tion this will be less extreme (Hypothesis's test suite runs at a
13288 higher number of examples than default), but you should still see a
13289 significant improvement.
13290 • Fix a bug in formatting of complex numbers where the string could get
13292 incorrectly truncated.
13293 0.4.1 - 2015-02-03
13295 Codename: Cruel and unusual edge cases
13296 This release is mostly about better test case generation.
13298 Enhancements:
13300 • Has a cool release name
13302 • text_type (str in python 3, unicode in python 2) example generation
13304 now actually produces interesting unicode instead of boring ascii
13305 strings.
13306 • floating point numbers are generated over a much wider range, with
13308 particular attention paid to generating nasty numbers - nan, infin‐
13309 ity, large and small values, etc.
13310 • examples can be generated using pieces of examples previously saved
13312 in the database. This allows interesting behaviour that has previ‐
13313 ously been discovered to be propagated to other examples.
13314 • improved parameter exploration algorithm which should allow it to
13316 more reliably hit interesting edge cases.
13317 • Timeout can now be disabled entirely by setting it to any value <= 0.
13319 Bug fixes:
13321 • The descriptor on a OneOfStrategy could be wrong if you had descrip‐
13323 tors which were equal but should not be coalesced. e.g. a strategy
13324 for one_of((frozenset({int}), {int})) would have reported its de‐
13325 scriptor as {int}. This is unlikely to have caused you any problems
13326 • If you had strategies that could produce NaN (which float previously
13328 couldn't but e.g. a Just(float('nan')) could) then this would have
13329 sent hypothesis into an infinite loop that would have only been ter‐
13330 minated when it hit the timeout.
13331 • Given elements that can take a long time to minimize, minimization of
13333 floats or tuples could be quadratic or worse in the that value. You
13334 should now see much better performance for simplification, albeit at
13335 some cost in quality.
13336 Other:
13338 • A lot of internals have been been rewritten. This shouldn't affect
13340 you at all, but it opens the way for certain of hypothesis's oddities
13341 to be a lot more extensible by users. Whether this is a good thing
13342 may be up for debate...
13343 0.4.0 - 2015-01-21
13345 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
13346 stores data in a local SQLite database and will reuse it for all tests
13347 of the same type.
13348 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
13350 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
13351 ther notice. The previous license remains applicable to all code prior
13352 to 0.4.0.
13353 Enhancements:
13355 • Printing of failing examples. I was finding that the pytest runner
13357 was not doing a good job of displaying these, and that Hypothesis it‐
13358 self could do much better.
13359 • Drop dependency on six for cross-version compatibility. It was easy
13361 enough to write the shim for the small set of features that we care
13362 about and this lets us avoid a moderately complex dependency.
13363 • Some improvements to statistical distribution of selecting from small
13365 (<= 3 elements)
13366 • Improvements to parameter selection for finding examples.
13368 Bugs fixed:
13370 • could_have_produced for lists, dicts and other collections would not
13372 have examined the elements and thus when using a union of different
13373 types of list this could result in Hypothesis getting confused and
13374 passing a value to the wrong strategy. This could potentially result
13375 in exceptions being thrown from within simplification.
13376 • sampled_from would not work correctly on a single element list.
13378 • Hypothesis could get very confused by values which are equal despite
13380 having different types being used in descriptors. Hypothesis now has
13381 its own more specific version of equality it uses for descriptors and
13382 tracking. It is always more fine grained than Python equality: Things
13383 considered != are not considered equal by hypothesis, but some things
13384 that are considered == are distinguished. If your test suite uses
13385 both frozenset and set tests this bug is probably affecting you.
13386 0.3.2 - 2015-01-16
13388 • Fix a bug where if you specified floats_in_range with integer argu‐
13389 ments Hypothesis would error in example simplification.
13390 • Improve the statistical distribution of the floats you get for the
13392 floats_in_range strategy. I'm not sure whether this will affect users
13393 in practice but it took my tests for various conditions from flaky to
13394 rock solid so it at the very least improves discovery of the artifi‐
13395 cial cases I'm looking for.
13396 • Improved repr() for strategies and RandomWithSeed instances.
13398 • Add detection for flaky test cases where hypothesis managed to find
13400 an example which breaks it but on the final invocation of the test it
13401 does not raise an error. This will typically happen with too much re‐
13402 cursion errors but could conceivably happen in other circumstances
13403 too.
13404 • Provide a "derandomized" mode. This allows you to run hypothesis with
13406 zero real randomization, making your build nice and deterministic.
13407 The tests run with a seed calculated from the function they're test‐
13408 ing so you should still get a good distribution of test cases.
13409 • Add a mechanism for more conveniently defining tests which just sam‐
13411 ple from some collection.
13412 • Fix for a really subtle bug deep in the internals of the strategy ta‐
13414 ble. In some circumstances if you were to define instance strategies
13415 for both a parent class and one or more of its subclasses you would
13416 under some circumstances get the strategy for the wrong superclass of
13417 an instance. It is very unlikely anyone has ever encountered this in
13418 the wild, but it is conceivably possible given that a mix of namedtu‐
13419 ple and tuple are used fairly extensively inside hypothesis which do
13420 exhibit this pattern of strategy.
13421 0.3.1 - 2015-01-13
13423 • Support for generation of frozenset and Random values
13424 • Correct handling of the case where a called function mutates it argu‐
13426 ment. This involved introducing a notion of a strategies knowing how
13427 to copy their argument. The default method should be entirely accept‐
13428 able and the worst case is that it will continue to have the old be‐
13429 haviour if you don't mark your strategy as mutable, so this shouldn't
13430 break anything.
13431 • Fix for a bug where some strategies did not correctly implement
13433 could_have_produced. It is very unlikely that any of these would have
13434 been seen in the wild, and the consequences if they had been would
13435 have been minor.
13436 • Re-export the @given decorator from the main hypothesis namespace.
13438 It's still available at the old location too.
13439 • Minor performance optimisation for simplifying long lists.
13441 0.3.0 - 2015-01-12
13443 • Complete redesign of the data generation system. Extreme breaking
13444 change for anyone who was previously writing their own SearchStrategy
13445 implementations. These will not work any more and you'll need to mod‐
13446 ify them.
13447 • New settings system allowing more global and modular control of Veri‐
13449 fier behaviour.
13450 • Decouple SearchStrategy from the StrategyTable. This leads to much
13452 more composable code which is a lot easier to understand.
13453 • A significant amount of internal API renaming and moving. This may
13455 also break your code.
13456 • Expanded available descriptors, allowing for generating integers or
13458 floats in a specific range.
13459 • Significantly more robust. A very large number of small bug fixes,
13461 none of which anyone is likely to have ever noticed.
13462 • Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
13464 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
13465 these to remain officially supported for a very long time. I would
13466 not be surprised to add pypy support back in later but I'm not going
13467 to do so until I know someone cares about it. In the meantime it will
13468 probably still work.
13469 0.2.2 - 2015-01-08
13471 • Fix an embarrassing complete failure of the installer caused by my
13472 being bad at version control
13473 0.2.1 - 2015-01-07
13475 • Fix a bug in the new stateful testing feature where you could make
13476 __init__ a @requires method. Simplification would not always work if
13477 the prune method was able to successfully shrink the test.
13478 0.2.0 - 2015-01-07
13480 • It's aliiive.
13481 • Improve python 3 support using six.
13483 • Distinguish between byte and unicode types.
13485 • Fix issues where FloatStrategy could raise.
13487 • Allow stateful testing to request constructor args.
13489 • Fix for issue where test annotations would timeout based on when the
13491 module was loaded instead of when the test started
13492 0.1.4 - 2013-12-14
13494 • Make verification runs time bounded with a configurable timeout
13495 0.1.3 - 2013-05-03
13497 • Bugfix: Stateful testing behaved incorrectly with subclassing.
13498 • Complex number support
13500 • support for recursive strategies
13502 • different error for hypotheses with unsatisfiable assumptions
13504 0.1.2 - 2013-03-24
13506 • Bugfix: Stateful testing was not minimizing correctly and could throw
13507 exceptions.
13508 • Better support for recursive strategies.
13510 • Support for named tuples.
13512 • Much faster integer generation.
13514 0.1.1 - 2013-03-24
13516 • Python 3.x support via 2to3.
13517 • Use new style classes (oops).
13519 0.1.0 - 2013-03-23
13521 • Introduce stateful testing.
13522 • Massive rewrite of internals to add flags and strategies.
13524 0.0.5 - 2013-03-13
13526 • No changes except trying to fix packaging
13527 0.0.4 - 2013-03-13
13529 • No changes except that I checked in a failing test case for 0.0.3 so
13530 had to replace the release. Doh
13531 0.0.3 - 2013-03-13
13533 • Improved a few internals.
13534 • Opened up creating generators from instances as a general API.
13536 • Test integration.
13538 0.0.2 - 2013-03-12
13540 • Starting to tighten up on the internals.
13541 • Change API to allow more flexibility in configuration.
13543 • More testing.
13545 0.0.1 - 2013-03-10
13547 • Initial release.
13548 • Basic working prototype. Demonstrates idea, probably shouldn't be
13550 used.
13551
13553 Hypothesis development is managed by David R. MacIver and Zac Hat‐
13554 field-Dodds, respectively the first author and lead maintainer.
13555 However, these roles don't include unpaid feature development on Hy‐
13557 pothesis. Our roles as leaders of the project are:
13558 1. Helping other people do feature development on Hypothesis
13560 2. Fixing bugs and other code health issues
13562 3. Improving documentation
13564 4. General release management work
13566 5. Planning the general roadmap of the project
13568 6. Doing sponsored development on tasks that are too large or in depth
13570 for other people to take on
13571 So all new features must either be sponsored or implemented by someone
13573 else. That being said, the maintenance team takes an active role in
13574 shepherding pull requests and helping people write a new feature (see
13575 CONTRIBUTING.rst for details and these examples of how the process
13576 goes). This isn't "patches welcome", it's "we will help you write a
13577 patch".
13578 Release policy
13580 Hypothesis releases follow semantic versioning.
13581 We maintain backwards-compatibility wherever possible, and use depreca‐
13583 tion warnings to mark features that have been superseded by a newer al‐
13584 ternative. If you want to detect this, you can upgrade warnings to er‐
13585 rors in the usual ways.
13586 We use continuous deployment to ensure that you can always use our new‐
13588 est and shiniest features - every change to the source tree is automat‐
13589 ically built and published on PyPI as soon as it's merged onto master,
13590 after code review and passing our extensive test suite.
13591 Project roadmap
13593 Hypothesis does not have a long-term release plan. We respond to bug
13594 reports as they are made; new features are released as and when someone
13595 volunteers to write and maintain them.
13596
13598 For questions you are happy to ask in public, the Hypothesis community
13599 is a friendly place where I or others will be more than happy to help
13600 you out. You're also welcome to ask questions on Stack Overflow. If you
13601 do, please tag them with 'python-hypothesis' so someone sees them.
13602 For bugs and enhancements, please file an issue on the GitHub issue
13604 tracker. Note that as per the development policy, enhancements will
13605 probably not get implemented unless you're willing to pay for develop‐
13606 ment or implement them yourself (with assistance from the maintainers).
13607 Bugs will tend to get fixed reasonably promptly, though it is of course
13608 on a best effort basis.
13609 To see the versions of Python, optional dependencies, test runners, and
13611 operating systems Hypothesis supports (meaning incompatibility is
13612 treated as a bug), see Compatibility.
13613 If you need to ask questions privately or want more of a guarantee of
13615 bugs being fixed promptly, please contact me on
13616 hypothesis-support@drmaciver.com to talk about availability of support
13617 contracts.
13618
13620 Downstream packagers often want to package Hypothesis. Here are some
13621 guidelines.
13622 The primary guideline is this: If you are not prepared to keep up with
13624 the Hypothesis release schedule, don't. You will annoy me and are doing
13625 your users a disservice.
13626 Hypothesis has a very frequent release schedule. It's rare that it goes
13628 a week without a release, and there are often multiple releases in a
13629 given week.
13630 If you are prepared to keep up with this schedule, you might find the
13632 rest of this document useful.
13633 Release tarballs
13635 These are available from the GitHub releases page. The tarballs on PyPI
13636 are intended for installation from a Python tool such as pip and should
13637 not be considered complete releases. Requests to include additional
13638 files in them will not be granted. Their absence is not a bug.
13639 Dependencies
13641 Python versions
13642 Hypothesis is designed to work with a range of Python versions - we
13643 support all versions of CPython with upstream support. We also support
13644 the latest versions of PyPy for Python 3.
13645 Other Python libraries
13647 Hypothesis has mandatory dependencies on the following libraries:
13648 • attrs
13650 • sortedcontainers
13652 Hypothesis has optional dependencies on the following libraries:
13654 extras_require = {
13656 "cli": ["click>=7.0", "black>=19.10b0", "rich>=9.0.0"],
13657 "codemods": ["libcst>=0.3.16"],
13658 "ghostwriter": ["black>=19.10b0"],
13659 "pytz": ["pytz>=2014.1"],
13660 "dateutil": ["python-dateutil>=1.4"],
13661 "lark": ["lark-parser>=0.6.5"],
13662 "numpy": ["numpy>=1.9.0"],
13663 "pandas": ["pandas>=0.25"],
13664 "pytest": ["pytest>=4.6"],
13665 "dpcontracts": ["dpcontracts>=0.4"],
13666 "redis": ["redis>=3.0.0"],
13667 # zoneinfo is an odd one: every dependency is conditional, because they're
13668 # only necessary on old versions of Python or Windows systems.
13669 "zoneinfo": [
13670 "tzdata>=2022.1 ; sys_platform == 'win32'",
13671 "backports.zoneinfo>=0.2.1 ; python_version<'3.9'",
13672 ],
13673 # We only support Django versions with upstream support - see
13674 # https://www.djangoproject.com/download/#supported-versions
13675 # We also leave the choice of timezone library to the user, since it
13676 # might be zoneinfo or pytz depending on version and configuration.
13677 "django": ["django>=2.2"],
13678 }
13679 The way this works when installing Hypothesis normally is that these
13682 features become available if the relevant library is installed.
13683 Specifically for pytest, our plugin supports versions of pytest which
13685 have been out of upstream support for some time. Hypothesis tests can
13686 still be executed by even older versions of pytest - you just won't
13687 have the plugin to provide automatic marks, helpful usage warnings, and
13688 per-test statistics.
13689 Testing Hypothesis
13691 If you want to test Hypothesis as part of your packaging you will prob‐
13692 ably not want to use the mechanisms Hypothesis itself uses for running
13693 its tests, because it has a lot of logic for installing and testing
13694 against different versions of Python.
13695 The tests must be run with fairly recent tooling; check the
13697 tree/master/requirements/ directory for details.
13698 The organisation of the tests is described in the
13700 hypothesis-python/tests/README.rst.
13701 Examples
13703 • arch linux
13704 • fedora
13706 • gentoo
13708
13710 One of the things that is often concerning for people using randomized
13711 testing is the question of how to reproduce failing test cases.
13712 NOTE:
13714 It is better to think about the data Hypothesis generates as being
13715 arbitrary, rather than random. We deliberately generate any valid
13716 data that seems likely to cause errors, so you shouldn't rely on any
13717 expected distribution of or relationships between generated data.
13718 You can read about "swarm testing" and "coverage guided fuzzing" if
13719 you're interested, because you don't need to know for Hypothesis!
13720 Fortunately Hypothesis has a number of features to support reproducing
13722 test failures. The one you will use most commonly when developing lo‐
13723 cally is the example database, which means that you shouldn't have to
13724 think about the problem at all for local use - test failures will just
13725 automatically reproduce without you having to do anything.
13726 The example database is perfectly suitable for sharing between ma‐
13728 chines, but there currently aren't very good work flows for that, so
13729 Hypothesis provides a number of ways to make examples reproducible by
13730 adding them to the source code of your tests. This is particularly use‐
13731 ful when e.g. you are trying to run an example that has failed on your
13732 CI, or otherwise share them between machines.
13733 Providing explicit examples
13735 The simplest way to reproduce a failed test is to ask Hypothesis to run
13736 the failing example it printed. For example, if Falsifying example:
13737 test(n=1) was printed you can decorate test with @example(n=1).
13738 @example can also be used to ensure a specific example is always exe‐
13740 cuted as a regression test or to cover some edge case - basically com‐
13741 bining a Hypothesis test and a traditional parametrized test.
13742 hypothesis.example(*args, **kwargs)
13744 A decorator which ensures a specific example is always tested.
13745 Hypothesis will run all examples you've asked for first. If any of them
13747 fail it will not go on to look for more examples.
13748 It doesn't matter whether you put the example decorator before or after
13750 given. Any permutation of the decorators in the above will do the same
13751 thing.
13752 Note that examples can be positional or keyword based. If they're posi‐
13754 tional then they will be filled in from the right when calling, so ei‐
13755 ther of the following styles will work as expected:
13756 @given(text())
13758 @example("Hello world")
13759 @example(x="Some very long string")
13760 def test_some_code(x):
13761 pass
13762 from unittest import TestCase
13765 class TestThings(TestCase):
13768 @given(text())
13769 @example("Hello world")
13770 @example(x="Some very long string")
13771 def test_some_code(self, x):
13772 pass
13773 As with @given, it is not permitted for a single example to be a mix of
13775 positional and keyword arguments. Either are fine, and you can use one
13776 in one example and the other in another example if for some reason you
13777 really want to, but a single example must be consistent.
13778 Reproducing a test run with @seed
13780 hypothesis.seed(seed)
13781 seed: Start the test execution from a specific seed.
13782 May be any hashable object. No exact meaning for seed is pro‐
13784 vided other than that for a fixed seed value Hypothesis will try
13785 the same actions (insofar as it can given external sources of
13786 non- determinism. e.g. timing and hash randomization).
13787 Overrides the derandomize setting, which is designed to enable
13789 deterministic builds rather than reproducing observed failures.
13790 When a test fails unexpectedly, usually due to a health check failure,
13792 Hypothesis will print out a seed that led to that failure, if the test
13793 is not already running with a fixed seed. You can then recreate that
13794 failure using either the @seed decorator or (if you are running pytest)
13795 with --hypothesis-seed. For example, the following test function and
13796 RuleBasedStateMachine will each check the same examples each time they
13797 are executed, thanks to @seed():
13798 @seed(1234)
13800 @given(x=...)
13801 def test(x):
13802 ...
13803 @seed(6789)
13806 class MyModel(RuleBasedStateMachine):
13807 ...
13808 The seed will not be printed if you could simply use @example instead.
13810 Reproducing an example with @reproduce_failure
13812 Hypothesis has an opaque binary representation that it uses for all ex‐
13813 amples it generates. This representation is not intended to be stable
13814 across versions or with respect to changes in the test, but can be used
13815 to to reproduce failures with the @reproduce_failure decorator.
13816 hypothesis.reproduce_failure(version, blob)
13818 Run the example that corresponds to this data blob in order to
13819 reproduce a failure.
13820 A test with this decorator always runs only one example and al‐
13822 ways fails. If the provided example does not cause a failure,
13823 or is in some way invalid for this test, then this will fail
13824 with a DidNotReproduce error.
13825 This decorator is not intended to be a permanent addition to
13827 your test suite. It's simply some code you can add to ease re‐
13828 production of a problem in the event that you don't have access
13829 to the test database. Because of this, no compatibility guaran‐
13830 tees are made between different versions of Hypothesis - its API
13831 may change arbitrarily from version to version.
13832 The intent is that you should never write this decorator by hand, but
13834 it is instead provided by Hypothesis. When a test fails with a falsi‐
13835 fying example, Hypothesis may print out a suggestion to use @repro‐
13836 duce_failure on the test to recreate the problem as follows:
13837 >>> from hypothesis import settings, given, PrintSettings
13839 >>> import hypothesis.strategies as st
13840 >>> @given(st.floats())
13841 ... @settings(print_blob=True)
13842 ... def test(f):
13843 ... assert f == f
13844 ...
13845 >>> try:
13846 ... test()
13847 ... except AssertionError:
13848 ... pass
13849 ...
13850 Falsifying example: test(f=nan)
13851 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
13853 Adding the suggested decorator to the test should reproduce the failure
13855 (as long as everything else is the same - changing the versions of
13856 Python or anything else involved, might of course affect the behaviour
13857 of the test! Note that changing the version of Hypothesis will result
13858 in a different error - each @reproduce_failure invocation is specific
13859 to a Hypothesis version).
13860 By default these messages are not printed. If you want to see these
13862 you must set the print_blob setting to True.
13863
13865 David R. MacIver
13866
13868 2013-2022, David R. MacIver
138696.45.0 Jul 22, 2022 HYPOTHESIS(1)
