1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property-based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the fu‐
41 ture.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 • Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 • If you delete an object, it is no longer visible.
53 • If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ""
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ""
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Let's see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127 Anyway, this test immediately finds a bug in the code:
129 Falsifying example: test_decode_inverts_encode(s='')
131 UnboundLocalError: local variable 'character' referenced before assignment
133 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136 If we fix that by just adding the following code to the beginning of
138 the function then Hypothesis tells us the code is correct (by doing
139 nothing as you'd expect a passing test to).
140 if not input_string:
142 return []
143 If we wanted to make sure this example was always checked we could add
145 it in explicitly by using the @example decorator:
146 from hypothesis import example, given, strategies as st
148 @given(st.text())
151 @example("")
152 def test_decode_inverts_encode(s):
153 assert decode(encode(s)) == s
154 This can be useful to show other developers (or your future self) what
156 kinds of data are valid inputs, or to ensure that particular edge cases
157 such as "" are tested every time. It's also great for regression tests
158 because although Hypothesis will remember failing examples, we don't
159 recommend distributing that database.
160 It's also worth noting that both example and given support keyword ar‐
162 guments as well as positional. The following would have worked just as
163 well:
164 @given(s=st.text())
166 @example(s="")
167 def test_decode_inverts_encode(s):
168 assert decode(encode(s)) == s
169 Suppose we had a more interesting bug and forgot to reset the count
171 each time. Say we missed a line in our encode method:
172 def encode(input_string):
174 count = 1
175 prev = ""
176 lst = []
177 for character in input_string:
178 if character != prev:
179 if prev:
180 entry = (prev, count)
181 lst.append(entry)
182 # count = 1 # Missing reset operation
183 prev = character
184 else:
185 count += 1
186 entry = (character, count)
187 lst.append(entry)
188 return lst
189 Hypothesis quickly informs us of the following example:
191 Falsifying example: test_decode_inverts_encode(s='001')
193 Note that the example provided is really quite simple. Hypothesis
195 doesn't just find any counter-example to your tests, it knows how to
196 simplify the examples it finds to produce small easy to understand
197 ones. In this case, two identical values are enough to set the count to
198 a number different from one, followed by another distinct value which
199 should have reset the count but in this case didn't.
200 The examples Hypothesis provides are valid Python code you can run. Any
202 arguments that you explicitly provide when calling the function are not
203 generated by Hypothesis, and if you explicitly provide all the argu‐
204 ments Hypothesis will just call the underlying function once rather
205 than running it multiple times.
206 Installing
208 Hypothesis is available on PyPI as "hypothesis". You can install it
209 with:
210 pip install hypothesis
212 You can install the dependencies for optional extensions with e.g. pip
214 install hypothesis[pandas,django].
215 If you want to install directly from the source code (e.g. because you
217 want to make changes and install the changed version), check out the
218 instructions in CONTRIBUTING.rst.
219 Running tests
221 In our example above we just let pytest discover and run our tests, but
222 we could also have run it explicitly ourselves:
223 if __name__ == "__main__":
225 test_decode_inverts_encode()
226 We could also have done this as a python:unittest.TestCase:
228 import unittest
230 class TestEncoding(unittest.TestCase):
233 @given(text())
234 def test_decode_inverts_encode(self, s):
235 self.assertEqual(decode(encode(s)), s)
236 if __name__ == "__main__":
239 unittest.main()
240 A detail: This works because Hypothesis ignores any arguments it hasn't
242 been told to provide (positional arguments start from the right), so
243 the self argument to the test is simply ignored and works as normal.
244 This also means that Hypothesis will play nicely with other ways of pa‐
245 rameterizing tests. e.g it works fine if you use pytest fixtures for
246 some arguments and Hypothesis for others.
247 Writing tests
249 A test in Hypothesis consists of two parts: A function that looks like
250 a normal test in your test framework of choice but with some additional
251 arguments, and a @given decorator that specifies how to provide those
252 arguments.
253 Here are some other examples of how you could use that:
255 from hypothesis import given, strategies as st
257 @given(st.integers(), st.integers())
260 def test_ints_are_commutative(x, y):
261 assert x + y == y + x
262 @given(x=st.integers(), y=st.integers())
265 def test_ints_cancel(x, y):
266 assert (x + y) - y == x
267 @given(st.lists(st.integers()))
270 def test_reversing_twice_gives_same_list(xs):
271 # This will generate lists of arbitrary length (usually between 0 and
272 # 100 elements) whose elements are integers.
273 ys = list(xs)
274 ys.reverse()
275 ys.reverse()
276 assert xs == ys
277 @given(st.tuples(st.booleans(), st.text()))
280 def test_look_tuples_work_too(t):
281 # A tuple is generated as the one you provided, with the corresponding
282 # types in those positions.
283 assert len(t) == 2
284 assert isinstance(t[0], bool)
285 assert isinstance(t[1], str)
286 Note that as we saw in the above example you can pass arguments to
288 @given either as positional or as keywords.
289 Where to start
291 You should now know enough of the basics to write some tests for your
292 code using Hypothesis. The best way to learn is by doing, so go have a
293 try.
294 If you're stuck for ideas for how to use this sort of test for your
296 code, here are some good starting points:
297 1. Try just calling functions with appropriate arbitrary data and see
299 if they crash. You may be surprised how often this works. e.g. note
300 that the first bug we found in the encoding example didn't even get
301 as far as our assertion: It crashed because it couldn't handle the
302 data we gave it, not because it did the wrong thing.
303 2. Look for duplication in your tests. Are there any cases where you're
305 testing the same thing with multiple different examples? Can you
306 generalise that to a single test using Hypothesis?
307 3. This piece is designed for an F# implementation, but is still very
309 good advice which you may find helps give you good ideas for using
310 Hypothesis.
311 If you have any trouble getting started, don't feel shy about asking
313 for help.
314
316 This is an account of slightly less common Hypothesis features that you
317 don't need to get started but will nevertheless make your life easier.
318 Additional test output
320 Normally the output of a failing test will look something like:
321 Falsifying example: test_a_thing(x=1, y="foo")
323 With the repr of each keyword argument being printed.
325 Sometimes this isn't enough, either because you have a value with a
327 __repr__() method that isn't very descriptive or because you need to
328 see the output of some intermediate steps of your test. That's where
329 the note function comes in:
330 hypothesis.note(value)
332 Report this value for the minimal failing example.
333 >>> from hypothesis import given, note, strategies as st
335 >>> @given(st.lists(st.integers()), st.randoms())
336 ... def test_shuffle_is_noop(ls, r):
337 ... ls2 = list(ls)
338 ... r.shuffle(ls2)
339 ... note(f"Shuffle: {ls2!r}")
340 ... assert ls == ls2
341 ...
342 >>> try:
343 ... test_shuffle_is_noop()
344 ... except AssertionError:
345 ... print("ls != ls2")
346 ...
347 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
348 Shuffle: [1, 0]
349 ls != ls2
350 The note is printed for the minimal failing example of the test in or‐
352 der to include any additional information you might need in your test.
353 Test statistics
355 If you are using pytest you can see a number of statistics about the
356 executed tests by passing the command line argument --hypothe‐
357 sis-show-statistics. This will include some general statistics about
358 the test:
359 For example if you ran the following with --hypothesis-show-statistics:
361 from hypothesis import given, strategies as st
363 @given(st.integers())
366 def test_integers(i):
367 pass
368 You would see:
370 - during generate phase (0.06 seconds):
372 - Typical runtimes: < 1ms, ~ 47% in data generation
373 - 100 passing examples, 0 failing examples, 0 invalid examples
374 - Stopped because settings.max_examples=100
375 The final "Stopped because" line is particularly important to note: It
377 tells you the setting value that determined when the test should stop
378 trying new examples. This can be useful for understanding the behaviour
379 of your tests. Ideally you'd always want this to be max_examples.
380 In some cases (such as filtered and recursive strategies) you will see
382 events mentioned which describe some aspect of the data generation:
383 from hypothesis import given, strategies as st
385 @given(st.integers().filter(lambda x: x % 2 == 0))
388 def test_even_integers(i):
389 pass
390 You would see something like:
392 test_even_integers:
394 - during generate phase (0.08 seconds):
396 - Typical runtimes: < 1ms, ~ 57% in data generation
397 - 100 passing examples, 0 failing examples, 12 invalid examples
398 - Events:
399 * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
400 * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
401 - Stopped because settings.max_examples=100
402 You can also mark custom events in a test using the event function:
404 hypothesis.event(value)
406 Record an event that occurred this test. Statistics on number of
407 test runs with each event will be reported at the end if you run
408 Hypothesis in statistics reporting mode.
409 Events should be strings or convertible to them.
411 from hypothesis import event, given, strategies as st
413 @given(st.integers().filter(lambda x: x % 2 == 0))
416 def test_even_integers(i):
417 event(f"i mod 3 = {i%3}")
418 You will then see output like:
420 test_even_integers:
422 - during generate phase (0.09 seconds):
424 - Typical runtimes: < 1ms, ~ 59% in data generation
425 - 100 passing examples, 0 failing examples, 32 invalid examples
426 - Events:
427 * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
428 * 31.06%, i mod 3 = 2
429 * 28.79%, i mod 3 = 0
430 * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
431 * 15.91%, i mod 3 = 1
432 - Stopped because settings.max_examples=100
433 Arguments to event can be any hashable type, but two events will be
435 considered the same if they are the same when converted to a string
436 with python:str.
437 Making assumptions
439 Sometimes Hypothesis doesn't give you exactly the right sort of data
440 you want - it's mostly of the right shape, but some examples won't work
441 and you don't want to care about them. You can just ignore these by
442 aborting the test early, but this runs the risk of accidentally testing
443 a lot less than you think you are. Also it would be nice to spend less
444 time on bad examples - if you're running 100 examples per test (the de‐
445 fault) and it turns out 70 of those examples don't match your needs,
446 that's a lot of wasted time.
447 hypothesis.assume(condition)
449 Calling assume is like an assert that marks the example as bad,
450 rather than failing the test.
451 This allows you to specify properties that you assume will be
453 true, and let Hypothesis try to avoid similar examples in fu‐
454 ture.
455 For example suppose you had the following test:
457 @given(floats())
459 def test_negation_is_self_inverse(x):
460 assert x == -(-x)
461 Running this gives us:
463 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
465 AssertionError
466 This is annoying. We know about NaN and don't really care about it, but
468 as soon as Hypothesis finds a NaN example it will get distracted by
469 that and tell us about it. Also the test will fail and we want it to
470 pass.
471 So let's block off this particular example:
473 from math import isnan
475 @given(floats())
478 def test_negation_is_self_inverse_for_non_nan(x):
479 assume(not isnan(x))
480 assert x == -(-x)
481 And this passes without a problem.
483 In order to avoid the easy trap where you assume a lot more than you
485 intended, Hypothesis will fail a test when it can't find enough exam‐
486 ples passing the assumption.
487 If we'd written:
489 @given(floats())
491 def test_negation_is_self_inverse_for_non_nan(x):
492 assume(False)
493 assert x == -(-x)
494 Then on running we'd have got the exception:
496 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
498 How good is assume?
500 Hypothesis has an adaptive exploration strategy to try to avoid things
501 which falsify assumptions, which should generally result in it still
502 being able to find examples in hard to find situations.
503 Suppose we had the following:
505 @given(lists(integers()))
507 def test_sum_is_positive(xs):
508 assert sum(xs) > 0
509 Unsurprisingly this fails and gives the falsifying example [].
511 Adding assume(xs) to this removes the trivial empty example and gives
513 us [0].
514 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
516 of positive integers is positive.
517 The reason that this should be surprising is not that it doesn't find a
519 counter-example, but that it finds enough examples at all.
520 In order to make sure something interesting is happening, suppose we
522 wanted to try this for long lists. e.g. suppose we added an as‐
523 sume(len(xs) > 10) to it. This should basically never find an example:
524 a naive strategy would find fewer than one in a thousand examples, be‐
525 cause if each element of the list is negative with probability
526 one-half, you'd have to have ten of these go the right way by chance.
527 In the default configuration Hypothesis gives up long before it's tried
528 1000 examples (by default it tries 200).
529 Here's what happens if we try to run this:
531 @given(lists(integers()))
533 def test_sum_is_positive(xs):
534 assume(len(xs) > 10)
535 assume(all(x > 0 for x in xs))
536 print(xs)
537 assert sum(xs) > 0
538 In: test_sum_is_positive()
540 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
542 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
543 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
544 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
545 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
546 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
547 As you can see, Hypothesis doesn't find many examples here, but it
549 finds some - enough to keep it happy.
550 In general if you can shape your strategies better to your tests you
552 should - for example integers(1, 1000) is a lot better than assume(1 <=
553 x <= 1000), but assume will take you a long way if you can't.
554 Defining strategies
556 The type of object that is used to explore the examples given to your
557 test function is called a SearchStrategy. These are created using the
558 functions exposed in the hypothesis.strategies module.
559 Many of these strategies expose a variety of arguments you can use to
561 customize generation. For example for integers you can specify min and
562 max values of integers you want. If you want to see exactly what a
563 strategy produces you can ask for an example:
564 >>> integers(min_value=0, max_value=10).example()
566 1
567 Many strategies are built out of other strategies. For example, if you
569 want to define a tuple you need to say what goes in each element:
570 >>> from hypothesis.strategies import tuples
572 >>> tuples(integers(), integers()).example()
573 (-24597, 12566)
574 Further details are available in a separate document.
576 The gory details of given parameters
578 hypothesis.given(*_given_arguments, **_given_kwargs)
579 A decorator for turning a test function that accepts arguments
580 into a randomized test.
581 This is the main entry point to Hypothesis.
583 The @given decorator may be used to specify which arguments of a func‐
585 tion should be parametrized over. You can use either positional or key‐
586 word arguments, but not a mixture of both.
587 For example all of the following are valid uses:
589 @given(integers(), integers())
591 def a(x, y):
592 pass
593 @given(integers())
596 def b(x, y):
597 pass
598 @given(y=integers())
601 def c(x, y):
602 pass
603 @given(x=integers())
606 def d(x, y):
607 pass
608 @given(x=integers(), y=integers())
611 def e(x, **kwargs):
612 pass
613 @given(x=integers(), y=integers())
616 def f(x, *args, **kwargs):
617 pass
618 class SomeTest(TestCase):
621 @given(integers())
622 def test_a_thing(self, x):
623 pass
624 The following are not:
626 @given(integers(), integers(), integers())
628 def g(x, y):
629 pass
630 @given(integers())
633 def h(x, *args):
634 pass
635 @given(integers(), x=integers())
638 def i(x, y):
639 pass
640 @given()
643 def j(x, y):
644 pass
645 The rules for determining what are valid uses of given are as follows:
647 1. You may pass any keyword argument to given.
649 2. Positional arguments to given are equivalent to the rightmost named
651 arguments for the test function.
652 3. Positional arguments may not be used if the underlying test function
654 has varargs, arbitrary keywords, or keyword-only arguments.
655 4. Functions tested with given may not have any defaults.
657 The reason for the "rightmost named arguments" behaviour is so that us‐
659 ing @given with instance methods works: self will be passed to the
660 function as normal and not be parametrized over.
661 The function returned by given has all the same arguments as the origi‐
663 nal test, minus those that are filled in by @given. Check the notes on
664 framework compatibility to see how this affects other testing libraries
665 you may be using.
666 Targeted example generation
668 Targeted property-based testing combines the advantages of both
669 search-based and property-based testing. Instead of being completely
670 random, T-PBT uses a search-based component to guide the input genera‐
671 tion towards values that have a higher probability of falsifying a
672 property. This explores the input space more effectively and requires
673 fewer tests to find a bug or achieve a high confidence in the system
674 being tested than random PBT. (Löscher and Sagonas)
675 This is not always a good idea - for example calculating the search
677 metric might take time better spent running more uniformly-random test
678 cases, or your target metric might accidentally lead Hypothesis away
679 from bugs - but if there is a natural metric like "floating-point er‐
680 ror", "load factor" or "queue length", we encourage you to experiment
681 with targeted testing.
682 hypothesis.target(observation, *, label='')
684 Calling this function with an int or float observation gives it
685 feedback with which to guide our search for inputs that will
686 cause an error, in addition to all the usual heuristics. Obser‐
687 vations must always be finite.
688 Hypothesis will try to maximize the observed value over several
690 examples; almost any metric will work so long as it makes sense
691 to increase it. For example, -abs(error) is a metric that in‐
692 creases as error approaches zero.
693 Example metrics:
695 • Number of elements in a collection, or tasks in a queue
697 • Mean or maximum runtime of a task (or both, if you use label)
699 • Compression ratio for data (perhaps per-algorithm or
701 per-level)
702 • Number of steps taken by a state machine
704 The optional label argument can be used to distinguish between
706 and therefore separately optimise distinct observations, such as
707 the mean and standard deviation of a dataset. It is an error to
708 call target() with any label more than once per test case.
709 NOTE:
711 The more examples you run, the better this technique works.
712 As a rule of thumb, the targeting effect is noticeable above
714 max_examples=1000, and immediately obvious by around ten
715 thousand examples per label used by your test.
716 Test statistics include the best score seen for each label,
718 which can help avoid the threshold problem when the minimal ex‐
719 ample shrinks right down to the threshold of failure (issue
720 #2180).
721 We recommend that users also skim the papers introducing targeted PBT;
723 from ISSTA 2017 and ICST 2018. For the curious, the initial implemen‐
724 tation in Hypothesis uses hill-climbing search via a mutating fuzzer,
725 with some tactics inspired by simulated annealing to avoid getting
726 stuck and endlessly mutating a local maximum.
727 Custom function execution
729 Hypothesis provides you with a hook that lets you control how it runs
730 examples.
731 This lets you do things like set up and tear down around each example,
733 run examples in a subprocess, transform coroutine tests into normal
734 tests, etc. For example, TransactionTestCase in the Django extra runs
735 each example in a separate database transaction.
736 The way this works is by introducing the concept of an executor. An ex‐
738 ecutor is essentially a function that takes a block of code and run it.
739 The default executor is:
740 def default_executor(function):
742 return function()
743 You define executors by defining a method execute_example on a class.
745 Any test methods on that class with @given used on them will use
746 self.execute_example as an executor with which to run tests. For exam‐
747 ple, the following executor runs all its code twice:
748 from unittest import TestCase
750 class TestTryReallyHard(TestCase):
753 @given(integers())
754 def test_something(self, i):
755 perform_some_unreliable_operation(i)
756 def execute_example(self, f):
758 f()
759 return f()
760 Note: The functions you use in map, etc. will run inside the executor.
762 i.e. they will not be called until you invoke the function passed to
763 execute_example.
764 An executor must be able to handle being passed a function which re‐
766 turns None, otherwise it won't be able to run normal test cases. So for
767 example the following executor is invalid:
768 from unittest import TestCase
770 class TestRunTwice(TestCase):
773 def execute_example(self, f):
774 return f()()
775 and should be rewritten as:
777 from unittest import TestCase
779 class TestRunTwice(TestCase):
782 def execute_example(self, f):
783 result = f()
784 if callable(result):
785 result = result()
786 return result
787 An alternative hook is provided for use by test runner extensions such
789 as pytest-trio, which cannot use the execute_example method. This is
790 not recommended for end-users - it is better to write a complete test
791 function directly, perhaps by using a decorator to perform the same
792 transformation before applying @given.
793 @given(x=integers())
795 @pytest.mark.trio
796 async def test(x):
797 ...
798 # Illustrative code, inside the pytest-trio plugin
801 test.hypothesis.inner_test = lambda x: trio.run(test, x)
802 For authors of test runners however, assigning to the inner_test attri‐
804 bute of the hypothesis attribute of the test will replace the interior
805 test.
806 NOTE:
808 The new inner_test must accept and pass through all the *args and
809 **kwargs expected by the original test.
810 If the end user has also specified a custom executor using the exe‐
812 cute_example method, it - and all other execution-time logic - will be
813 applied to the new inner test assigned by the test runner.
814 Making random code deterministic
816 While Hypothesis' example generation can be used for nondeterministic
817 tests, debugging anything nondeterministic is usually a very frustrat‐
818 ing exercise. To make things worse, our example shrinking relies on
819 the same input causing the same failure each time - though we show the
820 un-shrunk failure and a decent error message if it doesn't.
821 By default, Hypothesis will handle the global random and numpy.random
823 random number generators for you, and you can register others:
824 hypothesis.register_random(r)
826 Register the given Random instance for management by Hypothesis.
827 You can pass random.Random instances (or other objects with
829 seed, getstate, and setstate methods) to register_random(r) to
830 have their states seeded and restored in the same way as the
831 global PRNGs from the random and numpy.random modules.
832 All global PRNGs, from e.g. simulation or scheduling frameworks,
834 should be registered to prevent flaky tests. Hypothesis will
835 ensure that the PRNG state is consistent for all test runs, or
836 reproducibly varied if you choose to use the random_module()
837 strategy.
838 Inferred strategies
840 In some cases, Hypothesis can work out what to do when you omit argu‐
841 ments. This is based on introspection, not magic, and therefore has
842 well-defined limits.
843 builds() will check the signature of the target (using signature()).
845 If there are required arguments with type annotations and no strategy
846 was passed to builds(), from_type() is used to fill them in. You can
847 also pass the special value hypothesis.infer as a keyword argument, to
848 force this inference for arguments with a default value.
849 >>> def func(a: int, b: str):
851 ... return [a, b]
852 ...
853 >>> builds(func).example()
854 [-6993, '']
855 hypothesis.infer
857 @given does not perform any implicit inference for required arguments,
859 as this would break compatibility with pytest fixtures. infer can be
860 used as a keyword argument to explicitly fill in an argument from its
861 type annotation.
862 @given(a=infer)
864 def test(a: int):
865 pass
866 # is equivalent to
869 @given(a=integers())
870 def test(a):
871 pass
872 Limitations
874 Hypothesis does not inspect PEP 484 type comments at runtime. While
875 from_type() will work as usual, inference in builds() and @given will
876 only work if you manually create the __annotations__ attribute (e.g. by
877 using @annotations(...) and @returns(...) decorators).
878 The python:typing module changes between different Python releases, in‐
880 cluding at minor versions. These are all supported on a best-effort
881 basis, but you may encounter problems. Please report them to us, and
882 consider updating to a newer version of Python as a workaround.
883 Type annotations in Hypothesis
885 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
886 patible tool, the type checker should automatically pick up our type
887 hints.
888 NOTE:
890 Hypothesis' type hints may make breaking changes between minor re‐
891 leases.
892 Upstream tools and conventions about type hints remain in flux - for
894 example the python:typing module itself is provisional, and Mypy has
895 not yet reached version 1.0 - and we plan to support the latest ver‐
896 sion of this ecosystem, as well as older versions where practical.
897 We may also find more precise ways to describe the type of various
899 interfaces, or change their type and runtime behaviour together in a
900 way which is otherwise backwards-compatible. We often omit type
901 hints for deprecated features or arguments, as an additional form of
902 warning.
903 There are known issues inferring the type of examples generated by de‐
905 ferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
906 ies(). We will fix these, and require correspondingly newer versions
907 of Mypy for type hinting, as the ecosystem improves.
908 Writing downstream type hints
910 Projects that provide Hypothesis strategies and use type hints may wish
911 to annotate their strategies too. This is a supported use-case, again
912 on a best-effort provisional basis. For example:
913 def foo_strategy() -> SearchStrategy[Foo]:
915 ...
916 class hypothesis.strategies.SearchStrategy
918 SearchStrategy is the type of all strategy objects. It is a generic
920 type, and covariant in the type of the examples it creates. For exam‐
921 ple:
922 • integers() is of type SearchStrategy[int].
924 • lists(integers()) is of type SearchStrategy[List[int]].
926 • SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
928 a subtype of Animal (as seems likely).
929 WARNING:
931 SearchStrategy should only be used in type hints. Please do not in‐
932 herit from, compare to, or otherwise use it in any way outside of
933 type hints. The only supported way to construct objects of this
934 type is to use the functions provided by the hypothesis.strategies
935 module!
936 The Hypothesis pytest plugin
938 Hypothesis includes a tiny plugin to improve integration with pytest,
939 which is activated by default (but does not affect other test runners).
940 It aims to improve the integration between Hypothesis and Pytest by
941 providing extra information and convenient access to config options.
942 • pytest --hypothesis-show-statistics can be used to display test and
944 data generation statistics.
945 • pytest --hypothesis-profile=<profile name> can be used to load a set‐
947 tings profile. pytest --hypothesis-verbosity=<level name> can be
948 used to override the current verbosity level.
949 • pytest --hypothesis-seed=<an int> can be used to reproduce a failure
951 with a particular seed.
952 • pytest --hypothesis-explain can be used to temporarily enable the ex‐
954 plain phase.
955 Finally, all tests that are defined with Hypothesis automatically have
957 @pytest.mark.hypothesis applied to them. See here for information on
958 working with markers.
959 NOTE:
961 Pytest will load the plugin automatically if Hypothesis is in‐
962 stalled. You don't need to do anything at all to use it.
963 Use with external fuzzers
965 TIP:
966 Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
967 Use HypoFuzz to fuzz your whole test suite,
968 and find more bugs without more tests!
969 Sometimes, you might want to point a traditional fuzzer such as
972 python-afl, pythonfuzz, or Google's atheris (for Python and native ex‐
973 tensions) at your code. Wouldn't it be nice if you could use any of
974 your @given tests as fuzz targets, instead of converting bytestrings
975 into your objects by hand?
976 @given(st.text())
978 def test_foo(s):
979 ...
980 # This is a traditional fuzz target - call it with a bytestring,
983 # or a binary IO object, and it runs the test once.
984 fuzz_target = test_foo.hypothesis.fuzz_one_input
985 # For example:
987 fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
988 fuzz_target(io.BytesIO(...))
989 Depending on the input to fuzz_one_input, one of three things will hap‐
991 pen:
992 • If the bytestring was invalid, for example because it was too short
994 or failed a filter or assume() too many times, fuzz_one_input returns
995 None.
996 • If the bytestring was valid and the test passed, fuzz_one_input re‐
998 turns a canonicalised and pruned buffer which will replay that test
999 case. This is provided as an option to improve the performance of
1000 mutating fuzzers, but can safely be ignored.
1001 • If the test failed, i.e. raised an exception, fuzz_one_input will add
1003 the pruned buffer to the Hypothesis example database and then
1004 re-raise that exception. All you need to do to reproduce, minimize,
1005 and de-duplicate all the failures found via fuzzing is run your test
1006 suite!
1007 Note that the interpretation of both input and output bytestrings is
1009 specific to the exact version of Hypothesis you are using and the
1010 strategies given to the test, just like the example database and @re‐
1011 produce_failure decorator.
1012 Interaction with settings
1014 fuzz_one_input uses just enough of Hypothesis' internals to drive your
1015 test function with a fuzzer-provided bytestring, and most settings
1016 therefore have no effect in this mode. We recommend running your tests
1017 the usual way before fuzzing to get the benefits of healthchecks, as
1018 well as afterwards to replay, shrink, deduplicate, and report whatever
1019 errors were discovered.
1020 • The database setting is used by fuzzing mode - adding failures to the
1022 database to be replayed when you next run your tests is our preferred
1023 reporting mechanism and response to the 'fuzzer taming' problem.
1024 • The verbosity and stateful_step_count settings work as usual.
1026 The deadline, derandomize, max_examples, phases, print_blob, re‐
1028 port_multiple_bugs, and suppress_health_check settings do not affect
1029 fuzzing mode.
1030 Thread-Safety Policy
1032 As discussed in issue #2719, Hypothesis is not truly thread-safe and
1033 that's unlikely to change in the future. This policy therefore de‐
1034 scribes what you can expect if you use Hypothesis with multiple
1035 threads.
1036 Running tests in multiple processes, e.g. with pytest -n auto, is fully
1038 supported and we test this regularly in CI - thanks to process isola‐
1039 tion, we only need to ensure that DirectoryBasedExampleDatabase can't
1040 tread on its own toes too badly. If you find a bug here we will fix it
1041 ASAP.
1042 Running separate tests in multiple threads is not something we design
1044 or test for, and is not formally supported. That said, anecdotally it
1045 does mostly work and we would like it to keep working - we accept rea‐
1046 sonable patches and low-priority bug reports. The main risks here are
1047 global state, shared caches, and cached strategies.
1048 Using multiple threads within a single test , or running a single test
1050 simultaneously in multiple threads, makes it pretty easy to trigger in‐
1051 ternal errors. We usually accept patches for such issues unless read‐
1052 ability or single-thread performance suffer.
1053 Hypothesis assumes that tests are single-threaded, or do a suffi‐
1055 ciently-good job of pretending to be single-threaded. Tests that use
1056 helper threads internally should be OK, but the user must be careful to
1057 ensure that test outcomes are still deterministic. In particular it
1058 counts as nondeterministic if helper-thread timing changes the sequence
1059 of dynamic draws using e.g. the data().
1060 Interacting with any Hypothesis APIs from helper threads might do
1062 weird/bad things, so avoid that too - we rely on thread-local variables
1063 in a few places, and haven't explicitly tested/audited how they respond
1064 to cross-thread API calls. While data() and equivalents are the most
1065 obvious danger, other APIs might also be subtly affected.
1066
1068 Hypothesis tries to have good defaults for its behaviour, but sometimes
1069 that's not enough and you need to tweak it.
1070 The mechanism for doing this is the settings object. You can set up a
1072 @given based test to use this using a settings decorator:
1073 @given invocation is as follows:
1075 from hypothesis import given, settings
1077 @given(integers())
1080 @settings(max_examples=500)
1081 def test_this_thoroughly(x):
1082 pass
1083 This uses a settings object which causes the test to receive a much
1085 larger set of examples than normal.
1086 This may be applied either before or after the given and the results
1088 are the same. The following is exactly equivalent:
1089 from hypothesis import given, settings
1091 @settings(max_examples=500)
1094 @given(integers())
1095 def test_this_thoroughly(x):
1096 pass
1097 Available settings
1099 class hypothesis.settings(parent=None, *, max_examples=not_set, deran‐
1100 domize=not_set, database=not_set, verbosity=not_set, phases=not_set,
1101 stateful_step_count=not_set, report_multiple_bugs=not_set, sup‐
1102 press_health_check=not_set, deadline=not_set, print_blob=not_set)
1103 A settings object controls a variety of parameters that are used
1104 in falsification. These may control both the falsification
1105 strategy and the details of the data that is generated.
1106 Default values are picked up from the settings.default object
1108 and changes made there will be picked up in newly created set‐
1109 tings.
1110 database
1112 An instance of ExampleDatabase that will be used to save
1113 examples to and load previous examples from. May be None
1114 in which case no storage will be used.
1115 See the example database documentation for a list of
1117 built-in example database implementations, and how to de‐
1118 fine custom implementations.
1119 default value: (dynamically calculated)
1121 deadline
1123 If set, a duration (as timedelta, or integer or float
1124 number of milliseconds) that each individual example
1125 (i.e. each time your test function is called, not the
1126 whole decorated test) within a test is not allowed to ex‐
1127 ceed. Tests which take longer than that may be converted
1128 into errors (but will not necessarily be if close to the
1129 deadline, to allow some variability in test run time).
1130 Set this to None to disable this behaviour entirely.
1132 default value: timedelta(milliseconds=200)
1134 derandomize
1136 If True, seed Hypothesis' random number generator using a
1137 hash of the test function, so that every run will test
1138 the same set of examples until you update Hypothesis,
1139 Python, or the test function.
1140 This allows you to check for regressions and look for
1142 bugs using separate settings profiles - for example run‐
1143 ning quick deterministic tests on every commit, and a
1144 longer non-deterministic nightly testing run.
1145 default value: False
1147 max_examples
1149 Once this many satisfying examples have been considered
1150 without finding any counter-example, falsification will
1151 terminate.
1152 The default value is chosen to suit a workflow where the
1154 test will be part of a suite that is regularly executed
1155 locally or on a CI server, balancing total running time
1156 against the chance of missing a bug.
1157 If you are writing one-off tests, running tens of thou‐
1159 sands of examples is quite reasonable as Hypothesis may
1160 miss uncommon bugs with default settings. For very com‐
1161 plex code, we have observed Hypothesis finding novel bugs
1162 after several million examples while testing SymPy. If
1163 you are running more than 100k examples for a test, con‐
1164 sider using our integration for coverage-guided fuzzing -
1165 it really shines when given minutes or hours to run.
1166 default value: 100
1168 phases Control which phases should be run. See the full documen‐
1170 tation for more details
1171 default value: (Phase.explicit, Phase.reuse, Phase.gener‐
1173 ate, Phase.target, Phase.shrink)
1174 print_blob
1176 If set to True, Hypothesis will print code for failing
1177 examples that can be used with @reproduce_failure to re‐
1178 produce the failing example. The default is True if the
1179 CI or TF_BUILD env vars are set, False otherwise.
1180 default value: (dynamically calculated)
1182 report_multiple_bugs
1184 Because Hypothesis runs the test many times, it can some‐
1185 times find multiple bugs in a single run. Reporting all
1186 of them at once is usually very useful, but replacing the
1187 exceptions can occasionally clash with debuggers. If
1188 disabled, only the exception with the smallest minimal
1189 example is raised.
1190 default value: True
1192 stateful_step_count
1194 Number of steps to run a stateful program for before giv‐
1195 ing up on it breaking.
1196 default value: 50
1198 suppress_health_check
1200 A list of HealthCheck items to disable.
1201 default value: ()
1203 verbosity
1205 Control the verbosity level of Hypothesis messages
1206 default value: Verbosity.normal
1208 Controlling what runs
1210 Hypothesis divides tests into logically distinct phases:
1211 1. Running explicit examples provided with the @example decorator.
1213 2. Rerunning a selection of previously failing examples to reproduce a
1215 previously seen error
1216 3. Generating new examples.
1218 4. Mutating examples for targeted property-based testing.
1220 5. Attempting to shrink an example found in previous phases (other than
1222 phase 1 - explicit examples cannot be shrunk). This turns poten‐
1223 tially large and complicated examples which may be hard to read into
1224 smaller and simpler ones.
1225 6. Attempting to explain the cause of the failure, by identifying sus‐
1227 picious lines of code (e.g. the earliest lines which are never run
1228 on passing inputs, and always run on failures). This relies on
1229 python:sys.settrace(), and is therefore automatically disabled on
1230 PyPy or if you are using coverage or a debugger. If there are no
1231 clearly suspicious lines of code, we refuse the temptation to guess.
1232 The phases setting provides you with fine grained control over which of
1234 these run, with each phase corresponding to a value on the Phase enum:
1235 class hypothesis.Phase
1237 1. Phase.explicit controls whether explicit examples are run.
1239 2. Phase.reuse controls whether previous examples will be reused.
1241 3. Phase.generate controls whether new examples will be generated.
1243 4. Phase.target controls whether examples will be mutated for target‐
1245 ing.
1246 5. Phase.shrink controls whether examples will be shrunk.
1248 6. Phase.explain controls whether Hypothesis attempts to explain test
1250 failures.
1251 The phases argument accepts a collection with any subset of these. e.g.
1253 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1254 ples and shrink them, but will not run explicit examples or reuse pre‐
1255 vious failures, while settings(phases=[Phase.explicit]) will only run
1256 the explicit examples.
1257 Seeing intermediate result
1259 To see what's going on while Hypothesis runs your tests, you can turn
1260 up the verbosity setting.
1261 >>> from hypothesis import find, settings, Verbosity
1263 >>> from hypothesis.strategies import lists, integers
1264 >>> @given(lists(integers()))
1265 ... @settings(verbosity=Verbosity.verbose)
1266 ... def f(x):
1267 ... assert not any(x)
1268 ... f()
1269 Trying example: []
1270 Falsifying example: [-1198601713, -67, 116, -29578]
1271 Shrunk example to [-1198601713]
1272 Shrunk example to [-1198601600]
1273 Shrunk example to [-1191228800]
1274 Shrunk example to [-8421504]
1275 Shrunk example to [-32896]
1276 Shrunk example to [-128]
1277 Shrunk example to [64]
1278 Shrunk example to [32]
1279 Shrunk example to [16]
1280 Shrunk example to [8]
1281 Shrunk example to [4]
1282 Shrunk example to [3]
1283 Shrunk example to [2]
1284 Shrunk example to [1]
1285 [1]
1286 The four levels are quiet, normal, verbose and debug. normal is the de‐
1288 fault, while in quiet mode Hypothesis will not print anything out, not
1289 even the final falsifying example. debug is basically verbose but a bit
1290 more so. You probably don't want it.
1291 If you are using pytest, you may also need to disable output capturing
1293 for passing tests.
1294 Building settings objects
1296 Settings can be created by calling settings with any of the available
1297 settings values. Any absent ones will be set to defaults:
1298 >>> from hypothesis import settings
1300 >>> settings().max_examples
1301 100
1302 >>> settings(max_examples=10).max_examples
1303 10
1304 You can also pass a 'parent' settings object as the first argument, and
1306 any settings you do not specify as keyword arguments will be copied
1307 from the parent settings:
1308 >>> parent = settings(max_examples=10)
1310 >>> child = settings(parent, deadline=None)
1311 >>> parent.max_examples == child.max_examples == 10
1312 True
1313 >>> parent.deadline
1314 200
1315 >>> child.deadline is None
1316 True
1317 Default settings
1319 At any given point in your program there is a current default settings,
1320 available as settings.default. As well as being a settings object in
1321 its own right, all newly created settings objects which are not explic‐
1322 itly based off another settings are based off the default, so will in‐
1323 herit any values that are not explicitly set from it.
1324 You can change the defaults by using profiles.
1326 Settings profiles
1328 Depending on your environment you may want different default settings.
1329 For example: during development you may want to lower the number of ex‐
1330 amples to speed up the tests. However, in a CI environment you may want
1331 more examples so you are more likely to find bugs.
1332 Hypothesis allows you to define different settings profiles. These pro‐
1334 files can be loaded at any time.
1335 static settings.register_profile(name, parent=None, **kwargs)
1337 Registers a collection of values to be used as a settings pro‐
1338 file.
1339 Settings profiles can be loaded by name - for example, you might
1341 create a 'fast' profile which runs fewer examples, keep the 'de‐
1342 fault' profile, and create a 'ci' profile that increases the
1343 number of examples and uses a different database to store fail‐
1344 ures.
1345 The arguments to this method are exactly as for settings: op‐
1347 tional parent settings, and keyword arguments for each setting
1348 that will be set differently to parent (or settings.default, if
1349 parent is None).
1350 static settings.get_profile(name)
1352 Return the profile with the given name.
1353 static settings.load_profile(name)
1355 Loads in the settings defined in the profile provided.
1356 If the profile does not exist, InvalidArgument will be raised.
1358 Any setting not defined in the profile will be the library de‐
1359 fined default for that setting.
1360 Loading a profile changes the default settings but will not change the
1362 behaviour of tests that explicitly change the settings.
1363 >>> from hypothesis import settings
1365 >>> settings.register_profile("ci", max_examples=1000)
1366 >>> settings().max_examples
1367 100
1368 >>> settings.load_profile("ci")
1369 >>> settings().max_examples
1370 1000
1371 Instead of loading the profile and overriding the defaults you can re‐
1373 trieve profiles for specific tests.
1374 >>> settings.get_profile("ci").max_examples
1376 1000
1377 Optionally, you may define the environment variable to load a profile
1379 for you. This is the suggested pattern for running your tests on CI.
1380 The code below should run in a conftest.py or any setup/initialization
1381 section of your test suite. If this variable is not defined the Hy‐
1382 pothesis defined defaults will be loaded.
1383 >>> import os
1385 >>> from hypothesis import settings, Verbosity
1386 >>> settings.register_profile("ci", max_examples=1000)
1387 >>> settings.register_profile("dev", max_examples=10)
1388 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1389 >>> settings.load_profile(os.getenv(u"HYPOTHESIS_PROFILE", "default"))
1390 If you are using the hypothesis pytest plugin and your profiles are
1392 registered by your conftest you can load one with the command line op‐
1393 tion --hypothesis-profile.
1394 $ pytest tests --hypothesis-profile <profile-name>
1396
1398 Most things should be easy to generate and everything should be possi‐
1399 ble.
1400 To support this principle Hypothesis provides strategies for most
1402 built-in types with arguments to constrain or adjust the output, as
1403 well as higher-order strategies that can be composed to generate more
1404 complex types.
1405 This document is a guide to what strategies are available for generat‐
1407 ing data and how to build them. Strategies have a variety of other im‐
1408 portant internal features, such as how they simplify, but the data they
1409 can generate is the only public part of their API.
1410 Core strategies
1412 Functions for building strategies are all available in the hypothe‐
1413 sis.strategies module. The salient functions from it are as follows:
1414 hypothesis.strategies.binary(*, min_size=0, max_size=None)
1416 Generates python:bytes.
1417 The generated python:bytes will have a length of at least
1419 min_size and at most max_size. If max_size is None there is no
1420 upper limit.
1421 Examples from this strategy shrink towards smaller strings and
1423 lower byte values.
1424 hypothesis.strategies.booleans()
1426 Returns a strategy which generates instances of python:bool.
1427 Examples from this strategy will shrink towards False (i.e.
1429 shrinking will replace True with False where possible).
1430 hypothesis.strategies.builds(target, /, *args, **kwargs)
1432 Generates values by drawing from args and kwargs and passing
1433 them to the callable (provided as the first positional argument)
1434 in the appropriate argument position.
1435 e.g. builds(target, integers(), flag=booleans()) would draw an
1437 integer i and a boolean b and call target(i, flag=b).
1438 If the callable has type annotations, they will be used to infer
1440 a strategy for required arguments that were not passed to
1441 builds. You can also tell builds to infer a strategy for an op‐
1442 tional argument by passing the special value hypothesis.infer as
1443 a keyword argument to builds, instead of a strategy for that ar‐
1444 gument to the callable.
1445 If the callable is a class defined with attrs, missing required
1447 arguments will be inferred from the attribute on a best-effort
1448 basis, e.g. by checking attrs standard validators. Dataclasses
1449 are handled natively by the inference from type hints.
1450 Examples from this strategy shrink by shrinking the argument
1452 values to the callable.
1453 hypothesis.strategies.characters(*, whitelist_categories=None, black‐
1455 list_categories=None, blacklist_characters=None, min_codepoint=None,
1456 max_codepoint=None, whitelist_characters=None)
1457 Generates characters, length-one python:strings, following spec‐
1458 ified filtering rules.
1459 • When no filtering rules are specified, any character can be
1461 produced.
1462 • If min_codepoint or max_codepoint is specified, then only
1464 characters having a codepoint in that range will be produced.
1465 • If whitelist_categories is specified, then only characters
1467 from those Unicode categories will be produced. This is a fur‐
1468 ther restriction, characters must also satisfy min_codepoint
1469 and max_codepoint.
1470 • If blacklist_categories is specified, then any character from
1472 those categories will not be produced. Any overlap between
1473 whitelist_categories and blacklist_categories will raise an
1474 exception, as each character can only belong to a single
1475 class.
1476 • If whitelist_characters is specified, then any additional
1478 characters in that list will also be produced.
1479 • If blacklist_characters is specified, then any characters in
1481 that list will be not be produced. Any overlap between
1482 whitelist_characters and blacklist_characters will raise an
1483 exception.
1484 The _codepoint arguments must be integers between zero and
1486 python:sys.maxunicode. The _characters arguments must be col‐
1487 lections of length-one unicode strings, such as a unicode
1488 string.
1489 The _categories arguments must be used to specify either the
1491 one-letter Unicode major category or the two-letter Unicode
1492 general category. For example, ('Nd', 'Lu') signifies "Number,
1493 decimal digit" and "Letter, uppercase". A single letter ('major
1494 category') can be given to match all corresponding categories,
1495 for example 'P' for characters in any punctuation category.
1496 Examples from this strategy shrink towards the codepoint for
1498 '0', or the first allowable codepoint after it if '0' is ex‐
1499 cluded.
1500 hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magni‐
1502 tude=None, allow_infinity=None, allow_nan=None)
1503 Returns a strategy that generates complex numbers.
1504 This strategy draws complex numbers with constrained magnitudes.
1506 The min_magnitude and max_magnitude parameters should be
1507 non-negative Real numbers; a value of None corresponds an infi‐
1508 nite upper bound.
1509 If min_magnitude is nonzero or max_magnitude is finite, it is an
1511 error to enable allow_nan. If max_magnitude is finite, it is an
1512 error to enable allow_infinity.
1513 The magnitude constraints are respected up to a relative error
1515 of (around) floating-point epsilon, due to implementation via
1516 the system sqrt function.
1517 Examples from this strategy shrink by shrinking their real and
1519 imaginary parts, as floats().
1520 If you need to generate complex numbers with particular real and
1522 imaginary parts or relationships between parts, consider using
1523 builds(complex, ...) or @composite respectively.
1524 hypothesis.strategies.composite(f)
1526 Defines a strategy that is built out of potentially arbitrarily
1527 many other strategies.
1528 This is intended to be used as a decorator. See the full docu‐
1530 mentation for more details about how to use this function.
1531 Examples from this strategy shrink by shrinking the output of
1533 each draw call.
1534 hypothesis.strategies.data()
1536 This isn't really a normal strategy, but instead gives you an
1537 object which can be used to draw data interactively from other
1538 strategies.
1539 See the rest of the documentation for more complete information.
1541 Examples from this strategy do not shrink (because there is only
1543 one), but the result of calls to each draw() call shrink as they
1544 normally would.
1545 class hypothesis.strategies.DataObject(data)
1547 This type only exists so that you can write type hints for tests
1548 using the data() strategy. Do not use it directly!
1549 hypothesis.strategies.dates(min_value=datetime.date.min,
1551 max_value=datetime.date.max)
1552 A strategy for dates between min_value and max_value.
1553 Examples from this strategy shrink towards January 1st 2000.
1555 hypothesis.strategies.datetimes(min_value=datetime.datetime.min,
1557 max_value=datetime.datetime.max, *, timezones=none(), allow_imagi‐
1558 nary=True)
1559 A strategy for generating datetimes, which may be time‐
1560 zone-aware.
1561 This strategy works by drawing a naive datetime between
1563 min_value and max_value, which must both be naive (have no time‐
1564 zone).
1565 timezones must be a strategy that generates either None, for
1567 naive datetimes, or tzinfo objects for 'aware' datetimes. You
1568 can construct your own, though we recommend using the dateutil
1569 package and hypothesis.extra.dateutil.timezones() strategy, and
1570 also provide hypothesis.extra.pytz.timezones().
1571 You may pass allow_imaginary=False to filter out "imaginary"
1573 datetimes which did not (or will not) occur due to daylight sav‐
1574 ings, leap seconds, timezone and calendar adjustments, etc.
1575 Imaginary datetimes are allowed by default, because malformed
1576 timestamps are a common source of bugs.
1577 Examples from this strategy shrink towards midnight on January
1579 1st 2000, local time.
1580 hypothesis.strategies.decimals(min_value=None, max_value=None, *, al‐
1582 low_nan=None, allow_infinity=None, places=None)
1583 Generates instances of python:decimal.Decimal, which may be:
1584 • A finite rational number, between min_value and max_value.
1586 • Not a Number, if allow_nan is True. None means "allow NaN,
1588 unless min_value and max_value are not None".
1589 • Positive or negative infinity, if max_value and min_value re‐
1591 spectively are None, and allow_infinity is not False. None
1592 means "allow infinity, unless excluded by the min and max val‐
1593 ues".
1594 Note that where floats have one NaN value, Decimals have four:
1596 signed, and either quiet or signalling. See the decimal module
1597 docs for more information on special values.
1598 If places is not None, all finite values drawn from the strategy
1600 will have that number of digits after the decimal place.
1601 Examples from this strategy do not have a well defined shrink
1603 order but try to maximize human readability when shrinking.
1604 hypothesis.strategies.deferred(definition)
1606 A deferred strategy allows you to write a strategy that refer‐
1607 ences other strategies that have not yet been defined. This al‐
1608 lows for the easy definition of recursive and mutually recursive
1609 strategies.
1610 The definition argument should be a zero-argument function that
1612 returns a strategy. It will be evaluated the first time the
1613 strategy is used to produce an example.
1614 Example usage:
1616 >>> import hypothesis.strategies as st
1618 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1619 >>> x.example()
1620 (((False, (True, True)), (False, True)), (True, True))
1621 >>> x.example()
1622 True
1623 Mutual recursion also works fine:
1625 >>> a = st.deferred(lambda: st.booleans() | b)
1627 >>> b = st.deferred(lambda: st.tuples(a, a))
1628 >>> a.example()
1629 True
1630 >>> b.example()
1631 (False, (False, ((False, True), False)))
1632 Examples from this strategy shrink as they normally would from
1634 the strategy returned by the definition.
1635 hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class
1637 'dict'>, min_size=0, max_size=None)
1638 Generates dictionaries of type dict_class with keys drawn from
1639 the keys argument and values drawn from the values argument.
1640 The size parameters have the same interpretation as for lists().
1642 Examples from this strategy shrink by trying to remove keys from
1644 the generated dictionary, and by shrinking each generated key
1645 and value.
1646 class hypothesis.strategies.DrawFn
1648 This type only exists so that you can write type hints for func‐
1649 tions decorated with @composite.
1650 @composite
1652 def list_and_index(draw: DrawFn) -> Tuple[int, str]:
1653 i = draw(integers()) # type inferred as 'int'
1654 s = draw(text()) # type inferred as 'str'
1655 hypothesis.strategies.emails()
1657 A strategy for generating email addresses as unicode strings.
1658 The address format is specified in RFC 5322#section-3.4.1. Val‐
1659 ues shrink towards shorter local-parts and host domains.
1660 This strategy is useful for generating "user data" for tests, as
1662 mishandling of email addresses is a common source of bugs.
1663 hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)
1665 Generates a dictionary of the same type as mapping with a fixed
1666 set of keys mapping to strategies. mapping must be a dict sub‐
1667 class.
1668 Generated values have all keys present in mapping, in iteration
1670 order, with the corresponding values drawn from mapping[key].
1671 If optional is passed, the generated value may or may not con‐
1673 tain each key from optional and a value drawn from the corre‐
1674 sponding strategy. Generated values may contain optional keys
1675 in an arbitrary order.
1676 Examples from this strategy shrink by shrinking each individual
1678 value in the generated dictionary, and omitting optional
1679 key-value pairs.
1680 hypothesis.strategies.floats(min_value=None, max_value=None, *, al‐
1682 low_nan=None, allow_infinity=None, allow_subnormal=None, width=64, ex‐
1683 clude_min=False, exclude_max=False)
1684 Returns a strategy which generates floats.
1685 • If min_value is not None, all values will be >= min_value (or
1687 > min_value if exclude_min).
1688 • If max_value is not None, all values will be <= max_value (or
1690 < max_value if exclude_max).
1691 • If min_value or max_value is not None, it is an error to en‐
1693 able allow_nan.
1694 • If both min_value and max_value are not None, it is an error
1696 to enable allow_infinity.
1697 • If inferred values range does not include subnormal values, it
1699 is an error to enable allow_subnormal.
1700 Where not explicitly ruled out by the bounds, subnormals, in‐
1702 finities, and NaNs are possible values generated by this strat‐
1703 egy.
1704 The width argument specifies the maximum number of bits of pre‐
1706 cision required to represent the generated float. Valid values
1707 are 16, 32, or 64. Passing width=32 will still use the builtin
1708 64-bit float class, but always for values which can be exactly
1709 represented as a 32-bit float.
1710 The exclude_min and exclude_max argument can be used to generate
1712 numbers from open or half-open intervals, by excluding the re‐
1713 spective endpoints. Excluding either signed zero will also ex‐
1714 clude the other. Attempting to exclude an endpoint which is
1715 None will raise an error; use allow_infinity=False to generate
1716 finite floats. You can however use e.g. min_value=-math.inf,
1717 exclude_min=True to exclude only one infinite endpoint.
1718 Examples from this strategy have a complicated and hard to ex‐
1720 plain shrinking behaviour, but it tries to improve "human read‐
1721 ability". Finite numbers will be preferred to infinity and in‐
1722 finity will be preferred to NaN.
1723 hypothesis.strategies.fractions(min_value=None, max_value=None, *,
1725 max_denominator=None)
1726 Returns a strategy which generates Fractions.
1727 If min_value is not None then all generated values are no less
1729 than min_value. If max_value is not None then all generated
1730 values are no greater than max_value. min_value and max_value
1731 may be anything accepted by the Fraction constructor.
1732 If max_denominator is not None then the denominator of any gen‐
1734 erated values is no greater than max_denominator. Note that
1735 max_denominator must be None or a positive integer.
1736 Examples from this strategy shrink towards smaller denominators,
1738 then closer to zero.
1739 hypothesis.strategies.from_regex(regex, *, fullmatch=False)
1741 Generates strings that contain a match for the given regex (i.e.
1742 ones for which python:re.search() will return a non-None re‐
1743 sult).
1744 regex may be a pattern or compiled regex. Both byte-strings and
1746 unicode strings are supported, and will generate examples of the
1747 same type.
1748 You can use regex flags such as python:re.IGNORECASE or
1750 python:re.DOTALL to control generation. Flags can be passed ei‐
1751 ther in compiled regex or inside the pattern with a (?iLmsux)
1752 group.
1753 Some regular expressions are only partly supported - the under‐
1755 lying strategy checks local matching and relies on filtering to
1756 resolve context-dependent expressions. Using too many of these
1757 constructs may cause health-check errors as too many examples
1758 are filtered out. This mainly includes (positive or negative)
1759 lookahead and lookbehind groups.
1760 If you want the generated string to match the whole regex you
1762 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1763 gle character string, while "." will return any string, and
1764 r"\A.$" will return a single character optionally followed by a
1765 "\n". Alternatively, passing fullmatch=True will ensure that
1766 the whole string is a match, as if you had used the \A and \Z
1767 markers.
1768 Examples from this strategy shrink towards shorter strings and
1770 lower character values, with exact behaviour that may depend on
1771 the pattern.
1772 hypothesis.strategies.from_type(thing)
1774 Looks up the appropriate search strategy for the given type.
1775 from_type is used internally to fill in missing arguments to
1777 builds() and can be used interactively to explore what strate‐
1778 gies are available or to debug type resolution.
1779 You can use register_type_strategy() to handle your custom
1781 types, or to globally redefine certain strategies - for example
1782 excluding NaN from floats, or use timezone-aware instead of
1783 naive time and datetime strategies.
1784 The resolution logic may be changed in a future version, but
1786 currently tries these five options:
1787 1. If thing is in the default lookup mapping or user-registered
1789 lookup, return the corresponding strategy. The default
1790 lookup covers all types with Hypothesis strategies, including
1791 extras where possible.
1792 2. If thing is from the python:typing module, return the corre‐
1794 sponding strategy (special logic).
1795 3. If thing has one or more subtypes in the merged lookup, re‐
1797 turn the union of the strategies for those types that are not
1798 subtypes of other elements in the lookup.
1799 4. Finally, if thing has type annotations for all required argu‐
1801 ments, and is not an abstract class, it is resolved via
1802 builds().
1803 5. Because abstract types cannot be instantiated, we treat ab‐
1805 stract types as the union of their concrete subclasses. Note
1806 that this lookup works via inheritance but not via register,
1807 so you may still need to use register_type_strategy().
1808 There is a valuable recipe for leveraging from_type() to gener‐
1810 ate "everything except" values from a specified type. I.e.
1811 def everything_except(excluded_types):
1813 return (
1814 from_type(type)
1815 .flatmap(from_type)
1816 .filter(lambda x: not isinstance(x, excluded_types))
1817 )
1818 For example, everything_except(int) returns a strategy that can
1820 generate anything that from_type() can ever generate, except for
1821 instances of python:int, and excluding instances of types added
1822 via register_type_strategy().
1823 This is useful when writing tests which check that invalid input
1825 is rejected in a certain way.
1826 hypothesis.strategies.frozensets(elements, *, min_size=0,
1828 max_size=None)
1829 This is identical to the sets function but instead returns
1830 frozensets.
1831 hypothesis.strategies.functions(*, like=lambda : ..., returns=infer,
1833 pure=False)
1834 A strategy for functions, which can be used in callbacks.
1835 The generated functions will mimic the interface of like, which
1837 must be a callable (including a class, method, or function).
1838 The return value for the function is drawn from the returns ar‐
1839 gument, which must be a strategy. If returns is not passed, we
1840 attempt to infer a strategy from the return-type annotation if
1841 present, falling back to none().
1842 If pure=True, all arguments passed to the generated function
1844 must be hashable, and if passed identical arguments the original
1845 return value will be returned again - not regenerated, so beware
1846 mutable values.
1847 If pure=False, generated functions do not validate their argu‐
1849 ments, and may return a different value if called again with the
1850 same arguments.
1851 Generated functions can only be called within the scope of the
1853 @given which created them. This strategy does not support .ex‐
1854 ample().
1855 hypothesis.strategies.integers(min_value=None, max_value=None)
1857 Returns a strategy which generates integers.
1858 If min_value is not None then all values will be >= min_value.
1860 If max_value is not None then all values will be <= max_value
1861 Examples from this strategy will shrink towards zero, and nega‐
1863 tive values will also shrink towards positive (i.e. -n may be
1864 replaced by +n).
1865 hypothesis.strategies.ip_addresses(*, v=None, network=None)
1867 Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Ad‐
1868 dresses, or leave unspecified to allow both versions.
1869 network may be an IPv4Network or IPv6Network, or a string repre‐
1871 senting a network such as "127.0.0.0/24" or "2001:db8::/32". As
1872 well as generating addresses within a particular routable net‐
1873 work, this can be used to generate addresses from a reserved
1874 range listed in the IANA registries.
1875 If you pass both v and network, they must be for the same ver‐
1877 sion.
1878 hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None,
1880 unique_by=None, unique=False)
1881 This has the same behaviour as lists, but returns iterables in‐
1882 stead.
1883 Some iterables cannot be indexed (e.g. sets) and some do not
1885 have a fixed length (e.g. generators). This strategy produces
1886 iterators, which cannot be indexed and do not have a fixed
1887 length. This ensures that you do not accidentally depend on se‐
1888 quence behaviour.
1889 hypothesis.strategies.just(value)
1891 Return a strategy which only generates value.
1892 Note: value is not copied. Be wary of using mutable values.
1894 If value is the result of a callable, you can use
1896 builds(callable) instead of just(callable()) to get a fresh
1897 value each time.
1898 Examples from this strategy do not shrink (because there is only
1900 one).
1901 hypothesis.strategies.lists(elements, *, min_size=0, max_size=None,
1903 unique_by=None, unique=False)
1904 Returns a list containing values drawn from elements with length
1905 in the interval [min_size, max_size] (no bounds in that direc‐
1906 tion if these are None). If max_size is 0, only the empty list
1907 will be drawn.
1908 If unique is True (or something that evaluates to True), we com‐
1910 pare direct object equality, as if unique_by was lambda x: x.
1911 This comparison only works for hashable types.
1912 If unique_by is not None it must be a callable or tuple of
1914 callables returning a hashable type when given a value drawn
1915 from elements. The resulting list will satisfy the condition
1916 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1917 If unique_by is a tuple of callables the uniqueness will be re‐
1919 spective to each callable.
1920 For example, the following will produce two columns of integers
1922 with both columns being unique respectively.
1923 >>> twoints = st.tuples(st.integers(), st.integers())
1925 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1926 Examples from this strategy shrink by trying to remove elements
1928 from the list, and by shrinking each individual element of the
1929 list.
1930 hypothesis.strategies.none()
1932 Return a strategy which only generates None.
1933 Examples from this strategy do not shrink (because there is only
1935 one).
1936 hypothesis.strategies.nothing()
1938 This strategy never successfully draws a value and will always
1939 reject on an attempt to draw.
1940 Examples from this strategy do not shrink (because there are
1942 none).
1943 hypothesis.strategies.one_of(*args)
1945 Return a strategy which generates values from any of the argu‐
1946 ment strategies.
1947 This may be called with one iterable argument instead of multi‐
1949 ple strategy arguments, in which case one_of(x) and one_of(*x)
1950 are equivalent.
1951 Examples from this strategy will generally shrink to ones that
1953 come from strategies earlier in the list, then shrink according
1954 to behaviour of the strategy that produced them. In order to get
1955 good shrinking behaviour, try to put simpler strategies first.
1956 e.g. one_of(none(), text()) is better than one_of(text(),
1957 none()).
1958 This is especially important when using recursive strategies.
1960 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1961 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1962 st.none()) will shrink very badly indeed.
1963 hypothesis.strategies.permutations(values)
1965 Return a strategy which returns permutations of the ordered col‐
1966 lection values.
1967 Examples from this strategy shrink by trying to become closer to
1969 the original order of values.
1970 hypothesis.strategies.random_module()
1972 The Hypothesis engine handles PRNG state for the stdlib and
1973 Numpy random modules internally, always seeding them to zero and
1974 restoring the previous state after the test.
1975 If having a fixed seed would unacceptably weaken your tests, and
1977 you cannot use a random.Random instance provided by randoms(),
1978 this strategy calls python:random.seed() with an arbitrary inte‐
1979 ger and passes you an opaque object whose repr displays the seed
1980 value for debugging. If numpy.random is available, that state
1981 is also managed.
1982 Examples from these strategy shrink to seeds closer to zero.
1984 hypothesis.strategies.randoms(*, note_method_calls=False, use_true_ran‐
1986 dom=False)
1987 Generates instances of random.Random. The generated Random in‐
1988 stances are of a special HypothesisRandom subclass.
1989 • If note_method_calls is set to True, Hypothesis will print the
1991 randomly drawn values in any falsifying test case. This can be
1992 helpful for debugging the behaviour of randomized algorithms.
1993 • If use_true_random is set to True then values will be drawn
1995 from their usual distribution, otherwise they will actually be
1996 Hypothesis generated values (and will be shrunk accordingly
1997 for any failing test case). Setting use_true_random=False will
1998 tend to expose bugs that would occur with very low probability
1999 when it is set to True, and this flag should only be set to
2000 True when your code relies on the distribution of values for
2001 correctness.
2002 hypothesis.strategies.recursive(base, extend, *, max_leaves=100)
2004 base: A strategy to start from.
2005 extend: A function which takes a strategy and returns a new
2007 strategy.
2008 max_leaves: The maximum number of elements to be drawn from base
2010 on a given run.
2011 This returns a strategy S such that S = extend(base | S). That
2013 is, values may be drawn from base, or from any strategy reach‐
2014 able by mixing applications of | and extend.
2015 An example may clarify: recursive(booleans(), lists) would re‐
2017 turn a strategy that may return arbitrarily nested and mixed
2018 lists of booleans. So e.g. False, [True], [False, []], and
2019 [[[[True]]]] are all valid values to be drawn from that strat‐
2020 egy.
2021 Examples from this strategy shrink by trying to reduce the
2023 amount of recursion and by shrinking according to the shrinking
2024 behaviour of base and the result of extend.
2025 hypothesis.strategies.register_type_strategy(custom_type, strategy)
2027 Add an entry to the global type-to-strategy lookup.
2028 This lookup is used in builds() and @given.
2030 builds() will be used automatically for classes with type anno‐
2032 tations on __init__ , so you only need to register a strategy if
2033 one or more arguments need to be more tightly defined than their
2034 type-based default, or if you want to supply a strategy for an
2035 argument with a default value.
2036 strategy may be a search strategy, or a function that takes a
2038 type and returns a strategy (useful for generic types).
2039 Note that you may not register a parametrised generic type (such
2041 as MyCollection[int]) directly, because the resolution logic
2042 does not handle this case correctly. Instead, you may register
2043 a function for MyCollection and inspect the type parameters
2044 within that function.
2045 hypothesis.strategies.runner(*, default=not_set)
2047 A strategy for getting "the current test runner", whatever that
2048 may be. The exact meaning depends on the entry point, but it
2049 will usually be the associated 'self' value for it.
2050 If there is no current test runner and a default is provided,
2052 return that default. If no default is provided, raises Invali‐
2053 dArgument.
2054 Examples from this strategy do not shrink (because there is only
2056 one).
2057 hypothesis.strategies.sampled_from(elements)
2059 Returns a strategy which generates any value present in ele‐
2060 ments.
2061 Note that as with just(), values will not be copied and thus you
2063 should be careful of using mutable data.
2064 sampled_from supports ordered collections, as well as Enum ob‐
2066 jects. Flag objects may also generate any combination of their
2067 members.
2068 Examples from this strategy shrink by replacing them with values
2070 earlier in the list. So e.g. sampled_from([10, 1]) will shrink
2071 by trying to replace 1 values with 10, and sampled_from([1, 10])
2072 will shrink by trying to replace 10 values with 1.
2073 It is an error to sample from an empty sequence, because return‐
2075 ing nothing() makes it too easy to silently drop parts of com‐
2076 pound strategies. If you need that behaviour, use sam‐
2077 pled_from(seq) if seq else nothing().
2078 hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)
2080 This has the same behaviour as lists, but returns sets instead.
2081 Note that Hypothesis cannot tell if values are drawn from ele‐
2083 ments are hashable until running the test, so you can define a
2084 strategy for sets of an unhashable type but it will fail at test
2085 time.
2086 Examples from this strategy shrink by trying to remove elements
2088 from the set, and by shrinking each individual element of the
2089 set.
2090 hypothesis.strategies.shared(base, *, key=None)
2092 Returns a strategy that draws a single shared value per run,
2093 drawn from base. Any two shared instances with the same key will
2094 share the same value, otherwise the identity of this strategy
2095 will be used. That is:
2096 >>> s = integers() # or any other strategy
2098 >>> x = shared(s)
2099 >>> y = shared(s)
2100 In the above x and y may draw different (or potentially the
2102 same) values. In the following they will always draw the same:
2103 >>> x = shared(s, key="hi")
2105 >>> y = shared(s, key="hi")
2106 Examples from this strategy shrink as per their base strategy.
2108 hypothesis.strategies.slices(size)
2110 Generates slices that will select indices up to the supplied
2111 size
2112 Generated slices will have start and stop indices that range
2114 from -size to size - 1 and will step in the appropriate direc‐
2115 tion. Slices should only produce an empty selection if the start
2116 and end are the same.
2117 Examples from this strategy shrink toward 0 and smaller values
2119 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
2121 gories=('Cs',)), *, min_size=0, max_size=None)
2122 Generates strings with characters drawn from alphabet, which
2123 should be a collection of length one strings or a strategy gen‐
2124 erating such strings.
2125 The default alphabet strategy can generate the full unicode
2127 range but excludes surrogate characters because they are invalid
2128 in the UTF-8 encoding. You can use characters() without argu‐
2129 ments to find surrogate-related bugs such as bpo-34454.
2130 min_size and max_size have the usual interpretations. Note that
2132 Python measures string length by counting codepoints: U+00C5 Å
2133 is a single character, while U+0041 U+030A Å is two - the A,
2134 and a combining ring above.
2135 Examples from this strategy shrink towards shorter strings, and
2137 with the characters in the text shrinking as per the alphabet
2138 strategy. This strategy does not normalize() examples, so gen‐
2139 erated strings may be in any or none of the 'normal forms'.
2140 hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min,
2142 max_value=datetime.timedelta.max)
2143 A strategy for timedeltas between min_value and max_value.
2144 Examples from this strategy shrink towards zero.
2146 hypothesis.strategies.times(min_value=datetime.time.min,
2148 max_value=datetime.time.max, *, timezones=none())
2149 A strategy for times between min_value and max_value.
2150 The timezones argument is handled as for datetimes().
2152 Examples from this strategy shrink towards midnight, with the
2154 timezone component shrinking as for the strategy that provided
2155 it.
2156 hypothesis.strategies.timezone_keys(*, allow_prefix=True)
2158 A strategy for IANA timezone names.
2159 As well as timezone names like "UTC", "Australia/Sydney", or
2161 "America/New_York", this strategy can generate:
2162 • Aliases such as "Antarctica/McMurdo", which links to "Pa‐
2164 cific/Auckland".
2165 • Deprecated names such as "Antarctica/South_Pole", which also
2167 links to "Pacific/Auckland". Note that most but not all dep‐
2168 recated timezone names are also aliases.
2169 • Timezone names with the "posix/" or "right/" prefixes, unless
2171 allow_prefix=False.
2172 These strings are provided separately from Tzinfo objects - such
2174 as ZoneInfo instances from the timezones() strategy - to facili‐
2175 tate testing of timezone logic without needing workarounds to
2176 access non-canonical names.
2177 NOTE:
2179 The python:zoneinfo module is new in Python 3.9, so you will
2180 need to install the backports.zoneinfo module on earlier ver‐
2181 sions.
2182 On Windows, you will also need to install the tzdata package.
2184 pip install hypothesis[zoneinfo] will install these condi‐
2186 tional dependencies if and only if they are needed.
2187 On Windows, you may need to access IANA timezone data via the
2189 tzdata package. For non-IANA timezones, such as Windows-native
2190 names or GNU TZ strings, we recommend using sampled_from() with
2191 the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().
2192 hypothesis.strategies.timezones(*, no_cache=False)
2194 A strategy for python:zoneinfo.ZoneInfo objects.
2195 If no_cache=True, the generated instances are constructed using
2197 ZoneInfo.no_cache instead of the usual constructor. This may
2198 change the semantics of your datetimes in surprising ways, so
2199 only use it if you know that you need to!
2200 NOTE:
2202 The python:zoneinfo module is new in Python 3.9, so you will
2203 need to install the backports.zoneinfo module on earlier ver‐
2204 sions.
2205 On Windows, you will also need to install the tzdata package.
2207 pip install hypothesis[zoneinfo] will install these condi‐
2209 tional dependencies if and only if they are needed.
2210 hypothesis.strategies.tuples(*args)
2212 Return a strategy which generates a tuple of the same length as
2213 args by generating the value at index i from args[i].
2214 e.g. tuples(integers(), integers()) would generate a tuple of
2216 length two with both values an integer.
2217 Examples from this strategy shrink by shrinking their component
2219 parts.
2220 hypothesis.strategies.uuids(*, version=None)
2222 Returns a strategy that generates UUIDs.
2223 If the optional version argument is given, value is passed
2225 through to UUID and only UUIDs of that version will be gener‐
2226 ated.
2227 All returned values from this will be unique, so e.g. if you do
2229 lists(uuids()) the resulting list will never contain duplicates.
2230 Examples from this strategy don't have any meaningful shrink or‐
2232 der.
2233 Provisional strategies
2235 This module contains various provisional APIs and strategies.
2236 It is intended for internal use, to ease code reuse, and is not stable.
2238 Point releases may move or break the contents at any time!
2239 Internet strategies should conform to RFC 3986 or the authoritative
2241 definitions it links to. If not, report the bug!
2242 hypothesis.provisional.domains(*, max_length=255, max_ele‐
2244 ment_length=63)
2245 Generate RFC 1035 compliant fully qualified domain names.
2246 hypothesis.provisional.urls()
2248 A strategy for RFC 3986, generating http/https URLs.
2249 Shrinking
2251 When using strategies it is worth thinking about how the data shrinks.
2252 Shrinking is the process by which Hypothesis tries to produce human
2253 readable examples when it finds a failure - it takes a complex example
2254 and turns it into a simpler one.
2255 Each strategy defines an order in which it shrinks - you won't usually
2257 need to care about this much, but it can be worth being aware of as it
2258 can affect what the best way to write your own strategies is.
2259 The exact shrinking behaviour is not a guaranteed part of the API, but
2261 it doesn't change that often and when it does it's usually because we
2262 think the new way produces nicer examples.
2263 Possibly the most important one to be aware of is one_of(), which has a
2265 preference for values produced by strategies earlier in its argument
2266 list. Most of the others should largely "do the right thing" without
2267 you having to think about it.
2268 Adapting strategies
2270 Often it is the case that a strategy doesn't produce exactly what you
2271 want it to and you need to adapt it. Sometimes you can do this in the
2272 test, but this hurts reuse because you then have to repeat the adaption
2273 in every test.
2274 Hypothesis gives you ways to build strategies from other strategies
2276 given functions for transforming the data.
2277 Mapping
2279 map is probably the easiest and most useful of these to use. If you
2280 have a strategy s and a function f, then an example s.map(f).example()
2281 is f(s.example()), i.e. we draw an example from s and then apply f to
2282 it.
2283 e.g.:
2285 >>> lists(integers()).map(sorted).example()
2287 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2288 Note that many things that you might use mapping for can also be done
2290 with builds(), and if you find yourself indexing into a tuple within
2291 .map() it's probably time to use that instead.
2292 Filtering
2294 filter lets you reject some examples. s.filter(f).example() is some ex‐
2295 ample of s such that f(example) is truthy.
2296 >>> integers().filter(lambda x: x > 11).example()
2298 26126
2299 >>> integers().filter(lambda x: x > 11).example()
2300 23324
2301 It's important to note that filter isn't magic and if your condition is
2303 too hard to satisfy then this can fail:
2304 >>> integers().filter(lambda x: False).example()
2306 Traceback (most recent call last):
2307 ...
2308 hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries
2309 In general you should try to use filter only to avoid corner cases that
2311 you don't want rather than attempting to cut out a large chunk of the
2312 search space.
2313 A technique that often works well here is to use map to first transform
2315 the data and then use filter to remove things that didn't work out. So
2316 for example if you wanted pairs of integers (x,y) such that x < y you
2317 could do the following:
2318 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2320 [-8543729478746591815, 3760495307320535691]
2321 Chaining strategies together
2323 Finally there is flatmap. flatmap draws an example, then turns that ex‐
2324 ample into a strategy, then draws an example from that strategy.
2325 It may not be obvious why you want this at first, but it turns out to
2327 be quite useful because it lets you generate different types of data
2328 with relationships to each other.
2329 For example suppose we wanted to generate a list of lists of the same
2331 length:
2332 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2334 ... lambda n: lists(lists(integers(), min_size=n, max_size=n))
2335 ... )
2336 >>> rectangle_lists.example()
2337 []
2338 >>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
2339 [[], [], [], [], [], [], [], [], [], []]
2340 >>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
2341 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2342 >>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
2343 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2344 In this example we first choose a length for our tuples, then we build
2346 a strategy which generates lists containing lists precisely of that
2347 length. The finds show what simple examples for this look like.
2348 Most of the time you probably don't want flatmap, but unlike filter and
2350 map which are just conveniences for things you could just do in your
2351 tests, flatmap allows genuinely new data generation that you wouldn't
2352 otherwise be able to easily do.
2353 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2355 don't know Haskell, ignore everything in these parentheses. You do not
2356 need to understand anything about monads to use this, or anything else
2357 in Hypothesis).
2358 Recursive data
2360 Sometimes the data you want to generate has a recursive definition.
2361 e.g. if you wanted to generate JSON data, valid JSON is:
2362 1. Any float, any boolean, any unicode string.
2364 2. Any list of valid JSON data
2366 3. Any dictionary mapping unicode strings to valid JSON data.
2368 The problem is that you cannot call a strategy recursively and expect
2370 it to not just blow up and eat all your memory. The other problem here
2371 is that not all unicode strings display consistently on different ma‐
2372 chines, so we'll restrict them in our doctest.
2373 The way Hypothesis handles this is with the recursive() strategy which
2375 you pass in a base case and a function that, given a strategy for your
2376 data type, returns a new strategy for it. So for example:
2377 >>> from string import printable
2379 ... from pprint import pprint
2380 >>> json = recursive(
2381 ... none() | booleans() | floats() | text(printable),
2382 ... lambda children: lists(children) | dictionaries(text(printable), children),
2383 ... )
2384 >>> pprint(json.example())
2385 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2386 >>> pprint(json.example())
2387 {'de(l': None,
2388 'nK': {'(Rt)': None,
2389 '+hoZh1YU]gy8': True,
2390 '8z]EIFA06^li^': 'LFE{Q',
2391 '9,': 'l{cA=/'}}
2392 That is, we start with our leaf data and then we augment it by allowing
2394 lists and dictionaries of anything we can generate as JSON data.
2395 The size control of this works by limiting the maximum number of values
2397 that can be drawn from the base strategy. So for example if we wanted
2398 to only generate really small JSON we could do this as:
2399 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2401 >>> small_lists.example()
2402 True
2403 >>> small_lists.example()
2404 [False]
2405 Composite strategies
2407 The @composite decorator lets you combine other strategies in more or
2408 less arbitrary ways. It's probably the main thing you'll want to use
2409 for complicated custom strategies.
2410 The composite decorator works by converting a function that returns one
2412 example into a function that returns a strategy that produces such ex‐
2413 amples - which you can pass to @given, modify with .map or .filter, and
2414 generally use like any other strategy.
2415 It does this by giving you a special function draw as the first argu‐
2417 ment, which can be used just like the corresponding method of the
2418 data() strategy within a test. In fact, the implementation is almost
2419 the same - but defining a strategy with @composite makes code reuse
2420 easier, and usually improves the display of failing examples.
2421 For example, the following gives you a list and an index into it:
2423 >>> @composite
2425 ... def list_and_index(draw, elements=integers()):
2426 ... xs = draw(lists(elements, min_size=1))
2427 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2428 ... return (xs, i)
2429 ...
2430 draw(s) is a function that should be thought of as returning s.exam‐
2432 ple(), except that the result is reproducible and will minimize cor‐
2433 rectly. The decorated function has the initial argument removed from
2434 the list, but will accept all the others in the expected order. De‐
2435 faults are preserved.
2436 >>> list_and_index()
2438 list_and_index()
2439 >>> list_and_index().example()
2440 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2441 >>> list_and_index(booleans())
2443 list_and_index(elements=booleans())
2444 >>> list_and_index(booleans()).example()
2445 ([True, False], 0)
2446 Note that the repr will work exactly like it does for all the built-in
2448 strategies: it will be a function that you can call to get the strategy
2449 in question, with values provided only if they do not match the de‐
2450 faults.
2451 You can use assume inside composite functions:
2453 @composite
2455 def distinct_strings_with_common_characters(draw):
2456 x = draw(text(min_size=1))
2457 y = draw(text(alphabet=x))
2458 assume(x != y)
2459 return (x, y)
2460 This works as assume normally would, filtering out any examples for
2462 which the passed in argument is falsey.
2463 Take care that your function can cope with adversarial draws, or ex‐
2465 plicitly rejects them using the .filter() method or assume() - our mu‐
2466 tation and shrinking logic can do some strange things, and a naive im‐
2467 plementation might lead to serious performance problems. For example:
2468 @composite
2470 def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
2471 # The bad way: if Hypothesis keeps generating e.g. zero,
2472 # we'll keep looping for a very long time.
2473 result = set()
2474 while len(result) < size:
2475 result.add(draw(elements))
2476 # The good way: use a filter, so Hypothesis can tell what's valid!
2477 for _ in range(size):
2478 result.add(draw(elements.filter(lambda x: x not in result)))
2479 return result
2480 If @composite is used to decorate a method or classmethod, the draw ar‐
2482 gument must come before self or cls. While we therefore recommend
2483 writing strategies as standalone functions and using the
2484 register_type_strategy() function to associate them with a class, meth‐
2485 ods are supported and the @composite decorator may be applied either
2486 before or after @classmethod or @staticmethod. See issue #2578 and
2487 pull request #2634 for more details.
2488 Drawing interactively in tests
2490 There is also the data() strategy, which gives you a means of using
2491 strategies interactively. Rather than having to specify everything up
2492 front in @given you can draw from strategies in the body of your test.
2493 This is similar to @composite, but even more powerful as it allows you
2495 to mix test code with example generation. The downside of this power
2496 is that data() is incompatible with explicit @example(...)s - and the
2497 mixed code is often harder to debug when something goes wrong.
2498 If you need values that are affected by previous draws but which don't
2500 depend on the execution of your test, stick to the simpler @composite.
2501 @given(data())
2503 def test_draw_sequentially(data):
2504 x = data.draw(integers())
2505 y = data.draw(integers(min_value=x))
2506 assert x < y
2507 If the test fails, each draw will be printed with the falsifying exam‐
2509 ple. e.g. the above is wrong (it has a boundary condition error), so
2510 will print:
2511 Falsifying example: test_draw_sequentially(data=data(...))
2513 Draw 1: 0
2514 Draw 2: 0
2515 As you can see, data drawn this way is simplified as usual.
2517 Optionally, you can provide a label to identify values generated by
2519 each call to data.draw(). These labels can be used to identify values
2520 in the output of a falsifying example.
2521 For instance:
2523 @given(data())
2525 def test_draw_sequentially(data):
2526 x = data.draw(integers(), label="First number")
2527 y = data.draw(integers(min_value=x), label="Second number")
2528 assert x < y
2529 will produce the output:
2531 Falsifying example: test_draw_sequentially(data=data(...))
2533 Draw 1 (First number): 0
2534 Draw 2 (Second number): 0
2535
2537 Hypothesis has minimal dependencies, to maximise compatibility and make
2538 installing Hypothesis as easy as possible.
2539 Our integrations with specific packages are therefore provided by extra
2541 modules that need their individual dependencies installed in order to
2542 work. You can install these dependencies using the setuptools extra
2543 feature as e.g. pip install hypothesis[django]. This will check in‐
2544 stallation of compatible versions.
2545 You can also just install hypothesis into a project using them, ignore
2547 the version constraints, and hope for the best.
2548 In general "Which version is Hypothesis compatible with?" is a hard
2550 question to answer and even harder to regularly test. Hypothesis is al‐
2551 ways tested against the latest compatible version and each package will
2552 note the expected compatibility range. If you run into a bug with any
2553 of these please specify the dependency version.
2554 There are separate pages for django and numpy.
2556 hypothesis[cli]
2558 $ hypothesis --help
2559 Usage: hypothesis [OPTIONS] COMMAND [ARGS]...
2560 Options:
2562 --version Show the version and exit.
2563 -h, --help Show this message and exit.
2564 Commands:
2566 codemod `hypothesis codemod` refactors deprecated or inefficient code.
2567 fuzz [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
2568 write `hypothesis write` writes property-based tests for you!
2569 This module requires the click package, and provides Hypothesis' com‐
2571 mand-line interface, for e.g. 'ghostwriting' tests via the terminal.
2572 It's also where HypoFuzz adds the hypothesis fuzz command (learn more
2573 about that here).
2574 hypothesis[lark]
2576 This extra can be used to generate strings matching any context-free
2577 grammar, using the Lark parser library.
2578 It currently only supports Lark's native EBNF syntax, but we plan to
2580 extend this to support other common syntaxes such as ANTLR and RFC 5234
2581 ABNF. Lark already supports loading grammars from nearley.js, so you
2582 may not have to write your own at all.
2583 Note that as Lark is at version 0.x, this module may break API compati‐
2585 bility in minor releases if supporting the latest version of Lark would
2586 otherwise be infeasible. We may also be quite aggressive in bumping
2587 the minimum version of Lark, unless someone volunteers to either fund
2588 or do the maintenance.
2589 hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None)
2591 A strategy for strings accepted by the given context-free gram‐
2592 mar.
2593 grammar must be a Lark object, which wraps an EBNF specifica‐
2595 tion. The Lark EBNF grammar reference can be found here.
2596 from_lark will automatically generate strings matching the non‐
2598 terminal start symbol in the grammar, which was supplied as an
2599 argument to the Lark class. To generate strings matching a dif‐
2600 ferent symbol, including terminals, you can override this by
2601 passing the start argument to from_lark. Note that Lark may re‐
2602 move unreachable productions when the grammar is compiled, so
2603 you should probably pass the same value for start to both.
2604 Currently from_lark does not support grammars that need custom
2606 lexing. Any lexers will be ignored, and any undefined terminals
2607 from the use of %declare will result in generation errors. To
2608 define strategies for such terminals, pass a dictionary mapping
2609 their name to a corresponding strategy as the explicit argument.
2610 The hypothesmith project includes a strategy for Python source,
2612 based on a grammar and careful post-processing.
2613 Example grammars, which may provide a useful starting point for your
2615 tests, can be found in the Lark repository and in this third-party col‐
2616 lection.
2617 hypothesis[pytz]
2619 This module provides pytz timezones.
2620 You can use this strategy to make hypothesis.strategies.datetimes() and
2622 hypothesis.strategies.times() produce timezone-aware values.
2623 hypothesis.extra.pytz.timezones()
2625 Any timezone in the Olsen database, as a pytz tzinfo object.
2626 This strategy minimises to UTC, or the smallest possible fixed
2628 offset, and is designed for use with hypothesis.strategies.date‐
2629 times().
2630 hypothesis[dateutil]
2632 This module provides dateutil timezones.
2633 You can use this strategy to make datetimes() and times() produce time‐
2635 zone-aware values.
2636 hypothesis.extra.dateutil.timezones()
2638 Any timezone from dateutil.
2639 This strategy minimises to UTC, or the timezone with the small‐
2641 est offset from UTC as of 2000-01-01, and is designed for use
2642 with datetimes().
2643 Note that the timezones generated by the strategy may vary de‐
2645 pending on the configuration of your machine. See the dateutil
2646 documentation for more information.
2647
2649 Writing tests with Hypothesis frees you from the tedium of deciding on
2650 and writing out specific inputs to test. Now, the hypothesis.ex‐
2651 tra.ghostwriter module can write your test functions for you too!
2652 The idea is to provide an easy way to start property-based testing, and
2654 a seamless transition to more complex test code - because ghostwritten
2655 tests are source code that you could have written for yourself.
2656 So just pick a function you'd like tested, and feed it to one of the
2658 functions below. They follow imports, use but do not require type an‐
2659 notations, and generally do their best to write you a useful test. You
2660 can also use our command-line interface:
2661 $ hypothesis write --help
2663 Usage: hypothesis write [OPTIONS] FUNC...
2664 `hypothesis write` writes property-based tests for you!
2666 Type annotations are helpful but not required for our advanced
2668 introspection and templating logic. Try running the examples below to see
2669 how it works:
2670 hypothesis write gzip
2672 hypothesis write numpy.matmul
2673 hypothesis write re.compile --except re.error
2674 hypothesis write --equivalent ast.literal_eval eval
2675 hypothesis write --roundtrip json.dumps json.loads
2676 hypothesis write --style=unittest --idempotent sorted
2677 hypothesis write --binary-op operator.add
2678 Options:
2680 --roundtrip start by testing write/read or encode/decode!
2681 --equivalent very useful when optimising or refactoring code
2682 --idempotent
2683 --binary-op
2684 --style [pytest|unittest] pytest-style function, or unittest-style method?
2685 -e, --except OBJ_NAME dotted name of exception(s) to ignore
2686 -h, --help Show this message and exit.
2687 TIP:
2689 Using a light theme? Hypothesis respects NO_COLOR and DJANGO_COL‐
2690 ORS=light.
2691 NOTE:
2693 The ghostwriter requires black, but the generated code only requires
2694 Hypothesis itself.
2695 NOTE:
2697 Legal questions? While the ghostwriter fragments and logic is under
2698 the MPL-2.0 license like the rest of Hypothesis, the output from the
2699 ghostwriter is made available under the Creative Commons Zero (CC0)
2700 public domain dedication, so you can use it without any restric‐
2701 tions.
2702 hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(),
2704 style='pytest')
2705 Guess which ghostwriters to use, for a module or collection of
2706 functions.
2707 As for all ghostwriters, the except_ argument should be an
2709 python:Exception or tuple of exceptions, and style may be either
2710 "pytest" to write test functions or "unittest" to write test
2711 methods and TestCase.
2712 After finding the public functions attached to any modules, the
2714 magic ghostwriter looks for pairs of functions to pass to
2715 roundtrip(), then checks for binary_operation() and ufunc()
2716 functions, and any others are passed to fuzz().
2717 For example, try hypothesis write gzip on the command line!
2719 hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest')
2721 Write source code for a property-based test of func.
2722 The resulting test checks that valid input only leads to ex‐
2724 pected exceptions. For example:
2725 from re import compile, error
2727 from hypothesis.extra import ghostwriter
2729 ghostwriter.fuzz(compile, except_=error)
2731 Gives:
2733 # This test code was written by the `hypothesis.extra.ghostwriter` module
2735 # and is provided under the Creative Commons Zero public domain dedication.
2736 import re
2737 from hypothesis import given, reject, strategies as st
2739 # TODO: replace st.nothing() with an appropriate strategy
2741 @given(pattern=st.nothing(), flags=st.just(0))
2744 def test_fuzz_compile(pattern, flags):
2745 try:
2746 re.compile(pattern=pattern, flags=flags)
2747 except re.error:
2748 reject()
2749 Note that it includes all the required imports. Because the
2751 pattern parameter doesn't have annotations or a default argu‐
2752 ment, you'll need to specify a strategy - for example text() or
2753 binary(). After that, you have a test!
2754 hypothesis.extra.ghostwriter.idempotent(func, *, except_=(),
2756 style='pytest')
2757 Write source code for a property-based test of func.
2758 The resulting test checks that if you call func on it's own out‐
2760 put, the result does not change. For example:
2761 from typing import Sequence
2763 from hypothesis.extra import ghostwriter
2765 def timsort(seq: Sequence[int]) -> Sequence[int]:
2768 return sorted(seq)
2769 ghostwriter.idempotent(timsort)
2772 Gives:
2774 # This test code was written by the `hypothesis.extra.ghostwriter` module
2776 # and is provided under the Creative Commons Zero public domain dedication.
2777 from hypothesis import given, strategies as st
2779 @given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
2782 def test_idempotent_timsort(seq):
2783 result = timsort(seq=seq)
2784 repeat = timsort(seq=result)
2785 assert result == repeat, (result, repeat)
2786 hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(),
2788 style='pytest')
2789 Write source code for a property-based test of funcs.
2790 The resulting test checks that if you call the first function,
2792 pass the result to the second (and so on), the final result is
2793 equal to the first input argument.
2794 This is a very powerful property to test, especially when the
2796 config options are varied along with the object to round-trip.
2797 For example, try ghostwriting a test for python:json.dumps() -
2798 would you have thought of all that?
2799 hypothesis write --roundtrip json.dumps json.loads
2801 hypothesis.extra.ghostwriter.equivalent(*funcs, except_=(),
2803 style='pytest')
2804 Write source code for a property-based test of funcs.
2805 The resulting test checks that calling each of the functions
2807 gives the same result. This can be used as a classic 'oracle',
2808 such as testing a fast sorting algorithm against the
2809 python:sorted() builtin, or for differential testing where none
2810 of the compared functions are fully trusted but any difference
2811 indicates a bug (e.g. running a function on different numbers of
2812 threads, or simply multiple times).
2813 The functions should have reasonably similar signatures, as only
2815 the common parameters will be passed the same arguments - any
2816 other parameters will be allowed to vary.
2817 hypothesis.extra.ghostwriter.binary_operation(func, *, associa‐
2819 tive=True, commutative=True, identity=infer, distributes_over=None, ex‐
2820 cept_=(), style='pytest')
2821 Write property tests for the binary operation func.
2822 While binary operations are not particularly common, they have
2824 such nice properties to test that it seems a shame not to demon‐
2825 strate them with a ghostwriter. For an operator f, test that:
2826 • if associative, f(a, f(b, c)) == f(f(a, b), c)
2828 • if commutative, f(a, b) == f(b, a)
2830 • if identity is not None, f(a, identity) == a
2832 • if distributes_over is +, f(a, b) + f(a, c) == f(a, b+c)
2834 For example:
2836 ghostwriter.binary_operation(
2838 operator.mul,
2839 identity=1,
2840 distributes_over=operator.add,
2841 style="unittest",
2842 )
2843 hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest')
2845 Write a property-based test for the array ufunc func.
2846 The resulting test checks that your ufunc or gufunc has the ex‐
2848 pected broadcasting and dtype casting behaviour. You will prob‐
2849 ably want to add extra assertions, but as with the other ghost‐
2850 writers this gives you a great place to start.
2851 hypothesis write numpy.matmul
2853
2855 Hypothesis offers a number of features specific for Django testing,
2856 available in the hypothesis[django] extra. This is tested against each
2857 supported series with mainstream or extended support - if you're still
2858 getting security patches, you can test with Hypothesis.
2859 class hypothesis.extra.django.TestCase
2861 Using it is quite straightforward: All you need to do is subclass
2863 hypothesis.extra.django.TestCase or
2864 hypothesis.extra.django.TransactionTestCase or LiveServerTestCase or
2865 StaticLiveServerTestCase and you can use @given as normal, and the
2866 transactions will be per example rather than per test function as they
2867 would be if you used @given with a normal django test suite (this is
2868 important because your test function will be called multiple times and
2869 you don't want them to interfere with each other). Test cases on these
2870 classes that do not use @given will be run as normal.
2871 class hypothesis.extra.django.TransactionTestCase
2873 class hypothesis.extra.django.LiveServerTestCase
2875 class hypothesis.extra.django.StaticLiveServerTestCase
2877 We recommend avoiding TransactionTestCase unless you really have to run
2879 each test case in a database transaction. Because Hypothesis runs this
2880 in a loop, the performance problems it normally has are significantly
2881 exacerbated and your tests will be really slow. If you are using
2882 TransactionTestCase, you may need to use @settings(sup‐
2883 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2884 example generation.
2885 Having set up a test class, you can now pass @given a strategy for
2887 Django models:
2888 hypothesis.extra.django.from_model(model, /, **field_strategies)
2890 Return a strategy for examples of model.
2891 WARNING:
2893 Hypothesis creates saved models. This will run inside your
2894 testing transaction when using the test runner, but if you
2895 use the dev console this will leave debris in your database.
2896 model must be an subclass of Model. Strategies for fields may
2898 be passed as keyword arguments, for example
2899 is_staff=st.just(False). In order to support models with fields
2900 named "model", this is a positional-only parameter.
2901 Hypothesis can often infer a strategy based the field type and
2903 validators, and will attempt to do so for any required fields.
2904 No strategy will be inferred for an AutoField, nullable field,
2905 foreign key, or field for which a keyword argument is passed to
2906 from_model(). For example, a Shop type with a foreign key to
2907 Company could be generated with:
2908 shop_strategy = from_model(Shop, company=from_model(Company))
2910 Like for builds(), you can pass infer as a keyword argument to
2912 infer a strategy for a field which has a default value instead
2913 of using the default.
2914 For example, using the trivial django project we have for testing:
2916 >>> from hypothesis.extra.django import from_model
2918 >>> from toystore.models import Customer
2919 >>> c = from_model(Customer).example()
2920 >>> c
2921 <Customer: Customer object>
2922 >>> c.email
2923 'jaime.urbina@gmail.com'
2924 >>> c.name
2925 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2926 >>> c.age
2927 -873375803
2928 Hypothesis has just created this with whatever the relevant type of
2930 data is.
2931 Obviously the customer's age is implausible, which is only possible be‐
2933 cause we have not used (eg) MinValueValidator to set the valid range
2934 for this field (or used a PositiveSmallIntegerField, which would only
2935 need a maximum value validator).
2936 If you do have validators attached, Hypothesis will only generate exam‐
2938 ples that pass validation. Sometimes that will mean that we fail a
2939 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2940 egy to skip validation at the strategy level:
2941 NOTE:
2943 Inference from validators will be much more powerful when issue
2944 #1116 is implemented, but there will always be some edge cases that
2945 require you to pass an explicit strategy.
2946 >>> from hypothesis.strategies import integers
2948 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2949 >>> c
2950 <Customer: Customer object>
2951 >>> c.age
2952 5
2953 hypothesis.extra.django.from_form(form, form_kwargs=None,
2955 **field_strategies)
2956 Return a strategy for examples of form.
2957 form must be an subclass of Form. Strategies for fields may be
2959 passed as keyword arguments, for example
2960 is_staff=st.just(False).
2961 Hypothesis can often infer a strategy based the field type and
2963 validators, and will attempt to do so for any required fields.
2964 No strategy will be inferred for a disabled field or field for
2965 which a keyword argument is passed to from_form().
2966 This function uses the fields of an unbound form instance to de‐
2968 termine field strategies, any keyword arguments needed to in‐
2969 stantiate the unbound form instance can be passed into
2970 from_form() as a dict with the keyword form_kwargs. E.g.:
2971 shop_strategy = from_form(Shop, form_kwargs={"company_id": 5})
2973 Like for builds(), you can pass infer as a keyword argument to
2975 infer a strategy for a field which has a default value instead
2976 of using the default.
2977 Tips and tricks
2979 Custom field types
2980 If you have a custom Django field type you can register it with Hypoth‐
2981 esis's model deriving functionality by registering a default strategy
2982 for it:
2983 >>> from toystore.models import CustomishField, Customish
2985 >>> from_model(Customish).example()
2986 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2987 customish for model Customish
2988 >>> from hypothesis.extra.django import register_field_strategy
2989 >>> from hypothesis.strategies import just
2990 >>> register_field_strategy(CustomishField, just("hi"))
2991 >>> x = from_model(Customish).example()
2992 >>> x.customish
2993 'hi'
2994 Note that this mapping is on exact type. Subtypes will not inherit it.
2996 hypothesis.extra.django.register_field_strategy(field_type, strategy)
2998 Add an entry to the global field-to-strategy lookup used by
2999 from_field().
3000 field_type must be a subtype of django.db.models.Field or
3002 django.forms.Field, which must not already be registered.
3003 strategy must be a SearchStrategy.
3004 hypothesis.extra.django.from_field(field)
3006 Return a strategy for values that fit the given field.
3007 This function is used by from_form() and from_model() for any
3009 fields that require a value, or for which you passed hypothe‐
3010 sis.infer.
3011 It's pretty similar to the core from_type() function, with a
3013 subtle but important difference: from_field takes a Field in‐
3014 stance, rather than a Field subtype, so that it has access to
3015 instance attributes such as string length and validators.
3016 Generating child models
3018 For the moment there's no explicit support in hypothesis-django for
3019 generating dependent models. i.e. a Company model will generate no
3020 Shops. However if you want to generate some dependent models as well,
3021 you can emulate this by using the flatmap function as follows:
3022 from hypothesis.strategies import just, lists
3024 def generate_with_shops(company):
3027 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
3028 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
3031 Let's unpack what this is doing:
3033 The way flatmap works is that we draw a value from the original strat‐
3035 egy, then apply a function to it which gives us a new strategy. We then
3036 draw a value from that strategy. So in this case we're first drawing a
3037 company, and then we're drawing a list of shops belonging to that com‐
3038 pany: The just strategy is a strategy such that drawing it always pro‐
3039 duces the individual value, so from_model(Shop, company=just(company))
3040 is a strategy that generates a Shop belonging to the original company.
3041 So the following code would give us a list of shops all belonging to
3043 the same company:
3044 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
3046 The only difference from this and the above is that we want the com‐
3048 pany, not the shops. This is where the inner map comes in. We build the
3049 list of shops and then throw it away, instead returning the company we
3050 started for. This works because the models that Hypothesis generates
3051 are saved in the database, so we're essentially running the inner
3052 strategy purely for the side effect of creating those children in the
3053 database.
3054 Generating primary key values
3056 If your model includes a custom primary key that you want to generate
3057 using a strategy (rather than a default auto-increment primary key)
3058 then Hypothesis has to deal with the possibility of a duplicate primary
3059 key.
3060 If a model strategy generates a value for the primary key field, Hy‐
3062 pothesis will create the model instance with update_or_create(), over‐
3063 writing any existing instance in the database for this test case with
3064 the same primary key.
3065 On the subject of MultiValueField
3067 Django forms feature the MultiValueField which allows for several
3068 fields to be combined under a single named field, the default example
3069 of this is the SplitDateTimeField.
3070 class CustomerForm(forms.Form):
3072 name = forms.CharField()
3073 birth_date_time = forms.SplitDateTimeField()
3074 from_form supports MultiValueField subclasses directly, however if you
3076 want to define your own strategy be forewarned that Django binds data
3077 for a MultiValueField in a peculiar way. Specifically each sub-field is
3078 expected to have its own entry in data addressed by the field name
3079 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
3080 ValueField, so form data for the example above might look like this:
3081 {
3083 "name": "Samuel John",
3084 "birth_date_time_0": "2018-05-19", # the date, as the first sub-field
3085 "birth_date_time_1": "15:18:00", # the time, as the second sub-field
3086 }
3087 Thus, if you want to define your own strategies for such a field you
3089 must address your sub-fields appropriately:
3090 from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))
3092
3094 numpy
3095 Hypothesis offers a number of strategies for NumPy testing, available
3096 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
3097 package.
3098 The centerpiece is the arrays() strategy, which generates arrays with
3100 any dtype, shape, and contents you can specify or give a strategy for.
3101 To make this as useful as possible, strategies are provided to generate
3102 array shapes and generate all kinds of fixed-size or compound dtypes.
3103 hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0,
3105 max_size=None, min_value=None, max_value=None, allow_nan=None, al‐
3106 low_infinity=None, allow_subnormal=None, exclude_min=None, ex‐
3107 clude_max=None)
3108 Creates a strategy which can generate any value of the given
3109 dtype.
3110 Compatible **kwargs are passed to the inferred strategy function
3112 for integers, floats, and strings. This allows you to customise
3113 the min and max values, control the length or contents of
3114 strings, or exclude non-finite numbers. This is particularly
3115 useful when kwargs are passed through from arrays() which allow
3116 a variety of numeric dtypes, as it seamlessly handles the width
3117 or representable bounds for you. See issue #2552 for more de‐
3118 tail.
3119 hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None,
3121 fill=None, unique=False)
3122 Returns a strategy for generating numpy:numpy.ndarrays.
3123 • dtype may be any valid input to dtype (this includes dtype ob‐
3125 jects), or a strategy that generates such values.
3126 • shape may be an integer >= 0, a tuple of such integers, or a
3128 strategy that generates such values.
3129 • elements is a strategy for generating values to put in the ar‐
3131 ray. If it is None a suitable value will be inferred based on
3132 the dtype, which may give any legal value (including eg NaN
3133 for floats). If a mapping, it will be passed as **kwargs to
3134 from_dtype()
3135 • fill is a strategy that may be used to generate a single back‐
3137 ground value for the array. If None, a suitable default will
3138 be inferred based on the other arguments. If set to nothing()
3139 then filling behaviour will be disabled entirely and every el‐
3140 ement will be generated independently.
3141 • unique specifies if the elements of the array should all be
3143 distinct from one another. Note that in this case multiple NaN
3144 values may still be allowed. If fill is also set, the only
3145 valid values for it to return are NaN values (anything for
3146 which numpy:numpy.isnan returns True. So e.g. for complex num‐
3147 bers (nan+1j) is also a valid fill). Note that if unique is
3148 set to True the generated values must be hashable.
3149 Arrays of specified dtype and shape are generated for example
3151 like this:
3152 >>> import numpy as np
3154 >>> arrays(np.int8, (2, 3)).example()
3155 array([[-8, 6, 3],
3156 [-6, 4, 6]], dtype=int8)
3157 • See What you can generate and how.
3159 >>> import numpy as np
3161 >>> from hypothesis.strategies import floats
3162 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
3163 array([ 0.88974794, 0.77387938, 0.1977879 ])
3164 Array values are generated in two parts:
3166 1. Some subset of the coordinates of the array are populated
3168 with a value drawn from the elements strategy (or its in‐
3169 ferred form).
3170 2. If any coordinates were not assigned in the previous step, a
3172 single value is drawn from the fill strategy and is assigned
3173 to all remaining places.
3174 You can set fill to nothing() if you want to disable this behav‐
3176 iour and draw a value for every element.
3177 If fill is set to None then it will attempt to infer the correct
3179 behaviour automatically: If unique is True, no filling will oc‐
3180 cur by default. Otherwise, if it looks safe to reuse the values
3181 of elements across multiple coordinates (this will be the case
3182 for any inferred strategy, and for most of the builtins, but is
3183 not the case for mutable values or strategies built with
3184 flatmap, map, composite, etc) then it will use the elements
3185 strategy as the fill, else it will default to having no fill.
3186 Having a fill helps Hypothesis craft high quality examples, but
3188 its main importance is when the array generated is large: Hy‐
3189 pothesis is primarily designed around testing small examples. If
3190 you have arrays with hundreds or more elements, having a fill
3191 value is essential if you want your tests to run in reasonable
3192 time.
3193 hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None,
3195 min_side=1, max_side=None)
3196 Return a strategy for array shapes (tuples of int >= 1).
3197 • min_dims is the smallest length that the generated shape can
3199 possess.
3200 • max_dims is the largest length that the generated shape can
3202 possess, defaulting to min_dims + 2.
3203 • min_side is the smallest size that a dimension can possess.
3205 • max_side is the largest size that a dimension can possess, de‐
3207 faulting to min_side + 5.
3208 hypothesis.extra.numpy.scalar_dtypes()
3210 Return a strategy that can return any non-flexible scalar dtype.
3211 hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?',
3213 sizes=(8, 16, 32, 64))
3214 Return a strategy for unsigned integer dtypes.
3215 endianness may be < for little-endian, > for big-endian, = for
3217 native byte order, or ? to allow either byte order. This argu‐
3218 ment only applies to dtypes of more than one byte.
3219 sizes must be a collection of integer sizes in bits. The de‐
3221 fault (8, 16, 32, 64) covers the full range of sizes.
3222 hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16,
3224 32, 64))
3225 Return a strategy for signed integer dtypes.
3226 endianness and sizes are treated as for
3228 unsigned_integer_dtypes().
3229 hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16,
3231 32, 64))
3232 Return a strategy for floating-point dtypes.
3233 sizes is the size in bits of floating-point number. Some ma‐
3235 chines support 96- or 128-bit floats, but these are not gener‐
3236 ated by default.
3237 Larger floats (96 and 128 bit real parts) are not supported on
3239 all platforms and therefore disabled by default. To generate
3240 these dtypes, include these values in the sizes argument.
3241 hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?',
3243 sizes=(64, 128))
3244 Return a strategy for complex-number dtypes.
3245 sizes is the total size in bits of a complex number, which con‐
3247 sists of two floats. Complex halves (a 16-bit real part) are
3248 not supported by numpy and will not be generated by this strat‐
3249 egy.
3250 hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_pe‐
3252 riod='ns', endianness='?')
3253 Return a strategy for datetime64 dtypes, with various precisions
3254 from year to attosecond.
3255 hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_pe‐
3257 riod='ns', endianness='?')
3258 Return a strategy for timedelta64 dtypes, with various preci‐
3259 sions from year to attosecond.
3260 hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1,
3262 max_len=16)
3263 Return a strategy for generating bytestring dtypes, of various
3264 lengths and byteorder.
3265 While Hypothesis' string strategies can generate empty strings,
3267 string dtypes with length 0 indicate that size is still to be
3268 determined, so the minimum length for string dtypes is 1.
3269 hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?',
3271 min_len=1, max_len=16)
3272 Return a strategy for generating unicode string dtypes, of vari‐
3273 ous lengths and byteorder.
3274 While Hypothesis' string strategies can generate empty strings,
3276 string dtypes with length 0 indicate that size is still to be
3277 determined, so the minimum length for string dtypes is 1.
3278 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
3280 *, min_size=1, max_size=5, allow_subarrays=False)
3281 Return a strategy for generating array (compound) dtypes, with
3282 members drawn from the given subtype strategy.
3283 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
3285 *, max_leaves=10, max_itemsize=None)
3286 Return the most-general dtype strategy.
3287 Elements drawn from this strategy may be simple (from the sub‐
3289 type_strategy), or several such values drawn from array_dtypes()
3290 with allow_subarrays=True. Subdtypes in an array dtype may be
3291 nested to any depth, subject to the max_leaves argument.
3292 hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0,
3294 max_size=None)
3295 Return a strategy for generating permissible tuple-values for
3296 the axis argument for a numpy sequential function (e.g.
3297 numpy:numpy.sum()), given an array of the specified dimensional‐
3298 ity.
3299 All tuples will have a length >= min_size and <= max_size. The
3301 default value for max_size is ndim.
3302 Examples from this strategy shrink towards an empty tuple, which
3304 render most sequential functions as no-ops.
3305 The following are some examples drawn from this strategy.
3307 >>> [valid_tuple_axes(3).example() for i in range(4)]
3309 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3310 valid_tuple_axes can be joined with other strategies to generate
3312 any type of valid axis object, i.e. integers, tuples, and None:
3313 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3315 hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0,
3317 max_dims=None, min_side=1, max_side=None)
3318 Return a strategy for shapes that are broadcast-compatible with
3319 the provided shape.
3320 Examples from this strategy shrink towards a shape with length
3322 min_dims. The size of an aligned dimension shrinks towards size
3323 1. The size of an unaligned dimension shrink towards min_side.
3324 • shape is a tuple of integers.
3326 • min_dims is the smallest length that the generated shape can
3328 possess.
3329 • max_dims is the largest length that the generated shape can
3331 possess, defaulting to max(len(shape), min_dims) + 2.
3332 • min_side is the smallest size that an unaligned dimension can
3334 possess.
3335 • max_side is the largest size that an unaligned dimension can
3337 possess, defaulting to 2 plus the size of the largest aligned
3338 dimension.
3339 The following are some examples drawn from this strategy.
3341 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3343 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3344 hypothesis.extra.numpy.mutually_broadcastable_shapes(*,
3346 num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0,
3347 max_dims=None, min_side=1, max_side=None)
3348 Return a strategy for a specified number of shapes N that are
3349 mutually-broadcastable with one another and with the provided
3350 base shape.
3351 • num_shapes is the number of mutually broadcast-compatible
3353 shapes to generate.
3354 • base_shape is the shape against which all generated shapes can
3356 broadcast. The default shape is empty, which corresponds to a
3357 scalar and thus does not constrain broadcasting at all.
3358 • min_dims is the smallest length that the generated shape can
3360 possess.
3361 • max_dims is the largest length that the generated shape can
3363 possess, defaulting to max(len(shape), min_dims) + 2.
3364 • min_side is the smallest size that an unaligned dimension can
3366 possess.
3367 • max_side is the largest size that an unaligned dimension can
3369 possess, defaulting to 2 plus the size of the largest aligned
3370 dimension.
3371 The strategy will generate a python:typing.NamedTuple contain‐
3373 ing:
3374 • input_shapes as a tuple of the N generated shapes.
3376 • result_shape as the resulting shape produced by broadcasting
3378 the N shapes with the base shape.
3379 The following are some examples drawn from this strategy.
3381 >>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
3383 ... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
3384 >>> for _ in range(5):
3385 ... print(strat.example())
3386 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3387 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3388 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
3389 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
3390 BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))
3391 Use with Generalised Universal Function signatures
3393 A universal function (or ufunc for short) is a function that op‐
3395 erates on ndarrays in an element-by-element fashion, supporting
3396 array broadcasting, type casting, and several other standard
3397 features. A generalised ufunc operates on sub-arrays rather
3398 than elements, based on the "signature" of the function. Com‐
3399 pare e.g. numpy.add() (ufunc) to numpy.matmul() (gufunc).
3400 To generate shapes for a gufunc, you can pass the signature ar‐
3402 gument instead of num_shapes. This must be a gufunc signature
3403 string; which you can write by hand or access as e.g. np.mat‐
3404 mul.signature on generalised ufuncs.
3405 In this case, the side arguments are applied to the 'core dimen‐
3407 sions' as well, ignoring any frozen dimensions. base_shape and
3408 the dims arguments are applied to the 'loop dimensions', and if
3409 necessary, the dimensionality of each shape is silently capped
3410 to respect the 32-dimension limit.
3411 The generated result_shape is the real result shape of applying
3413 the gufunc to arrays of the generated input_shapes, even where
3414 this is different to broadcasting the loop dimensions.
3415 gufunc-compatible shapes shrink their loop dimensions as above,
3417 towards omitting optional core dimensions, and smaller-size core
3418 dimensions.
3419 >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
3421 >>> for _ in range(3):
3422 ... mutually_broadcastable_shapes(signature=np.matmul.signature).example()
3423 BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
3424 BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
3425 BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))
3426 hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0,
3428 max_dims=None, allow_newaxis=False, allow_ellipsis=True)
3429 Return a strategy for basic indexes of arrays with the specified
3430 shape, which may include dimensions of size zero.
3431 It generates tuples containing some mix of integers,
3433 python:slice objects, ... (an Ellipsis), and None. When a
3434 length-one tuple would be generated, this strategy may instead
3435 return the element which will index the first axis, e.g. 5 in‐
3436 stead of (5,).
3437 • shape is the shape of the array that will be indexed, as a tu‐
3439 ple of positive integers. This must be at least two-dimen‐
3440 sional for a tuple to be a valid index; for one-dimensional
3441 arrays use slices() instead.
3442 • min_dims is the minimum dimensionality of the resulting array
3444 from use of the generated index. When min_dims == 0, scalars
3445 and zero-dimensional arrays are both allowed.
3446 • max_dims is the the maximum dimensionality of the resulting
3448 array, defaulting to len(shape) if not allow_newaxis else
3449 max(len(shape), min_dims) + 2.
3450 • allow_newaxis specifies whether None is allowed in the index.
3452 • allow_ellipsis specifies whether ... is allowed in the index.
3454 hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=ar‐
3456 ray_shapes(), dtype='int')
3457 Return a search strategy for tuples of integer-arrays that, when
3458 used to index into an array of shape shape, given an array whose
3459 shape was drawn from result_shape.
3460 Examples from this strategy shrink towards the tuple of in‐
3462 dex-arrays:
3463 len(shape) * (np.zeros(drawn_result_shape, dtype), )
3465 • shape a tuple of integers that indicates the shape of the ar‐
3467 ray, whose indices are being generated.
3468 • result_shape a strategy for generating tuples of integers,
3470 which describe the shape of the resulting index arrays. The
3471 default is array_shapes(). The shape drawn from this strategy
3472 determines the shape of the array that will be produced when
3473 the corresponding example from integer_array_indices is used
3474 as an index.
3475 • dtype the integer data type of the generated index-arrays.
3477 Negative integer indices can be generated if a signed integer
3478 type is specified.
3479 Recall that an array can be indexed using a tuple of integer-ar‐
3481 rays to access its members in an arbitrary order, producing an
3482 array with an arbitrary shape. For example:
3483 >>> from numpy import array
3485 >>> x = array([-0, -1, -2, -3, -4])
3486 >>> ind = (array([[4, 0], [0, 1]]),) # a tuple containing a 2D integer-array
3487 >>> x[ind] # the resulting array is commensurate with the indexing array(s)
3488 array([[-4, 0],
3489 [0, -1]])
3490 Note that this strategy does not accommodate all variations of
3492 so-called 'advanced indexing', as prescribed by NumPy's nomen‐
3493 clature. Combinations of basic and advanced indexes are too
3494 complex to usefully define in a standard strategy; we leave ap‐
3495 plication-specific strategies to the user. Advanced-boolean in‐
3496 dexing can be defined as arrays(shape=..., dtype=bool), and is
3497 similarly left to the user.
3498 pandas
3500 Hypothesis provides strategies for several of the core pandas data
3501 types: pandas.Index, pandas.Series and pandas.DataFrame.
3502 The general approach taken by the pandas module is that there are mul‐
3504 tiple strategies for generating indexes, and all of the other strate‐
3505 gies take the number of entries they contain from their index strategy
3506 (with sensible defaults). So e.g. a Series is specified by specifying
3507 its numpy.dtype (and/or a strategy for generating elements for it).
3508 hypothesis.extra.pandas.indexes(*, elements=None, dtype=None,
3510 min_size=0, max_size=None, unique=True)
3511 Provides a strategy for producing a pandas.Index.
3512 Arguments:
3514 • elements is a strategy which will be used to generate the in‐
3516 dividual values of the index. If None, it will be inferred
3517 from the dtype. Note: even if the elements strategy produces
3518 tuples, the generated value will not be a MultiIndex, but in‐
3519 stead be a normal index whose elements are tuples.
3520 • dtype is the dtype of the resulting index. If None, it will be
3522 inferred from the elements strategy. At least one of dtype or
3523 elements must be provided.
3524 • min_size is the minimum number of elements in the index.
3526 • max_size is the maximum number of elements in the index. If
3528 None then it will default to a suitable small size. If you
3529 want larger indexes you should pass a max_size explicitly.
3530 • unique specifies whether all of the elements in the resulting
3532 index should be distinct.
3533 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
3535 Provides a strategy which generates an Index whose values are 0,
3536 1, ..., n for some n.
3537 Arguments:
3539 • min_size is the smallest number of elements the index can
3541 have.
3542 • max_size is the largest number of elements the index can have.
3544 If None it will default to some suitable value based on
3545 min_size.
3546 hypothesis.extra.pandas.series(*, elements=None, dtype=None, in‐
3548 dex=None, fill=None, unique=False)
3549 Provides a strategy for producing a pandas.Series.
3550 Arguments:
3552 • elements: a strategy that will be used to generate the indi‐
3554 vidual values in the series. If None, we will attempt to infer
3555 a suitable default from the dtype.
3556 • dtype: the dtype of the resulting series and may be any value
3558 that can be passed to numpy.dtype. If None, will use pandas's
3559 standard behaviour to infer it from the type of the elements
3560 values. Note that if the type of values that comes out of your
3561 elements strategy varies, then so will the resulting dtype of
3562 the series.
3563 • index: If not None, a strategy for generating indexes for the
3565 resulting Series. This can generate either pandas.Index ob‐
3566 jects or any sequence of values (which will be passed to the
3567 Index constructor).
3568 You will probably find it most convenient to use the indexes()
3570 or range_indexes() function to produce values for this argu‐
3571 ment.
3572 Usage:
3574 >>> series(dtype=int).example()
3576 0 -2001747478
3577 1 1153062837
3578 class hypothesis.extra.pandas.column(name=None, elements=None,
3580 dtype=None, fill=None, unique=False)
3581 Data object for describing a column in a DataFrame.
3582 Arguments:
3584 • name: the column name, or None to default to the column posi‐
3586 tion. Must be hashable, but can otherwise be any value sup‐
3587 ported as a pandas column name.
3588 • elements: the strategy for generating values in this column,
3590 or None to infer it from the dtype.
3591 • dtype: the dtype of the column, or None to infer it from the
3593 element strategy. At least one of dtype or elements must be
3594 provided.
3595 • fill: A default value for elements of the column. See arrays()
3597 for a full explanation.
3598 • unique: If all values in this column should be distinct.
3600 hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, ele‐
3602 ments=None, fill=None, unique=False)
3603 A convenience function for producing a list of column objects of
3604 the same general shape.
3605 The names_or_number argument is either a sequence of values, the
3607 elements of which will be used as the name for individual column
3608 objects, or a number, in which case that many unnamed columns
3609 will be created. All other arguments are passed through verbatim
3610 to create the columns.
3611 hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, in‐
3613 dex=None)
3614 Provides a strategy for producing a pandas.DataFrame.
3615 Arguments:
3617 • columns: An iterable of column objects describing the shape of
3619 the generated DataFrame.
3620 • rows: A strategy for generating a row object. Should generate
3622 either dicts mapping column names to values or a sequence map‐
3623 ping column position to the value in that position (note that
3624 unlike the pandas.DataFrame constructor, single values are not
3625 allowed here. Passing e.g. an integer is an error, even if
3626 there is only one column).
3627 At least one of rows and columns must be provided. If both are
3629 provided then the generated rows will be validated against the
3630 columns and an error will be raised if they don't match.
3631 Caveats on using rows:
3633 • In general you should prefer using columns to rows, and only
3635 use rows if the columns interface is insufficiently flexible
3636 to describe what you need - you will get better performance
3637 and example quality that way.
3638 • If you provide rows and not columns, then the shape and
3640 dtype of the resulting DataFrame may vary. e.g. if you have
3641 a mix of int and float in the values for one column in your
3642 row entries, the column will sometimes have an integral
3643 dtype and sometimes a float.
3644 • index: If not None, a strategy for generating indexes for the
3646 resulting DataFrame. This can generate either pandas.Index ob‐
3647 jects or any sequence of values (which will be passed to the
3648 Index constructor).
3649 You will probably find it most convenient to use the indexes()
3651 or range_indexes() function to produce values for this argu‐
3652 ment.
3653 Usage:
3655 The expected usage pattern is that you use column and columns()
3657 to specify a fixed shape of the DataFrame you want as follows.
3658 For example the following gives a two column data frame:
3659 >>> from hypothesis.extra.pandas import column, data_frames
3661 >>> data_frames([
3662 ... column('A', dtype=int), column('B', dtype=float)]).example()
3663 A B
3664 0 2021915903 1.793898e+232
3665 1 1146643993 inf
3666 2 -2096165693 1.000000e+07
3667 If you want the values in different columns to interact in some
3669 way you can use the rows argument. For example the following
3670 gives a two column DataFrame where the value in the first column
3671 is always at most the value in the second:
3672 >>> from hypothesis.extra.pandas import column, data_frames
3674 >>> import hypothesis.strategies as st
3675 >>> data_frames(
3676 ... rows=st.tuples(st.floats(allow_nan=False),
3677 ... st.floats(allow_nan=False)).map(sorted)
3678 ... ).example()
3679 0 1
3680 0 -3.402823e+38 9.007199e+15
3681 1 -1.562796e-298 5.000000e-01
3682 You can also combine the two:
3684 >>> from hypothesis.extra.pandas import columns, data_frames
3686 >>> import hypothesis.strategies as st
3687 >>> data_frames(
3688 ... columns=columns(["lo", "hi"], dtype=float),
3689 ... rows=st.tuples(st.floats(allow_nan=False),
3690 ... st.floats(allow_nan=False)).map(sorted)
3691 ... ).example()
3692 lo hi
3693 0 9.314723e-49 4.353037e+45
3694 1 -9.999900e-01 1.000000e+07
3695 2 -2.152861e+134 -1.069317e-73
3696 (Note that the column dtype must still be specified and will not
3698 be inferred from the rows. This restriction may be lifted in fu‐
3699 ture).
3700 Combining rows and columns has the following behaviour:
3702 • The column names and dtypes will be used.
3704 • If the column is required to be unique, this will be enforced.
3706 • Any values missing from the generated rows will be provided
3708 using the column's fill.
3709 • Any values in the row not present in the column specification
3711 (if dicts are passed, if there are keys with no corresponding
3712 column name, if sequences are passed if there are too many
3713 items) will result in InvalidArgument being raised.
3714 Supported versions
3716 There is quite a lot of variation between pandas versions. We only com‐
3717 mit to supporting the latest version of pandas, but older minor ver‐
3718 sions are supported on a "best effort" basis. Hypothesis is currently
3719 tested against and confirmed working with every Pandas minor version
3720 from 0.25 through to 1.3.
3721 Releases that are not the latest patch release of their minor version
3723 are not tested or officially supported, but will probably also work un‐
3724 less you hit a pandas bug.
3725 Array API
3727 TIP:
3728 The Array API standard is not yet finalised, so this module will
3729 make breaking changes if necessary to support newer versions of the
3730 standard.
3731 Hypothesis offers strategies for Array API adopting libraries in the
3733 hypothesis.extra.array_api package. See issue #3037 for more details.
3734 If you want to test with CuPy, Dask, JAX, MXNet, PyTorch, TensorFlow,
3735 or Xarray - or just numpy.array_api - this is the extension for you!
3736 hypothesis.extra.array_api.make_strategies_namespace(xp)
3738 Creates a strategies namespace for the given array module.
3739 • xp is the Array API library to automatically pass to the
3741 namespaced methods.
3742 A python:types.SimpleNamespace is returned which contains all
3744 the strategy methods in this module but without requiring the xp
3745 argument. Creating and using a strategies namespace for NumPy's
3746 Array API implementation would go like this:
3747 >>> from numpy import array_api as xp
3749 >>> xps = make_strategies_namespace(xp)
3750 >>> x = xps.arrays(xp.int8, (2, 3)).example()
3751 >>> x
3752 Array([[-8, 6, 3],
3753 [-6, 4, 6]], dtype=int8)
3754 >>> x.__array_namespace__() is xp
3755 True
3756 The resulting namespace contains all our familiar strategies like
3758 arrays() and from_dtype(), but based on the Array API standard seman‐
3759 tics and returning objects from the xp module:
3760 xps.from_dtype(dtype, *, min_value=None, max_value=None, al‐
3762 low_nan=None, allow_infinity=None, allow_subnormal=None, ex‐
3763 clude_min=None, exclude_max=None)
3764 Return a strategy for any value of the given dtype.
3765 Values generated are of the Python scalar which is promotable to
3767 dtype, where the values do not exceed its bounds.
3768 • dtype may be a dtype object or the string name of a valid
3770 dtype.
3771 Compatible **kwargs are passed to the inferred strategy function
3773 for integers and floats. This allows you to customise the min
3774 and max values, and exclude non-finite numbers. This is particu‐
3775 larly useful when kwargs are passed through from arrays(), as it
3776 seamlessly handles the width or other representable bounds for
3777 you.
3778 xps.arrays(dtype, shape, *, elements=None, fill=None, unique=False)
3780 Returns a strategy for arrays.
3781 • dtype may be a valid dtype object or name, or a strategy that
3783 generates such values.
3784 • shape may be an integer >= 0, a tuple of such integers, or a
3786 strategy that generates such values.
3787 • elements is a strategy for values to put in the array. If None
3789 then a suitable value will be inferred based on the dtype,
3790 which may give any legal value (including e.g. NaN for
3791 floats). If a mapping, it will be passed as **kwargs to
3792 from_dtype() when inferring based on the dtype.
3793 • fill is a strategy that may be used to generate a single back‐
3795 ground value for the array. If None, a suitable default will
3796 be inferred based on the other arguments. If set to nothing()
3797 then filling behaviour will be disabled entirely and every el‐
3798 ement will be generated independently.
3799 • unique specifies if the elements of the array should all be
3801 distinct from one another; if fill is also set, the only valid
3802 values for fill to return are NaN values.
3803 Arrays of specified dtype and shape are generated for example
3805 like this:
3806 >>> from numpy import array_api as xp
3808 >>> xps.arrays(xp, xp.int8, (2, 3)).example()
3809 Array([[-8, 6, 3],
3810 [-6, 4, 6]], dtype=int8)
3811 Specifying element boundaries by a python:dict of the kwargs to
3813 pass to from_dtype() will ensure dtype bounds will be respected.
3814 >>> xps.arrays(xp, xp.int8, 3, elements={"min_value": 10}).example()
3816 Array([125, 13, 79], dtype=int8)
3817 Refer to What you can generate and how for passing your own ele‐
3819 ments strategy.
3820 >>> xps.arrays(xp, xp.float32, 3, elements=floats(0, 1, width=32)).example()
3822 Array([ 0.88974794, 0.77387938, 0.1977879 ], dtype=float32)
3823 Array values are generated in two parts:
3825 1. A single value is drawn from the fill strategy and is used to
3827 create a filled array.
3828 2. Some subset of the coordinates of the array are populated
3830 with a value drawn from the elements strategy (or its in‐
3831 ferred form).
3832 You can set fill to nothing() if you want to disable this behav‐
3834 iour and draw a value for every element.
3835 By default arrays will attempt to infer the correct fill behav‐
3837 iour: if unique is also True, no filling will occur. Otherwise,
3838 if it looks safe to reuse the values of elements across multiple
3839 coordinates (this will be the case for any inferred strategy,
3840 and for most of the builtins, but is not the case for mutable
3841 values or strategies built with flatmap, map, composite, etc.)
3842 then it will use the elements strategy as the fill, else it will
3843 default to having no fill.
3844 Having a fill helps Hypothesis craft high quality examples, but
3846 its main importance is when the array generated is large: Hy‐
3847 pothesis is primarily designed around testing small examples. If
3848 you have arrays with hundreds or more elements, having a fill
3849 value is essential if you want your tests to run in reasonable
3850 time.
3851 xps.array_shapes(*, min_dims=1, max_dims=None, min_side=1,
3853 max_side=None)
3854 Return a strategy for array shapes (tuples of int >= 1).
3855 • min_dims is the smallest length that the generated shape can
3857 possess.
3858 • max_dims is the largest length that the generated shape can
3860 possess, defaulting to min_dims + 2.
3861 • min_side is the smallest size that a dimension can possess.
3863 • max_side is the largest size that a dimension can possess, de‐
3865 faulting to min_side + 5.
3866 xps.scalar_dtypes()
3868 Return a strategy for all valid dtype objects.
3869 xps.boolean_dtypes()
3871 Return a strategy for just the boolean dtype object.
3872 xps.numeric_dtypes()
3874 Return a strategy for all numeric dtype objects.
3875 xps.integer_dtypes(*, sizes=(8, 16, 32, 64))
3877 Return a strategy for signed integer dtype objects.
3878 sizes contains the signed integer sizes in bits, defaulting to
3880 (8, 16, 32, 64) which covers all valid sizes.
3881 xps.unsigned_integer_dtypes(*, sizes=(8, 16, 32, 64))
3883 Return a strategy for unsigned integer dtype objects.
3884 sizes contains the unsigned integer sizes in bits, defaulting to
3886 (8, 16, 32, 64) which covers all valid sizes.
3887 xps.floating_dtypes(*, sizes=(32, 64))
3889 Return a strategy for floating-point dtype objects.
3890 sizes contains the floating-point sizes in bits, defaulting to
3892 (32, 64) which covers all valid sizes.
3893 xps.valid_tuple_axes(ndim, *, min_size=0, max_size=None)
3895 Return a strategy for permissible tuple-values for the axis ar‐
3896 gument in Array API sequential methods e.g. sum, given the spec‐
3897 ified dimensionality.
3898 All tuples will have a length >= min_size and <= max_size. The
3900 default value for max_size is ndim.
3901 Examples from this strategy shrink towards an empty tuple, which
3903 render most sequential functions as no-ops.
3904 The following are some examples drawn from this strategy.
3906 >>> [valid_tuple_axes(3).example() for i in range(4)]
3908 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3909 valid_tuple_axes can be joined with other strategies to generate
3911 any type of valid axis object, i.e. integers, tuples, and None:
3912 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3914 xps.broadcastable_shapes(shape, *, min_dims=0, max_dims=None,
3916 min_side=1, max_side=None)
3917 Return a strategy for shapes that are broadcast-compatible with
3918 the provided shape.
3919 Examples from this strategy shrink towards a shape with length
3921 min_dims. The size of an aligned dimension shrinks towards size
3922 1. The size of an unaligned dimension shrink towards min_side.
3923 • shape is a tuple of integers.
3925 • min_dims is the smallest length that the generated shape can
3927 possess.
3928 • max_dims is the largest length that the generated shape can
3930 possess, defaulting to max(len(shape), min_dims) + 2.
3931 • min_side is the smallest size that an unaligned dimension can
3933 possess.
3934 • max_side is the largest size that an unaligned dimension can
3936 possess, defaulting to 2 plus the size of the largest aligned
3937 dimension.
3938 The following are some examples drawn from this strategy.
3940 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3942 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3943 xps.mutually_broadcastable_shapes(num_shapes, *, base_shape=(),
3945 min_dims=0, max_dims=None, min_side=1, max_side=None)
3946 Return a strategy for a specified number of shapes N that are
3947 mutually-broadcastable with one another and with the provided
3948 base shape.
3949 • num_shapes is the number of mutually broadcast-compatible
3951 shapes to generate.
3952 • base_shape is the shape against which all generated shapes can
3954 broadcast. The default shape is empty, which corresponds to a
3955 scalar and thus does not constrain broadcasting at all.
3956 • min_dims is the smallest length that the generated shape can
3958 possess.
3959 • max_dims is the largest length that the generated shape can
3961 possess, defaulting to max(len(shape), min_dims) + 2.
3962 • min_side is the smallest size that an unaligned dimension can
3964 possess.
3965 • max_side is the largest size that an unaligned dimension can
3967 possess, defaulting to 2 plus the size of the largest aligned
3968 dimension.
3969 The strategy will generate a python:typing.NamedTuple contain‐
3971 ing:
3972 • input_shapes as a tuple of the N generated shapes.
3974 • result_shape as the resulting shape produced by broadcasting
3976 the N shapes with the base shape.
3977 The following are some examples drawn from this strategy.
3979 >>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
3981 ... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
3982 >>> for _ in range(5):
3983 ... print(strat.example())
3984 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3985 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3986 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
3987 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
3988 BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))
3989 xps.indices(shape, *, min_dims=0, max_dims=None, allow_ellipsis=True)
3991 Return a strategy for valid indices of arrays with the specified
3992 shape, which may include dimensions of size zero.
3993 It generates tuples containing some mix of integers,
3995 python:slice objects, and ... (an Ellipsis). When a length-one
3996 tuple would be generated, this strategy may instead return the
3997 element which will index the first axis, e.g. 5 instead of (5,).
3998 • shape is the shape of the array that will be indexed, as a tu‐
4000 ple of integers >= 0. This must be at least two-dimensional
4001 for a tuple to be a valid index; for one-dimensional arrays
4002 use slices() instead.
4003 • min_dims is the minimum dimensionality of the resulting array
4005 from use of the generated index.
4006 • max_dims is the the maximum dimensionality of the resulting
4008 array, defaulting to len(shape).
4009 • allow_ellipsis specifies whether ... is allowed in the index.
4011
4013 Hypothesis tries to detect common mistakes and things that will cause
4014 difficulty at run time in the form of a number of 'health checks'.
4015 These include detecting and warning about:
4017 • Strategies with very slow data generation
4019 • Strategies which filter out too much
4021 • Recursive strategies which branch too much
4023 • Tests that are unlikely to complete in a reasonable amount of time.
4025 If any of these scenarios are detected, Hypothesis will emit a warning
4027 about them.
4028 The general goal of these health checks is to warn you about things
4030 that you are doing that might appear to work but will either cause Hy‐
4031 pothesis to not work correctly or to perform badly.
4032 To selectively disable health checks, use the suppress_health_check
4034 setting. The argument for this parameter is a list with elements drawn
4035 from any of the class-level attributes of the HealthCheck class. Using
4036 a value of HealthCheck.all() will disable all health checks.
4037 class hypothesis.HealthCheck(value)
4039 Arguments for suppress_health_check.
4040 Each member of this enum is a type of health check to suppress.
4042 data_too_large = 1
4044 Checks if too many examples are aborted for being too
4045 large.
4046 This is measured by the number of random choices that Hy‐
4048 pothesis makes in order to generate something, not the
4049 size of the generated object. For example, choosing a
4050 100MB object from a predefined list would take only a few
4051 bits, while generating 10KB of JSON from scratch might
4052 trigger this health check.
4053 filter_too_much = 2
4055 Check for when the test is filtering out too many exam‐
4056 ples, either through use of assume() or filter(), or oc‐
4057 casionally for Hypothesis internal reasons.
4058 too_slow = 3
4060 Check for when your data generation is extremely slow and
4061 likely to hurt testing.
4062 return_value = 5
4064 Checks if your tests return a non-None value (which will
4065 be ignored and is unlikely to do what you want).
4066 large_base_example = 7
4068 Checks if the natural example to shrink towards is very
4069 large.
4070 not_a_test_method = 8
4072 Checks if @given has been applied to a method defined by
4073 python:unittest.TestCase (i.e. not a test).
4074 function_scoped_fixture = 9
4076 Checks if @given has been applied to a test with a pytest
4077 function-scoped fixture. Function-scoped fixtures run
4078 once for the whole function, not once per example, and
4079 this is usually not what you want.
4080 Because of this limitation, tests that need to set up or
4082 reset state for every example need to do so manually
4083 within the test itself, typically using an appropriate
4084 context manager.
4085 Suppress this health check only in the rare case that you
4087 are using a function-scoped fixture that does not need to
4088 be reset between individual examples, but for some reason
4089 you cannot use a wider fixture scope (e.g. session scope,
4090 module scope, class scope).
4091 This check requires the Hypothesis pytest plugin, which
4093 is enabled by default when running Hypothesis inside
4094 pytest.
4095 Deprecations
4097 We also use a range of custom exception and warning types, so you can
4098 see exactly where an error came from - or turn only our warnings into
4099 errors.
4100 class hypothesis.errors.HypothesisDeprecationWarning
4102 A deprecation warning issued by Hypothesis.
4103 Actually inherits from FutureWarning, because DeprecationWarning
4105 is hidden by the default warnings filter.
4106 You can configure the Python python:warnings to handle these
4108 warnings differently to others, either turning them into errors
4109 or suppressing them entirely. Obviously we would prefer the
4110 former!
4111 Deprecated features will be continue to emit warnings for at least six
4113 months, and then be removed in the following major release. Note how‐
4114 ever that not all warnings are subject to this grace period; sometimes
4115 we strengthen validation by adding a warning and these may become er‐
4116 rors immediately at a major release.
4117
4119 When Hypothesis finds a bug it stores enough information in its data‐
4120 base to reproduce it. This enables you to have a classic testing work‐
4121 flow of find a bug, fix a bug, and be confident that this is actually
4122 doing the right thing because Hypothesis will start by retrying the ex‐
4123 amples that broke things last time.
4124 Limitations
4126 The database is best thought of as a cache that you never need to in‐
4127 validate: Information may be lost when you upgrade a Hypothesis version
4128 or change your test, so you shouldn't rely on it for correctness - if
4129 there's an example you want to ensure occurs each time then there's a
4130 feature for including them in your source code - but it helps the de‐
4131 velopment workflow considerably by making sure that the examples you've
4132 just found are reproduced.
4133 The database also records examples that exercise less-used parts of
4135 your code, so the database may update even when no failing examples
4136 were found.
4137 Upgrading Hypothesis and changing your tests
4139 The design of the Hypothesis database is such that you can put arbi‐
4140 trary data in the database and not get wrong behaviour. When you up‐
4141 grade Hypothesis, old data might be invalidated, but this should happen
4142 transparently. It can never be the case that e.g. changing the strategy
4143 that generates an argument gives you data from the old strategy.
4144 ExampleDatabase implementations
4146 Hypothesis' default database setting creates a
4147 DirectoryBasedExampleDatabase in your current working directory, under
4148 .hypothesis/examples. If this location is unusable, e.g. because you
4149 do not have read or write permissions, Hypothesis will emit a warning
4150 and fall back to an InMemoryExampleDatabase.
4151 Hypothesis provides the following ExampleDatabase implementations:
4153 class hypothesis.database.InMemoryExampleDatabase
4155 A non-persistent example database, implemented in terms of a
4156 dict of sets.
4157 This can be useful if you call a test function several times in
4159 a single session, or for testing other database implementations,
4160 but because it does not persist between runs we do not recommend
4161 it for general use.
4162 class hypothesis.database.DirectoryBasedExampleDatabase(path)
4164 Use a directory to store Hypothesis examples as files.
4165 Each test corresponds to a directory, and each example to a file
4167 within that directory. While the contents are fairly opaque, a
4168 DirectoryBasedExampleDatabase can be shared by checking the di‐
4169 rectory into version control, for example with the following
4170 .gitignore:
4171 # Ignore files cached by Hypothesis...
4173 .hypothesis/*
4174 # except for the examples directory
4175 !.hypothesis/examples/
4176 Note however that this only makes sense if you also pin to an
4178 exact version of Hypothesis, and we would usually recommend im‐
4179 plementing a shared database with a network datastore - see
4180 ExampleDatabase, and the MultiplexedDatabase helper.
4181 class hypothesis.database.ReadOnlyDatabase(db)
4183 A wrapper to make the given database read-only.
4184 The implementation passes through fetch, and turns save, delete,
4186 and move into silent no-ops.
4187 Note that this disables Hypothesis' automatic discarding of
4189 stale examples. It is designed to allow local machines to ac‐
4190 cess a shared database (e.g. from CI servers), without propagat‐
4191 ing changes back from a local or in-development branch.
4192 class hypothesis.database.MultiplexedDatabase(*dbs)
4194 A wrapper around multiple databases.
4195 Each save, fetch, move, or delete operation will be run against
4197 all of the wrapped databases. fetch does not yield duplicate
4198 values, even if the same value is present in two or more of the
4199 wrapped databases.
4200 This combines well with a ReadOnlyDatabase, as follows:
4202 local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
4204 shared = CustomNetworkDatabase()
4205 settings.register_profile("ci", database=shared)
4207 settings.register_profile(
4208 "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
4209 )
4210 settings.load_profile("ci" if os.environ.get("CI") else "dev")
4211 So your CI system or fuzzing runs can populate a central shared
4213 database; while local runs on development machines can reproduce
4214 any failures from CI but will only cache their own failures lo‐
4215 cally and cannot remove examples from the shared database.
4216 class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_af‐
4218 ter=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:')
4219 Store Hypothesis examples as sets in the given Redis datastore.
4220 This is particularly useful for shared databases, as per the
4222 recipe for a MultiplexedDatabase.
4223 NOTE:
4225 If a test has not been run for expire_after, those examples
4226 will be allowed to expire. The default time-to-live persists
4227 examples between weekly runs.
4228 Defining your own ExampleDatabase
4230 You can define your ExampleDatabase, for example to use a shared datas‐
4231 tore, with just a few methods:
4232 class hypothesis.database.ExampleDatabase(*args, **kwargs)
4234 An abstract base class for storing examples in Hypothesis' in‐
4235 ternal format.
4236 An ExampleDatabase maps each bytes key to many distinct bytes
4238 values, like a Mapping[bytes, AbstractSet[bytes]].
4239 abstract save(key, value)
4241 Save value under key.
4242 If this value is already present for this key, silently
4244 do nothing.
4245 abstract fetch(key)
4247 Return an iterable over all values matching this key.
4248 abstract delete(key, value)
4250 Remove this value from this key.
4251 If this value is not present, silently do nothing.
4253 move(src, dest, value)
4255 Move value from key src to key dest. Equivalent to
4256 delete(src, value) followed by save(src, value), but may
4257 have a more efficient implementation.
4258 Note that value will be inserted at dest regardless of
4260 whether it is currently present at src.
4261
4263 With @given, your tests are still something that you mostly write your‐
4264 self, with Hypothesis providing some data. With Hypothesis's stateful
4265 testing, Hypothesis instead tries to generate not just data but entire
4266 tests. You specify a number of primitive actions that can be combined
4267 together, and then Hypothesis will try to find sequences of those ac‐
4268 tions that result in a failure.
4269 TIP:
4271 Before reading this reference documentation, we recommend reading
4272 How not to Die Hard with Hypothesis and An Introduction to
4273 Rule-Based Stateful Testing, in that order. The implementation de‐
4274 tails will make more sense once you've seen them used in practice,
4275 and know why each method or decorator is available.
4276 NOTE:
4278 This style of testing is often called model-based testing, but in
4279 Hypothesis is called stateful testing (mostly for historical reasons
4280 - the original implementation of this idea in Hypothesis was more
4281 closely based on ScalaCheck's stateful testing where the name is
4282 more apt). Both of these names are somewhat misleading: You don't
4283 really need any sort of formal model of your code to use this, and
4284 it can be just as useful for pure APIs that don't involve any state
4285 as it is for stateful ones.
4286 It's perhaps best to not take the name of this sort of testing too
4288 seriously. Regardless of what you call it, it is a powerful form of
4289 testing which is useful for most non-trivial APIs.
4290 You may not need state machines
4292 The basic idea of stateful testing is to make Hypothesis choose actions
4293 as well as values for your test, and state machines are a great declar‐
4294 ative way to do just that.
4295 For simpler cases though, you might not need them at all - a standard
4297 test with @given might be enough, since you can use data() in branches
4298 or loops. In fact, that's how the state machine explorer works inter‐
4299 nally. For more complex workloads though, where a higher level API
4300 comes into it's own, keep reading!
4301 Rule-based state machines
4303 class hypothesis.stateful.RuleBasedStateMachine
4304 A RuleBasedStateMachine gives you a structured way to define
4305 state machines.
4306 The idea is that a state machine carries a bunch of types of
4308 data divided into Bundles, and has a set of rules which may read
4309 data from bundles (or just from normal strategies) and push data
4310 onto bundles. At any given point a random applicable rule will
4311 be executed.
4312 A rule is very similar to a normal @given based test in that it takes
4314 values drawn from strategies and passes them to a user defined test
4315 function. The key difference is that where @given based tests must be
4316 independent, rules can be chained together - a single test run may in‐
4317 volve multiple rule invocations, which may interact in various ways.
4318 Rules can take normal strategies as arguments, or a specific kind of
4320 strategy called a Bundle. A Bundle is a named collection of generated
4321 values that can be reused by other operations in the test. They are
4322 populated with the results of rules, and may be used as arguments to
4323 rules, allowing data to flow from one rule to another, and rules to
4324 work on the results of previous computations or actions.
4325 You can think of each value that gets added to any Bundle as being as‐
4327 signed to a new variable. Drawing a value from the bundle strategy
4328 means choosing one of the corresponding variables and using that value,
4329 and consumes() as a del statement for that variable. If you can re‐
4330 place use of Bundles with instance attributes of the class that is of‐
4331 ten simpler, but often Bundles are strictly more powerful.
4332 The following rule based state machine example is a simplified version
4334 of a test for Hypothesis's example database implementation. An example
4335 database maps keys to sets of values, and in this test we compare one
4336 implementation of it to a simplified in memory model of its behaviour,
4337 which just stores the same values in a Python dict. The test then runs
4338 operations against both the real database and the in-memory representa‐
4339 tion of it and looks for discrepancies in their behaviour.
4340 import shutil
4342 import tempfile
4343 from collections import defaultdict
4344 import hypothesis.strategies as st
4346 from hypothesis.database import DirectoryBasedExampleDatabase
4347 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
4348 class DatabaseComparison(RuleBasedStateMachine):
4351 def __init__(self):
4352 super().__init__()
4353 self.tempd = tempfile.mkdtemp()
4354 self.database = DirectoryBasedExampleDatabase(self.tempd)
4355 self.model = defaultdict(set)
4356 keys = Bundle("keys")
4358 values = Bundle("values")
4359 @rule(target=keys, k=st.binary())
4361 def add_key(self, k):
4362 return k
4363 @rule(target=values, v=st.binary())
4365 def add_value(self, v):
4366 return v
4367 @rule(k=keys, v=values)
4369 def save(self, k, v):
4370 self.model[k].add(v)
4371 self.database.save(k, v)
4372 @rule(k=keys, v=values)
4374 def delete(self, k, v):
4375 self.model[k].discard(v)
4376 self.database.delete(k, v)
4377 @rule(k=keys)
4379 def values_agree(self, k):
4380 assert set(self.database.fetch(k)) == self.model[k]
4381 def teardown(self):
4383 shutil.rmtree(self.tempd)
4384 TestDBComparison = DatabaseComparison.TestCase
4387 In this we declare two bundles - one for keys, and one for values. We
4389 have two trivial rules which just populate them with data (k and v),
4390 and three non-trivial rules: save saves a value under a key and delete
4391 removes a value from a key, in both cases also updating the model of
4392 what should be in the database. values_agree then checks that the con‐
4393 tents of the database agrees with the model for a particular key.
4394 We can then integrate this into our test suite by getting a unittest
4396 TestCase from it:
4397 TestTrees = DatabaseComparison.TestCase
4399 # Or just run with pytest's unittest support
4401 if __name__ == "__main__":
4402 unittest.main()
4403 This test currently passes, but if we comment out the line where we
4405 call self.model[k].discard(v), we would see the following output when
4406 run under pytest:
4407 AssertionError: assert set() == {b''}
4409 ------------ Hypothesis ------------
4411 state = DatabaseComparison()
4413 var1 = state.add_key(k=b'')
4414 var2 = state.add_value(v=var1)
4415 state.save(k=var1, v=var2)
4416 state.delete(k=var1, v=var2)
4417 state.values_agree(k=var1)
4418 state.teardown()
4419 Note how it's printed out a very short program that will demonstrate
4421 the problem. The output from a rule based state machine should gener‐
4422 ally be pretty close to Python code - if you have custom repr implemen‐
4423 tations that don't return valid Python then it might not be, but most
4424 of the time you should just be able to copy and paste the code into a
4425 test to reproduce it.
4426 You can control the detailed behaviour with a settings object on the
4428 TestCase (this is a normal hypothesis settings object using the de‐
4429 faults at the time the TestCase class was first referenced). For exam‐
4430 ple if you wanted to run fewer examples with larger programs you could
4431 change the settings to:
4432 DatabaseComparison.TestCase.settings = settings(
4434 max_examples=50, stateful_step_count=100
4435 )
4436 Which doubles the number of steps each program runs and halves the num‐
4438 ber of test cases that will be run.
4439 Rules
4441 As said earlier, rules are the most common feature used in RuleBased‐
4442 StateMachine. They are defined by applying the rule() decorator on a
4443 function. Note that RuleBasedStateMachine must have at least one rule
4444 defined and that a single function cannot be used to define multiple
4445 rules (this to avoid having multiple rules doing the same things). Due
4446 to the stateful execution method, rules generally cannot take arguments
4447 from other sources such as fixtures or pytest.mark.parametrize - con‐
4448 sider providing them via a strategy such as sampled_from() instead.
4449 hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)
4451 Decorator for RuleBasedStateMachine. Any Bundle present in tar‐
4452 get or targets will define where the end result of this function
4453 should go. If both are empty then the end result will be dis‐
4454 carded.
4455 target must be a Bundle, or if the result should go to multiple
4457 bundles you can pass a tuple of them as the targets argument.
4458 It is invalid to use both arguments for a single rule. If the
4459 result should go to exactly one of several bundles, define a
4460 separate rule for each case.
4461 kwargs then define the arguments that will be passed to the
4463 function invocation. If their value is a Bundle, or if it is
4464 consumes(b) where b is a Bundle, then values that have previ‐
4465 ously been produced for that bundle will be provided. If con‐
4466 sumes is used, the value will also be removed from the bundle.
4467 Any other kwargs should be strategies and values from them will
4469 be provided.
4470 hypothesis.stateful.consumes(bundle)
4472 When introducing a rule in a RuleBasedStateMachine, this func‐
4473 tion can be used to mark bundles from which each value used in a
4474 step with the given rule should be removed. This function re‐
4475 turns a strategy object that can be manipulated and combined
4476 like any other.
4477 For example, a rule declared with
4479 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
4481 sumes(b3)))
4482 will consume a value from Bundle b2 and several values from Bun‐
4484 dle b3 to populate value2 and value3 each time it is executed.
4485 hypothesis.stateful.multiple(*args)
4487 This function can be used to pass multiple results to the tar‐
4488 get(s) of a rule. Just use return multiple(result1, result2,
4489 ...) in your rule.
4490 It is also possible to use return multiple() with no arguments
4492 in order to end a rule without passing any result.
4493 Initializes
4495 Initializes are a special case of rules that are guaranteed to be run
4496 at most once at the beginning of a run (i.e. before any normal rule is
4497 called). Note if multiple initialize rules are defined, they may be
4498 called in any order, and that order will vary from run to run.
4499 Initializes are typically useful to populate bundles:
4501 hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)
4503 Decorator for RuleBasedStateMachine.
4504 An initialize decorator behaves like a rule, but all @initial‐
4506 ize() decorated methods will be called before any @rule() deco‐
4507 rated methods, in an arbitrary order. Each @initialize() method
4508 will be called exactly once per run, unless one raises an excep‐
4509 tion - after which only the .teardown() method will be run.
4510 @initialize() methods may not have preconditions.
4511 import hypothesis.strategies as st
4513 from hypothesis.stateful import Bundle, RuleBasedStateMachine, initialize, rule
4514 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
4516 class NumberModifier(RuleBasedStateMachine):
4519 folders = Bundle("folders")
4521 files = Bundle("files")
4522 @initialize(target=folders)
4524 def init_folders(self):
4525 return "/"
4526 @rule(target=folders, name=name_strategy)
4528 def create_folder(self, parent, name):
4529 return f"{parent}/{name}"
4530 @rule(target=files, name=name_strategy)
4532 def create_file(self, parent, name):
4533 return f"{parent}/{name}"
4534 Preconditions
4536 While it's possible to use assume() in RuleBasedStateMachine rules, if
4537 you use it in only a few rules you can quickly run into a situation
4538 where few or none of your rules pass their assumptions. Thus, Hypothe‐
4539 sis provides a precondition() decorator to avoid this problem. The
4540 precondition() decorator is used on rule-decorated functions, and must
4541 be given a function that returns True or False based on the RuleBased‐
4542 StateMachine instance.
4543 hypothesis.stateful.precondition(precond)
4545 Decorator to apply a precondition for rules in a RuleBased‐
4546 StateMachine. Specifies a precondition for a rule to be consid‐
4547 ered as a valid step in the state machine, which is more effi‐
4548 cient than using assume() within the rule. The precond function
4549 will be called with the instance of RuleBasedStateMachine and
4550 should return True or False. Usually it will need to look at at‐
4551 tributes on that instance.
4552 For example:
4554 class MyTestMachine(RuleBasedStateMachine):
4556 state = 1
4557 @precondition(lambda self: self.state != 0)
4559 @rule(numerator=integers())
4560 def divide_with(self, numerator):
4561 self.state = numerator / self.state
4562 If multiple preconditions are applied to a single rule, it is
4564 only considered a valid step when all of them return True. Pre‐
4565 conditions may be applied to invariants as well as rules.
4566 from hypothesis.stateful import RuleBasedStateMachine, precondition, rule
4568 class NumberModifier(RuleBasedStateMachine):
4571 num = 0
4573 @rule()
4575 def add_one(self):
4576 self.num += 1
4577 @precondition(lambda self: self.num != 0)
4579 @rule()
4580 def divide_with_one(self):
4581 self.num = 1 / self.num
4582 By using precondition() here instead of assume(), Hypothesis can filter
4584 the inapplicable rules before running them. This makes it much more
4585 likely that a useful sequence of steps will be generated.
4586 Note that currently preconditions can't access bundles; if you need to
4588 use preconditions, you should store relevant data on the instance in‐
4589 stead.
4590 Invariants
4592 Often there are invariants that you want to ensure are met after every
4593 step in a process. It would be possible to add these as rules that are
4594 run, but they would be run zero or multiple times between other rules.
4595 Hypothesis provides a decorator that marks a function to be run after
4596 every step.
4597 hypothesis.stateful.invariant(*, check_during_init=False)
4599 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
4600 chine. The decorated function will be run after every rule and
4601 can raise an exception to indicate failed invariants.
4602 For example:
4604 class MyTestMachine(RuleBasedStateMachine):
4606 state = 1
4607 @invariant()
4609 def is_nonzero(self):
4610 assert self.state != 0
4611 By default, invariants are only checked after all @initialize()
4613 rules have been run. Pass check_during_init=True for invariants
4614 which can also be checked during initialization.
4615 from hypothesis.stateful import RuleBasedStateMachine, invariant, rule
4617 class NumberModifier(RuleBasedStateMachine):
4620 num = 0
4622 @rule()
4624 def add_two(self):
4625 self.num += 2
4626 if self.num > 50:
4627 self.num += 1
4628 @invariant()
4630 def divide_with_one(self):
4631 assert self.num % 2 == 0
4632 NumberTest = NumberModifier.TestCase
4635 Invariants can also have precondition()s applied to them, in which case
4637 they will only be run if the precondition function returns true.
4638 Note that currently invariants can't access bundles; if you need to use
4640 invariants, you should store relevant data on the instance instead.
4641 More fine grained control
4643 If you want to bypass the TestCase infrastructure you can invoke these
4644 manually. The stateful module exposes the function run_state_ma‐
4645 chine_as_test, which takes an arbitrary function returning a RuleBased‐
4646 StateMachine and an optional settings parameter and does the same as
4647 the class based runTest provided.
4648 This is not recommended as it bypasses some important internal func‐
4650 tions, including reporting of statistics such as runtimes and event()
4651 calls. It was originally added to support custom __init__ methods, but
4652 you can now use initialize() rules instead.
4653
4655 Hypothesis does its level best to be compatible with everything you
4656 could possibly need it to be compatible with. Generally you should just
4657 try it and expect it to work. If it doesn't, you can be surprised and
4658 check this document for the details.
4659 Hypothesis versions
4661 Backwards compatibility is better than backporting fixes, so we use se‐
4662 mantic versioning and only support the most recent version of Hypothe‐
4663 sis. See support for more information.
4664 Documented APIs will not break except between major version bumps. All
4666 APIs mentioned in this documentation are public unless explicitly noted
4667 as provisional, in which case they may be changed in minor releases.
4668 Undocumented attributes, modules, and behaviour may include breaking
4669 changes in patch releases.
4670 Python versions
4672 Hypothesis is supported and tested on CPython 3.7+, i.e. all versions
4673 of CPython with upstream support, along with PyPy for the same ver‐
4674 sions. 32-bit builds of CPython also work, though we only test them on
4675 Windows.
4676 In general Hypothesis does not officially support anything except the
4678 latest patch release of any version of Python it supports. Earlier re‐
4679 leases should work and bugs in them will get fixed if reported, but
4680 they're not tested in CI and no guarantees are made.
4681 Operating systems
4683 In theory Hypothesis should work anywhere that Python does. In practice
4684 it is only known to work and regularly tested on OS X, Windows and
4685 Linux, and you may experience issues running it elsewhere.
4686 If you're using something else and it doesn't work, do get in touch and
4688 I'll try to help, but unless you can come up with a way for me to run a
4689 CI server on that operating system it probably won't stay fixed due to
4690 the inevitable march of time.
4691 Testing frameworks
4693 In general Hypothesis goes to quite a lot of effort to generate things
4694 that look like normal Python test functions that behave as closely to
4695 the originals as possible, so it should work sensibly out of the box
4696 with every test framework.
4697 If your testing relies on doing something other than calling a function
4699 and seeing if it raises an exception then it probably won't work out of
4700 the box. In particular things like tests which return generators and
4701 expect you to do something with them (e.g. nose's yield based tests)
4702 will not work. Use a decorator or similar to wrap the test to take this
4703 form, or ask the framework maintainer to support our hooks for insert‐
4704 ing such a wrapper later.
4705 In terms of what's actually known to work:
4707 • Hypothesis integrates as smoothly with pytest and unittest as we
4709 can make it, and this is verified as part of the CI.
4710 • pytest fixtures work in the usual way for tests that have been
4712 decorated with @given - just avoid passing a strategy for each ar‐
4713 gument that will be supplied by a fixture. However, each fixture
4714 will run once for the whole function, not once per example. Deco‐
4715 rating a fixture function with @given is meaningless.
4716 • The python:unittest.mock.patch() decorator works with @given, but
4718 we recommend using it as a context manager within the decorated
4719 test to ensure that the mock is per-test-case and avoid poor in‐
4720 teractions with Pytest fixtures.
4721 • Nose works fine with Hypothesis, and this is tested as part of the
4723 CI. yield based tests simply won't work.
4724 • Integration with Django's testing requires use of the hypothe‐
4726 sis-django extra. The issue is that in Django's tests' normal
4727 mode of execution it will reset the database once per test rather
4728 than once per example, which is not what you want.
4729 • Coverage works out of the box with Hypothesis; our own test suite
4731 has 100% branch coverage.
4732 Optional packages
4734 The supported versions of optional packages, for strategies in hypothe‐
4735 sis.extra, are listed in the documentation for that extra. Our general
4736 goal is to support all versions that are supported upstream.
4737 Regularly verifying this
4739 Everything mentioned above as explicitly supported is checked on every
4740 commit with GitHub Actions. Our continuous delivery pipeline runs all
4741 of these checks before publishing each release, so when we say they're
4742 supported we really mean it.
4743
4745 This is a collection of examples of how to use Hypothesis in interest‐
4746 ing ways. It's small for now but will grow over time.
4747 All of these examples are designed to be run under pytest, and nose
4749 should work too.
4750 How not to sort by a partial order
4752 The following is an example that's been extracted and simplified from a
4753 real bug that occurred in an earlier version of Hypothesis. The real
4754 bug was a lot harder to find.
4755 Suppose we've got the following type:
4757 class Node:
4759 def __init__(self, label, value):
4760 self.label = label
4761 self.value = tuple(value)
4762 def __repr__(self):
4764 return f"Node({self.label!r}, {self.value!r})"
4765 def sorts_before(self, other):
4767 if len(self.value) >= len(other.value):
4768 return False
4769 return other.value[: len(self.value)] == self.value
4770 Each node is a label and a sequence of some data, and we have the rela‐
4772 tionship sorts_before meaning the data of the left is an initial seg‐
4773 ment of the right. So e.g. a node with value [1, 2] will sort before a
4774 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
4775 before the other.
4776 We have a list of nodes, and we want to topologically sort them with
4778 respect to this ordering. That is, we want to arrange the list so that
4779 if x.sorts_before(y) then x appears earlier in the list than y. We
4780 naively think that the easiest way to do this is to extend the partial
4781 order defined here to a total order by breaking ties arbitrarily and
4782 then using a normal sorting algorithm. So we define the following code:
4783 from functools import total_ordering
4785 @total_ordering
4788 class TopoKey:
4789 def __init__(self, node):
4790 self.value = node
4791 def __lt__(self, other):
4793 if self.value.sorts_before(other.value):
4794 return True
4795 if other.value.sorts_before(self.value):
4796 return False
4797 return self.value.label < other.value.label
4799 def sort_nodes(xs):
4802 xs.sort(key=TopoKey)
4803 This takes the order defined by sorts_before and extends it by breaking
4805 ties by comparing the node labels.
4806 But now we want to test that it works.
4808 First we write a function to verify that our desired outcome holds:
4810 def is_prefix_sorted(xs):
4812 for i in range(len(xs)):
4813 for j in range(i + 1, len(xs)):
4814 if xs[j].sorts_before(xs[i]):
4815 return False
4816 return True
4817 This will return false if it ever finds a pair in the wrong order and
4819 return true otherwise.
4820 Given this function, what we want to do with Hypothesis is assert that
4822 for all sequences of nodes, the result of calling sort_nodes on it is
4823 sorted.
4824 First we need to define a strategy for Node:
4826 import hypothesis.strategies as st
4828 NodeStrategy = st.builds(Node, st.integers(), st.lists(st.booleans(), max_size=10))
4830 We want to generate short lists of values so that there's a decent
4832 chance of one being a prefix of the other (this is also why the choice
4833 of bool as the elements). We then define a strategy which builds a node
4834 out of an integer and one of those short lists of booleans.
4835 We can now write a test:
4837 from hypothesis import given
4839 @given(st.lists(NodeStrategy))
4842 def test_sorting_nodes_is_prefix_sorted(xs):
4843 sort_nodes(xs)
4844 assert is_prefix_sorted(xs)
4845 this immediately fails with the following example:
4847 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
4849 The reason for this is that because False is not a prefix of (True,
4851 True) nor vice versa, sorting things the first two nodes are equal be‐
4852 cause they have equal labels. This makes the whole order non-transi‐
4853 tive and produces basically nonsense results.
4854 But this is pretty unsatisfying. It only works because they have the
4856 same label. Perhaps we actually wanted our labels to be unique. Let's
4857 change the test to do that.
4858 def deduplicate_nodes_by_label(nodes):
4860 table = {node.label: node for node in nodes}
4861 return list(table.values())
4862 We define a function to deduplicate nodes by labels, and can now map
4864 that over a strategy for lists of nodes to give us a strategy for lists
4865 of nodes with unique labels:
4866 @given(st.lists(NodeStrategy).map(deduplicate_nodes_by_label))
4868 def test_sorting_nodes_is_prefix_sorted(xs):
4869 sort_nodes(xs)
4870 assert is_prefix_sorted(xs)
4871 Hypothesis quickly gives us an example of this still being wrong:
4873 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]
4875 Now this is a more interesting example. None of the nodes will sort
4877 equal. What is happening here is that the first node is strictly less
4878 than the last node because (False,) is a prefix of (False, False). This
4879 is in turn strictly less than the middle node because neither is a pre‐
4880 fix of the other and -2 < -1. The middle node is then less than the
4881 first node because -1 < 0.
4882 So, convinced that our implementation is broken, we write a better one:
4884 def sort_nodes(xs):
4886 for i in range(1, len(xs)):
4887 j = i - 1
4888 while j >= 0:
4889 if xs[j].sorts_before(xs[j + 1]):
4890 break
4891 xs[j], xs[j + 1] = xs[j + 1], xs[j]
4892 j -= 1
4893 This is just insertion sort slightly modified - we swap a node back‐
4895 wards until swapping it further would violate the order constraints.
4896 The reason this works is because our order is a partial order already
4897 (this wouldn't produce a valid result for a general topological sorting
4898 - you need the transitivity).
4899 We now run our test again and it passes, telling us that this time
4901 we've successfully managed to sort some nodes without getting it com‐
4902 pletely wrong. Go us.
4903 Time zone arithmetic
4905 This is an example of some tests for pytz which check that various
4906 timezone conversions behave as you would expect them to. These tests
4907 should all pass, and are mostly a demonstration of some useful sorts of
4908 thing to test with Hypothesis, and how the datetimes() strategy works.
4909 from datetime import timedelta
4911 # The datetimes strategy is naive by default, so tell it to use timezones
4913 aware_datetimes = st.datetimes(timezones=st.timezones())
4914 @given(aware_datetimes, st.timezones(), st.timezones())
4917 def test_convert_via_intermediary(dt, tz1, tz2):
4918 """Test that converting between timezones is not affected
4919 by a detour via another timezone.
4920 """
4921 assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
4922 @given(aware_datetimes, st.timezones())
4925 def test_convert_to_and_fro(dt, tz2):
4926 """If we convert to a new timezone and back to the old one
4927 this should leave the result unchanged.
4928 """
4929 tz1 = dt.tzinfo
4930 assert dt == dt.astimezone(tz2).astimezone(tz1)
4931 @given(aware_datetimes, st.timezones())
4934 def test_adding_an_hour_commutes(dt, tz):
4935 """When converting between timezones it shouldn't matter
4936 if we add an hour here or add an hour there.
4937 """
4938 an_hour = timedelta(hours=1)
4939 assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
4940 @given(aware_datetimes, st.timezones())
4943 def test_adding_a_day_commutes(dt, tz):
4944 """When converting between timezones it shouldn't matter
4945 if we add a day here or add a day there.
4946 """
4947 a_day = timedelta(days=1)
4948 assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
4949 Condorcet's paradox
4951 A classic paradox in voting theory, called Condorcet's paradox, is that
4952 majority preferences are not transitive. That is, there is a population
4953 and a set of three candidates A, B and C such that the majority of the
4954 population prefer A to B, B to C and C to A.
4955 Wouldn't it be neat if we could use Hypothesis to provide an example of
4957 this?
4958 Well as you can probably guess from the presence of this section, we
4960 can! The main trick is to decide how we want to represent the result
4961 of an election - for this example, we'll use a list of "votes", where
4962 each vote is a list of candidates in the voters preferred order. With‐
4963 out further ado, here is the code:
4964 from collections import Counter
4966 from hypothesis import given
4968 from hypothesis.strategies import lists, permutations
4969 # We need at least three candidates and at least three voters to have a
4972 # paradox; anything less can only lead to victories or at worst ties.
4973 @given(lists(permutations(["A", "B", "C"]), min_size=3))
4974 def test_elections_are_transitive(election):
4975 all_candidates = {"A", "B", "C"}
4976 # First calculate the pairwise counts of how many prefer each candidate
4978 # to the other
4979 counts = Counter()
4980 for vote in election:
4981 for i in range(len(vote)):
4982 for j in range(i + 1, len(vote)):
4983 counts[(vote[i], vote[j])] += 1
4984 # Now look at which pairs of candidates one has a majority over the
4986 # other and store that.
4987 graph = {}
4988 for i in all_candidates:
4989 for j in all_candidates:
4990 if counts[(i, j)] > counts[(j, i)]:
4991 graph.setdefault(i, set()).add(j)
4992 # Now for each triple assert that it is transitive.
4994 for x in all_candidates:
4995 for y in graph.get(x, ()):
4996 for z in graph.get(y, ()):
4997 assert x not in graph.get(z, ())
4998 The example Hypothesis gives me on my first run (your mileage may of
5000 course vary) is:
5001 [["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]
5003 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
5005 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
5006 and 2) prefer C to A. This is in fact basically the canonical example
5007 of the voting paradox.
5008 Fuzzing an HTTP API
5010 Hypothesis's support for testing HTTP services is somewhat nascent.
5011 There are plans for some fully featured things around this, but right
5012 now they're probably quite far down the line.
5013 But you can do a lot yourself without any explicit support! Here's a
5015 script I wrote to throw arbitrary data against the API for an entirely
5016 fictitious service called Waspfinder (this is only lightly obfuscated
5017 and you can easily figure out who I'm actually talking about, but I
5018 don't want you to run this code and hammer their API without their per‐
5019 mission).
5020 All this does is use Hypothesis to generate arbitrary JSON data match‐
5022 ing the format their API asks for and check for 500 errors. More ad‐
5023 vanced tests which then use the result and go on to do other things are
5024 definitely also possible. The schemathesis package provides an excel‐
5025 lent example of this!
5026 import math
5028 import os
5029 import random
5030 import time
5031 import unittest
5032 from collections import namedtuple
5033 import requests
5035 from hypothesis import assume, given, strategies as st
5037 Goal = namedtuple("Goal", ("slug",))
5039 # We just pass in our API credentials via environment variables.
5042 waspfinder_token = os.getenv("WASPFINDER_TOKEN")
5043 waspfinder_user = os.getenv("WASPFINDER_USER")
5044 assert waspfinder_token is not None
5045 assert waspfinder_user is not None
5046 GoalData = st.fixed_dictionaries(
5048 {
5049 "title": st.text(),
5050 "goal_type": st.sampled_from(
5051 ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
5052 ),
5053 "goaldate": st.one_of(st.none(), st.floats()),
5054 "goalval": st.one_of(st.none(), st.floats()),
5055 "rate": st.one_of(st.none(), st.floats()),
5056 "initval": st.floats(),
5057 "panic": st.floats(),
5058 "secret": st.booleans(),
5059 "datapublic": st.booleans(),
5060 }
5061 )
5062 needs2 = ["goaldate", "goalval", "rate"]
5065 class WaspfinderTest(unittest.TestCase):
5068 @given(GoalData)
5069 def test_create_goal_dry_run(self, data):
5070 # We want slug to be unique for each run so that multiple test runs
5071 # don't interfere with each other. If for some reason some slugs trigger
5072 # an error and others don't we'll get a Flaky error, but that's OK.
5073 slug = hex(random.getrandbits(32))[2:]
5074 # Use assume to guide us through validation we know about, otherwise
5076 # we'll spend a lot of time generating boring examples.
5077 # Title must not be empty
5079 assume(data["title"])
5080 # Exactly two of these values should be not None. The other will be
5082 # inferred by the API.
5083 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
5085 for v in data.values():
5086 if isinstance(v, float):
5087 assume(not math.isnan(v))
5088 data["slug"] = slug
5089 # The API nicely supports a dry run option, which means we don't have
5091 # to worry about the user account being spammed with lots of fake goals
5092 # Otherwise we would have to make sure we cleaned up after ourselves
5093 # in this test.
5094 data["dryrun"] = True
5095 data["auth_token"] = waspfinder_token
5096 for d, v in data.items():
5097 if v is None:
5098 data[d] = "null"
5099 else:
5100 data[d] = str(v)
5101 result = requests.post(
5102 "https://waspfinder.example.com/api/v1/users/"
5103 "%s/goals.json" % (waspfinder_user,),
5104 data=data,
5105 )
5106 # Let's not hammer the API too badly. This will of course make the
5108 # tests even slower than they otherwise would have been, but that's
5109 # life.
5110 time.sleep(1.0)
5111 # For the moment all we're testing is that this doesn't generate an
5113 # internal error. If we didn't use the dry run option we could have
5114 # then tried doing more with the result, but this is a good start.
5115 self.assertNotEqual(result.status_code, 500)
5116 if __name__ == "__main__":
5119 unittest.main()
5120
5122 The Hypothesis community is small for the moment but is full of excel‐
5123 lent people who can answer your questions and help you out. Please do
5124 join us. The two major places for community discussion are:
5125 • The mailing list.
5127 • An IRC channel, #hypothesis on freenode, which is more active than
5129 the mailing list.
5130 Feel free to use these to ask for help, provide feedback, or discuss
5132 anything remotely Hypothesis related at all. If you post a question on
5133 Stack Overflow, please use the python-hypothesis tag!
5134 Please note that the Hypothesis code of conduct applies in all Hypothe‐
5136 sis community spaces.
5137 If you would like to cite Hypothesis, please consider our suggested ci‐
5139 tation.
5140 If you like repo badges, we suggest the following badge, which you can
5142 add with reStructuredText or Markdown, respectively: [image]
5143 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
5145 :alt: Tested with Hypothesis
5146 :target: https://hypothesis.readthedocs.io
5147 [](https://hypothesis.readthedocs.io/)
5149 Finally, we have a beautiful logo which appears online, and often on
5151 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
5152 [image]
5153 As well as being beautiful, dragonflies actively hunt down bugs for a
5155 living! You can find the images and a usage guide in the brand direc‐
5156 tory on GitHub, or find us at conferences where we often have stickers
5157 and sometimes other swag.
5158
5160 What is Hypothesis for?
5161 From the perspective of a user, the purpose of Hypothesis is to make it
5163 easier for you to write better tests.
5164 From my perspective as the author, that is of course also a purpose of
5166 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
5167 mania for a moment), the larger purpose of Hypothesis is to drag the
5168 world kicking and screaming into a new and terrifying age of high qual‐
5169 ity software.
5170 Software is, as they say, eating the world. Software is also terrible.
5172 It's buggy, insecure and generally poorly thought out. This combination
5173 is clearly a recipe for disaster.
5174 And the state of software testing is even worse. Although it's fairly
5176 uncontroversial at this point that you should be testing your code, can
5177 you really say with a straight face that most projects you've worked on
5178 are adequately tested?
5179 A lot of the problem here is that it's too hard to write good tests.
5181 Your tests encode exactly the same assumptions and fallacies that you
5182 had when you wrote the code, so they miss exactly the same bugs that
5183 you missed when you wrote the code.
5184 Meanwhile, there are all sorts of tools for making testing better that
5186 are basically unused. The original Quickcheck is from 1999 and the ma‐
5187 jority of developers have not even heard of it, let alone used it.
5188 There are a bunch of half-baked implementations for most languages, but
5189 very few of them are worth using.
5190 The goal of Hypothesis is to bring advanced testing techniques to the
5192 masses, and to provide an implementation that is so high quality that
5193 it is easier to use them than it is not to use them. Where I can, I
5194 will beg, borrow and steal every good idea I can find that someone has
5195 had to make software testing better. Where I can't, I will invent new
5196 ones.
5197 Quickcheck is the start, but I also plan to integrate ideas from fuzz
5199 testing (a planned future feature is to use coverage information to
5200 drive example selection, and the example saving database is already in‐
5201 spired by the workflows people use for fuzz testing), and am open to
5202 and actively seeking out other suggestions and ideas.
5203 The plan is to treat the social problem of people not using these ideas
5205 as a bug to which there is a technical solution: Does property-based
5206 testing not match your workflow? That's a bug, let's fix it by figur‐
5207 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
5208 tom data for your application? That's a bug. Let's fix it by figuring
5209 out how to make it easier, or how to take something you're already us‐
5210 ing to specify your data and derive a generator from that automati‐
5211 cally. Find the explanations of these advanced ideas hopelessly obtuse
5212 and hard to follow? That's a bug. Let's provide you with an easy API
5213 that lets you test your code better without a PhD in software verifica‐
5214 tion.
5215 Grand ambitions, I know, and I expect ultimately the reality will be
5217 somewhat less grand, but so far in about three months of development,
5218 Hypothesis has become the most solid implementation of Quickcheck ever
5219 seen in a mainstream language (as long as we don't count Scala as main‐
5220 stream yet), and at the same time managed to significantly push forward
5221 the state of the art, so I think there's reason to be optimistic.
5222
5224 This is a page for listing people who are using Hypothesis and how ex‐
5225 cited they are about that. If that's you and your name is not on the
5226 list, this file is in Git and I'd love it if you sent me a pull request
5227 to fix that.
5228 Stripe
5230 At Stripe we use Hypothesis to test every piece of our machine learning
5231 model training pipeline (powered by scikit). Before we migrated, our
5232 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
5233 resentative at all of our actual very complex data. Because we needed
5234 to craft examples for each test, we took the easy way out and lived
5235 with extremely low test coverage.
5236 Hypothesis changed all that. Once we had our strategies for generating
5238 Dataframes of features it became trivial to slightly customize each
5239 strategy for new tests. Our coverage is now close to 90%.
5240 Full-stop, property-based testing is profoundly more powerful - and has
5242 caught or prevented far more bugs - than our old style of example-based
5243 testing.
5244 Kristian Glass - Director of Technology at LaterPay GmbH
5246 Hypothesis has been brilliant for expanding the coverage of our test
5247 cases, and also for making them much easier to read and understand, so
5248 we're sure we're testing the things we want in the way we want.
5249 Seth Morton
5251 When I first heard about Hypothesis, I knew I had to include it in my
5252 two open-source Python libraries, natsort and fastnumbers . Quite
5253 frankly, I was a little appalled at the number of bugs and "holes" I
5254 found in the code. I can now say with confidence that my libraries are
5255 more robust to "the wild." In addition, Hypothesis gave me the confi‐
5256 dence to expand these libraries to fully support Unicode input, which I
5257 never would have had the stomach for without such thorough testing ca‐
5258 pabilities. Thanks!
5259 Sixty North
5261 At Sixty North we use Hypothesis for testing Segpy an open source
5262 Python library for shifting data between Python data structures and SEG
5263 Y files which contain geophysical data from the seismic reflection sur‐
5264 veys used in oil and gas exploration.
5265 This is our first experience of property-based testing – as opposed to
5267 example-based testing. Not only are our tests more powerful, they are
5268 also much better explanations of what we expect of the production code.
5269 In fact, the tests are much closer to being specifications. Hypothesis
5270 has located real defects in our code which went undetected by tradi‐
5271 tional test cases, simply because Hypothesis is more relentlessly devi‐
5272 ous about test case generation than us mere humans! We found Hypothe‐
5273 sis particularly beneficial for Segpy because SEG Y is an antiquated
5274 format that uses legacy text encodings (EBCDIC) and even a legacy
5275 floating point format we implemented from scratch in Python.
5276 Hypothesis is sure to find a place in most of our future Python code‐
5278 bases and many existing ones too.
5279 mulkieran
5281 Just found out about this excellent QuickCheck for Python implementa‐
5282 tion and ran up a few tests for my bytesize package last night. Refuted
5283 a few hypotheses in the process.
5284 Looking forward to using it with a bunch of other projects as well.
5286 Adam Johnson
5288 I have written a small library to serialize dicts to MariaDB's dynamic
5289 columns binary format, mariadb-dyncol. When I first developed it, I
5290 thought I had tested it really well - there were hundreds of test
5291 cases, some of them even taken from MariaDB's test suite itself. I was
5292 ready to release.
5293 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
5295 Wow! It found bug after bug after bug. Even after a first release, I
5296 thought of a way to make the tests do more validation, which revealed a
5297 further round of bugs! Most impressively, Hypothesis found a compli‐
5298 cated off-by-one error in a condition with 4095 versus 4096 bytes of
5299 data - something that I would never have found.
5300 Long live Hypothesis! (Or at least, property-based testing).
5302 Josh Bronson
5304 Adopting Hypothesis improved bidict's test coverage and significantly
5305 increased our ability to make changes to the code with confidence that
5306 correct behavior would be preserved. Thank you, David, for the great
5307 testing tool.
5308 Cory Benfield
5310 Hypothesis is the single most powerful tool in my toolbox for working
5311 with algorithmic code, or any software that produces predictable output
5312 from a wide range of sources. When using it with Priority, Hypothesis
5313 consistently found errors in my assumptions and extremely subtle bugs
5314 that would have taken months of real-world use to locate. In some
5315 cases, Hypothesis found subtle deviations from the correct output of
5316 the algorithm that may never have been noticed at all.
5317 When it comes to validating the correctness of your tools, nothing
5319 comes close to the thoroughness and power of Hypothesis.
5320 Jon Moore
5322 One extremely satisfied user here. Hypothesis is a really solid imple‐
5323 mentation of property-based testing, adapted well to Python, and with
5324 good features such as failure-case shrinkers. I first used it on a
5325 project where we needed to verify that a vendor's Python and non-Python
5326 implementations of an algorithm matched, and it found about a dozen
5327 cases that previous example-based testing and code inspections had not.
5328 Since then I've been evangelizing for it at our firm.
5329 Russel Winder
5331 I am using Hypothesis as an integral part of my Python workshops. Test‐
5332 ing is an integral part of Python programming and whilst unittest and,
5333 better, pytest can handle example-based testing, property-based testing
5334 is increasingly far more important than example-base testing, and Hy‐
5335 pothesis fits the bill.
5336 Wellfire Interactive
5338 We've been using Hypothesis in a variety of client projects, from test‐
5339 ing Django-related functionality to domain-specific calculations. It
5340 both speeds up and simplifies the testing process since there's so much
5341 less tedious and error-prone work to do in identifying edge cases. Test
5342 coverage is nice but test depth is even nicer, and it's much easier to
5343 get meaningful test depth using Hypothesis.
5344 Cody Kochmann
5346 Hypothesis is being used as the engine for random object generation
5347 with my open source function fuzzer battle_tested which maps all behav‐
5348 iors of a function allowing you to minimize the chance of unexpected
5349 crashes when running code in production.
5350 With how efficient Hypothesis is at generating the edge cases that
5352 cause unexpected behavior occur, battle_tested is able to map out the
5353 entire behavior of most functions in less than a few seconds.
5354 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
5356 ing it.
5357 Merchise Autrement
5359 Just minutes after our first use of hypothesis we uncovered a subtle
5360 bug in one of our most used library. Since then, we have increasingly
5361 used hypothesis to improve the quality of our testing in libraries and
5362 applications as well.
5363 Florian Kromer
5365 At Roboception GmbH I use Hypothesis to implement fully automated
5366 stateless and stateful reliability tests for the 3D sensor rc_visard
5367 and robotic software components .
5368 Thank you very much for creating the (probably) most powerful prop‐
5370 erty-based testing framework.
5371 Reposit Power
5373 With a micro-service architecture, testing between services is made
5374 easy using Hypothesis in integration testing. Ensuring everything is
5375 running smoothly is vital to help maintain a secure network of Virtual
5376 Power Plants.
5377 It allows us to find potential bugs and edge cases with relative ease
5379 and minimal overhead. As our architecture relies on services communi‐
5380 cating effectively, Hypothesis allows us to strictly test for the kind
5381 of data which moves around our services, particularly our backend
5382 Python applications.
5383 Your name goes here
5385 I know there are many more, because I keep finding out about new people
5386 I'd never even heard of using Hypothesis. If you're looking to way to
5387 give back to a tool you love, adding your name here only takes a moment
5388 and would really help a lot. As per instructions at the top, just send
5389 me a pull request and I'll add you to the list.
5390
5392 The following is a non-exhaustive list of open source projects I know
5393 are using Hypothesis. If you're aware of any others please add them to
5394 the list! The only inclusion criterion right now is that if it's a
5395 Python library then it should be available on PyPI.
5396 You can find hundreds more from the Hypothesis page at libraries.io,
5398 and thousands on GitHub. Hypothesis has over 100,000 downloads per
5399 week, and was used by more than 4% of Python users surveyed by the PSF
5400 in 2020.
5401 • aur
5403 • argon2_cffi
5405 • attrs
5407 • axelrod
5409 • bidict
5411 • binaryornot
5413 • brotlicffi
5415 • chardet
5417 • cmph-cffi
5419 • cryptography
5421 • dbus-signature-pyparsing
5423 • dry-python/returns
5425 • fastnumbers
5427 • flocker
5429 • flownetpy
5431 • funsize
5433 • fusion-index
5435 • hyper-h2
5437 • into-dbus-python
5439 • justbases
5441 • justbytes
5443 • loris
5445 • mariadb-dyncol
5447 • mercurial
5449 • natsort
5451 • poliastro
5453 • pretext
5455 • priority
5457 • PyCEbox
5459 • PyPy
5461 • pyrsistent
5463 • python-humble-utils
5465 • pyudev
5467 • qutebrowser
5469 • RubyMarshal
5471 • Segpy
5473 • simoa
5475 • srt
5477 • tchannel
5479 • vdirsyncer
5481 • wcag-contrast-ratio
5483 • yacluster
5485 • yturl
5487
5489 Hypothesis has been eagerly used and extended by the open source commu‐
5490 nity. This page lists extensions and applications; you can find more
5491 or newer packages by searching PyPI by keyword or filter by classifier,
5492 or search libraries.io.
5493 If there's something missing which you think should be here, let us
5495 know!
5496 NOTE:
5498 Being listed on this page does not imply that the Hypothesis main‐
5499 tainers endorse a package.
5500 External strategies
5502 Some packages provide strategies directly:
5503 • hypothesis-fspaths - strategy to generate filesystem paths.
5505 • hypothesis-geojson - strategy to generate GeoJson.
5507 • hypothesis-geometry - strategies to generate geometric objects.
5509 • hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
5511 • hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
5513 • hypothesis-ros - strategies to generate messages and parameters for
5515 the Robot Operating System.
5516 • hypothesis-csv - strategy to generate CSV files.
5518 • hypothesis-networkx - strategy to generate networkx graphs.
5520 • hypothesis-bio - strategies for bioinformatics data, such as DNA,
5522 codons, FASTA, and FASTQ formats.
5523 • hypothesmith - strategy to generate syntatically-valid Python code.
5525 Others provide a function to infer a strategy from some other schema:
5527 • hypothesis-jsonschema - infer strategies from JSON schemas.
5529 • lollipop-hypothesis - infer strategies from lollipop schemas.
5531 • hypothesis-drf - infer strategies from a djangorestframework seri‐
5533 aliser.
5534 • hypothesis-graphql - infer strategies from GraphQL schemas.
5536 • hypothesis-mongoengine - infer strategies from a mongoengine model.
5538 • hypothesis-pb - infer strategies from Protocol Buffer schemas.
5540 Or some other custom integration, such as a "hypothesis" entry point:
5542 • deal is a design-by-contract library with built-in Hypothesis sup‐
5544 port.
5545 • icontract-hypothesis infers strategies from icontract code contracts.
5547 • Pandera schemas all have a .strategy() method, which returns a strat‐
5549 egy for matching DataFrames.
5550 • Pydantic automatically registers constrained types - so builds() and
5552 from_type() "just work" regardless of the underlying implementation.
5553 Other cool things
5555 schemathesis is a tool for testing web applications built with Open API
5556 / Swagger specifications. It reads the schema and generates test cases
5557 which will ensure that the application is compliant with its schema.
5558 The application under test could be written in any language, the only
5559 thing you need is a valid API schema in a supported format. Includes
5560 CLI and convenient pytest integration. Powered by Hypothesis and
5561 hypothesis-jsonschema, inspired by the earlier swagger-conformance li‐
5562 brary.
5563 Trio is an async framework with "an obsessive focus on usability and
5565 correctness", so naturally it works with Hypothesis! pytest-trio in‐
5566 cludes a custom hook that allows @given(...) to work with Trio-style
5567 async test functions, and hypothesis-trio includes stateful testing ex‐
5568 tensions to support concurrent programs.
5569 pymtl3 is "an open-source Python-based hardware generation, simulation,
5571 and verification framework with multi-level hardware modeling support",
5572 which ships with Hypothesis integrations to check that all of those
5573 levels are equivalent, from function-level to register-transfer level
5574 and even to hardware.
5575 libarchimedes makes it easy to use Hypothesis in the Hy language, a
5577 Lisp embedded in Python.
5578 battle_tested is a fuzzing tool that will show you how your code can
5580 fail - by trying all kinds of inputs and reporting whatever happens.
5581 pytest-subtesthack functions as a workaround for issue #377.
5583 returns uses Hypothesis to verify that Higher Kinded Types correctly
5585 implement functor, applicative, monad, and other laws; allowing a
5586 declarative approach to be combined with traditional pythonic code.
5587 icontract-hypothesis includes a ghostwriter for test files and IDE in‐
5589 tegrations such as icontract-hypothesis-vim,
5590 icontract-hypothesis-pycharm, and icontract-hypothesis-vscode - you can
5591 run a quick 'smoke test' with only a few keystrokes for any type-anno‐
5592 tated function, even if it doesn't have any contracts!
5593 Writing an extension
5595 See CONTRIBUTING.rst for more information.
5596 New strategies can be added to Hypothesis, or published as an external
5598 package on PyPI - either is fine for most strategies. If in doubt, ask!
5599 It's generally much easier to get things working outside, because
5601 there's more freedom to experiment and fewer requirements in stability
5602 and API style. We're happy to review and help with external packages as
5603 well as pull requests!
5604 If you're thinking about writing an extension, please name it hypothe‐
5606 sis-{something} - a standard prefix makes the community more visible
5607 and searching for extensions easier. And make sure you use the Frame‐
5608 work :: Hypothesis trove classifier!
5609 On the other hand, being inside gets you access to some deeper imple‐
5611 mentation features (if you need them) and better long-term guarantees
5612 about maintenance. We particularly encourage pull requests for new
5613 composable primitives that make implementing other strategies easier,
5614 or for widely used types in the standard library. Strategies for other
5615 things are also welcome; anything with external dependencies just goes
5616 in hypothesis.extra.
5617 Tools such as assertion helpers may also need to check whether the cur‐
5619 rent test is using Hypothesis:
5620 hypothesis.currently_in_test_context()
5622 Return True if the calling code is currently running inside an
5623 @given or stateful test, False otherwise.
5624 This is useful for third-party integrations and assertion
5626 helpers which may be called from traditional or property-based
5627 tests, but can only use assume() or target() in the latter case.
5628 Hypothesis integration via setuptools entry points
5630 If you would like to ship Hypothesis strategies for a custom type - ei‐
5631 ther as part of the upstream library, or as a third-party extension,
5632 there's a catch: from_type() only works after the corresponding call to
5633 register_type_strategy(), and you'll have the same problem with regis‐
5634 ter_random(). This means that either
5635 • you have to try importing Hypothesis to register the strategy when
5637 your library is imported, though that's only useful at test time, or
5638 • the user has to call a 'register the strategies' helper that you pro‐
5640 vide before running their tests
5641 Entry points are Python's standard way of automating the latter: when
5643 you register a "hypothesis" entry point in your setup.py, we'll import
5644 and run it automatically when hypothesis is imported. Nothing happens
5645 unless Hypothesis is already in use, and it's totally seamless for
5646 downstream users!
5647 Let's look at an example. You start by adding a function somewhere in
5649 your package that does all the Hypothesis-related setup work:
5650 # mymodule.py
5652 class MyCustomType:
5655 def __init__(self, x: int):
5656 assert x >= 0, f"got {x}, but only positive numbers are allowed"
5657 self.x = x
5658 def _hypothesis_setup_hook():
5661 import hypothesis.strategies as st
5662 st.register_type_strategy(MyCustomType, st.integers(min_value=0))
5664 and then tell setuptools that this is your "hypothesis" entry point:
5666 # setup.py
5668 # You can list a module to import by dotted name
5670 entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}
5671 # Or name a specific function too, and Hypothesis will call it for you
5673 entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}
5674 And that's all it takes!
5676 NOTE:
5678 On Python 3.7, where the importlib.metadata module is not in the
5679 standard library, loading entry points requires either the
5680 importlib_metadata (preferred) or setuptools (fallback) package to
5681 be installed.
5682 Interaction with pytest-cov
5684 Because pytest does not load plugins from entrypoints in any particular
5685 order, using the Hypothesis entrypoint may import your module before
5686 pytest-cov starts. This is a known issue, but there are workarounds.
5687 You can use coverage run pytest ... instead of pytest --cov ..., opting
5689 out of the pytest plugin entirely. Alternatively, you can ensure that
5690 Hypothesis is loaded after coverage measurement is started by disabling
5691 the entrypoint, and loading our pytest plugin from your conftest.py in‐
5692 stead:
5693 echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
5695 pytest -p "no:hypothesispytest" ...
5696
5698 This is a record of all past Hypothesis releases and what went into
5699 them, in reverse chronological order. All previous releases should
5700 still be available on PyPI.
5701 Hypothesis 6.x
5703 6.39.1 - 2022-03-03
5704 This patch fixes a regression where the bound inner function
5705 (your_test.hypothesis.inner_test) would be invoked with positional ar‐
5706 guments rather than passing them by name, which broke pytest-asyncio (‐
5707 issue #3245).
5708 6.39.0 - 2022-03-01
5710 This release improves Hypothesis' handling of positional-only argu‐
5711 ments, which are now allowed @st.composite strategies.
5712 On Python 3.8 and later, the first arguments to builds() and
5714 from_model() are now natively positional-only. In cases which were al‐
5715 ready errors, the TypeError from incorrect usage will therefore be
5716 raises immediately when the function is called, rather than when the
5717 strategy object is used.
5718 6.38.0 - 2022-02-26
5720 This release makes floats() error consistently when your floating-point
5721 hardware has been configured to violate IEEE-754 for subnormal numbers,
5722 instead of only when an internal assertion was tripped (issue #3092).
5723 If this happens to you, passing allow_subnormal=False will suppress the
5725 explicit error. However, we strongly recommend fixing the root cause
5726 by disabling global-effect unsafe-math compiler options instead, or at
5727 least consulting e.g. Simon Byrne's Beware of fast-math explainer
5728 first.
5729 6.37.2 - 2022-02-21
5731 This patch fixes a bug in stateful testing, where returning a single
5732 value wrapped in multiple() would be printed such that the assigned
5733 variable was a tuple rather than the single element (issue #3236).
5734 6.37.1 - 2022-02-21
5736 This patch fixes a warning under pytest 7 relating to our rich trace‐
5737 back display logic (issue #3223).
5738 6.37.0 - 2022-02-18
5740 When distinguishing multiple errors, Hypothesis now looks at the inner
5741 exceptions of PEP 654 ExceptionGroups.
5742 6.36.2 - 2022-02-13
5744 This patch updates our vendored list of top-level domains, which is
5745 used by the provisional domains() strategy.
5746 6.36.1 - 2022-01-31
5748 This patch fixes some deprecation warnings from pytest 7.0, along with
5749 some code formatting and docs updates.
5750 6.36.0 - 2022-01-19
5752 This release disallows using python:typing.Final with from_type() and
5753 register_type_strategy().
5754 Why? Because Final can only be used during class definition. We don't
5756 generate class attributes.
5757 It also does not make sense as a runtime type on its own.
5759 6.35.1 - 2022-01-17
5761 This patch fixes hypothesis write output highlighting with rich version
5762 12.0 and later.
5763 6.35.0 - 2022-01-08
5765 This release disallows using python:typing.ClassVar with from_type()
5766 and register_type_strategy().
5767 Why? Because ClassVar can only be used during class definition. We
5769 don't generate class attributes.
5770 It also does not make sense as a runtime type on its own.
5772 6.34.2 - 2022-01-05
5774 This patch updates our vendored list of top-level domains, which is
5775 used by the provisional domains() strategy.
5776 6.34.1 - 2021-12-31
5778 This patch fixes issue #3169, an extremely rare bug which would trigger
5779 if an internal least-recently-reused cache dropped a newly added entry
5780 immediately after it was added.
5781 6.34.0 - 2021-12-31
5783 This release fixes issue #3133 and issue #3144, where attempting to
5784 generate Pandas series of lists or sets would fail with confusing er‐
5785 rors if you did not specify dtype=object.
5786 6.33.0 - 2021-12-30
5788 This release disallows using python:typing.TypeAlias with from_type()
5789 and register_type_strategy().
5790 Why? Because TypeAlias is not really a type, it is a tag for type
5792 checkers that some expression is a type alias, not something else.
5793 It does not make sense for Hypothesis to resolve it as a strategy.
5795 References issue #2978.
5796 6.32.1 - 2021-12-23
5798 This patch updates our autoformatting tools, improving our code style
5799 without any API changes.
5800 6.32.0 - 2021-12-23
5802 This release drops support for Python 3.6, which reached end of life
5803 upstream on 2021-12-23.
5804 6.31.6 - 2021-12-15
5806 This patch adds a temporary hook for a downstream tool, which is not
5807 part of the public API.
5808 6.31.5 - 2021-12-14
5810 This release updates our copyright headers to use a general authorship
5811 statement and omit the year.
5812 6.31.4 - 2021-12-11
5814 This patch makes the .example() method more representative of test-time
5815 data generation, albeit often at a substantial cost to readability (‐
5816 issue #3182).
5817 6.31.3 - 2021-12-10
5819 This patch improves annotations on some of Hypothesis' internal func‐
5820 tions, in order to deobfuscate the signatures of some strategies. In
5821 particular, strategies shared between hypothesis.extra.numpy and the
5822 hypothesis.extra.array_api extra will benefit from this patch.
5823 6.31.2 - 2021-12-10
5825 This patch fix invariants display in stateful falsifying examples (‐
5826 issue #3185).
5827 6.31.1 - 2021-12-10
5829 This patch updates xps.indices() so no flat indices are generated, i.e.
5830 generated indices will now always explicitly cover each axes of an ar‐
5831 ray if no ellipsis is present. This is to be consistent with a specifi‐
5832 cation change that dropped support for flat indexing (#272).
5833 6.31.0 - 2021-12-09
5835 This release makes us compatible with Django 4.0, in particular by
5836 adding support for use of zoneinfo timezones (though we respect the new
5837 USE_DEPRECATED_PYTZ setting if you need it).
5838 6.30.1 - 2021-12-05
5840 This patch updates our vendored list of top-level domains, which is
5841 used by the provisional domains() strategy.
5842 6.30.0 - 2021-12-03
5844 This release adds an allow_subnormal argument to the floats() strategy,
5845 which can explicitly toggle the generation of subnormal floats (issue
5846 #3155). Disabling such generation is useful when testing flush-to-zero
5847 builds of libraries.
5848 nps.from_dtype() and xps.from_dtype() can also accept the allow_subnor‐
5850 mal argument, and xps.from_dtype() or xps.arrays() will disable subnor‐
5851 mals by default if the array module xp is detected to flush-to-zero
5852 (like is typical with CuPy).
5853 6.29.3 - 2021-12-02
5855 This patch fixes a bug in mutually_broadcastable_shapes(), which re‐
5856 stricted the patterns of singleton dimensions that could be generated
5857 for dimensions that extended beyond base_shape (issue #3170).
5858 6.29.2 - 2021-12-02
5860 This patch clarifies our pretty-printing of DataFrames (issue #3114).
5861 6.29.1 - 2021-12-02
5863 This patch documents timezones() Windows-only requirement for the
5864 tzdata package, and ensures that pip install hypothesis[zoneinfo] will
5865 install the latest version.
5866 6.29.0 - 2021-11-29
5868 This release teaches builds() to use deferred() when resolving unrecog‐
5869 nised type hints, so that you can conveniently register strategies for
5870 recursive types with constraints on some arguments (issue #3026):
5871 class RecursiveClass:
5873 def __init__(self, value: int, next_node: typing.Optional["SomeClass"]):
5874 assert value > 0
5875 self.value = value
5876 self.next_node = next_node
5877 st.register_type_strategy(
5880 RecursiveClass, st.builds(RecursiveClass, value=st.integers(min_value=1))
5881 )
5882 6.28.1 - 2021-11-28
5884 This release fixes some internal calculations related to collection
5885 sizes (issue #3143).
5886 6.28.0 - 2021-11-28
5888 This release modifies our pytest plugin, to avoid importing Hypothesis
5889 and therefore triggering Hypothesis' entry points for test suites where
5890 Hypothesis is installed but not actually used (issue #3140).
5891 6.27.3 - 2021-11-28
5893 This release fixes issue #3080, where from_type() failed on unions con‐
5894 taining PEP 585 builtin generic types (like list[int]) in Python 3.9
5895 and later.
5896 6.27.2 - 2021-11-26
5898 This patch makes the hypothesis codemod command somewhat faster.
5899 6.27.1 - 2021-11-22
5901 This patch changes the backing datastructures of register_random() and
5902 a few internal caches to use weakref.WeakKeyDictionary. This reduces
5903 memory usage and may improve performance when registered Random in‐
5904 stances are only used for a subset of your tests (issue #3131).
5905 6.27.0 - 2021-11-22
5907 This release teaches Hypothesis' multiple-error reporting to format
5908 tracebacks using pytest or better-exceptions, if they are installed and
5909 enabled (issue #3116).
5910 6.26.0 - 2021-11-21
5912 Did you know that of the 264 possible floating-point numbers, 253 of
5913 them are nan - and Python prints them all the same way?
5914 While nans usually have all zeros in the sign bit and mantissa, this
5916 isn't always true, and 'signaling' nans might trap or error. To help
5917 distinguish such errors in e.g. CI logs, Hypothesis now prints -nan for
5918 negative nans, and adds a comment like # Saw 3 signaling NaNs if appli‐
5919 cable.
5920 6.25.0 - 2021-11-19
5922 This release adds special filtering logic to make a few special cases
5923 like s.map(lambda x: x) and lists().filter(len) more efficient (issue
5924 #2701).
5925 6.24.6 - 2021-11-18
5927 This patch makes floats() generate "subnormal" floating point numbers
5928 more often, as these rare values can have strange interactions with
5929 unsafe compiler optimisations like -ffast-math (issue #2976).
5930 6.24.5 - 2021-11-16
5932 This patch fixes a rare internal error in the datetimes() strategy,
5933 where the implementation of allow_imaginary=False crashed when checking
5934 a time during the skipped hour of a DST transition if the DST offset is
5935 negative - only true of Europe/Dublin, who we presume have their rea‐
5936 sons - and the tzinfo object is a pytz timezone (which predates PEP
5937 495).
5938 6.24.4 - 2021-11-15
5940 This patch gives Hypothesis it's own internal Random instance, ensuring
5941 that test suites which reset the global random state don't induce weird
5942 correlations between property-based tests (issue #2135).
5943 6.24.3 - 2021-11-13
5945 This patch updates documentation of note() (issue #3147).
5946 6.24.2 - 2021-11-05
5948 This patch updates internal testing for the Array API extra to be con‐
5949 sistent with new specification changes: sum() not accepting boolean ar‐
5950 rays (#234), unique() split into separate functions (#275), and treat‐
5951 ing NaNs as distinct (#310). It has no user visible impact.
5952 6.24.1 - 2021-11-01
5954 This patch updates our vendored list of top-level domains, which is
5955 used by the provisional domains() strategy.
5956 6.24.0 - 2021-10-23
5958 This patch updates our vendored list of top-level domains, which is
5959 used by the provisional domains() strategy.
5960 (did you know that gTLDs can be both added and removed?)
5962 6.23.4 - 2021-10-20
5964 This patch adds an error for when shapes in xps.arrays() is not passed
5965 as either a valid shape or strategy.
5966 6.23.3 - 2021-10-18
5968 This patch updates our formatting with shed.
5969 6.23.2 - 2021-10-08
5971 This patch replaces external links to NumPy API docs with
5972 sphinx.ext.intersphinx cross-references. It is purely a documentation
5973 improvement.
5974 6.23.1 - 2021-09-29
5976 This patch cleans up internal logic for xps.arrays(). There is no
5977 user-visible change.
5978 6.23.0 - 2021-09-26
5980 This release follows pytest in considering SystemExit and GeneratorExit
5981 exceptions to be test failures, meaning that we will shink to minimal
5982 examples and check for flakiness even though they subclass BaseExcep‐
5983 tion directly (issue #2223).
5984 KeyboardInterrupt continues to interrupt everything, and will be
5986 re-raised immediately.
5987 6.22.0 - 2021-09-24
5989 This release adds LiveServerTestCase and StaticLiveServerTestCase for
5990 django test. Thanks to Ivan Tham for this feature!
5991 6.21.6 - 2021-09-19
5993 This patch fixes some new linter warnings such as flake8-bugbear's B904
5994 for explicit exception chaining, so tracebacks might be a bit nicer.
5995 6.21.5 - 2021-09-16
5997 This release fixes None being inferred as the float64 dtype in
5998 from_dtype() and arrays() from the Array API extra.
5999 6.21.4 - 2021-09-16
6001 This release fixes the type hint for the @given() decorator when deco‐
6002 rating an async function (issue #3099).
6003 6.21.3 - 2021-09-15
6005 This release improves Ghostwritten tests for builtins (issue #2977).
6006 6.21.2 - 2021-09-15
6008 This release deprecates use of both min_dims > len(shape) and max_dims
6009 > len(shape) when allow_newaxis == False in basic_indices() (issue
6010 #3091).
6011 6.21.1 - 2021-09-13
6013 This release improves the behaviour of builds() and from_type() in cer‐
6014 tain situations involving decorators (issue #2495 and issue #3029).
6015 6.21.0 - 2021-09-11
6017 This release introduces strategies for array/tensor libraries adopting
6018 the Array API standard (issue #3037). They are available in the hy‐
6019 pothesis.extra.array_api extra, and work much like the existing strate‐
6020 gies for NumPy.
6021 6.20.1 - 2021-09-10
6023 This patch fixes issue #961, where calling given() inline on a bound
6024 method would fail to handle the self argument correctly.
6025 6.20.0 - 2021-09-09
6027 This release allows slices() to generate step=None, and fixes an
6028 off-by-one error where the start index could be equal to size. This
6029 works fine for all Python sequences and Numpy arrays, but is undefined
6030 behaviour in the Array API standard (see pull request #3065).
6031 6.19.0 - 2021-09-08
6033 This release makes stateful testing more likely to tell you if you do
6034 something unexpected and unsupported:
6035 • The return_value health check now applies to rule() and initialize()
6037 rules, if they don't have target bundles, as well as invariant().
6038 • Using a consumes() bundle as a target is deprecated, and will be an
6040 error in a future version.
6041 If existing code triggers these new checks, check for related bugs and
6043 misunderstandings - these patterns never had any effect.
6044 6.18.0 - 2021-09-06
6046 This release teaches from_type() a neat trick: when resolving an
6047 python:typing.Annotated type, if one of the annotations is a strategy
6048 object we use that as the inferred strategy. For example:
6049 PositiveInt = Annotated[int, st.integers(min_value=1)]
6051 If there are multiple strategies, we use the last outer-most annota‐
6053 tion. See issue #2978 and pull request #3082 for discussion.
6054 Requires Python 3.9 or later for get_type_hints(..., include_ex‐
6056 tras=False).
6057 6.17.4 - 2021-08-31
6059 This patch makes unique arrays() much more efficient, especially when
6060 there are only a few valid elements - such as for eight-bit integers (‐
6061 issue #3066).
6062 6.17.3 - 2021-08-30
6064 This patch fixes the repr of array_shapes().
6065 6.17.2 - 2021-08-30
6067 This patch wraps some internal helper code in our proxies decorator to
6068 prevent mutations of method docstrings carrying over to other instances
6069 of the respective methods.
6070 6.17.1 - 2021-08-29
6072 This patch moves some internal helper code in preparation for issue
6073 #3065. There is no user-visible change, unless you depended on undocu‐
6074 mented internals.
6075 6.17.0 - 2021-08-27
6077 This release adds type annotations to the stateful testing API.
6078 Thanks to Ruben Opdebeeck for this contribution!
6080 6.16.0 - 2021-08-27
6082 This release adds the DrawFn type as a reusable type hint for the draw
6083 argument of @composite functions.
6084 Thanks to Ruben Opdebeeck for this contribution!
6086 6.15.0 - 2021-08-22
6088 This release emits a more useful error message when @given() is applied
6089 to a coroutine function, i.e. one defined using async def (issue
6090 #3054).
6091 This was previously only handled by the generic return_value health
6093 check, which doesn't direct you to use either a custom executor or a
6094 library such as pytest-trio or pytest-asyncio to handle it for you.
6095 6.14.9 - 2021-08-20
6097 This patch fixes a regression in Hypothesis 6.14.8, where from_type()
6098 failed to resolve types which inherit from multiple parametrised
6099 generic types, affecting the returns package (issue #3060).
6100 6.14.8 - 2021-08-16
6102 This patch ensures that registering a strategy for a subclass of a a
6103 parametrised generic type such as class Lines(Sequence[str]): will not
6104 "leak" into unrelated strategies such as st.from_type(Sequence[int]) (‐
6105 issue #2951). Unfortunately this fix requires PEP 560, meaning Python
6106 3.7 or later.
6107 6.14.7 - 2021-08-14
6109 This patch fixes issue #3050, where attrs classes could cause an inter‐
6110 nal error in the ghostwriter.
6111 6.14.6 - 2021-08-07
6113 This patch improves the error message for issue #3016, where PEP 585
6114 builtin generics with self-referential forward-reference strings cannot
6115 be resolved to a strategy by from_type().
6116 6.14.5 - 2021-07-27
6118 This patch fixes hypothesis.strategies._internal.types.is_a_new_type.
6119 It was failing on Python 3.10.0b4, where NewType is a function.
6120 6.14.4 - 2021-07-26
6122 This patch fixes from_type() and register_type_strategy() for
6123 python:typing.NewType on Python 3.10, which changed the underlying im‐
6124 plementation (see bpo-44353 for details).
6125 6.14.3 - 2021-07-18
6127 This patch updates our autoformatting tools, improving our code style
6128 without any API changes.
6129 6.14.2 - 2021-07-12
6131 This patch ensures that we shorten tracebacks for tests which fail due
6132 to inconsistent data generation between runs (i.e. raise Flaky).
6133 6.14.1 - 2021-07-02
6135 This patch updates some internal type annotations. There is no
6136 user-visible change.
6137 6.14.0 - 2021-06-09
6139 The explain phase now requires shrinking to be enabled, and will be au‐
6140 tomatically skipped for deadline-exceeded errors.
6141 6.13.14 - 2021-06-04
6143 This patch improves the tuples() strategy type annotations, to preserve
6144 the element types for up to length-five tuples (issue #3005).
6145 As for one_of(), this is the best we can do before a planned extension
6147 to PEP 646 is released, hopefully in Python 3.11.
6148 6.13.13 - 2021-06-04
6150 This patch teaches the Ghostwriter how to find custom ufuncs from any
6151 module that defines them, and that yaml.unsafe_load() does not undo
6152 yaml.safe_load().
6153 6.13.12 - 2021-06-03
6155 This patch reduces the amount of internal code excluded from our test
6156 suite's code coverage checks.
6157 There is no user-visible change.
6159 6.13.11 - 2021-06-02
6161 This patch removes some old internal helper code that previously ex‐
6162 isted to make Python 2 compatibility easier.
6163 There is no user-visible change.
6165 6.13.10 - 2021-05-30
6167 This release adjusts some internal code to help make our test suite
6168 more reliable.
6169 There is no user-visible change.
6171 6.13.9 - 2021-05-30
6173 This patch cleans up some internal code related to filtering strate‐
6174 gies.
6175 There is no user-visible change.
6177 6.13.8 - 2021-05-28
6179 This patch slightly improves the performance of some internal code for
6180 generating integers.
6181 6.13.7 - 2021-05-27
6183 This patch fixes a bug in from_regex() that caused from_regex("", full‐
6184 match=True) to unintentionally generate non-empty strings (issue
6185 #4982).
6186 The only strings that completely match an empty regex pattern are empty
6188 strings.
6189 6.13.6 - 2021-05-26
6191 This patch fixes a bug that caused integers() to shrink towards nega‐
6192 tive values instead of positive values in some cases.
6193 6.13.5 - 2021-05-24
6195 This patch fixes rare cases where hypothesis write --binary-op could
6196 print reproducing instructions from the internal search for an identity
6197 element.
6198 6.13.4 - 2021-05-24
6200 This patch removes some unnecessary intermediate list-comprehensions,
6201 using the latest versions of pyupgrade and shed.
6202 6.13.3 - 2021-05-23
6204 This patch adds a .hypothesis property to invalid test functions,
6205 bringing them inline with valid tests and fixing a bug where
6206 pytest-asyncio would swallow the real error message and mistakenly
6207 raise a version incompatibility error.
6208 6.13.2 - 2021-05-23
6210 Some of Hypothesis's numpy/pandas strategies use a fill argument to
6211 speed up generating large arrays, by generating a single fill value and
6212 sharing that value among many array slots instead of filling every sin‐
6213 gle slot individually.
6214 When no fill argument is provided, Hypothesis tries to detect whether
6216 it is OK to automatically use the elements argument as a fill strategy,
6217 so that it can still use the faster approach.
6218 This patch fixes a bug that would cause that optimization to trigger in
6220 some cases where it isn't 100% guaranteed to be OK.
6221 If this makes some of your numpy/pandas tests run more slowly, try
6223 adding an explicit fill argument to the relevant strategies to ensure
6224 that Hypothesis always uses the faster approach.
6225 6.13.1 - 2021-05-20
6227 This patch strengthens some internal import-time consistency checks for
6228 the built-in strategies.
6229 There is no user-visible change.
6231 6.13.0 - 2021-05-18
6233 This release adds URL fragment generation to the urls() strategy (issue
6234 #2908). Thanks to Pax (R. Margret) for contributing this patch at the
6235 PyCon US Mentored Sprints!
6236 6.12.1 - 2021-05-17
6238 This patch fixes issue #2964, where .map() and .filter() methods were
6239 omitted from the repr() of just() and sampled_from() strategies, since
6240 version 5.43.7.
6241 6.12.0 - 2021-05-06
6243 This release automatically rewrites some simple filters, such as inte‐
6244 gers().filter(lambda x: x > 9) to the more efficient inte‐
6245 gers(min_value=10), based on the AST of the predicate.
6246 We continue to recommend using the efficient form directly wherever
6248 possible, but this should be useful for e.g. pandera "Checks" where you
6249 already have a simple predicate and translating manually is really an‐
6250 noying. See issue #2701 for ideas about floats and simple text strate‐
6251 gies.
6252 6.11.0 - 2021-05-06
6254 hypothesis.target() now returns the observation value, allowing it to
6255 be conveniently used inline in expressions such as assert target(abs(a
6256 - b)) < 0.1.
6257 6.10.1 - 2021-04-26
6259 This patch fixes a deprecation warning if you're using recent versions
6260 of importlib-metadata (issue #2934), which we use to load third-party
6261 plugins such as Pydantic's integration. On older versions of
6262 importlib-metadata, there is no change and you don't need to upgrade.
6263 6.10.0 - 2021-04-17
6265 This release teaches the Ghostwriter to read parameter types from
6266 Sphinx, Google, or Numpy-style structured docstrings, and improves some
6267 related heuristics about how to test scientific and numerical programs.
6268 6.9.2 - 2021-04-15
6270 This release improves the Ghostwriter's handling of exceptions, by
6271 reading :raises ...: entries in function docstrings and ensuring that
6272 we don't suppresss the error raised by test assertions.
6273 6.9.1 - 2021-04-12
6275 This patch updates our autoformatting tools, improving our code style
6276 without any API changes.
6277 6.9.0 - 2021-04-11
6279 This release teaches from_type() how to see through python:typing.Anno‐
6280 tated. Thanks to Vytautas Strimaitis for reporting and fixing issue
6281 #2919!
6282 6.8.12 - 2021-04-11
6284 If rich is installed, the hypothesis write command will use it to syn‐
6285 tax-highlight the Ghostwritten code.
6286 6.8.11 - 2021-04-11
6288 This patch improves an error message from builds() when from_type()
6289 would be more suitable (issue #2930).
6290 6.8.10 - 2021-04-11
6292 This patch updates the type annotations for arrays() to reflect that
6293 shape: SearchStrategy[int] is supported.
6294 6.8.9 - 2021-04-07
6296 This patch fixes from_type() with abstract types which have either re‐
6297 quired but non-type-annotated arguments to __init__, or where
6298 from_type() can handle some concrete subclasses but not others.
6299 6.8.8 - 2021-04-07
6301 This patch teaches hypothesis write to check for possible roundtrips in
6302 several more cases, such as by looking for an inverse in the module
6303 which defines the function to test.
6304 6.8.7 - 2021-04-07
6306 This patch adds a more helpful error message if you try to call sam‐
6307 pled_from() on an Enum which has no members, but does have data‐
6308 class()-style annotations (issue #2923).
6309 6.8.6 - 2021-04-06
6311 The fixed_dictionaries() strategy now preserves dict iteration order
6312 instead of sorting the keys. This also affects the pretty-printing of
6313 keyword arguments to @given() (issue #2913).
6314 6.8.5 - 2021-04-05
6316 This patch teaches hypothesis write to default to ghostwriting tests
6317 with --style=pytest only if pytest is installed, or --style=unittest
6318 otherwise.
6319 6.8.4 - 2021-04-01
6321 This patch adds type annotations for the settings decorator, to avoid
6322 an error when running mypy in strict mode.
6323 6.8.3 - 2021-03-28
6325 This patch improves the Ghostwriter's handling of strategies to gener‐
6326 ate various fiddly types including frozensets, keysviews, valuesviews,
6327 regex matches and patterns, and so on.
6328 6.8.2 - 2021-03-27
6330 This patch fixes some internal typos. There is no user-visible change.
6331 6.8.1 - 2021-03-14
6333 This patch lays more groundwork for filter rewriting (issue #2701).
6334 There is no user-visible change... yet.
6335 6.8.0 - 2021-03-11
6337 This release registers the remaining builtin types, and teaches
6338 from_type() to try resolving ForwardRef and Type references to built-in
6339 types.
6340 6.7.0 - 2021-03-10
6342 This release teaches RuleBasedStateMachine to avoid checking invari‐
6343 ant()s until all initialize() rules have been run. You can enable
6344 checking of specific invariants for incompletely initialized machines
6345 by using @invariant(check_during_init=True) (issue #2868).
6346 In previous versions, it was possible if awkward to implement this be‐
6348 haviour using precondition() and an auxiliary variable.
6349 6.6.1 - 2021-03-09
6351 This patch improves the error message when from_type() fails to resolve
6352 a forward-reference inside a python:typing.Type such as Type["int"] (‐
6353 issue #2565).
6354 6.6.0 - 2021-03-07
6356 This release makes it an explicit error to apply invariant() to a
6357 rule() or initialize() rule in stateful testing. Such a combination
6358 had unclear semantics, especially in combination with precondition(),
6359 and was never meant to be allowed (issue #2681).
6360 6.5.0 - 2021-03-07
6362 This release adds the explain phase, in which Hypothesis attempts to
6363 explain why your test failed by pointing to suspicious lines of code
6364 (i.e. those which were always, and only, run on failing inputs). We
6365 plan to include "generalising" failing examples in this phase in a fu‐
6366 ture release (issue #2192).
6367 6.4.3 - 2021-03-04
6369 This patch fixes issue #2794, where nesting deferred() strategies
6370 within recursive() strategies could trigger an internal assertion.
6371 While it was always possible to get the same results from a more sensi‐
6372 ble strategy, the convoluted form now works too.
6373 6.4.2 - 2021-03-04
6375 This patch fixes several problems with mypy when --no-implicit-reexport
6376 was activated in user projects.
6377 Thanks to Nikita Sobolev for fixing issue #2884!
6379 6.4.1 - 2021-03-04
6381 This patch fixes an exception that occurs when using type unions of the
6382 typing_extensions Literal backport on Python 3.6.
6383 Thanks to Ben Anhalt for identifying and fixing this bug.
6385 6.4.0 - 2021-03-02
6387 This release fixes stateful testing methods with multiple precondi‐
6388 tion() decorators. Previously, only the outer-most precondition was
6389 checked (issue #2681).
6390 6.3.4 - 2021-02-28
6392 This patch refactors some internals of RuleBasedStateMachine. There is
6393 no change to the public API or behaviour.
6394 6.3.3 - 2021-02-26
6396 This patch moves some internal code, so that future work can avoid cre‐
6397 ating import cycles. There is no user-visible change.
6398 6.3.2 - 2021-02-25
6400 This patch enables register_type_strategy() for subclasses of
6401 python:typing.TypedDict. Previously, from_type() would ignore the reg‐
6402 istered strategy (issue #2872).
6403 Thanks to Ilya Lebedev for identifying and fixing this bug!
6405 6.3.1 - 2021-02-24
6407 This release lays the groundwork for automatic rewriting of simple fil‐
6408 ters, for example converting integers().filter(lambda x: x > 9) to in‐
6409 tegers(min_value=10).
6410 Note that this is not supported yet, and we will continue to recommend
6412 writing the efficient form directly wherever possible - predicate
6413 rewriting is provided mainly for the benefit of downstream libraries
6414 which would otherwise have to implement it for themselves (e.g. pandera
6415 and icontract-hypothesis). See issue #2701 for details.
6416 6.3.0 - 2021-02-20
6418 The Hypothesis pytest plugin now requires pytest version 4.6 or later.
6419 If the plugin detects an earlier version of pytest, it will automati‐
6420 cally deactivate itself.
6421 (4.6.x is the earliest pytest branch that still accepts community bug‐
6423 fixes.)
6424 Hypothesis-based tests should continue to work in earlier versions of
6426 pytest, but enhanced integrations provided by the plugin (such as --hy‐
6427 pothesis-show-statistics and other command-line flags) will no longer
6428 be available in obsolete pytest versions.
6429 6.2.0 - 2021-02-12
6431 If you use pytest-html, Hypothesis now includes the summary statistics
6432 for each test in the HTML report, whether or not the --hypothe‐
6433 sis-show-statistics argument was passed to show them in the com‐
6434 mand-line output.
6435 6.1.1 - 2021-01-31
6437 This patch updates our automatic code formatting to use shed, which in‐
6438 cludes autoflake, black, isort, and pyupgrade (issue #2780).
6439 6.1.0 - 2021-01-29
6441 This release teaches Hypothesis to distinguish between errors based on
6442 the __cause__ or __context__ of otherwise identical exceptions, which
6443 is particularly useful when internal errors can be wrapped by a li‐
6444 brary-specific or semantically appropriate exception such as:
6445 try:
6447 do_the_thing(foo, timeout=10)
6448 except Exception as err:
6449 raise FooError("Failed to do the thing") from err
6450 Earlier versions of Hypothesis only see the FooError, while we can now
6452 distinguish a FooError raised because of e.g. an internal assertion
6453 from one raised because of a TimeoutExceeded exception.
6454 6.0.4 - 2021-01-27
6456 This release prevents a race condition inside recursive() strategies.
6457 The race condition occurs when the same recursive() strategy is shared
6458 among tests that are running in multiple threads (issue #2717).
6459 6.0.3 - 2021-01-23
6461 This patch improves the type annotations for one_of(), by adding over‐
6462 loads to handle up to five distinct arguments as Union before falling
6463 back to Any, as well as annotating the | (__or__) operator for strate‐
6464 gies (issue #2765).
6465 6.0.2 - 2021-01-14
6467 This release makes some small improvements to how filtered strategies
6468 work. It should improve the performance of shrinking filtered strate‐
6469 gies, and may under some (probably rare) circumstances improve the di‐
6470 versity of generated examples.
6471 6.0.1 - 2021-01-13
6473 This patch fixes an interaction where our test statistics handling made
6474 Pytest's --junit-xml output fail to validate against the strict xunit2
6475 schema (issue #1975).
6476 6.0.0 - 2021-01-08
6478 Welcome to the next major version of Hypothesis!
6479 There are no new features here, as we release those in minor versions.
6481 Instead, 6.0 is a chance for us to remove deprecated features (many al‐
6482 ready converted into no-ops), and turn a variety of warnings into er‐
6483 rors.
6484 If you were running on the last version of Hypothesis 5.x without any
6486 Hypothesis deprecation warnings, this will be a very boring upgrade.
6487 In fact, nothing will change for you at all.
6488 Changes
6490 • Many functions now use PEP 3102 keyword-only arguments where passing
6491 positional arguments was deprecated since 5.5.
6492 • hypothesis.extra.django.from_model() no longer accepts model as a
6494 keyword argument, where it could conflict with fields named "model".
6495 • randoms() now defaults to use_true_random=False.
6497 • complex_numbers() no longer accepts min_magnitude=None; either use
6499 min_magnitude=0 or just omit the argument.
6500 • hypothesis.provisional.ip4_addr_strings and ip6_addr_strings are re‐
6502 moved in favor of ip_addresses(v=...).map(str).
6503 • register_type_strategy() no longer accepts generic types with type
6505 arguments, which were always pretty badly broken.
6506 • Using function-scoped pytest fixtures is now a health-check error,
6508 instead of a warning.
6509 TIP:
6511 The hypothesis codemod command can automatically refactor your code,
6512 particularly to convert positional to keyword arguments where those
6513 are now required.
6514 Hypothesis 5.x
6516 5.49.0 - 2021-01-07
6517 This release adds the function_scoped_fixture health check value, which
6518 can be used to suppress the existing warning that appears when @given
6519 is applied to a test that uses pytest function-scoped fixtures.
6520 (This warning exists because function-scoped fixtures only run once per
6522 function, not once per example, which is usually unexpected and can
6523 cause subtle problems.)
6524 When this warning becomes a health check error in a future release,
6526 suppressing it via Python warning settings will no longer be possible.
6527 In the rare case that once-per-function behaviour is intended, it will
6528 still be possible to use function_scoped_fixture to opt out of the
6529 health check error for specific tests.
6530 5.48.0 - 2021-01-06
6532 This release adds hypothesis.currently_in_test_context(), which can be
6533 used to check whether the calling code is currently running inside an
6534 @given or stateful test.
6535 This is most useful for third-party integrations and assertion helpers
6537 which may wish to use assume() or target(), without also requiring that
6538 the helper only be used from property-based tests (issue #2581).
6539 5.47.0 - 2021-01-05
6541 This release upgrades the import logic for ghostwritten tests, handling
6542 many cases where imports would previously be missing or from unexpected
6543 locations.
6544 5.46.0 - 2021-01-04
6546 This release upgrades from_type(), to infer strategies for type-anno‐
6547 tated arguments even if they have defaults when it otherwise falls back
6548 to builds() (issue #2708).
6549 5.45.0 - 2021-01-04
6551 This release adds the codemods extra, which you can use to check for
6552 and automatically fix issues such as use of deprecated Hypothesis APIs
6553 (issue #2705).
6554 5.44.0 - 2021-01-03
6556 This patch fixes from_type() with the typing_extensions Literal back‐
6557 port on Python 3.6.
6558 5.43.9 - 2021-01-02
6560 This patch fixes issue #2722, where certain orderings of regis‐
6561 ter_type_strategy(), ForwardRef, and from_type() could trigger an in‐
6562 ternal error.
6563 5.43.8 - 2021-01-02
6565 This patch makes some strategies for collections with a uniqueness con‐
6566 straint much more efficient, including dictionaries(keys=sam‐
6567 pled_from(...), values=..) and lists(tuples(sampled_from(...), ...),
6568 unique_by=lambda x: x[0]). (related to issue #2036)
6569 5.43.7 - 2021-01-02
6571 This patch extends our faster special case for sampled_from() elements
6572 in unique lists() to account for chains of .map(...) and .filter(...)
6573 calls (issue #2036).
6574 5.43.6 - 2021-01-02
6576 This patch improves the type annotations on assume() and @repro‐
6577 duce_failure().
6578 5.43.5 - 2021-01-01
6580 This patch updates our copyright headers to include 2021. Happy new
6581 year!
6582 5.43.4 - 2020-12-24
6584 This change fixes a documentation error in the database setting.
6585 The previous documentation suggested that callers could specify a data‐
6587 base path string, or the special string ":memory:", but this setting
6588 has never actually allowed string arguments.
6589 Permitted values are None, and instances of ExampleDatabase.
6591 5.43.3 - 2020-12-11
6593 This patch fixes issue #2696, an internal error triggered when the @ex‐
6594 ample decorator was used and the verbosity setting was quiet.
6595 5.43.2 - 2020-12-10
6597 This patch improves the error message from the data_frames() strategy
6598 when both the rows and columns arguments are given, but there is a
6599 missing entry in rows and the corresponding column has no fill value (‐
6600 issue #2678).
6601 5.43.1 - 2020-12-10
6603 This patch improves the error message if builds() is passed an Enum
6604 which cannot be called without arguments, to suggest using sam‐
6605 pled_from() (issue #2693).
6606 5.43.0 - 2020-12-09
6608 This release adds new timezones() and timezone_keys() strategies (issue
6609 #2630) based on the new python:zoneinfo module in Python 3.9.
6610 pip install hypothesis[zoneinfo] will ensure that you have the appro‐
6612 priate backports installed if you need them.
6613 5.42.3 - 2020-12-09
6615 This patch fixes an internal error in datetimes() with allow_imagi‐
6616 nary=False where the timezones argument can generate tzinfo=None (issue
6617 #2662).
6618 5.42.2 - 2020-12-09
6620 This patch teaches hypothesis.extra.django.from_field() to infer more
6621 efficient strategies by inspecting (not just filtering by) field val‐
6622 idators for numeric and string fields (issue #1116).
6623 5.42.1 - 2020-12-09
6625 This patch refactors hypothesis.settings to use type-annotated keyword
6626 arguments instead of **kwargs, which makes tab-completion much more
6627 useful - as well as type-checkers like mypy.
6628 5.42.0 - 2020-12-09
6630 This patch teaches the magic() ghostwriter to recognise "en/de" func‐
6631 tion roundtrips other than the common encode/decode pattern, such as
6632 encrypt/decrypt or, encipher/decipher.
6633 5.41.5 - 2020-12-05
6635 This patch adds a performance optimisation to avoid saving redundant
6636 seeds when using the .fuzz_one_input hook.
6637 5.41.4 - 2020-11-28
6639 This patch fixes issue #2657, where passing unicode patterns compiled
6640 with python:re.IGNORECASE to from_regex() could trigger an internal er‐
6641 ror when casefolding a character creates a longer string (e.g.
6642 "\u0130".lower() -> "i\u0370").
6643 5.41.3 - 2020-11-18
6645 This patch adds a final fallback clause to our plugin logic to fail
6646 with a warning rather than error on Python < 3.8 when neither the
6647 importlib_metadata (preferred) or setuptools (fallback) packages are
6648 available.
6649 5.41.2 - 2020-11-08
6651 This patch fixes urls() strategy ensuring that ~ (tilde) is treated as
6652 one of the url-safe characters (issue #2658).
6653 5.41.1 - 2020-11-03
6655 This patch improves our CLI help and documentation.
6656 5.41.0 - 2020-10-30
6658 Hypothesis now shrinks examples where the error is raised while drawing
6659 from a strategy. This makes complicated custom strategies much easier
6660 to debug, at the cost of a slowdown for use-cases where you catch and
6661 ignore such errors.
6662 5.40.0 - 2020-10-30
6664 This release teaches from_type() how to handle ChainMap, Counter,
6665 Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).
6666 5.39.0 - 2020-10-30
6668 from_type() now knows how to resolve PEP 585 parameterized standard
6669 collection types, which are new in Python 3.9 (issue #2629).
6670 5.38.1 - 2020-10-26
6672 This patch fixes builds(), so that when passed infer for an argument
6673 with a non-Optional type annotation and a default value of None to
6674 build a class which defines an explicit __signature__ attribute, either
6675 None or that type may be generated.
6676 This is unlikely to happen unless you are using pydantic (issue #2648).
6678 5.38.0 - 2020-10-24
6680 This release improves our support for @st.composite on a python:class‐
6681 method or python:staticmethod (issue #2578).
6682 5.37.5 - 2020-10-24
6684 This patch fixes from_type() with Iterable[T] (issue #2645).
6685 5.37.4 - 2020-10-20
6687 This patch teaches the magic() ghostwriter to recognise that pairs of
6688 functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().
6689 5.37.3 - 2020-10-15
6691 This patch improves builds() and from_type() support for explicitly de‐
6692 fined __signature__ attributes, from version 5.8.3, to support generic
6693 types from the python:typing module.
6694 Thanks to Rónán Carrigan for identifying and fixing this problem!
6696 5.37.2 - 2020-10-14
6698 This patch fixes from_lark() with version 0.10.1+ of the lark-parser
6699 package.
6700 5.37.1 - 2020-10-07
6702 This patch fixes some broken links in the lark extra documentation.
6703 5.37.0 - 2020-10-03
6705 This release adds a new RedisExampleDatabase, along with the ReadOnly‐
6706 Database and MultiplexedDatabase helpers, to support team workflows
6707 where failing examples can be seamlessly shared between everyone on the
6708 team - and your CI servers or buildbots.
6709 5.36.2 - 2020-10-02
6711 This patch ensures that if the "hypothesis" entry point is callable, we
6712 call it after importing it. You can still use non-callable entry
6713 points (like modules), which are only imported.
6714 We also prefer importlib.metadata or the backport over pkg_resources,
6716 which makes import hypothesis around 200 milliseconds faster (issue
6717 #2571).
6718 5.36.1 - 2020-09-25
6720 This patch adds some helpful suggestions to error messages you might
6721 see while learning to use the @example() decorator (issue #2611) or the
6722 one_of() strategy.
6723 5.36.0 - 2020-09-24
6725 This release upgrades the from_dtype() strategy to pass optional
6726 **kwargs to the inferred strategy, and upgrades the arrays() strategy
6727 to accept an elements=kwargs dict to pass through to from_dtype().
6728 arrays(floating_dtypes(), shape, elements={"min_value": -10,
6730 "max_value": 10}) is a particularly useful pattern, as it allows for
6731 any floating dtype without triggering the roundoff warning for smaller
6732 types or sacrificing variety for larger types (issue #2552).
6733 5.35.4 - 2020-09-21
6735 This patch reformats our code with the latest black to take advantage
6736 of the support for magic trailing commas.
6737 5.35.3 - 2020-09-15
6739 This release significantly improves the performance of Hypothesis's in‐
6740 ternal implementation of automaton learning. However this code does not
6741 run as part of the user-accessible API so this has no user-visible im‐
6742 pact.
6743 5.35.2 - 2020-09-14
6745 This patch ensures that, when the generate phases is disabled, we can
6746 replay up to max_examples examples from the database - which is very
6747 useful when using Hypothesis with a fuzzer.
6748 Thanks to Afrida Tabassum for fixing issue #2585!
6750 5.35.1 - 2020-09-14
6752 This patch changes some internal python:struct.Struct.format strings
6753 from bytes to str, to avoid python:BytesWarning when running python
6754 -bb.
6755 Thanks to everyone involved in pytest-xdist issue 596, bpo-16349,
6757 bpo-21071, and bpo-41777 for their work on this - it was a remarkably
6758 subtle issue!
6759 5.35.0 - 2020-09-11
6761 The target() function now accepts integers as well as floats.
6762 5.34.1 - 2020-09-11
6764 This patch adds explicit Optional annotations to our public API, to
6765 better support users who run mypy with --strict or no_implicit_op‐
6766 tional=True.
6767 Thanks to Krzysztof Przybyła for bringing this to our attention and
6769 writing the patch!
6770 5.34.0 - 2020-09-11
6772 This release drops support for Python 3.5, which reached end of life
6773 upstream on 2020-09-13.
6774 5.33.2 - 2020-09-09
6776 This patch fixes a problem with builds() that was not able to generate
6777 valid data for annotated classes with constructors.
6778 Thanks to Nikita Sobolev for fixing issue #2603!
6780 5.33.1 - 2020-09-07
6782 This patch improves the error message from the hypothesis write command
6783 if black (required for the ghostwriter) is not installed.
6784 Thanks to Nikita Sobolev for fixing issue #2604!
6786 5.33.0 - 2020-09-06
6788 When reporting failing examples, or tried examples in verbose mode, Hy‐
6789 pothesis now identifies which were from @example(...) explicit exam‐
6790 ples.
6791 5.32.1 - 2020-09-06
6793 This patch contains some internal refactoring. Thanks to Felix Sheldon
6794 for fixing issue #2516!
6795 5.32.0 - 2020-09-04
6797 An array drawn from arrays() will own its own memory; previously most
6798 arrays returned by this strategy were views.
6799 5.31.0 - 2020-09-04
6801 builds() will use the __signature__ attribute of the target, if it ex‐
6802 ists, to retrieve type hints. Previously python:typ‐
6803 ing.get_type_hints(), was used by default. If argument names varied
6804 between the __annotations__ and __signature__, they would not be sup‐
6805 plied to the target.
6806 This was particularly an issue for pydantic models which use an alias
6808 generator.
6809 5.30.1 - 2020-09-04
6811 This patch makes the ghostwriter much more robust when passed unusual
6812 modules.
6813 • improved support for non-resolvable type annotations
6815 • magic() can now write equivalent() tests
6817 • running magic() on modules where some names in __all__ are undefined
6819 skips such names, instead of raising an error
6820 • magic() now knows to skip mocks
6822 • improved handling of import-time errors found by the ghostwriter CLI
6824 5.30.0 - 2020-08-30
6826 register_type_strategy() now supports python:typing.TypeVar, which was
6827 previously hard-coded, and allows a variety of types to be generated
6828 for an unconstrained TypeVar instead of just text().
6829 Thanks again to Nikita Sobolev for all your work on advanced types!
6831 5.29.4 - 2020-08-28
6833 This release fixes some hard to trigger bugs in Hypothesis's automata
6834 learning code. This code is only run as part of the Hypothesis build
6835 process, and not for user code, so this release has no user visible im‐
6836 pact.
6837 5.29.3 - 2020-08-27
6839 This patch adds type annotations to the hypothesis.database module.
6840 There is no runtime change, but your typechecker might notice.
6841 5.29.2 - 2020-08-27
6843 This patch tracks some additional information in Hypothesis internals,
6844 and has no user-visible impact.
6845 5.29.1 - 2020-08-27
6847 This release fixes a bug in some Hypothesis internal support code for
6848 learning automata. This mostly doesn't have any user visible impact,
6849 although it slightly affects the learned shrink passes so shrinking may
6850 be subtly different.
6851 5.29.0 - 2020-08-24
6853 This release adds support for entry-points, which allows for smoother
6854 integration of third-party Hypothesis extensions and external li‐
6855 braries. Unless you're publishing a library with Hypothesis integra‐
6856 tion, you'll probably only ever use this indirectly!
6857 5.28.0 - 2020-08-24
6859 from_type() can now resolve TypeVar instances when the bound is a For‐
6860 wardRef, so long as that name is in fact defined in the same module as
6861 the typevar (no TYPE_CHECKING tricks, sorry). This feature requires
6862 Python 3.7 or later.
6863 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
6865 5.27.0 - 2020-08-20
6867 This patch adds two new ghostwriters to test binary operations, like
6868 python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().
6869 5.26.1 - 2020-08-19
6871 This release improves the performance of some methods in Hypothesis's
6872 internal automaton library. These are currently only lightly used by
6873 user code, but this may result in slightly faster shrinking.
6874 5.26.0 - 2020-08-17
6876 register_type_strategy() no longer accepts parametrised user-defined
6877 generic types, because the resolution logic was quite badly broken (‐
6878 issue #2537).
6879 Instead of registering a strategy for e.g. MyCollection[int], you
6881 should register a function for MyCollection and inspect the type param‐
6882 eters within that function.
6883 Thanks to Nikita Sobolev for the bug report, design assistance, and
6885 pull request to implement this feature!
6886 5.25.0 - 2020-08-16
6888 Tired of writing tests? Or new to Hypothesis and not sure where to
6889 start?
6890 This release is for you! With our new Ghostwriter functions and hy‐
6892 pothesis write ... command-line interface, you can stop writing tests
6893 entirely... or take the source code Hypothesis writes for you as a
6894 starting point.
6895 This has been in the works for months, from issue #2118 to versions
6897 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who
6898 reviewed pull requests or commented on demos, and to Timothy Crosley's
6899 hypothesis-auto project for inspiration.
6900 5.24.4 - 2020-08-14
6902 This patch adds yet more internal functions to support a new feature
6903 we're working on, like version 5.18.3 and version 5.23.6. We promise
6904 it's worth the wait!
6905 5.24.3 - 2020-08-13
6907 This release fixes a small internal bug in Hypothesis's internal autom‐
6908 aton library. Fortunately this bug was currently impossible to hit in
6909 user facing code, so this has no user visible impact.
6910 5.24.2 - 2020-08-12
6912 This release improves shrink quality by allowing Hypothesis to automat‐
6913 ically learn new shrink passes for difficult to shrink tests.
6914 The automatic learning is not currently accessible in user code (it
6916 still needs significant work on robustness and performance before it is
6917 ready for that), but this release includes learned passes that should
6918 improve shrinking quality for tests which use any of the text(),
6919 floats(), datetimes(), emails(), and complex_numbers() strategies.
6920 5.24.1 - 2020-08-12
6922 This patch updates some docstrings, without changing runtime behaviour.
6923 5.24.0 - 2020-08-10
6925 The functions() strategy has a new argument pure=True, which ensures
6926 that the same return value is generated for identical calls to the gen‐
6927 erated function (issue #2538).
6928 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
6930 5.23.12 - 2020-08-10
6932 This release removes a number of Hypothesis's internal "shrink passes"
6933 - transformations it makes to a generated test case during shrinking -
6934 which appeared to be redundant with other transformations.
6935 It is unlikely that you will see much impact from this. If you do, it
6937 will likely show up as a change in shrinking performance (probably
6938 slower, maybe faster), or possibly in worse shrunk results. If you en‐
6939 counter the latter, please let us know.
6940 5.23.11 - 2020-08-04
6942 This release fixes a bug in some internal Hypothesis support code. It
6943 has no user visible impact.
6944 5.23.10 - 2020-08-04
6946 This release improves the quality of shrunk test cases in some special
6947 cases. Specifically, it should get shrinking unstuck in some scenarios
6948 which require simultaneously changing two parts of the generated test
6949 case.
6950 5.23.9 - 2020-08-03
6952 This release improves the performance of some internal support code. It
6953 has no user visible impact, as that code is not currently run during
6954 normal Hypothesis operation.
6955 5.23.8 - 2020-07-31
6957 This release adds a heuristic to detect when shrinking has finished de‐
6958 spite the fact that there are many more possible transformations to
6959 try. This will be particularly useful for tests where the minimum fail‐
6960 ing test case is very large despite there being many smaller test cases
6961 possible, where it is likely to speed up shrinking dramatically.
6962 In some cases it is likely that this will result in worse shrunk test
6964 cases. In those cases rerunning the test will result in further shrink‐
6965 ing.
6966 5.23.7 - 2020-07-29
6968 This release makes some performance improvements to shrinking. They
6969 should only be noticeable for tests that are currently particularly
6970 slow to shrink.
6971 5.23.6 - 2020-07-29
6973 This patch adds some more internal functions to support a new feature
6974 we're working on, like version 5.18.3. There is still no user-visible
6975 change... yet.
6976 5.23.5 - 2020-07-29
6978 This release makes some changes to internal support code that is not
6979 currently used in production Hypothesis. It has no user visible effect
6980 at present.
6981 5.23.4 - 2020-07-29
6983 This release improves shrinking quality in some special cases.
6984 5.23.3 - 2020-07-27
6986 This release fixes issue #2507, where lazy evaluation meant that the
6987 values drawn from a sampled_from() strategy could depend on mutations
6988 of the sampled sequence that happened after the strategy was construc‐
6989 ted.
6990 5.23.2 - 2020-07-27
6992 This patch fixes issue #2462, a bug in our handling of unittest.Test‐
6993 Case.subTest(). Thanks to Israel Fruchter for fixing this at the Eu‐
6994 roPython sprints!
6995 5.23.1 - 2020-07-26
6997 This release improves the behaviour of the characters() strategy when
6998 shrinking, by changing which characters are considered smallest to pre‐
6999 fer more "normal" ascii characters where available.
7000 5.23.0 - 2020-07-26
7002 The default print_blob setting is now smarter. It defaults to True in
7003 CI and False for local development.
7004 Thanks to Hugo van Kemenade for implementing this feature at the Eu‐
7006 roPython sprints!
7007 5.22.0 - 2020-07-25
7009 The slices() strategy can now generate slices for empty sequences,
7010 slices with negative start and stop indices (from the end of the se‐
7011 quence), and step=None in place of step=1.
7012 Thanks to Sangarshanan for implementing this feature at the EuroPython
7014 sprints!
7015 5.21.0 - 2020-07-23
7017 This release ensures that tests which raise RecursionError are not re‐
7018 ported as flaky simply because we run them from different initial stack
7019 depths (issue #2494).
7020 5.20.4 - 2020-07-23
7022 This release improves the performance of the sample method on objects
7023 obtained from randoms() when use_true_random=False. This should mostly
7024 only be noticeable when the sample size is a large fraction of the pop‐
7025 ulation size, but may also help avoid health check failures in some
7026 other cases.
7027 5.20.3 - 2020-07-21
7029 This release makes some internal changes for testing purposes and
7030 should have no user visible effect.
7031 5.20.2 - 2020-07-18
7033 This release fixes a small caching bug in Hypothesis internals that may
7034 under some circumstances have resulted in a less diverse set of test
7035 cases being generated than was intended.
7036 Fixing this problem revealed some performance problems that could occur
7038 during targeted property based testing, so this release also fixes
7039 those. Targeted property-based testing should now be significantly
7040 faster in some cases, but this may be at the cost of reduced effective‐
7041 ness.
7042 5.20.1 - 2020-07-17
7044 This patch updates our formatting to use isort 5. There is no
7045 user-visible change.
7046 5.20.0 - 2020-07-17
7048 The basic_indices() strategy can now generate bare indexers in place of
7049 length-one tuples. Thanks to Andrea for this patch!
7050 5.19.3 - 2020-07-15
7052 This patch removes an internal use of distutils in order to avoid this
7053 setuptools warning for some users.
7054 5.19.2 - 2020-07-13
7056 This patch contains a small internal refactoring with no user-visible
7057 impact.
7058 Thanks to Andrea for writing this at the SciPy 2020 Sprints!
7060 5.19.1 - 2020-07-12
7062 This release slightly improves shrinking behaviour. This should mainly
7063 only impact stateful tests, but may have some minor positive impact on
7064 shrinking collections (lists, sets, etc).
7065 5.19.0 - 2020-06-30
7067 This release improves the randoms() strategy by adding support for Ran‐
7068 dom instances where Hypothesis generates the random values rather than
7069 having them be "truly" random.
7070 5.18.3 - 2020-06-27
7072 This patch adds some internal functions to support a new feature we're
7073 working on. There is no user-visible change... yet.
7074 5.18.2 - 2020-06-26
7076 This patch improves our docs for the derandomize setting.
7077 5.18.1 - 2020-06-25
7079 This release consists of some internal refactoring to the shrinker in
7080 preparation for future work. It has no user visible impact.
7081 5.18.0 - 2020-06-22
7083 This release teaches Hypothesis to shorten tracebacks for explicit ex‐
7084 amples, as we already do for generated examples, so that you can focus
7085 on your code rather than ours.
7086 If you have multiple failing explicit examples, they will now all be
7088 reported. To report only the first failure, you can use the re‐
7089 port_multiple_bugs=False setting as for generated examples.
7090 5.17.0 - 2020-06-22
7092 This patch adds strategy inference for the Literal, NewType, Type, De‐
7093 faultDict, and TypedDict types from the typing_extensions backport on
7094 PyPI.
7095 5.16.3 - 2020-06-21
7097 This patch precomputes some of the setup logic for our external fuzzer
7098 integration and sets deadline=None in fuzzing mode, saving around 150us
7099 on each iteration.
7100 This is around two-thirds the runtime to fuzz an empty test with
7102 @given(st.none()), and nice to have even as a much smaller fraction of
7103 the runtime for non-trivial tests.
7104 5.16.2 - 2020-06-19
7106 This patch fixes an internal error when warning about the use of func‐
7107 tion-scoped fixtures for parametrised tests where the parametrised
7108 value contained a % character. Thanks to Bryant for reporting and fix‐
7109 ing this bug!
7110 5.16.1 - 2020-06-10
7112 If you pass a python:list or python:tuple where a strategy was ex‐
7113 pected, the error message now mentions sampled_from() as an example
7114 strategy.
7115 Thanks to the enthusiastic participants in the PyCon Mentored Sprints
7117 who suggested adding this hint.
7118 5.16.0 - 2020-05-27
7120 functions() can now infer the appropriate returns strategy if you pass
7121 a like function with a return-type annotation. Before, omitting the
7122 returns argument would generate functions that always returned None.
7123 5.15.1 - 2020-05-21
7125 Fix from_type() with generic types under Python 3.9.
7126 5.15.0 - 2020-05-19
7128 This patch fixes an error that happens when multiple threads create new
7129 strategies.
7130 5.14.0 - 2020-05-13
7132 Passing min_magnitude=None to complex_numbers() is now deprecated - you
7133 can explicitly pass min_magnitude=0, or omit the argument entirely.
7134 5.13.1 - 2020-05-13
7136 This patch fixes an internal error in from_type() for python:typ‐
7137 ing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and
7138 fixing issue #2427!
7139 5.13.0 - 2020-05-12
7141 This release upgrades the test statistics available via the --hypothe‐
7142 sis-show-statistics option to include separate information on each of
7143 the phases (issue #1555).
7144 5.12.2 - 2020-05-12
7146 This patch teaches the from_type() internals to return slightly more
7147 efficient strategies for some generic sets and mappings.
7148 5.12.1 - 2020-05-12
7150 This patch adds a # noqa comment for flake8 3.8.0, which disagrees with
7151 mypy about how to write the type of ....
7152 5.12.0 - 2020-05-10
7154 This release limits the maximum duration of the shrinking phase to five
7155 minutes, so that Hypothesis does not appear to hang when making very
7156 slow progress shrinking a failing example (issue #2340).
7157 If one of your tests triggers this logic, we would really appreciate a
7159 bug report to help us improve the shrinker for difficult but realistic
7160 workloads.
7161 5.11.0 - 2020-05-07
7163 This release improves the interaction between assume() and the @exam‐
7164 ple() decorator, so that the following test no longer fails with Unsat‐
7165 isfiedAssumption (issue #2125):
7166 @given(value=floats(0, 1))
7168 @example(value=0.56789) # used to make the test fail!
7169 @pytest.mark.parametrize("threshold", [0.5, 1])
7170 def test_foo(threshold, value):
7171 assume(value < threshold)
7172 ...
7173 5.10.5 - 2020-05-04
7175 If you have django installed but don't use it, this patch will make im‐
7176 port hypothesis a few hundred milliseconds faster (e.g. 0.704s ->
7177 0.271s).
7178 Thanks to importtime-waterfall for highlighting this problem and Jake
7180 Vanderplas for the solution - it's impossible to misuse code from a
7181 module you haven't imported!
7182 5.10.4 - 2020-04-24
7184 This patch improves the internals of builds() type inference, to handle
7185 recursive forward references in certain dataclasses. This is useful
7186 for e.g. hypothesmith's forthcoming LibCST mode.
7187 5.10.3 - 2020-04-22
7189 This release reverses the order in which some operations are tried dur‐
7190 ing shrinking. This should generally be a slight performance improve‐
7191 ment, but most tests are unlikely to notice much difference.
7192 5.10.2 - 2020-04-22
7194 This patch fixes issue #2406, where use of pandas:pandas.Timestamp ob‐
7195 jects as bounds for the datetimes() strategy caused an internal error.
7196 This bug was introduced in version 5.8.1.
7197 5.10.1 - 2020-04-19
7199 This release is a small internal refactoring to how shrinking interacts
7200 with targeted property-based testing that should have no user user vis‐
7201 ible impact.
7202 5.10.0 - 2020-04-18
7204 This release improves our support for datetimes and times around DST
7205 transitions.
7206 times() and datetimes() are now sometimes generated with fold=1, indi‐
7208 cating that they represent the second occurrence of a given wall-time
7209 when clocks are set backwards. This may be set even when there is no
7210 transition, in which case the fold value should be ignored.
7211 For consistency, timezones provided by the pytz package can now gener‐
7213 ate imaginary times (such as the hour skipped over when clocks 'spring
7214 forward' to daylight saving time, or during some historical timezone
7215 transitions). All other timezones have always supported generation of
7216 imaginary times.
7217 If you prefer the previous behaviour, datetimes() now takes an argument
7219 allow_imaginary which defaults to True but can be set to False for any
7220 timezones strategy.
7221 5.9.1 - 2020-04-16
7223 This patch fixes the rendering of binary() docstring by using the
7224 proper backticks syntax.
7225 5.9.0 - 2020-04-15
7227 Failing tests which use target() now report the highest score observed
7228 for each target alongside the failing example(s), even without explic‐
7229 itly showing test statistics.
7230 This improves the debugging workflow for tests of accuracy, which as‐
7232 sert that the total imprecision is within some error budget - for exam‐
7233 ple, abs(a - b) < 0.5. Previously, shrinking to a minimal failing ex‐
7234 ample could often make errors seem smaller or more subtle than they re‐
7235 ally are (see the threshold problem, and issue #2180).
7236 5.8.6 - 2020-04-15
7238 This patch improves the docstring of binary(), the python:repr() of
7239 sampled_from() on an python:enum.Enum subclass, and a warning in our
7240 pytest plugin. There is no change in runtime behaviour.
7241 5.8.5 - 2020-04-15
7243 This release (potentially very significantly) improves the performance
7244 of failing tests in some rare cases, mostly only relevant when using
7245 targeted property-based testing, by stopping further optimisation of
7246 unrelated test cases once a failing example is found.
7247 5.8.4 - 2020-04-14
7249 This release fixes issue #2395, where under some circumstances targeted
7250 property-based testing could cause Hypothesis to get caught in an infi‐
7251 nite loop.
7252 5.8.3 - 2020-04-12
7254 This patch teaches builds() and from_type() to use the __signature__
7255 attribute of classes where it has been set, improving our support for
7256 Pydantic models (in pydantic >= 1.5).
7257 5.8.2 - 2020-04-12
7259 This release improves the performance of the part of the core engine
7260 that deliberately generates duplicate values.
7261 5.8.1 - 2020-04-12
7263 This patch improves dates() shrinking, to simplify year, month, and day
7264 like datetimes() rather than minimizing the number of days since
7265 2000-01-01.
7266 5.8.0 - 2020-03-24
7268 This release adds a .hypothesis.fuzz_one_input attribute to @given
7269 tests, for easy integration with external fuzzers such as python-afl
7270 (supporting issue #171).
7271 5.7.2 - 2020-03-24
7273 This patch fixes issue #2341, ensuring that the printed output from a
7274 stateful test cannot use variable names before they are defined.
7275 5.7.1 - 2020-03-23
7277 This patch fixes issue #2375, preventing incorrect failure when a func‐
7278 tion scoped fixture is overridden with a higher scoped fixture.
7279 5.7.0 - 2020-03-19
7281 This release allows the array_dtypes() strategy to generate Numpy
7282 dtypes which have field titles in addition to field names. We expect
7283 this to expose latent bugs where code expects that set(dtype.names) ==
7284 set(dtype.fields), though the latter may include titles.
7285 5.6.1 - 2020-03-18
7287 This makes model a positional-only argument to from_model(), to support
7288 models with a field literally named "model" (issue #2369).
7289 5.6.0 - 2020-02-29
7291 This release adds an explicit warning for tests that are both decorated
7292 with @given(...) and request a function-scoped pytest fixture, because
7293 such fixtures are only executed once for all Hypothesis test cases and
7294 that often causes trouble (issue #377).
7295 It's very difficult to fix this on the pytest side, so since 2015 our
7297 advice has been "just don't use function-scoped fixtures with Hypothe‐
7298 sis". Now we detect and warn about the issue at runtime!
7299 5.5.5 - 2020-02-29
7301 This release cleans up the internal machinery for stateful, after we
7302 dropped the legacy APIs in Hypothesis 5.0 (issue #2218). There is no
7303 user-visible change.
7304 5.5.4 - 2020-02-16
7306 This patch fixes issue #2351, arrays() would raise a confusing error if
7307 we inferred a strategy for datetime64 or timedelta64 values with vary‐
7308 ing time units.
7309 We now infer an internally-consistent strategy for such arrays, and
7311 have a more helpful error message if an inconsistent strategy is ex‐
7312 plicitly specified.
7313 5.5.3 - 2020-02-14
7315 This patch improves the signature of builds() by specifying target as a
7316 positional-only argument on Python 3.8 (see PEP 570). The semantics of
7317 builds() have not changed at all - this just clarifies the documenta‐
7318 tion.
7319 5.5.2 - 2020-02-13
7321 This release makes Hypothesis faster at generating test cases that con‐
7322 tain duplicated values in their inputs.
7323 5.5.1 - 2020-02-07
7325 This patch has some tiny internal code clean-ups, with no user-visible
7326 change.
7327 5.5.0 - 2020-02-07
7329 Our style guide suggests that optional parameters should usually be
7330 keyword-only arguments (see PEP 3102) to prevent confusion based on po‐
7331 sitional arguments - for example, hypothesis.strategies.floats() takes
7332 up to four boolean flags and many of the Numpy strategies have both
7333 dims and side bounds.
7334 This release converts most optional parameters in our API to use key‐
7336 word-only arguments - and adds a compatibility shim so you get warnings
7337 rather than errors everywhere (issue #2130).
7338 5.4.2 - 2020-02-06
7340 This patch fixes compatibility with Python 3.5.2 (issue #2334). Note
7341 that we only test the latest patch of each minor version, though as in
7342 this case we usually accept pull requests for older patch versions.
7343 5.4.1 - 2020-02-01
7345 This patch improves the repr of from_type(), so that in most cases it
7346 will display the strategy it resolves to rather than from_type(...).
7347 The latter form will continue to be used where resolution is not imme‐
7348 diately successful, e.g. invalid arguments or recursive type defini‐
7349 tions involving forward references.
7350 5.4.0 - 2020-01-30
7352 This release removes support for Python 3.5.0 and 3.5.1, where the
7353 python:typing module was quite immature (e.g. missing overload() and
7354 Type).
7355 Note that Python 3.5 will reach its end-of-life in September 2020, and
7357 new releases of Hypothesis may drop support somewhat earlier.
7358 NOTE:
7360 pip install hypothesis should continue to give you the latest com‐
7361 patible version. If you have somehow ended up with an incompatible
7362 version, you need to update your packaging stack to pip >= 9.0 and
7363 setuptools >= 24.2 - see here for details. Then pip uninstall hy‐
7364 pothesis && pip install hypothesis will get you back to a compatible
7365 version.
7366 5.3.1 - 2020-01-26
7368 This patch does some minor internal cleanup; there is no user-visible
7369 change.
7370 5.3.0 - 2020-01-21
7372 The standard library ipaddress module is new in Python 3, and this re‐
7373 lease adds the new ip_addresses() strategy to generate IPv4Addresses
7374 and/or IPv6Addresses (depending on the v and network arguments).
7375 If you use them in type annotations, from_type() now has strategies
7377 registered for ipaddress address, network, and interface types.
7378 The provisional strategies for IP address strings are therefore depre‐
7380 cated.
7381 5.2.1 - 2020-01-21
7383 This patch reverts version 5.2, due to a strange issue where indexing
7384 an array of strings can raise an error instead of returning an item
7385 which contains certain surrogate characters.
7386 5.2.0 - 2020-01-19
7388 This release allows from_dtype() to generate Unicode strings which can‐
7389 not be encoded in UTF-8, but are valid in Numpy arrays (which use
7390 UTF-32).
7391 5.1.6 - 2020-01-19
7393 This patch fixes issue #2320, where from_type(Set[Hashable]) could
7394 raise an internal error because Decimal("snan") is of a hashable type,
7395 but raises an error when hashed. We now ensure that set elements and
7396 dict keys in generic types can actually be hashed.
7397 5.1.5 - 2020-01-12
7399 This patch fixes an internal error when running in an IPython repl or
7400 Jupyter notebook on Windows (issue #2319), and an internal error on
7401 Python 3.5.1 (issue #2318).
7402 5.1.4 - 2020-01-11
7404 This patch fixes a bug where errors in third-party extensions such as
7405 hypothesis-trio or hypothesis-jsonschema were incorrectly considered to
7406 be Hypothesis internal errors, which could result in confusing error
7407 messages.
7408 Thanks to Vincent Michel for reporting and fixing the bug!
7410 5.1.3 - 2020-01-11
7412 This release converts the type hint comments on our public API to PEP
7413 484 type annotations.
7414 Thanks to Ivan Levkivskyi for com2ann - with the refactoring tools from
7416 5.0.1 it made this process remarkably easy!
7417 5.1.2 - 2020-01-09
7419 This patch makes multiple() iterable, so that output like a, b =
7420 state.some_rule() is actually executable and can be used to reproduce
7421 failing examples.
7422 Thanks to Vincent Michel for reporting and fixing issue #2311!
7424 5.1.1 - 2020-01-06
7426 This patch contains many small refactorings to replace our Python 2
7427 compatibility functions with their native Python 3 equivalents. Since
7428 Hypothesis is now Python 3 only, there is no user-visible change.
7429 5.1.0 - 2020-01-03
7431 This release teaches from_type() how to generate python:datetime.time‐
7432 zone. As a result, you can now generate python:datetime.tzinfo objects
7433 without having pytz installed.
7434 If your tests specifically require pytz timezones, you should be using
7436 hypothesis.extra.pytz.timezones() instead of st.from_type(tzinfo).
7437 5.0.1 - 2020-01-01
7439 This patch contains mostly-automated refactorings to remove code that
7440 we only needed to support Python 2. Since Hypothesis is now Python 3
7441 only (hurray!), there is no user-visible change.
7442 Our sincere thanks to the authors of autoflake, black, isort, and
7444 pyupgrade, who have each and collectively made this kind of update
7445 enormously easier.
7446 5.0.0 - 2020-01-01
7448 Welcome to the next major version of Hypothesis!
7449 There are no new features here, as we release those in minor versions.
7451 Instead, 5.0 is a chance for us to remove deprecated features (many al‐
7452 ready converted into no-ops), and turn a variety of warnings into er‐
7453 rors.
7454 If you were running on the last version of Hypothesis 4.x without any
7456 Hypothesis deprecation warnings, this will be a very boring upgrade.
7457 In fact, nothing will change for you at all.
7458 NOTE:
7460 This release drops support for Python 2, which has passed its end of
7461 life date. The Python 3 Statement outlines our reasons, and lists
7462 many other packages that have made the same decision.
7463 pip install hypothesis should continue to give you the latest com‐
7465 patible version. If you have somehow ended up with Hypothesis 5.0
7466 on Python 2, you need to update your packaging stack to pip >= 9.0
7467 and setuptools >= 24.2 - see here for details. Then pip uninstall
7468 hypothesis && pip install hypothesis will get you back to a compati‐
7469 ble version.
7470 Strategies
7472 • integers() bounds must be equal to an integer, though they can still
7473 be other types.
7474 • If fractions() is passed a max_denominator, the bounds must have at
7476 most that denominator.
7477 • floats() bounds must be exactly representable as a floating-point
7479 number with the given width. If not, the error message includes the
7480 nearest such number.
7481 • sampled_from([]) is now an error.
7483 • The values from the elements and fill strategies for hypothesis.ex‐
7485 tra.numpy.arrays() must be losslessly representable in an array of
7486 the given dtype.
7487 • The min_size and max_size arguments to all collection strategies must
7489 be of type python:int (or max_size may be None).
7490 Miscellaneous
7492 • The .example() method of strategies (intended for interactive explo‐
7493 ration) no longer takes a random argument.
7494 • It is now an error to apply @example, @seed, or @reproduce_failure
7496 without also applying @given.
7497 • You may pass either the target or targets argument to stateful rules,
7499 but not both.
7500 • deadline must be None (to disable), a timedelta, or an integer or
7502 float number of milliseconds.
7503 • Both of derandomize and print_blob must be either True or False,
7505 where they previously accepted other values.
7506 • stateful_step_count must be at least one.
7508 • max_examples must be at least one. To disable example generation,
7510 use the phases setting.
7511 Removals
7513 • hypothesis.stateful.GenericStateMachine in favor of hypothesis.state‐
7514 ful.RuleBasedStateMachine
7515 • hypothesis.extra.django.models.models in favor of hypothesis.ex‐
7517 tra.django.from_model() and hypothesis.extra.django.models.add_de‐
7518 fault_field_mapping in favor of hypothesis.extra.django.regis‐
7519 ter_field_strategy()
7520 • hypothesis.HealthCheck.hung_test, without replacement
7522 • hypothesis.settings.buffer, without replacement
7524 • hypothesis.PrintSettings, because hypothesis.settings.print_blob
7526 takes True or False
7527 • hypothesis.settings.timeout, in favor of hypothesis.settings.deadline
7529 • hypothesis.unlimited without replacement (only only useful as argu‐
7531 ment to timeout)
7532 Hypothesis 4.x
7534 4.57.1 - 2019-12-29
7535 This patch improves the type hints and documentation for the django ex‐
7536 tra. There is no runtime change.
7537 4.57.0 - 2019-12-28
7539 This release improves support for the SupportsOp protocols from the
7540 python:typing module when using on from_type() as outlined in issue
7541 #2292. The following types now generate much more varied strategies
7542 when called with from_type():
7543 • python:typing.SupportsAbs
7545 • python:typing.SupportsBytes
7547 • python:typing.SupportsComplex
7549 • python:typing.SupportsInt
7551 • python:typing.SupportsFloat
7553 • python:typing.SupportsRound
7555 Note that using from_type() with one of the above strategies will not
7557 ensure that the the specified function will execute successfully (ie :
7558 the strategy returned for from_type(typing.SupportsAbs) may include
7559 NaNs or things which cause the python:abs() function to error. )
7560 Thanks to Lea Provenzano for this patch.
7562 4.56.3 - 2019-12-22
7564 This release fixes a small internal bug in shrinking which could have
7565 caused it to perform slightly more tests than were necessary. Fixing
7566 this shouldn't have much effect but it will make shrinking slightly
7567 faster.
7568 4.56.2 - 2019-12-21
7570 This release removes an internal heuristic that was no longer providing
7571 much benefit. It is unlikely that there will be any user visible ef‐
7572 fect.
7573 4.56.1 - 2019-12-19
7575 This release further improves the optimisation algorithm for targeted
7576 property-based testing.
7577 4.56.0 - 2019-12-18
7579 This release enables deprecation warnings even when the verbosity set‐
7580 ting is quiet, in preparation for Hypothesis 5.0 (issue #2218).
7581 Warnings can still be filtered by the standard mechanisms provided in
7583 the standard-library python:warnings module.
7584 4.55.4 - 2019-12-18
7586 This release improves Hypothesis's management of the set of test cases
7587 it tracks between runs. It will only do anything if you have Phase.tar‐
7588 get enabled and an example database set. In those circumstances it
7589 should result in a more thorough and faster set of examples that are
7590 tried on each run.
7591 4.55.3 - 2019-12-18
7593 This release makes Hypothesis better at generating test cases where
7594 generated values are duplicated in different parts of the test case.
7595 This will be especially noticeable with reasonably complex values, as
7596 it was already able to do this for simpler ones such as integers or
7597 floats.
7598 4.55.2 - 2019-12-17
7600 This release expands the set of test cases that Hypothesis saves in its
7601 database for future runs to include a representative set of "struc‐
7602 turally different" test cases - e.g. it might try to save test cases
7603 where a given list is empty or not.
7604 Currently this is unlikely to have much user visible impact except to
7606 produce slightly more consistent behaviour between consecutive runs of
7607 a test suite. It is mostly groundwork for future improvements which
7608 will exploit this functionality more effectively.
7609 4.55.1 - 2019-12-16
7611 This patch fixes issue #2257, where from_type() could incorrectly gen‐
7612 erate bytestrings when passed a generic python:typing.Sequence such as
7613 Sequence[set].
7614 4.55.0 - 2019-12-16
7616 This release adds database support for targeted property-based testing,
7617 so the best examples based on the targeting will be saved and reused
7618 between runs. This is mostly laying groundwork for future features in
7619 this area, but will also make targeted property-based tests more useful
7620 during development, where the same tests tend to get run over and over
7621 again.
7622 If max_examples is large, this may increase memory usage significantly
7624 under some circumstances, but these should be relatively rare.
7625 This release also adds a dependency on the sortedcontainers package.
7627 4.54.2 - 2019-12-16
7629 This release improves the optimisation algorithm for targeted prop‐
7630 erty-based testing, so that it will find higher quality results more
7631 reliably. Specifically, in cases where it would previously have got
7632 near a local optimum, it will now tend to achieve the locally optimal
7633 value.
7634 4.54.1 - 2019-12-16
7636 This release is mostly internal changes in support of better testing of
7637 the core engine. You are unlikely to see much effect, although some in‐
7638 ternal heuristics have changed slightly.
7639 4.54.0 - 2019-12-15
7641 This release adds a dedicated phase for targeted property-based test‐
7642 ing, and (somewhat) improves the targeting algorithm so that it will
7643 find higher quality results more reliably. This comes at a cost of
7644 making it more likely to get stuck in a local optimum.
7645 4.53.3 - 2019-12-15
7647 This patch fixes from_type() with python:typing.Hashable and
7648 python:typing.Sized, which previously failed with an internal error on
7649 Python 3.7 or later.
7650 Thanks to Lea Provenzano for both reporting issue #2272 and writing the
7652 patch!
7653 4.53.2 - 2019-12-11
7655 This release reorganises a number of the Hypothesis internal modules
7656 into a package structure. If you are only depending on the public API
7657 it should have no effect. If you are depending on the internal API
7658 (which you shouldn't be, and which we don't guarantee compatibility on)
7659 you may have to rename some imports.
7660 4.53.1 - 2019-12-09
7662 This release changes the size distribution of the number of steps run
7663 in stateful testing: It will now almost always run the maximum number
7664 of steps permitted.
7665 4.53.0 - 2019-12-09
7667 statistics now include the best score seen for each label, which can
7668 help avoid the threshold problem when the minimal example shrinks
7669 right down to the threshold of failure (issue #2180).
7670 4.52.0 - 2019-12-09
7672 This release changes the stateful_step_count setting to raise an error
7673 if set to 0. This is a backwards compatible change because a value of 0
7674 would never have worked and attempting to run it would have resulted in
7675 an internal assertion error.
7676 4.51.1 - 2019-12-09
7678 This release makes a small internal change to the distribution of test
7679 cases. It is unlikely to have much user visible impact.
7680 4.51.0 - 2019-12-07
7682 This release deprecates use of @example, @seed, or @reproduce_failure
7683 without @given.
7684 Thanks to Nick Anyos for the patch!
7686 4.50.8 - 2019-12-05
7688 This patch makes certain uses of Bundles more efficient in stateful
7689 testing (issue #2078).
7690 4.50.7 - 2019-12-05
7692 This release refactors some of Hypothesis's internal interfaces for
7693 representing data generation. It should have no user visible effect.
7694 4.50.6 - 2019-12-02
7696 This patch removes some old debugging helpers in our Numpy extra which
7697 have not been needed since issue #1963 and issue #2245.
7698 4.50.5 - 2019-12-01
7700 This patch fixes issue #2229, where Numpy arrays of unsized strings
7701 would only ever have strings of size one due to an interaction between
7702 our generation logic and Numpy's allocation strategy.
7703 4.50.4 - 2019-12-01
7705 This patch fixes a rare internal error in strategies for a list of
7706 unique items sampled from a short non-unique sequence (issue #2247).
7707 The bug was discovered via hypothesis-jsonschema.
7708 4.50.3 - 2019-12-01
7710 This release improves the error message when @settings tries to inherit
7711 settings from a parent argument that isn't a settings instance.
7712 4.50.2 - 2019-11-29
7714 This release improves Hypothesis's "Falsifying example" output, by
7715 breaking output across multiple lines where necessary, and by removing
7716 irrelevant information from the stateful testing output.
7717 4.50.1 - 2019-11-29
7719 This patch adds flake8-comprehensions to our linter suite. There is no
7720 user-visible change - expect perhaps via some strange microbenchmarks -
7721 but certain parts of the code now have a clear and more consistent
7722 style.
7723 4.50.0 - 2019-11-28
7725 This release fixes some cases where we might previously have failed to
7726 run the validation logic for some strategies. As a result tests which
7727 would previously have been silently testing significantly less than
7728 they should may now start to raise InvalidArgument now that these er‐
7729 rors are caught.
7730 4.49.0 - 2019-11-28
7732 This release significantly improves the data distribution in rule based
7733 stateful testing, by using a technique called Swarm Testing (Groce,
7734 Alex, et al. "Swarm testing." Proceedings of the 2012 International
7735 Symposium on Software Testing and Analysis. ACM, 2012.) to select
7736 which rules are run in any given test case. This should allow it to
7737 find many issues that it would previously have missed.
7738 This change is likely to be especially beneficial for stateful tests
7740 with large numbers of rules.
7741 4.48.1 - 2019-11-28
7743 This release adds some heuristics to test case generation that try to
7744 ensure that test cases generated early on will be relatively small.
7745 This fixes a bug introduced in Hypothesis 4.42.0 which would cause oc‐
7747 casional too_slow failures on some tests.
7748 4.48.0 - 2019-11-28
7750 This release revokes the deprecation of find, as we've now rebuilt it
7751 on top of @given, which means it has minimal maintenance burden and
7752 we're happy to support it.
7753 4.47.5 - 2019-11-28
7755 This release rebuilds find() on top of @given in order to have more
7756 code in common. It should have minimal user visible effect.
7757 4.47.4 - 2019-11-27
7759 This patch removes an internal compatibility shim that we no longer
7760 need.
7761 4.47.3 - 2019-11-26
7763 This patch fixes several typos in our docstrings and comments, with no
7764 change in behaviour. Thanks to Dmitry Dygalo for identifying and fix‐
7765 ing them!
7766 4.47.2 - 2019-11-25
7768 This release fixes an internal issue where Hypothesis would sometimes
7769 generate test cases that were above its intended maximum size. This
7770 would only have happened rarely and probably would not have caused ma‐
7771 jor problems when it did.
7772 Users of the new targeted property-based testing might see minor im‐
7774 pact (possibly slightly faster tests and slightly worse target scores),
7775 but only in the unlikely event that they were hitting this problem.
7776 Other users should not see any effect at all.
7777 4.47.1 - 2019-11-24
7779 This release removes some unused code from the core engine. There is
7780 no user-visible change.
7781 4.47.0 - 2019-11-24
7783 This release commonizes some code between running explicit examples and
7784 normal test execution. The main user visible impact of this is that
7785 deadlines are now enforced when running explicit examples.
7786 4.46.1 - 2019-11-23
7788 This patch ensures that a KeyboardInterrupt received during example
7789 generation is not treated as a mystery test failure but instead propa‐
7790 gates to the top level, not recording the interrupted generation in the
7791 conjecture data tree. Thanks to Anne Archibald for identifying and
7792 fixing the problem.
7793 4.46.0 - 2019-11-22
7795 This release changes the behaviour of floats() when excluding signed
7796 zeros - floats(max_value=0.0, exclude_max=True) can no longer generate
7797 -0.0 nor the much rarer floats(min_value=-0.0, exclude_min=True) gener‐
7798 ate +0.0.
7799 The correct interaction between signed zeros and exclusive endpoints
7801 was unclear; we now enforce the invariant that floats() will never gen‐
7802 erate a value equal to an excluded endpoint (issue #2201).
7803 If you prefer the old behaviour, you can pass floats(max_value=-0.0) or
7805 floats(min_value=0.0) which is exactly equivalent and has not changed.
7806 If you had two endpoints equal to zero, we recommend clarifying your
7807 tests by using just() or sampled_from() instead of floats().
7808 4.45.1 - 2019-11-20
7810 This patch improves the error message when invalid arguments are passed
7811 to rule() or invariant() (issue #2149).
7812 Thanks to Benjamin Palmer for this bugfix!
7814 4.45.0 - 2019-11-20
7816 This release supports python:typing.Final and python:typing.TypedDict
7817 in from_type().
7818 4.44.5 - 2019-11-20
7820 This patch disables our pytest plugin when running on versions of
7821 pytest before 4.3, the oldest our plugin supports. Note that at time
7822 of writing the Pytest developers only support 4.6 and later!
7823 Hypothesis tests using @given() work on any test runner, but our inte‐
7825 grations to e.g. avoid example database collisions when combined with
7826 @pytest.mark.parametrize eventually drop support for obsolete versions.
7827 4.44.4 - 2019-11-20
7829 This patch adds some internal comments and clarifications to the Hy‐
7830 pothesis implementation. There is no user-visible change.
7831 4.44.3 - 2019-11-20
7833 This patch avoids importing test runners such as pytest, unittest2, or
7834 nose solely to access their special "skip test" exception types - if
7835 the module is not in sys.modules, the exception can't be raised anyway.
7836 This fixes a problem where importing an otherwise unused module could
7838 cause spurious errors due to import-time side effects (and possibly
7839 -Werror).
7840 4.44.2 - 2019-11-12
7842 This release fixes @given to only complain about missing keyword-only
7843 arguments if the associated test function is actually called.
7844 This matches the behaviour of other InvalidArgument errors produced by
7846 @given.
7847 4.44.1 - 2019-11-11
7849 This patch allows Hypothesis to run in environments that do not specify
7850 a __file__, such as a python:zipapp (issue #2196).
7851 4.44.0 - 2019-11-11
7853 This release adds a signature argument to mutually_broad‐
7854 castable_shapes() (issue #2174), which allows us to generate shapes
7855 which are valid for functions like np.matmul() that require shapes
7856 which are not simply broadcastable.
7857 Thanks to everyone who has contributed to this feature over the last
7859 year, and a particular shout-out to Zac Hatfield-Dodds and Ryan Sok‐
7860 laski for mutually_broadcastable_shapes() and to Ryan Turner for the
7861 downstream hypothesis-gufunc project.
7862 4.43.9 - 2019-11-11
7864 This patch fixes issue #2108, where the first test using data() to draw
7865 from characters() or text() would be flaky due to unreliable test tim‐
7866 ings.
7867 Time taken by lazy instantiation of strategies is now counted towards
7869 drawing from the strategy, rather than towards the deadline for the
7870 test function.
7871 4.43.8 - 2019-11-08
7873 This release ensures that the strategies passed to @given are properly
7874 validated when applied to a test method inside a test class.
7875 This should result in clearer error messages when some of those strate‐
7877 gies are invalid.
7878 4.43.7 - 2019-11-08
7880 This release changes how Hypothesis manages its search space in cases
7881 where it generates redundant data. This should cause it to generate
7882 significantly fewer duplicated examples (especially with short integer
7883 ranges), and may cause it to produce more useful examples in some cases
7884 (especially ones where there is a significant amount of filtering).
7885 4.43.6 - 2019-11-07
7887 This patch refactors width handling in floats(); you may notice small
7888 performance improvements but the main purpose is to enable work on
7889 issue #1704 (improving shrinking of bounded floats).
7890 4.43.5 - 2019-11-06
7892 This patch removes an unused internal flag. There is no user-visible
7893 change.
7894 4.43.4 - 2019-11-05
7896 This patch corrects the exception type and error message you get if you
7897 attempt to use data() to draw from something which is not a strategy.
7898 This never worked, but the error is more helpful now.
7899 4.43.3 - 2019-11-05
7901 We've adopted flake8-bugbear to check for a few more style issues, and
7902 this patch implements the minor internal cleanups it suggested. There
7903 is no user-visible change.
7904 4.43.2 - 2019-11-05
7906 This patch fixes the formatting of some documentation, but there is no
7907 change to any executed code.
7908 4.43.1 - 2019-11-04
7910 Python 3.8's new python:typing.Literal type - see PEP 586 for details -
7911 is now supported in from_type().
7912 4.43.0 - 2019-11-04
7914 This release adds the strategy mutually_broadcastable_shapes(), which
7915 generates multiple array shapes that are mutually broadcast-compatible
7916 with an optional user-specified base-shape.
7917 This is a generalisation of broadcastable_shapes(). It relies heavily
7919 on non-public internals for performance when generating and shrinking
7920 examples. We intend to support generating shapes matching a ufunc sig‐
7921 nature in a future version (issue #2174).
7922 Thanks to Ryan Soklaski, Zac Hatfield-Dodds, and @rdturnermtl who con‐
7924 tributed to this new feature.
7925 4.42.10 - 2019-11-03
7927 This release fixes from_type() when used with bounded or constrained
7928 python:typing.TypeVar objects (issue #2094).
7929 Previously, distinct typevars with the same constraints would be
7931 treated as all single typevar, and in cases where a typevar bound was
7932 resolved to a union of subclasses this could result in mixed types be‐
7933 ing generated for that typevar.
7934 4.42.9 - 2019-11-03
7936 This patch ensures that the default value broadcastable_shapes()
7937 chooses for max_dims is always valid (at most 32), even if you pass
7938 min_dims=32.
7939 4.42.8 - 2019-11-02
7941 This patch ensures that we only add profile information to the pytest
7942 header if running either pytest or Hypothesis in verbose mode, matching
7943 the builtin cache plugin (issue #2155).
7944 4.42.7 - 2019-11-02
7946 This patch makes stateful step printing expand the result of a step
7947 into multiple variables when you return multiple() (issue #2139).
7948 Thanks to Joseph Weston for reporting and fixing this bug!
7949 4.42.6 - 2019-11-02
7951 This release fixes a bug (issue #2166) where a Unicode character info
7952 cache file was generated but never used on subsequent test runs, caus‐
7953 ing tests to run more slowly than they should have.
7954 Thanks to Robert Knight for this bugfix!
7956 4.42.5 - 2019-11-01
7958 This patch corrects some internal documentation. There is no user-vis‐
7959 ible change.
7960 4.42.4 - 2019-11-01
7962 This release fixes a bug (issue #2160) where decorators applied after
7963 @settings and before @given were ignored.
7964 Thanks to Tom Milligan for this bugfix!
7966 4.42.3 - 2019-10-30
7968 This release updates Hypothesis's formatting to the new version of
7969 black, and has absolutely no user visible effect.
7970 4.42.2 - 2019-10-30
7972 This release fixes a bug in recursive() which would have meant that in
7973 practice max_leaves was treated as if it was lower than it actually is
7974 - specifically it would be capped at the largest power of two smaller
7975 than it. It is now handled correctly.
7976 4.42.1 - 2019-10-30
7978 Python 3.8's new python:typing.SupportsIndex type - see PEP 357 for de‐
7979 tails - is now supported in from_type().
7980 Thanks to Grigorios Giannakopoulos for the patch!
7982 4.42.0 - 2019-10-27
7984 This release significantly simplifies Hypothesis's internal logic for
7985 data generation, by removing a number of heuristics of questionable or
7986 unproven value.
7987 The results of this change will vary significantly from test to test.
7989 Most test suites will see significantly faster data generation and
7990 lower memory usage. The "quality" of the generated data may go up or
7991 down depending on your particular test suites.
7992 If you see any significant regressions in Hypothesis's ability to find
7994 bugs in your code as a result of this release, please file an issue to
7995 let us know.
7996 Users of the new targeted property-based testing functionality are
7998 reasonably likely to see improvements in data generation, as this re‐
7999 lease changes the search algorithm for targeted property based testing
8000 to one that is more likely to be productive than the existing approach.
8001 4.41.3 - 2019-10-21
8003 This patch is to ensure that our internals remain comprehensible to
8004 mypy 0.740 - there is no user-visible change.
8005 4.41.2 - 2019-10-17
8007 This patch changes some internal hashes to SHA384, to better support
8008 users subject to FIPS-140. There is no user-visible API change.
8009 Thanks to Paul Kehrer for this contribution!
8011 4.41.1 - 2019-10-16
8013 This release makes --hypothesis-show-statistics much more useful for
8014 tests using a RuleBasedStateMachine, by simplifying the reprs so that
8015 events are aggregated correctly.
8016 4.41.0 - 2019-10-16
8018 This release upgrades the fixed_dictionaries() strategy to support op‐
8019 tional keys (issue #1913).
8020 4.40.2 - 2019-10-16
8022 This release makes some minor internal changes in support of improving
8023 the Hypothesis test suite. It should not have any user visible impact.
8024 4.40.1 - 2019-10-14
8026 This release changes how Hypothesis checks if a parameter to a test
8027 function is a mock object. It is unlikely to have any noticeable ef‐
8028 fect, but may result in a small performance improvement, especially for
8029 test functions where a mock object is being passed as the first argu‐
8030 ment.
8031 4.40.0 - 2019-10-09
8033 This release fixes a bug where our example database logic did not dis‐
8034 tinguish between failing examples based on arguments from a
8035 @pytest.mark.parametrize(...). This could in theory cause data loss if
8036 a common failure overwrote a rare one, and in practice caused occa‐
8037 sional file-access collisions in highly concurrent workloads (e.g. dur‐
8038 ing a 300-way parametrize on 16 cores).
8039 For internal reasons this also involves bumping the minimum supported
8041 version of pytest to 4.3
8042 Thanks to Peter C Kroon for the Hacktoberfest patch!
8044 4.39.3 - 2019-10-09
8046 This patch improves our type hints on the emails(), functions(), inte‐
8047 gers(), iterables(), and slices() strategies, as well as the .filter()
8048 method.
8049 There is no runtime change, but if you use mypy or a similar
8051 type-checker on your tests the results will be a bit more precise.
8052 4.39.2 - 2019-10-09
8054 This patch improves the performance of unique collections such as
8055 sets() of just() or booleans() strategies. They were already pretty
8056 good though, so you're unlikely to notice much!
8057 4.39.1 - 2019-10-09
8059 If a value in a dict passed to fixed_dictionaries() is not a strategy,
8060 Hypothesis now tells you which one.
8061 4.39.0 - 2019-10-07
8063 This release adds the basic_indices() strategy, to generate basic in‐
8064 dexes for arrays of the specified shape (issue #1930).
8065 It generates tuples containing some mix of integers, python:slice ob‐
8067 jects, ... (Ellipsis), and numpy:numpy.newaxis; which when used to in‐
8068 dex an array of the specified shape produce either a scalar or a
8069 shared-memory view of the array. Note that the index tuple may be
8070 longer or shorter than the array shape, and may produce a view with an‐
8071 other dimensionality again!
8072 Thanks to Lampros Mountrakis, Ryan Soklaski, and Zac Hatfield-Dodds for
8074 their collaboration on this surprisingly subtle strategy!
8075 4.38.3 - 2019-10-04
8077 This patch defers creation of the .hypothesis directory until we have
8078 something to store in it, meaning that it will appear when Hypothesis
8079 is used rather than simply installed.
8080 Thanks to Peter C Kroon for the Hacktoberfest patch!
8082 4.38.2 - 2019-10-02
8084 This patch bumps our dependency on attrs to >=19.2.0; but there are no
8085 user-visible changes to Hypothesis.
8086 4.38.1 - 2019-10-01
8088 This is a comment-only patch which tells mypy 0.730 to ignore some in‐
8089 ternal compatibility shims we use to support older Pythons.
8090 4.38.0 - 2019-10-01
8092 This release adds the hypothesis.target() function, which implements
8093 targeted property-based testing (issue #1779).
8094 By calling target() in your test function, Hypothesis can do a
8096 hill-climbing search for bugs. If you can calculate a suitable metric
8097 such as the load factor or length of a queue, this can help you find
8098 bugs with inputs that are highly improbably from unguided generation -
8099 however good our heuristics, example diversity, and deduplication logic
8100 might be. After all, those features are at work in targeted PBT too!
8101 4.37.0 - 2019-09-28
8103 This release emits a warning if you use the .example() method of a
8104 strategy in a non-interactive context.
8105 given() is a much better choice for writing tests, whether you care
8107 about performance, minimal examples, reproducing failures, or even just
8108 the variety of inputs that will be tested!
8109 4.36.2 - 2019-09-20
8111 This patch disables part of the typing-based inference for the attrs
8112 package under Python 3.5.0, which has some incompatible internal de‐
8113 tails (issue #2095).
8114 4.36.1 - 2019-09-17
8116 This patch fixes a bug in strategy inference for attrs classes where
8117 Hypothesis would fail to infer a strategy for attributes of a generic
8118 type such as Union[int, str] or List[bool] (issue #2091).
8119 Thanks to Jonathan Gayvallet for the bug report and this patch!
8121 4.36.0 - 2019-09-09
8123 This patch deprecates min_len or max_len of 0 in byte_string_dtypes()
8124 and unicode_string_dtypes(). The lower limit is now 1.
8125 Numpy uses a length of 0 in these dtypes to indicate an undetermined
8127 size, chosen from the data at array creation. However, as the arrays()
8128 strategy creates arrays before filling them, strings were truncated to
8129 1 byte.
8130 4.35.1 - 2019-09-09
8132 This patch improves the messaging that comes from invalid size argu‐
8133 ments to collection strategies such as lists().
8134 4.35.0 - 2019-09-04
8136 This release improves the from_lark() strategy, tightening argument
8137 validation and adding the explicit argument to allow use with terminals
8138 that use @declare instead of a string or regular expression.
8139 This feature is required to handle features such as indent and dedent
8141 tokens in Python code, which can be generated with the hypothesmith
8142 package.
8143 4.34.0 - 2019-08-23
8145 The from_type() strategy now knows to look up the subclasses of ab‐
8146 stract types, which cannot be instantiated directly.
8147 This is very useful for hypothesmith to support libCST.
8149 4.33.1 - 2019-08-21
8151 This patch works around a crash when an incompatible version of Numpy
8152 is installed under PyPy 5.10 (Python 2.7).
8153 If you are still using Python 2, please upgrade to Python 3 as soon as
8155 possible - it will be unsupported at the end of this year.
8156 4.33.0 - 2019-08-20
8158 This release improves the domains() strategy, as well as the urls() and
8159 the emails() strategies which use it. These strategies now use the
8160 full IANA list of Top Level Domains and are correct as per RFC 1035.
8161 Passing tests using these strategies may now fail.
8163 Thanks to TechDragon for this improvement.
8165 4.32.3 - 2019-08-05
8167 This patch tidies up the repr of several settings-related objects, at
8168 runtime and in the documentation, and deprecates the undocumented edge
8169 case that phases=None was treated like phases=tuple(Phase).
8170 It also fixes from_lark() with lark 0.7.2 and later.
8172 4.32.2 - 2019-07-30
8174 This patch updates some internal comments for mypy 0.720. There is no
8175 user-visible impact.
8176 4.32.1 - 2019-07-29
8178 This release changes how the shrinker represents its progress inter‐
8179 nally. For large generated test cases this should result in signifi‐
8180 cantly less memory usage and possibly faster shrinking. Small generated
8181 test cases may be slightly slower to shrink but this shouldn't be very
8182 noticeable.
8183 4.32.0 - 2019-07-28
8185 This release makes arrays() more pedantic about elements strategies
8186 that cannot be exactly represented as array elements.
8187 In practice, you will see new warnings if you were using a float16 or
8189 float32 dtype without passing floats() the width=16 or width=32 argu‐
8190 ments respectively.
8191 The previous behaviour could lead to silent truncation, and thus some
8193 elements being equal to an explicitly excluded bound (issue #1899).
8194 4.31.1 - 2019-07-28
8196 This patch changes an internal use of MD5 to SHA hashes, to better sup‐
8197 port users subject to FIPS-140. There is no user-visible or API
8198 change.
8199 Thanks to Alex Gaynor for this patch.
8201 4.31.0 - 2019-07-24
8203 This release simplifies the logic of the print_blob setting by removing
8204 the option to set it to PrintSettings.INFER. As a result the
8205 print_blob setting now takes a single boolean value, and the use of
8206 PrintSettings is deprecated.
8207 4.28.2 - 2019-07-14
8209 This patch improves the docstrings of several Hypothesis strategies, by
8210 clarifying markup and adding cross-references. There is no runtime
8211 change.
8212 Thanks to Elizabeth Williams and Serah Njambi Rono for their contribu‐
8214 tions at the SciPy 2019 sprints!
8215 4.28.1 - 2019-07-12
8217 This patch improves the behaviour of the text() strategy when passed an
8218 alphabet which is not a strategy. The value is now interpreted as
8219 whitelist_characters to characters() instead of a sequence for sam‐
8220 pled_from(), which standardises the distribution of examples and the
8221 shrinking behaviour.
8222 You can get the previous behaviour by using lists(sampled_from(alpha‐
8224 bet)).map("".map) instead.
8225 4.28.0 - 2019-07-11
8227 This release deprecates find(). The .example() method is a better re‐
8228 placement if you want an example, and for the rare occasions where you
8229 want the minimal example you can get it from @given.
8230 @given has steadily outstripped find() in both features and performance
8232 over recent years, and as we do not have the resources to maintain and
8233 test both we think it is better to focus on just one.
8234 4.27.0 - 2019-07-08
8236 This release refactors the implementation of the .example() method, to
8237 more accurately represent the data which will be generated by @given.
8238 As a result, calling s.example() on an empty strategy s (such as noth‐
8240 ing()) now raises Unsatisfiable instead of the old NoExamples excep‐
8241 tion.
8242 4.26.4 - 2019-07-07
8244 This patch ensures that the Pandas extra will keep working when Python
8245 3.8 removes abstract base classes from the top-level python:collections
8246 namespace. This also fixes the relevant warning in Python 3.7, but
8247 there is no other difference in behaviour and you do not need to do
8248 anything.
8249 4.26.3 - 2019-07-05
8251 This release fixes issue #2027, by changing the way Hypothesis tries
8252 to generate distinct examples to be more efficient.
8253 This may result in slightly different data distribution, and should im‐
8255 prove generation performance in general, but should otherwise have min‐
8256 imal user impact.
8257 4.26.2 - 2019-07-04
8259 This release fixes issue #1864, where some simple tests would perform
8260 very slowly, because they would run many times with each subsequent run
8261 being progressively slower. They will now stop after a more reasonable
8262 number of runs without hitting this problem.
8263 Unless you are hitting exactly this issue, it is unlikely that this re‐
8265 lease will have any effect, but certain classes of custom generators
8266 that are currently very slow may become a bit faster, or start to trig‐
8267 ger health check failures.
8268 4.26.1 - 2019-07-04
8270 This release adds the strategy integer_array_indices(), which generates
8271 tuples of Numpy arrays that can be used for advanced indexing to select
8272 an array of a specified shape.
8273 4.26.0 - 2019-07-04
8275 This release significantly improves the performance of drawing unique
8276 collections whose elements are drawn from sampled_from() strategies.
8277 As a side effect, this detects an error condition that would previously
8279 have passed silently: When the min_size argument on a collection with
8280 distinct elements is greater than the number of elements being sampled,
8281 this will now raise an error.
8282 4.25.1 - 2019-07-03
8284 This release removes some defunct internal functionality that was only
8285 being used for testing. It should have no user visible impact.
8286 4.25.0 - 2019-07-03
8288 This release deprecates and disables the buffer_size setting, which
8289 should have been treated as a private implementation detail all along.
8290 We recommend simply deleting this settings argument.
8291 4.24.6 - 2019-06-26
8293 This patch makes datetimes() more efficient, as it now handles short
8294 months correctly by construction instead of filtering.
8295 4.24.5 - 2019-06-23
8297 This patch improves the development experience by simplifying the
8298 tracebacks you will see when e.g. you have used the .map(...) method of
8299 a strategy and the mapped function raises an exception.
8300 No new exceptions can be raised, nor existing exceptions change any‐
8302 thing but their traceback. We're simply using if-statements rather
8303 than exceptions for control flow in a certain part of the internals!
8304 4.24.4 - 2019-06-21
8306 This patch fixes issue #2014, where our compatibility layer broke with
8307 version 3.7.4 of the typing module backport on PyPI.
8308 This issue only affects Python 2. We remind users that Hypothesis,
8310 like many other packages, will drop Python 2 support on 2020-01-01 and
8311 already has several features that are only available on Python 3.
8312 4.24.3 - 2019-06-07
8314 This patch improves the implementation of an internal wrapper on Python
8315 3.8 beta1 (and will break on the alphas; but they're not meant to be
8316 stable). On other versions, there is no change at all.
8317 Thanks to Daniel Hahler for the patch, and Victor Stinner for his work
8319 on bpo-37032 that made it possible.
8320 4.24.2 - 2019-06-06
8322 Deprecation messages for functions in hypothesis.extra.django.models
8323 now explicitly name the deprecated function to make it easier to track
8324 down usages. Thanks to Kristian Glass for this contribution!
8325 4.24.1 - 2019-06-04
8327 This patch fixes issue #1999, a spurious bug raised when a @st.compos‐
8328 ite function was passed a keyword-only argument.
8329 Thanks to Jim Nicholls for his fantastic bug report.
8331 4.24.0 - 2019-05-29
8333 This release deprecates GenericStateMachine, in favor of RuleBased‐
8334 StateMachine. Rule-based stateful testing is significantly faster, es‐
8335 pecially during shrinking.
8336 If your use-case truly does not fit rule-based stateful testing, we
8338 recommend writing a custom test function which drives your specific
8339 control-flow using data().
8340 4.23.9 - 2019-05-28
8342 This patch fixes a very rare example database issue with file permis‐
8343 sions.
8344 When running a test that uses both @given and pytest.mark.parametrize,
8346 using pytest-xdist on Windows, with failing examples in the database,
8347 two attempts to read a file could overlap and we caught FileNotFound
8348 but not other OSErrors.
8349 4.23.8 - 2019-05-26
8351 This patch has a minor cleanup of the internal engine. There is no
8352 user-visible impact.
8353 4.23.7 - 2019-05-26
8355 This patch clarifies some error messages when the test function signa‐
8356 ture is incompatible with the arguments to @given, especially when the
8357 @settings() decorator is also used (issue #1978).
8358 4.23.6 - 2019-05-19
8360 This release adds the pyupgrade fixer to our code style, for consistent
8361 use of dict and set literals and comprehensions.
8362 4.23.5 - 2019-05-16
8364 This release slightly simplifies a small part of the core engine.
8365 There is no user-visible change.
8366 4.23.4 - 2019-05-09
8368 Fixes a minor formatting issue the docstring of from_type()
8369 4.23.3 - 2019-05-09
8371 Adds a recipe to the docstring of from_type() that describes a means
8372 for drawing values for "everything except" a specified type. This
8373 recipe is especially useful for writing tests that perform input-type
8374 validation.
8375 4.23.2 - 2019-05-08
8377 This patch uses autoflake to remove some pointless pass statements,
8378 which improves our workflow but has no user-visible impact.
8379 4.23.1 - 2019-05-08
8381 This patch fixes an OverflowError in from_type(xrange) on Python 2.
8382 It turns out that not only do the start and stop values have to fit in
8384 a C long, but so does stop - start. We now handle this even on 32bit
8385 platforms, but remind users that Python2 will not be supported after
8386 2019 without specific funding.
8387 4.23.0 - 2019-05-08
8389 This release implements the slices() strategy, to generate slices of a
8390 length-size sequence.
8391 Thanks to Daniel J. West for writing this patch at the PyCon 2019
8393 sprints!
8394 4.22.3 - 2019-05-07
8396 This patch exposes DataObject, solely to support more precise type
8397 hints. Objects of this type are provided by data(), and can be used to
8398 draw examples from strategies intermixed with your test code.
8399 4.22.2 - 2019-05-07
8401 This patch fixes the very rare issue #1798 in array_dtypes(), which
8402 caused an internal error in our tests.
8403 4.22.1 - 2019-05-07
8405 This patch fixes a rare bug in from_type(range).
8406 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
8408 Sprints.
8409 4.22.0 - 2019-05-07
8411 The unique_by argument to lists now accepts a tuple of callables such
8412 that every element of the generated list will be unique with respect to
8413 each callable in the tuple (issue #1916).
8414 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
8416 4.21.1 - 2019-05-06
8418 This patch cleans up the internals of one_of(). You may see a slight
8419 change to the distribution of examples from this strategy but there is
8420 no change to the public API.
8421 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
8423 sprints!
8424 4.21.0 - 2019-05-05
8426 The from_type() strategy now supports python:slice objects.
8427 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
8429 Mentored Sprints.
8430 4.20.0 - 2019-05-05
8432 This release improves the array_shapes() strategy, to choose an appro‐
8433 priate default for max_side based on the min_side, and max_dims based
8434 on the min_dims. An explicit error is raised for dimensions greater
8435 than 32, which are not supported by Numpy, as for other invalid combi‐
8436 nations of arguments.
8437 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
8439 tored Sprints.
8440 4.19.0 - 2019-05-05
8442 The from_type() strategy now supports python:range objects (or xrange
8443 on Python 2).
8444 Thanks to Katrina Durance for writing this feature at the PyCon 2019
8446 Mentored Sprints.
8447 4.18.3 - 2019-04-30
8449 This release fixes a very rare edge case in the test-case mutator,
8450 which could cause an internal error with certain unusual tests.
8451 4.18.2 - 2019-04-30
8453 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
8454 changed the representation of code objects to support positional-only
8455 arguments. Note however that Hypothesis does not (yet) support such
8456 functions as e.g. arguments to builds() or inputs to @given.
8457 Thanks to Paul Ganssle for identifying and fixing this bug.
8459 4.18.1 - 2019-04-29
8461 This patch improves the performance of unique collections such as
8462 sets() when the elements are drawn from a sampled_from() strategy (‐
8463 issue #1115).
8464 4.18.0 - 2019-04-24
8466 This release adds the functions() strategy, which can be used to imi‐
8467 tate your 'real' function for callbacks.
8468 4.17.2 - 2019-04-19
8470 This release refactors stateful rule selection to share the new machin‐
8471 ery with sampled_from() instead of using the original independent im‐
8472 plementation.
8473 4.17.1 - 2019-04-16
8475 This patch allows Hypothesis to try a few more examples after finding
8476 the first bug, in hopes of reporting multiple distinct bugs. The
8477 heuristics described in issue #847 ensure that we avoid wasting time on
8478 fruitless searches, while still surfacing each bug as soon as possible.
8479 4.17.0 - 2019-04-16
8481 This release adds the strategy broadcastable_shapes(), which generates
8482 array shapes that are broadcast-compatible with a provided shape.
8483 4.16.0 - 2019-04-12
8485 This release allows register_type_strategy() to be used with
8486 python:typing.NewType instances. This may be useful to e.g. provide
8487 only positive integers for from_type(UserId) with a UserId = New‐
8488 Type('UserId', int) type.
8489 Thanks to PJCampi for suggesting and writing the patch!
8491 4.15.0 - 2019-04-09
8493 This release supports passing a timedelta as the deadline setting, so
8494 you no longer have to remember that the number is in milliseconds (‐
8495 issue #1900).
8496 Thanks to Damon Francisco for this change!
8498 4.14.7 - 2019-04-09
8500 This patch makes the type annotations on hypothesis.extra.dateutil com‐
8501 patible with mypy 0.700.
8502 4.14.6 - 2019-04-07
8504 This release fixes a bug introduced in Hypothesis 4.14.3 that would
8505 sometimes cause sampled_from(...).filter(...) to encounter an internal
8506 assertion failure when there are three or fewer elements, and every el‐
8507 ement is rejected by the filter.
8508 4.14.5 - 2019-04-05
8510 This patch takes the previous efficiency improvements to sam‐
8511 pled_from(...).filter(...) strategies that reject most elements, and
8512 generalises them to also apply to sampled_from(...).filter(...).fil‐
8513 ter(...) and longer chains of filters.
8514 4.14.4 - 2019-04-05
8516 This release fixes a bug that prevented random_module() from correctly
8517 restoring the previous state of the random module.
8518 The random state was instead being restored to a temporary determinis‐
8520 tic state, which accidentally caused subsequent tests to see the same
8521 random values across multiple test runs.
8522 4.14.3 - 2019-04-03
8524 This patch adds an internal special case to make sampled_from(...).fil‐
8525 ter(...) much more efficient when the filter rejects most elements (‐
8526 issue #1885).
8527 4.14.2 - 2019-03-31
8529 This patch improves the error message if the function f in s.flatmap(f)
8530 does not return a strategy.
8531 Thanks to Kai Chen for this change!
8533 4.14.1 - 2019-03-30
8535 This release modifies how Hypothesis selects operations to run during
8536 shrinking, by causing it to deprioritise previously useless classes of
8537 shrink until others have reached a fixed point.
8538 This avoids certain pathological cases where the shrinker gets very
8540 close to finishing and then takes a very long time to finish the last
8541 small changes because it tries many useless shrinks for each useful one
8542 towards the end. It also should cause a more modest improvement (prob‐
8543 ably no more than about 30%) in shrinking performance for most tests.
8544 4.14.0 - 2019-03-19
8546 This release blocks installation of Hypothesis on Python 3.4, which
8547 reached its end of life date on 2019-03-18.
8548 This should not be of interest to anyone but downstream maintainers -
8550 if you are affected, migrate to a secure version of Python as soon as
8551 possible or at least seek commercial support.
8552 4.13.0 - 2019-03-19
8554 This release makes it an explicit error to call floats(min_value=inf,
8555 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
8556 are no possible values that can be generated (issue #1859).
8557 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
8559 -0. and we could thus generate either if comparing by value, violating
8560 the sequence ordering of floats is a special case we don't want or
8561 need.
8562 4.12.1 - 2019-03-18
8564 This release should significantly reduce the amount of memory that Hy‐
8565 pothesis uses for representing large test cases, by storing information
8566 in a more compact representation and only unpacking it lazily when it
8567 is first needed.
8568 4.12.0 - 2019-03-18
8570 This update adds the report_multiple_bugs setting, which you can use to
8571 disable multi-bug reporting and only raise whichever bug had the small‐
8572 est minimal example. This is occasionally useful when using a debugger
8573 or tools that annotate tracebacks via introspection.
8574 4.11.7 - 2019-03-18
8576 This change makes a tiny improvement to the core engine's bookkeeping.
8577 There is no user-visible change.
8578 4.11.6 - 2019-03-15
8580 This release changes some of Hypothesis's internal shrinking behaviour
8581 in order to reduce memory usage and hopefully improve performance.
8582 4.11.5 - 2019-03-13
8584 This release adds a micro-optimisation to how Hypothesis handles debug
8585 reporting internally. Hard to shrink test may see a slight performance
8586 improvement, but in most common scenarios it is unlikely to be notice‐
8587 able.
8588 4.11.4 - 2019-03-13
8590 This release removes some redundant code that was no longer needed but
8591 was still running a significant amount of computation and allocation on
8592 the hot path. This should result in a modest speed improvement for
8593 most tests, especially those with large test cases.
8594 4.11.3 - 2019-03-13
8596 This release adds a micro-optimisation to how Hypothesis caches test
8597 cases. This will cause a small improvement in speed and memory usage
8598 for large test cases, but in most common scenarios it is unlikely to be
8599 noticeable.
8600 4.11.2 - 2019-03-13
8602 This release removes some internal code that populates a field that is
8603 no longer used anywhere. This should result in some modest performance
8604 and speed improvements and no other user visible effects.
8605 4.11.1 - 2019-03-13
8607 This is a formatting-only patch, enabled by a new version of isort.
8608 4.11.0 - 2019-03-12
8610 This release deprecates sampled_from() with empty sequences. This re‐
8611 turns nothing(), which gives a clear error if used directly... but sim‐
8612 ply vanishes if combined with another strategy.
8613 Tests that silently generate less than expected are a serious problem
8615 for anyone relying on them to find bugs, and we think reliability more
8616 important than convenience in this case.
8617 4.10.0 - 2019-03-11
8619 This release improves Hypothesis's to detect flaky tests, by noticing
8620 when the behaviour of the test changes between runs. In particular
8621 this will notice many new cases where data generation depends on exter‐
8622 nal state (e.g. external sources of randomness) and flag those as flaky
8623 sooner and more reliably.
8624 The basis of this feature is a considerable reengineering of how Hy‐
8626 pothesis stores its history of test cases, so on top of this its memory
8627 usage should be considerably reduced.
8628 4.9.0 - 2019-03-09
8630 This release adds the strategy valid_tuple_axes(), which generates tu‐
8631 ples of axis-indices that can be passed to the axis argument in NumPy's
8632 sequential functions (e.g. numpy:numpy.sum()).
8633 Thanks to Ryan Soklaski for this strategy.
8635 4.8.0 - 2019-03-06
8637 This release significantly tightens validation in hypothesis.settings.
8638 max_examples, buffer_size, and stateful_step_count must be positive in‐
8639 tegers; deadline must be a positive number or None; and derandomize
8640 must be either True or False.
8641 As usual, this replaces existing errors with a more helpful error and
8643 starts new validation checks as deprecation warnings.
8644 4.7.19 - 2019-03-04
8646 This release makes some micro-optimisations to certain calculations
8647 performed in the shrinker. These should particularly speed up large
8648 test cases where the shrinker makes many small changes. It will also
8649 reduce the amount allocated, but most of this is garbage that would
8650 have been immediately thrown away, so you probably won't see much ef‐
8651 fect specifically from that.
8652 4.7.18 - 2019-03-03
8654 This patch removes some overhead from arrays() with a constant shape
8655 and dtype. The resulting performance improvement is modest, but worth‐
8656 while for small arrays.
8657 4.7.17 - 2019-03-01
8659 This release makes some micro-optimisations within Hypothesis's inter‐
8660 nal representation of test cases. This should cause heavily nested
8661 test cases to allocate less during generation and shrinking, which
8662 should speed things up slightly.
8663 4.7.16 - 2019-02-28
8665 This changes the order in which Hypothesis runs certain operations dur‐
8666 ing shrinking. This should significantly decrease memory usage and
8667 speed up shrinking of large examples.
8668 4.7.15 - 2019-02-28
8670 This release allows Hypothesis to calculate a number of attributes of
8671 generated test cases lazily. This should significantly reduce memory
8672 usage and modestly improve performance, especially for large test
8673 cases.
8674 4.7.14 - 2019-02-28
8676 This release reduces the number of operations the shrinker will try
8677 when reordering parts of a test case. This should in some circum‐
8678 stances significantly speed up shrinking. It may result in different
8679 final test cases, and if so usually slightly worse ones, but it should
8680 not generally have much impact on the end result as the operations re‐
8681 moved were typically useless.
8682 4.7.13 - 2019-02-27
8684 This release changes how Hypothesis reorders examples within a test
8685 case during shrinking. This should make shrinking considerably faster.
8686 4.7.12 - 2019-02-27
8688 This release slightly improves the shrinker's ability to replace parts
8689 of a test case with their minimal version, by allowing it to do so in
8690 bulk rather than one at a time. Where this is effective, shrinker per‐
8691 formance should be modestly improved.
8692 4.7.11 - 2019-02-25
8694 This release makes some micro-optimisations to common operations per‐
8695 formed during shrinking. Shrinking should now be slightly faster, es‐
8696 pecially for large examples with relatively fast test functions.
8697 4.7.10 - 2019-02-25
8699 This release is a purely internal refactoring of Hypothesis's API for
8700 representing test cases. There should be no user visible effect.
8701 4.7.9 - 2019-02-24
8703 This release changes certain shrink passes to make them more efficient
8704 when they aren't making progress.
8705 4.7.8 - 2019-02-23
8707 This patch removes some unused code, which makes the internals a bit
8708 easier to understand. There is no user-visible impact.
8709 4.7.7 - 2019-02-23
8711 This release reduces the number of operations the shrinker will try
8712 when reordering parts of a test case. This should in some circum‐
8713 stances significantly speed up shrinking. It may result in different
8714 final test cases, and if so usually slightly worse ones, but it should
8715 not generally have much impact on the end result as the operations re‐
8716 moved were typically useless.
8717 4.7.6 - 2019-02-23
8719 This patch removes some unused code from the shrinker. There is no
8720 user-visible change.
8721 4.7.5 - 2019-02-23
8723 This release changes certain shrink passes to make them adaptive - that
8724 is, in cases where they are successfully making progress they may now
8725 do so significantly faster.
8726 4.7.4 - 2019-02-22
8728 This is a docs-only patch, noting that because the lark-parser is under
8729 active development at version 0.x, hypothesis[lark] APIs may break in
8730 minor releases if necessary to keep up with the upstream package.
8731 4.7.3 - 2019-02-22
8733 This changes Hypothesis to no longer import various test frameworks by
8734 default (if they are installed). which will speed up the initial im‐
8735 port hypothesis call.
8736 4.7.2 - 2019-02-22
8738 This release changes Hypothesis's internal representation of a test
8739 case to calculate some expensive structural information on demand
8740 rather than eagerly. This should reduce memory usage a fair bit, and
8741 may make generation somewhat faster.
8742 4.7.1 - 2019-02-21
8744 This release refactors the internal representation of previously run
8745 test cases. The main thing you should see as a result is that Hypothe‐
8746 sis becomes somewhat less memory hungry.
8747 4.7.0 - 2019-02-21
8749 This patch allows array_shapes() to generate shapes with side-length or
8750 even dimension zero, though the minimum still defaults to one. These
8751 shapes are rare and have some odd behavior, but are particularly impor‐
8752 tant to test for just that reason!
8753 In a related bigfix, arrays() now supports generating zero-dimensional
8755 arrays with dtype=object and a strategy for iterable elements. Previ‐
8756 ously, the array element would incorrectly be set to the first item in
8757 the generated iterable.
8758 Thanks to Ryan Turner for continuing to improve our Numpy support.
8760 4.6.1 - 2019-02-19
8762 This release is a trivial micro-optimisation inside Hypothesis which
8763 should result in it using significantly less memory.
8764 4.6.0 - 2019-02-18
8766 This release changes some inconsistent behavior of arrays() from the
8767 Numpy extra when asked for an array of shape=(). arrays() will now al‐
8768 ways return a Numpy ndarray, and the array will always be of the re‐
8769 quested dtype.
8770 Thanks to Ryan Turner for this change.
8772 4.5.12 - 2019-02-18
8774 This release fixes a minor typo in an internal comment. There is no
8775 user-visible change.
8776 4.5.11 - 2019-02-15
8778 This release fixes issue #1813, a bug introduced in 3.59.1, which
8779 caused random_module() to no longer affect the body of the test: Al‐
8780 though Hypothesis would claim to be seeding the random module in fact
8781 tests would always run with a seed of zero.
8782 4.5.10 - 2019-02-14
8784 This patch fixes an off-by-one error in the maximum length of emails().
8785 Thanks to Krzysztof Jurewicz for pull request #1812.
8786 4.5.9 - 2019-02-14
8788 This patch removes some unused code from the shrinker. There is no
8789 user-visible change.
8790 4.5.8 - 2019-02-12
8792 This release fixes an internal IndexError in Hypothesis that could
8793 sometimes be triggered during shrinking.
8794 4.5.7 - 2019-02-11
8796 This release modifies the shrinker to interleave different types of re‐
8797 duction operations, e.g. switching between deleting data and lowering
8798 scalar values rather than trying entirely deletions then entirely low‐
8799 ering.
8800 This may slow things down somewhat in the typical case, but has the ma‐
8802 jor advantage that many previously difficult to shrink examples should
8803 become much faster, because the shrinker will no longer tend to stall
8804 when trying some ineffective changes to the shrink target but will in‐
8805 stead interleave it with other more effective operations.
8806 4.5.6 - 2019-02-11
8808 This release makes a number of internal changes to the implementation
8809 of hypothesis.extra.lark.from_lark(). These are primarily intended as
8810 a refactoring, but you may see some minor improvements to performance
8811 when generating large strings, and possibly to shrink quality.
8812 4.5.5 - 2019-02-10
8814 This patch prints an explanatory note when issue #1798 is triggered,
8815 because the error message from Numpy is too terse to locate the prob‐
8816 lem.
8817 4.5.4 - 2019-02-08
8819 In Python 2, long integers are not allowed in the shape argument to ar‐
8820 rays(). Thanks to Ryan Turner for fixing this.
8821 4.5.3 - 2019-02-08
8823 This release makes a small internal refactoring to clarify how Hypothe‐
8824 sis instructs tests to stop running when appropriate. There is no
8825 user-visible change.
8826 4.5.2 - 2019-02-06
8828 This release standardises all of the shrinker's internal operations on
8829 running in a random order.
8830 The main effect you will see from this that it should now be much less
8832 common for the shrinker to stall for a long time before making further
8833 progress. In some cases this will correspond to shrinking more slowly,
8834 but on average it should result in faster shrinking.
8835 4.5.1 - 2019-02-05
8837 This patch updates some docstrings, but has no runtime changes.
8838 4.5.0 - 2019-02-03
8840 This release adds exclude_min and exclude_max arguments to floats(), so
8841 that you can easily generate values from open or half-open intervals (‐
8842 issue #1622).
8843 4.4.6 - 2019-02-03
8845 This patch fixes a bug where from_regex() could throw an internal error
8846 if the python:re.IGNORECASE flag was used (issue #1786).
8847 4.4.5 - 2019-02-02
8849 This release removes two shrink passes that Hypothesis runs late in the
8850 process. These were very expensive when the test function was slow and
8851 often didn't do anything useful.
8852 Shrinking should get faster for most failing tests. If you see any re‐
8854 gression in example quality as a result of this release, please let us
8855 know.
8856 4.4.4 - 2019-02-02
8858 This release modifies the way that Hypothesis deletes data during
8859 shrinking. It will primarily be noticeable for very large examples,
8860 which should now shrink faster.
8861 The shrinker is now also able to perform some deletions that it could
8863 not previously, but this is unlikely to be very noticeable.
8864 4.4.3 - 2019-01-25
8866 This release fixes an open file leak that used to cause ResourceWarn‐
8867 ings.
8868 4.4.2 - 2019-01-24
8870 This release changes Hypothesis's internal approach to caching the re‐
8871 sults of executing test cases. The result should be that it is now
8872 significantly less memory hungry, especially when shrinking large test
8873 cases.
8874 Some tests may get slower or faster depending on whether the new or old
8876 caching strategy was well suited to them, but any change in speed in
8877 either direction should be minor.
8878 4.4.1 - 2019-01-24
8880 This patch tightens up some of our internal heuristics to deal with
8881 shrinking floating point numbers, which will now run in fewer circum‐
8882 stances.
8883 You are fairly unlikely to see much difference from this, but if you do
8885 you are likely to see shrinking become slightly faster and/or producing
8886 slightly worse results.
8887 4.4.0 - 2019-01-24
8889 This release adds the from_form() function, which allows automatic
8890 testing against Django forms. (issue #35)
8891 Thanks to Paul Stiverson for this feature, which resolves our oldest
8893 open issue!
8894 4.3.0 - 2019-01-24
8896 This release deprecates HealthCheck.hung_test and disables the associ‐
8897 ated runtime check for tests that ran for more than five minutes. Such
8898 a check is redundant now that we enforce the deadline and max_examples
8899 setting, which can be adjusted independently.
8900 4.2.0 - 2019-01-23
8902 This release adds a new module, hypothesis.extra.lark, which you can
8903 use to generate strings matching a context-free grammar.
8904 In this initial version, only lark-parser EBNF grammars are supported,
8906 by the new hypothesis.extra.lark.from_lark() function.
8907 4.1.2 - 2019-01-23
8909 This patch fixes a very rare overflow bug (issue #1748) which could
8910 raise an InvalidArgument error in complex_numbers() even though the ar‐
8911 guments were valid.
8912 4.1.1 - 2019-01-23
8914 This release makes some improvements to internal code organisation and
8915 documentation and has no impact on behaviour.
8916 4.1.0 - 2019-01-22
8918 This release adds register_random(), which registers random.Random in‐
8919 stances or compatible objects to be seeded and reset by Hypothesis to
8920 ensure that test cases are deterministic.
8921 We still recommend explicitly passing a random.Random instance from
8923 randoms() if possible, but registering a framework-global state for Hy‐
8924 pothesis to manage is better than flaky tests!
8925 4.0.2 - 2019-01-22
8927 This patch fixes issue #1387, where bounded integers() with a very
8928 large range would almost always generate very large numbers. Now, we
8929 usually use the same tuned distribution as unbounded integers().
8930 4.0.1 - 2019-01-16
8932 This release randomizes the order in which the shrinker tries some of
8933 its initial normalization operations. You are unlikely to see much
8934 difference as a result unless your generated examples are very large.
8935 In this case you may see some performance improvements in shrinking.
8936 4.0.0 - 2019-01-14
8938 Welcome to the next major version of Hypothesis!
8939 There are no new features here, as we release those in minor versions.
8941 Instead, 4.0 is a chance for us to remove deprecated features (many al‐
8942 ready converted into no-ops), and turn a variety of warnings into er‐
8943 rors.
8944 If you were running on the last version of Hypothesis 3.x without any
8946 Hypothesis deprecation warnings (or using private APIs), this will be a
8947 very boring upgrade. In fact, nothing will change for you at all. Per
8948 our deprecation policy, warnings added in the last six months (after
8949 2018-07-05) have not been converted to errors.
8950 Removals
8952 • hypothesis.extra.datetime has been removed, replaced by the core date
8953 and time strategies.
8954 • hypothesis.extra.fakefactory has been removed, replaced by general
8956 expansion of Hypothesis' strategies and the third-party ecosystem.
8957 • The SQLite example database backend has been removed.
8959 Settings
8961 • The deadline is now enforced by default, rather than just emitting a
8962 warning when the default (200 milliseconds per test case) deadline is
8963 exceeded.
8964 • The database_file setting has been removed; use database.
8966 • The perform_health_check setting has been removed; use sup‐
8968 press_health_check.
8969 • The max_shrinks setting has been removed; use phases to disable
8971 shrinking.
8972 • The min_satisfying_examples, max_iterations, strict, timeout, and
8974 use_coverage settings have been removed without user-configurable re‐
8975 placements.
8976 Strategies
8978 • The elements argument is now required for collection strategies.
8979 • The average_size argument was a no-op and has been removed.
8981 • Date and time strategies now only accept min_value and max_value for
8983 bounds.
8984 • builds() now requires that the thing to build is passed as the first
8986 positional argument.
8987 • Alphabet validation for text() raises errors, not warnings, as does
8989 category validation for characters().
8990 • The choices() strategy has been removed. Instead, you can use data()
8992 with sampled_from(), so choice(elements) becomes data.draw(sam‐
8993 pled_from(elements)).
8994 • The streaming() strategy has been removed. Instead, you can use
8996 data() and replace iterating over the stream with data.draw() calls.
8997 • sampled_from() and permutations() raise errors instead of warnings if
8999 passed a collection that is not a sequence.
9000 Miscellaneous
9002 • Applying @given to a test function multiple times was really ineffi‐
9003 cient, and now it's also an error.
9004 • Using the .example() method of a strategy (intended for interactive
9006 exploration) within another strategy or a test function always weak‐
9007 ened data generation and broke shrinking, and now it's an error too.
9008 • The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
9010 ported, as the database_file setting has been removed.
9011 • The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
9013 ported. You can use the --hypothesis-verbosity pytest argument in‐
9014 stead, or write your own setup code using the settings profile system
9015 to replace it.
9016 • Using @seed or derandomize=True now forces database=None to ensure
9018 results are in fact reproducible. If database is not None, doing so
9019 also emits a HypothesisWarning.
9020 • Unused exception types have been removed from hypothesis.errors;
9022 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
9023 ple, Timeout, and WrongFormat.
9024 Hypothesis 3.x
9026 3.88.3 - 2019-01-11
9027 This changes the order that the shrinker tries certain operations in
9028 its "emergency" phase which runs late in the process. The new order
9029 should be better at avoiding long stalls where the shrinker is failing
9030 to make progress, which may be helpful if you have difficult to shrink
9031 test cases. However this will not be noticeable in the vast majority
9032 of use cases.
9033 3.88.2 - 2019-01-11
9035 This is a pure refactoring release that extracts some logic from the
9036 core Hypothesis engine into its own class and file. It should have no
9037 user visible impact.
9038 3.88.1 - 2019-01-11
9040 This patch fixes some markup in our documentation.
9041 3.88.0 - 2019-01-10
9043 Introduces hypothesis.stateful.multiple(), which allows rules in rule
9044 based state machines to send multiple results at once to their target
9045 Bundle, or none at all.
9046 3.87.0 - 2019-01-10
9048 This release contains a massive cleanup of the Hypothesis for Django
9049 extra:
9050 • hypothesis.extra.django.models.models() is deprecated in favor of hy‐
9052 pothesis.extra.django.from_model().
9053 • hypothesis.extra.django.models.add_default_field_mapping() is depre‐
9055 cated in favor of hypothesis.extra.django.register_field_strategy().
9056 • from_model() does not infer a strategy for nullable fields or fields
9058 with a default unless passed infer, like builds(). models.models()
9059 would usually but not always infer, and a special default_value
9060 marker object was required to disable inference.
9061 3.86.9 - 2019-01-09
9063 This release improves some internal logic about when a test case in Hy‐
9064 pothesis's internal representation could lead to a valid test case. In
9065 some circumstances this can lead to a significant speed up during
9066 shrinking. It may have some minor negative impact on the quality of
9067 the final result due to certain shrink passes now having access to less
9068 information about test cases in some circumstances, but this should
9069 rarely matter.
9070 3.86.8 - 2019-01-09
9072 This release has no user visible changes but updates our URLs to use
9073 HTTPS.
9074 3.86.7 - 2019-01-08
9076 Hypothesis can now automatically generate values for Django models with
9077 a URLfield, thanks to a new provisional strategy for URLs (issue
9078 #1388).
9079 3.86.6 - 2019-01-07
9081 This release is a pure refactoring that extracts some internal code
9082 into its own file. It should have no user visible effect.
9083 3.86.5 - 2019-01-06
9085 This is a docs-only patch, which fixes some typos and removes a few hy‐
9086 perlinks for deprecated features.
9087 3.86.4 - 2019-01-04
9089 This release changes the order in which the shrinker tries to delete
9090 data. For large and slow tests this may significantly improve the per‐
9091 formance of shrinking.
9092 3.86.3 - 2019-01-04
9094 This release fixes a bug where certain places Hypothesis internal er‐
9095 rors could be raised during shrinking when a user exception occurred
9096 that suppressed an exception Hypothesis uses internally in its genera‐
9097 tion.
9098 The two known ways to trigger this problem were:
9100 • Errors raised in stateful tests' teardown function.
9102 • Errors raised in finally blocks that wrapped a call to data.draw.
9104 These cases will now be handled correctly.
9106 3.86.2 - 2019-01-04
9108 This patch is a docs-only change to fix a broken hyperlink.
9109 3.86.1 - 2019-01-04
9111 This patch fixes issue #1732, where integers() would always return long
9112 values on Python 2.
9113 3.86.0 - 2019-01-03
9115 This release ensures that infinite numbers are never generated by
9116 floats() with allow_infinity=False, which could previously happen in
9117 some cases where one bound was also provided.
9118 The trivially inconsistent min_value=inf, allow_infinity=False now
9120 raises an InvalidArgumentError, as does the inverse with max_value.
9121 You can still use just(inf) to generate inf without violating other
9122 constraints.
9123 3.85.3 - 2019-01-02
9125 Happy new year everyone! This release has no user visible changes but
9126 updates our copyright headers to include 2019.
9127 3.85.2 - 2018-12-31
9129 This release makes a small change to the way the shrinker works. You
9130 may see some improvements to speed of shrinking on especially large and
9131 hard to shrink examples, but most users are unlikely to see much dif‐
9132 ference.
9133 3.85.1 - 2018-12-30
9135 This patch fixes issue #1700, where a line that contained a Unicode
9136 character before a lambda definition would cause an internal exception.
9137 3.85.0 - 2018-12-29
9139 Introduces the hypothesis.stateful.consumes() function. When defining a
9140 rule in stateful testing, it can be used to mark bundles from which
9141 values should be consumed, i. e. removed after use in the rule. This
9142 has been proposed in issue #136.
9143 Thanks to Jochen Müller for this long-awaited feature.
9145 3.84.6 - 2018-12-28
9147 This patch makes a small internal change to fix an issue in Hypothe‐
9148 sis's own coverage tests (issue #1718).
9149 There is no user-visible change.
9151 3.84.5 - 2018-12-21
9153 This patch refactors the hypothesis.strategies module, so that private
9154 names should no longer appear in tab-completion lists. We previously
9155 relied on __all__ for this, but not all editors respect it.
9156 3.84.4 - 2018-12-21
9158 This is a follow-up patch to ensure that the deprecation date is auto‐
9159 matically recorded for any new deprecations. There is no user-visible
9160 effect.
9161 3.84.3 - 2018-12-20
9163 This patch updates the Hypothesis pytest plugin to avoid a recently
9164 deprecated hook interface. There is no user-visible change.
9165 3.84.2 - 2018-12-19
9167 This patch fixes the internals for integers() with one bound. Values
9168 from this strategy now always shrink towards zero instead of towards
9169 the bound, and should shrink much more efficiently too. On Python 2,
9170 providing a bound incorrectly excluded long integers, which can now be
9171 generated.
9172 3.84.1 - 2018-12-18
9174 This patch adds information about when features were deprecated, but
9175 this is only recorded internally and has no user-visible effect.
9176 3.84.0 - 2018-12-18
9178 This release changes the stateful testing backend from find() to use
9179 @given (issue #1300). This doesn't change how you create stateful
9180 tests, but does make them run more like other Hypothesis tests.
9181 @reproduce_failure and @seed now work for stateful tests.
9183 Stateful tests now respect the deadline and suppress_health_check set‐
9185 tings, though they are disabled by default. You can enable them by us‐
9186 ing @settings(...) as a class decorator with whatever arguments you
9187 prefer.
9188 3.83.2 - 2018-12-17
9190 Hypothesis has adopted Black as our code formatter (issue #1686).
9191 There are no functional changes to the source, but it's prettier!
9192 3.83.1 - 2018-12-13
9194 This patch increases the variety of examples generated by from_type().
9195 3.83.0 - 2018-12-12
9197 Our pytest plugin now warns you when strategy functions have been col‐
9198 lected as tests, which may happen when e.g. using the @composite deco‐
9199 rator when you should be using @given(st.data()) for inline draws.
9200 Such functions always pass when treated as tests, because the lazy cre‐
9201 ation of strategies mean that the function body is never actually exe‐
9202 cuted!
9203 3.82.6 - 2018-12-11
9205 Hypothesis can now show statistics when running under pytest-xdist.
9206 Previously, statistics were only reported when all tests were run in a
9207 single process (issue #700).
9208 3.82.5 - 2018-12-08
9210 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
9211 to integers() could cause errors on Python 3 due to internal rounding.
9212 3.82.4 - 2018-12-08
9214 Hypothesis now seeds and resets the global state of np.random for each
9215 test case, to ensure that tests are reproducible.
9216 This matches and complements the existing handling of the python:random
9218 module - Numpy simply maintains an independent PRNG for performance
9219 reasons.
9220 3.82.3 - 2018-12-08
9222 This is a no-op release to add the new Framework :: Hypothesis trove
9223 classifier to hypothesis on PyPI.
9224 You can use it as a filter to find Hypothesis-related packages such as
9226 extensions as they add the tag over the coming weeks, or simply visit
9227 our curated list.
9228 3.82.2 - 2018-12-08
9230 The Hypothesis for Pandas extension is now listed in setup.py, so you
9231 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
9232 tion.
9233 3.82.1 - 2018-10-29
9235 This patch fixes from_type() on Python 2 for classes where cls.__init__
9236 is object.__init__. Thanks to ccxcz for reporting issue #1656.
9237 3.82.0 - 2018-10-29
9239 The alphabet argument for text() now uses its default value of charac‐
9240 ters(blacklist_categories=('Cs',)) directly, instead of hiding that be‐
9241 hind alphabet=None and replacing it within the function. Passing None
9242 is therefore deprecated.
9243 3.81.0 - 2018-10-27
9245 GenericStateMachine and RuleBasedStateMachine now raise an explicit er‐
9246 ror when instances of settings are assigned to the classes' settings
9247 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
9248 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
9249 tor to handle this automatically.
9250 3.80.0 - 2018-10-25
9252 Since version 3.68.0, arrays() checks that values drawn from the ele‐
9253 ments and fill strategies can be safely cast to the dtype of the array,
9254 and emits a warning otherwise.
9255 This release expands the checks to cover overflow for finite complex64
9257 elements and string truncation caused by too-long elements or trailing
9258 null characters (issue #1591).
9259 3.79.4 - 2018-10-25
9261 Tests using @given now shrink errors raised from pytest helper func‐
9262 tions, instead of reporting the first example found.
9263 This was previously fixed in version 3.56.0, but only for stateful
9265 testing.
9266 3.79.3 - 2018-10-23
9268 Traceback elision is now disabled on Python 2, to avoid an import-time
9269 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
9270 3.79.2: issue #1648).
9271 We encourage all users to upgrade to Python 3 before the end of 2019.
9273 3.79.2 - 2018-10-23
9275 This patch shortens tracebacks from Hypothesis, so you can see exactly
9276 happened in your code without having to skip over irrelevant details
9277 about our internals (issue #848).
9278 In the example test (see pull request #1582), this reduces tracebacks
9280 from nine frames to just three - and for a test with multiple errors,
9281 from seven frames per error to just one!
9282 If you do want to see the internal details, you can disable frame eli‐
9284 sion by setting verbosity to debug.
9285 3.79.1 - 2018-10-22
9287 The abstract number classes Number, Complex, Real, Rational, and Inte‐
9288 gral are now supported by the from_type() strategy. Previously, you
9289 would have to use register_type_strategy() before they could be re‐
9290 solved (issue #1636)
9291 3.79.0 - 2018-10-18
9293 This release adds a CLI flag for verbosity --hypothesis-verbosity to
9294 the Hypothesis pytest plugin, applied after loading the profile speci‐
9295 fied by --hypothesis-profile. Valid options are the names of verbosity
9296 settings, quiet, normal, verbose or debug.
9297 Thanks to Bex Dunn for writing this patch at the PyCon Australia
9299 sprints!
9300 The pytest header now correctly reports the current profile if --hy‐
9302 pothesis-profile has been used.
9303 Thanks to Mathieu Paturel for the contribution at the Canberra Python
9305 Hacktoberfest.
9306 3.78.0 - 2018-10-16
9308 This release has deprecated the generation of integers, floats and
9309 fractions when the conversion of the upper and/ or lower bound is not
9310 100% exact, e.g. when an integer gets passed a bound that is not a
9311 whole number. (issue #1625)
9312 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
9314 3.77.0 - 2018-10-16
9316 This minor release adds functionality to settings allowing it to be
9317 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
9318 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
9320 3.76.1 - 2018-10-16
9322 This patch fixes some warnings added by recent releases of pydocstyle
9323 and mypy.
9324 3.76.0 - 2018-10-11
9326 This release deprecates using floats for min_size and max_size.
9327 The type hint for average_size arguments has been changed from Op‐
9329 tional[int] to None, because non-None values are always ignored and
9330 deprecated.
9331 3.75.4 - 2018-10-10
9333 This patch adds more internal comments to the core engine's se‐
9334 quence-length shrinker. There should be no user-visible change.
9335 3.75.3 - 2018-10-09
9337 This patch adds additional comments to some of the core engine's inter‐
9338 nal data structures. There is no user-visible change.
9339 3.75.2 - 2018-10-09
9341 This patch avoids caching a trivial case, fixing issue #493.
9342 3.75.1 - 2018-10-09
9344 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
9345 for this contribution!
9346 3.75.0 - 2018-10-08
9348 This release deprecates the use of min_size=None, setting the default
9349 min_size to 0 (issue #1618).
9350 3.74.3 - 2018-10-08
9352 This patch makes some small internal changes to comply with a new lint
9353 setting in the build. There should be no user-visible change.
9354 3.74.2 - 2018-10-03
9356 This patch fixes issue #1153, where time spent reifying a strategy was
9357 also counted in the time spent generating the first example. Strate‐
9358 gies are now fully constructed and validated before the timer is
9359 started.
9360 3.74.1 - 2018-10-03
9362 This patch fixes some broken formatting and links in the documentation.
9363 3.74.0 - 2018-10-01
9365 This release checks that the value of the print_blob setting is a
9366 PrintSettings instance.
9367 Being able to specify a boolean value was not intended, and is now dep‐
9369 recated. In addition, specifying True will now cause the blob to al‐
9370 ways be printed, instead of causing it to be suppressed.
9371 Specifying any value that is not a PrintSettings or a boolean is now an
9373 error.
9374 3.73.5 - 2018-10-01
9376 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
9377 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
9378 eter names min_value and max_value instead of the deprecated names
9379 3.73.4 - 2018-09-30
9381 This patch ensures that Hypothesis deprecation warnings display the
9382 code that emitted them when you're not running in -Werror mode (issue
9383 #652).
9384 3.73.3 - 2018-09-27
9386 Tracebacks involving @composite are now slightly shorter due to some
9387 internal refactoring.
9388 3.73.2 - 2018-09-26
9390 This patch fixes errors in the internal comments for one of the
9391 shrinker passes. There is no user-visible change.
9392 3.73.1 - 2018-09-25
9394 This patch substantially improves the distribution of data generated
9395 with recursive(), and fixes a rare internal error (issue #1502).
9396 3.73.0 - 2018-09-24
9398 This release adds the fulfill() function, which is designed for testing
9399 code that uses dpcontracts 0.4 or later for input validation. This
9400 provides some syntactic sugar around use of assume(), to automatically
9401 filter out and retry calls that cause a precondition check to fail (‐
9402 issue #1474).
9403 3.72.0 - 2018-09-24
9405 This release makes setting attributes of the hypothesis.settings class
9406 an explicit error. This has never had any effect, but could mislead
9407 users who confused it with the current settings instance hypothe‐
9408 sis.settings.default (which is also immutable). You can change the
9409 global settings with settings profiles.
9410 3.71.11 - 2018-09-24
9412 This patch factors out some common code in the shrinker for iterating
9413 over pairs of data blocks. There should be no user-visible change.
9414 3.71.10 - 2018-09-18
9416 This patch allows from_type() to handle the empty tuple type, typ‐
9417 ing.Tuple[()].
9418 3.71.9 - 2018-09-17
9420 This patch updates some internal comments for mypy. There is no
9421 user-visible effect, even for Mypy users.
9422 3.71.8 - 2018-09-17
9424 This patch fixes a rare bug that would cause a particular shrinker pass
9425 to raise an IndexError, if a shrink improvement changed the underlying
9426 data in an unexpected way.
9427 3.71.7 - 2018-09-17
9429 This release fixes the broken cross-references in our docs, and adds a
9430 CI check so we don't add new ones.
9431 3.71.6 - 2018-09-16
9433 This patch fixes two bugs (issue #944 and issue #1521), where messages
9434 about @seed did not check the current verbosity setting, and the wrong
9435 settings were active while executing explicit examples.
9436 3.71.5 - 2018-09-15
9438 This patch fixes a DeprecationWarning added in Python 3.8 (issue
9439 #1576).
9440 Thanks to tirkarthi for this contribution!
9442 3.71.4 - 2018-09-14
9444 This is a no-op release, which implements automatic DOI minting and
9445 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
9446 zon 2020 programme for providing this service!
9447 Check our CITATION.cff file for details, or head right on over to
9449 doi.org/10.5281/zenodo.1412597
9450 3.71.3 - 2018-09-10
9452 This release adds the test name to some deprecation warnings, for eas‐
9453 ier debugging.
9454 Thanks to Sanyam Khurana for the patch!
9456 3.71.2 - 2018-09-10
9458 This release makes Hypothesis's memory usage substantially smaller for
9459 tests with many examples, by bounding the number of past examples it
9460 keeps around.
9461 You will not see much difference unless you are running tests with
9463 max_examples set to well over 1000, but if you do have such tests then
9464 you should see memory usage mostly plateau where previously it would
9465 have grown linearly with time.
9466 3.71.1 - 2018-09-09
9468 This patch adds internal comments to some tree traversals in the core
9469 engine. There is no user-visible change.
9470 3.71.0 - 2018-09-08
9472 This release deprecates the coverage-guided testing functionality, as
9473 it has proven brittle and does not really pull its weight.
9474 We intend to replace it with something more useful in the future, but
9476 the feature in its current form does not seem to be worth the cost of
9477 using, and whatever replaces it will likely look very different.
9478 3.70.4 - 2018-09-08
9480 This patch changes the behaviour of reproduce_failure() so that blobs
9481 are only printed in quiet mode when the print_blob setting is set to
9482 ALWAYS.
9483 Thanks to Cameron McGill for writing this patch at the PyCon Australia
9485 sprints!
9486 3.70.3 - 2018-09-03
9488 This patch removes some unnecessary code from the internals. There is
9489 no user-visible change.
9490 3.70.2 - 2018-09-03
9492 This patch fixes an internal bug where a corrupted argument to @repro‐
9493 duce_failure could raise the wrong type of error. Thanks again to
9494 Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
9495 tently provides excellent bug reports including issue #1558.
9496 3.70.1 - 2018-09-03
9498 This patch updates hypothesis to report its version and settings when
9499 run with pytest. (issue #1223).
9500 Thanks to Jack Massey for this feature.
9502 3.70.0 - 2018-09-01
9504 This release adds a fullmatch argument to from_regex(). When full‐
9505 match=True, the whole example will match the regex pattern as for
9506 python:re.fullmatch().
9507 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
9509 sprints!
9510 3.69.12 - 2018-08-30
9512 This release reverts the changes to logging handling in 3.69.11, which
9513 broke test that use the pytest caplog fixture internally because all
9514 logging was disabled (issue #1546).
9515 3.69.11 - 2018-08-29
9517 This patch will hide all logging messages produced by test cases before
9518 the final, minimal, failing test case (issue #356).
9519 Thanks to Gary Donovan for writing this patch at the PyCon Australia
9521 sprints!
9522 3.69.10 - 2018-08-29
9524 This patch fixes a bug that prevents coverage from reporting unexecuted
9525 Python files (issue #1085).
9526 Thanks to Gary Donovan for writing this patch at the PyCon Australia
9528 sprints!
9529 3.69.9 - 2018-08-28
9531 This patch improves the packaging of the Python package by adding LI‐
9532 CENSE.txt to the sdist (issue #1311), clarifying the minimum supported
9533 versions of pytz and dateutil (issue #1383), and adds keywords to the
9534 metadata (issue #1520).
9535 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
9537 tralia sprints!
9538 3.69.8 - 2018-08-28
9540 This is an internal change which replaces pickle with json to prevent
9541 possible security issues.
9542 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
9544 sprints!
9545 3.69.7 - 2018-08-28
9547 This patch ensures that note() prints the note for every test case when
9548 the verbosity setting is Verbosity.verbose. At normal verbosity it
9549 only prints from the final test case.
9550 Thanks to Tom McDermott for writing this patch at the PyCon Australia
9552 sprints!
9553 3.69.6 - 2018-08-27
9555 This patch improves the testing of some internal caching. It should
9556 have no user-visible effect.
9557 3.69.5 - 2018-08-27
9559 This change performs a small rename and refactoring in the core engine.
9560 There is no user-visible change.
9561 3.69.4 - 2018-08-27
9563 This change improves the core engine's ability to avoid unnecessary
9564 work, by consulting its cache of previously-tried inputs in more cases.
9565 3.69.3 - 2018-08-27
9567 This patch handles passing an empty python:enum.Enum to from_type() by
9568 returning nothing(), instead of raising an internal python:Assertion‐
9569 Error.
9570 Thanks to Paul Amazona for writing this patch at the PyCon Australia
9572 sprints!
9573 3.69.2 - 2018-08-23
9575 This patch fixes a small mistake in an internal comment. There is no
9576 user-visible change.
9577 3.69.1 - 2018-08-21
9579 This change fixes a small bug in how the core engine consults its cache
9580 of previously-tried inputs. There is unlikely to be any user-visible
9581 change.
9582 3.69.0 - 2018-08-20
9584 This release improves argument validation for stateful testing.
9585 • If the target or targets of a rule() are invalid, we now raise a use‐
9587 ful validation error rather than an internal exception.
9588 • Passing both the target and targets arguments is deprecated - append
9590 the target bundle to the targets tuple of bundles instead.
9591 • Passing the name of a Bundle rather than the Bundle itself is also
9593 deprecated.
9594 3.68.3 - 2018-08-20
9596 This is a docs-only patch, fixing some typos and formatting issues.
9597 3.68.2 - 2018-08-19
9599 This change fixes a small bug in how the core engine caches the results
9600 of previously-tried inputs. The effect is unlikely to be noticeable,
9601 but it might avoid unnecessary work in some cases.
9602 3.68.1 - 2018-08-18
9604 This patch documents the from_dtype() function, which infers a strategy
9605 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
9606 directly when creating e.g. Pandas objects.
9607 3.68.0 - 2018-08-15
9609 arrays() now checks that integer and float values drawn from elements
9610 and fill strategies can be safely cast to the dtype of the array, and
9611 emits a warning otherwise (issue #1385).
9612 Elements in the resulting array could previously violate constraints on
9614 the elements strategy due to floating-point overflow or truncation of
9615 integers to fit smaller types.
9616 3.67.1 - 2018-08-14
9618 This release contains a tiny refactoring of the internals. There is no
9619 user-visible change.
9620 3.67.0 - 2018-08-10
9622 This release adds a width argument to floats(), to generate lower-pre‐
9623 cision floating point numbers for e.g. Numpy arrays.
9624 The generated examples are always instances of Python's native float
9626 type, which is 64bit, but passing width=32 will ensure that all values
9627 can be exactly represented as 32bit floats. This can be useful to
9628 avoid overflow (to +/- infinity), and for efficiency of generation and
9629 shrinking.
9630 Half-precision floats (width=16) are also supported, but require Numpy
9632 if you are running Python 3.5 or earlier.
9633 3.66.33 - 2018-08-10
9635 This release fixes a bug in floats(), where setting allow_infin‐
9636 ity=False and exactly one of min_value and max_value would allow infi‐
9637 nite values to be generated.
9638 3.66.32 - 2018-08-09
9640 This release adds type hints to the example() and seed() decorators,
9641 and fixes the type hint on register_type_strategy(). The second argu‐
9642 ment to register_type_strategy() must either be a SearchStrategy, or a
9643 callable which takes a type and returns a SearchStrategy.
9644 3.66.31 - 2018-08-08
9646 Another set of changes designed to improve the performance of shrinking
9647 on large examples. In particular the shrinker should now spend consid‐
9648 erably less time running useless shrinks.
9649 3.66.30 - 2018-08-06
9651 "Bug fixes and performance improvements".
9652 This release is a fairly major overhaul of the shrinker designed to im‐
9654 prove its behaviour on large examples, especially around stateful test‐
9655 ing. You should hopefully see shrinking become much faster, with little
9656 to no quality degradation (in some cases quality may even improve).
9657 3.66.29 - 2018-08-05
9659 This release fixes two very minor bugs in the core engine:
9660 • it fixes a corner case that was missing in 3.66.28, which should
9662 cause shrinking to work slightly better.
9663 • it fixes some logic for how shrinking interacts with the database
9665 that was causing Hypothesis to be insufficiently aggressive about
9666 clearing out old keys.
9667 3.66.28 - 2018-08-05
9669 This release improves how Hypothesis handles reducing the size of inte‐
9670 gers' representation. This change should mostly be invisible as it's
9671 purely about the underlying representation and not the generated value,
9672 but it may result in some improvements to shrink performance.
9673 3.66.27 - 2018-08-05
9675 This release changes the order in which Hypothesis chooses parts of the
9676 test case to shrink. For typical usage this should be a significant
9677 performance improvement on large examples. It is unlikely to have a ma‐
9678 jor impact on example quality, but where it does change the result it
9679 should usually be an improvement.
9680 3.66.26 - 2018-08-05
9682 This release improves the debugging information that the shrinker emits
9683 about the operations it performs, giving better summary statistics
9684 about which passes resulted in test executions and whether they were
9685 successful.
9686 3.66.25 - 2018-08-05
9688 This release fixes several bugs that were introduced to the shrinker in
9689 3.66.24 which would have caused it to behave significantly less well
9690 than advertised. With any luck you should actually see the promised
9691 benefits now.
9692 3.66.24 - 2018-08-03
9694 This release changes how Hypothesis deletes data when shrinking in or‐
9695 der to better handle deletion of large numbers of contiguous sequences.
9696 Most tests should see little change, but this will hopefully provide a
9697 significant speed up for stateful testing.
9698 3.66.23 - 2018-08-02
9700 This release makes some internal changes to enable further improvements
9701 to the shrinker. You may see some changes in the final shrunk examples,
9702 but they are unlikely to be significant.
9703 3.66.22 - 2018-08-01
9705 This release adds some more internal caching to the shrinker. This
9706 should cause a significant speed up for shrinking, especially for
9707 stateful testing and large example sizes.
9708 3.66.21 - 2018-08-01
9710 This patch is for downstream packagers - our tests now pass under
9711 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
9712 of Hypothesis itself.
9713 3.66.20 - 2018-08-01
9715 This release removes some functionality from the shrinker that was tak‐
9716 ing a considerable amount of time and does not appear to be useful any
9717 more due to a number of quality improvements in the shrinker.
9718 You may see some degradation in shrink quality as a result of this, but
9720 mostly shrinking should just get much faster.
9721 3.66.19 - 2018-08-01
9723 This release slightly changes the format of some debugging information
9724 emitted during shrinking, and refactors some of the internal interfaces
9725 around that.
9726 3.66.18 - 2018-07-31
9728 This release is a very small internal refactoring which should have no
9729 user visible impact.
9730 3.66.17 - 2018-07-31
9732 This release fixes a bug that could cause an IndexError to be raised
9733 from inside Hypothesis during shrinking. It is likely that it was im‐
9734 possible to trigger this bug in practice - it was only made visible by
9735 some currently unreleased work.
9736 3.66.16 - 2018-07-31
9738 This release is a very small internal refactoring which should have no
9739 user visible impact.
9740 3.66.15 - 2018-07-31
9742 This release makes Hypothesis's shrinking faster by removing some re‐
9743 dundant work that it does when minimizing values in its internal repre‐
9744 sentation.
9745 3.66.14 - 2018-07-30
9747 This release expands the deprecation of timeout from 3.16.0 to also
9748 emit the deprecation warning in find or stateful testing.
9749 3.66.13 - 2018-07-30
9751 This release adds an additional shrink pass that is able to reduce the
9752 size of examples in some cases where the transformation is non-obvious.
9753 In particular this will improve the quality of some examples which
9754 would have regressed in 3.66.12.
9755 3.66.12 - 2018-07-28
9757 This release changes how we group data together for shrinking. It
9758 should result in improved shrinker performance, especially in stateful
9759 testing.
9760 3.66.11 - 2018-07-28
9762 This patch modifies how which rule to run is selected during rule based
9763 stateful testing. This should result in a slight performance increase
9764 during generation and a significant performance and quality improvement
9765 when shrinking.
9766 As a result of this change, some state machines which would previously
9768 have thrown an InvalidDefinition are no longer detected as invalid.
9769 3.66.10 - 2018-07-28
9771 This release weakens some minor functionality in the shrinker that had
9772 only modest benefit and made its behaviour much harder to reason about.
9773 This is unlikely to have much user visible effect, but it is possible
9775 that in some cases shrinking may get slightly slower. It is primarily
9776 to make it easier to work on the shrinker and pave the way for future
9777 work.
9778 3.66.9 - 2018-07-26
9780 This release improves the information that Hypothesis emits about its
9781 shrinking when verbosity is set to debug.
9782 3.66.8 - 2018-07-24
9784 This patch includes some minor fixes in the documentation, and updates
9785 the minimum version of pytest to 3.0 (released August 2016).
9786 3.66.7 - 2018-07-24
9788 This release fixes a bug where difficult to shrink tests could some‐
9789 times trigger an internal assertion error inside the shrinker.
9790 3.66.6 - 2018-07-23
9792 This patch ensures that Hypothesis fully supports Python 3.7, by up‐
9793 grading from_type() (issue #1264) and fixing some minor issues in our
9794 test suite (issue #1148).
9795 3.66.5 - 2018-07-22
9797 This patch fixes the online docs for various extras, by ensuring that
9798 their dependencies are installed on readthedocs.io (issue #1326).
9799 3.66.4 - 2018-07-20
9801 This release improves the shrinker's ability to reorder examples.
9802 For example, consider the following test:
9804 import hypothesis.strategies as st
9806 from hypothesis import given
9807 @given(st.text(), st.text())
9810 def test_non_equal(x, y):
9811 assert x != y
9812 Previously this could have failed with either of x="", y="0" or x="0",
9814 y="". Now it should always fail with x="", y="0".
9815 This will allow the shrinker to produce more consistent results, espe‐
9817 cially in cases where test cases contain some ordered collection whose
9818 actual order does not matter.
9819 3.66.3 - 2018-07-20
9821 This patch fixes inference in the builds() strategy with subtypes of
9822 python:typing.NamedTuple, where the __init__ method is not useful for
9823 introspection. We now use the field types instead - thanks to James
9824 Uther for identifying this bug.
9825 3.66.2 - 2018-07-19
9827 This release improves the shrinker's ability to handle situations where
9828 there is an additive constraint between two values.
9829 For example, consider the following test:
9831 import hypothesis.strategies as st
9833 from hypothesis import given
9834 @given(st.integers(), st.integers())
9837 def test_does_not_exceed_100(m, n):
9838 assert m + n < 100
9839 Previously this could have failed with almost any pair (m, n) with 0 <=
9841 m <= n and m + n == 100. Now it should almost always fail with m=0,
9842 n=100.
9843 This is a relatively niche specialisation, but can be useful in situa‐
9845 tions where e.g. a bug is triggered by an integer overflow.
9846 3.66.1 - 2018-07-09
9848 This patch fixes a rare bug where an incorrect percentage drawtime
9849 could be displayed for a test, when the system clock was changed during
9850 a test running under Python 2 (we use python:time.monotonic() where it
9851 is available to avoid such problems). It also fixes a possible
9852 zero-division error that can occur when the underlying C library dou‐
9853 ble-rounds an intermediate value in python:math.fsum() and gets the
9854 least significant bit wrong.
9855 3.66.0 - 2018-07-05
9857 This release improves validation of the alphabet argument to the text()
9858 strategy. The following misuses are now deprecated, and will be an er‐
9859 ror in a future version:
9860 • passing an unordered collection (such as set('abc')), which violates
9862 invariants about shrinking and reproducibility
9863 • passing an alphabet sequence with elements that are not strings
9865 • passing an alphabet sequence with elements that are not of length
9867 one, which violates any size constraints that may apply
9868 Thanks to Sushobhit for adding these warnings (issue #1329).
9870 3.65.3 - 2018-07-04
9872 This release fixes a mostly theoretical bug where certain usage of the
9873 internal API could trigger an assertion error inside Hypothesis. It is
9874 unlikely that this problem is even possible to trigger through the pub‐
9875 lic API.
9876 3.65.2 - 2018-07-04
9878 This release fixes dependency information for coverage. Previously Hy‐
9879 pothesis would allow installing coverage with any version, but it only
9880 works with coverage 4.0 or later.
9881 We now specify the correct metadata in our setup.py, so Hypothesis will
9883 only allow installation with compatible versions of coverage.
9884 3.65.1 - 2018-07-03
9886 This patch ensures that stateful tests which raise an error from a
9887 pytest helper still print the sequence of steps taken to reach that
9888 point (issue #1372). This reporting was previously broken because the
9889 helpers inherit directly from python:BaseException, and therefore re‐
9890 quire special handling to catch without breaking e.g. the use of ctrl-C
9891 to quit the test.
9892 3.65.0 - 2018-06-30
9894 This release deprecates the max_shrinks setting in favor of an internal
9895 heuristic. If you need to avoid shrinking examples, use the phases
9896 setting instead. (issue #1235)
9897 3.64.2 - 2018-06-27
9899 This release fixes a bug where an internal assertion error could some‐
9900 times be triggered while shrinking a failing test.
9901 3.64.1 - 2018-06-27
9903 This patch fixes type-checking errors in our vendored pretty-printer,
9904 which were ignored by our mypy config but visible for anyone else
9905 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
9906 3.64.0 - 2018-06-26
9908 This release adds an interface which can be used to insert a wrapper
9909 between the original test function and @given (issue #1257). This will
9910 be particularly useful for test runner extensions such as pytest-trio,
9911 but is not recommended for direct use by other users of Hypothesis.
9912 3.63.0 - 2018-06-26
9914 This release adds a new mechanism to infer strategies for classes de‐
9915 fined using attrs, based on the the type, converter, or validator of
9916 each attribute. This inference is now built in to builds() and
9917 from_type().
9918 On Python 2, from_type() no longer generates instances of int when
9920 passed long, or vice-versa.
9921 3.62.0 - 2018-06-26
9923 This release adds PEP 484 type hints to Hypothesis on a provisional ba‐
9924 sis, using the comment-based syntax for Python 2 compatibility. You
9925 can read more about our type hints here.
9926 It also adds the py.typed marker specified in PEP 561. After you pip
9928 install hypothesis, mypy 0.590 or later will therefore type-check your
9929 use of our public interface!
9930 3.61.0 - 2018-06-24
9932 This release deprecates the use of settings as a context manager, the
9933 use of which is somewhat ambiguous.
9934 Users should define settings with global state or with the @set‐
9936 tings(...) decorator.
9937 3.60.1 - 2018-06-20
9939 Fixed a bug in generating an instance of a Django model from a strategy
9940 where the primary key is generated as part of the strategy. See details
9941 here.
9942 Thanks to Tim Martin for this contribution.
9944 3.60.0 - 2018-06-20
9946 This release adds the @initialize decorator for stateful testing (orig‐
9947 inally discussed in issue #1216). All @initialize rules will be called
9948 once each in an arbitrary order before any normal rule is called.
9949 3.59.3 - 2018-06-19
9951 This is a no-op release to take into account some changes to the re‐
9952 lease process. It should have no user visible effect.
9953 3.59.2 - 2018-06-18
9955 This adds support for partially sorting examples which cannot be fully
9956 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
9957 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
9958 0].
9959 Thanks to Luke for contributing.
9961 3.59.1 - 2018-06-16
9963 This patch uses python:random.getstate() and python:random.setstate()
9964 to restore the PRNG state after @given runs deterministic tests. With‐
9965 out restoring state, you might have noticed problems such as issue
9966 #1266. The fix also applies to stateful testing (issue #702).
9967 3.59.0 - 2018-06-14
9969 This release adds the emails() strategy, which generates unicode
9970 strings representing an email address.
9971 Thanks to Sushobhit for moving this to the public API (issue #162).
9973 3.58.1 - 2018-06-13
9975 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
9976 1 2 3.
9977 Thanks to Luke for contributing.
9979 3.58.0 - 2018-06-13
9981 This adds a new extra timezones() strategy that generates dateutil
9982 timezones.
9983 Thanks to Conrad for contributing.
9985 3.57.0 - 2018-05-20
9987 Using an unordered collection with the permutations() strategy has been
9988 deprecated because the order in which e.g. a set shrinks is arbitrary.
9989 This may cause different results between runs.
9990 3.56.10 - 2018-05-16
9992 This release makes hypothesis.settings.define_setting a private method,
9993 which has the effect of hiding it from the documentation.
9994 3.56.9 - 2018-05-11
9996 This is another release with no functionality changes as part of
9997 changes to Hypothesis's new release tagging scheme.
9998 3.56.8 - 2018-05-10
10000 This is a release with no functionality changes that moves Hypothesis
10001 over to a new release tagging scheme.
10002 3.56.7 - 2018-05-10
10004 This release provides a performance improvement for most tests, but in
10005 particular users of sampled_from() who don't have numpy installed
10006 should see a significant performance improvement.
10007 3.56.6 - 2018-05-09
10009 This patch contains further internal work to support Mypy. There are
10010 no user-visible changes... yet.
10011 3.56.5 - 2018-04-22
10013 This patch contains some internal refactoring to run mypy in CI. There
10014 are no user-visible changes.
10015 3.56.4 - 2018-04-21
10017 This release involves some very minor internal clean up and should have
10018 no user visible effect at all.
10019 3.56.3 - 2018-04-20
10021 This release fixes a problem introduced in 3.56.0 where setting the hy‐
10022 pothesis home directory (through currently undocumented means) would no
10023 longer result in the default database location living in the new home
10024 directory.
10025 3.56.2 - 2018-04-20
10027 This release fixes a problem introduced in 3.56.0 where setting max_ex‐
10028 amples to 1 would result in tests failing with Unsatisfiable. This
10029 problem could also occur in other harder to trigger circumstances (e.g.
10030 by setting it to a low value, having a hard to satisfy assumption, and
10031 disabling health checks).
10032 3.56.1 - 2018-04-20
10034 This release fixes a problem that was introduced in 3.56.0: Use of the
10035 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
10036 cated, actually broken due to being read before various setup the dep‐
10037 recation path needed was done. It now works correctly (and emits a dep‐
10038 recation warning).
10039 3.56.0 - 2018-04-17
10041 This release deprecates several redundant or internally oriented set‐
10042 tings, working towards an orthogonal set of configuration options that
10043 are widely useful without requiring any knowledge of our internals (‐
10044 issue #535).
10045 • Deprecated settings that no longer have any effect are no longer
10047 shown in the __repr__ unless set to a non-default value.
10048 • hypothesis.settings.perform_health_check is deprecated, as it dupli‐
10050 cates suppress_health_check.
10051 • hypothesis.settings.max_iterations is deprecated and disabled, be‐
10053 cause we can usually get better behaviour from an internal heuristic
10054 than a user-controlled setting.
10055 • hypothesis.settings.min_satisfying_examples is deprecated and dis‐
10057 abled, due to overlap with the filter_too_much healthcheck and poor
10058 interaction with max_examples.
10059 • HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
10061 the profile system instead.
10062 • Examples tried by find() are now reported at debug verbosity level
10064 (as well as verbose level).
10065 3.55.6 - 2018-04-14
10067 This release fixes a somewhat obscure condition (issue #1230) under
10068 which you could occasionally see a failing test trigger an assertion
10069 error inside Hypothesis instead of failing normally.
10070 3.55.5 - 2018-04-14
10072 This patch fixes one possible cause of issue #966. When running Python
10073 2 with hash randomisation, passing a python:bytes object to python:ran‐
10074 dom.seed() would use version=1, which broke derandomize (because the
10075 seed depended on a randomised hash). If derandomize is still nondeter‐
10076 ministic for you, please open an issue.
10077 3.55.4 - 2018-04-13
10079 This patch makes a variety of minor improvements to the documentation,
10080 and improves a few validation messages for invalid inputs.
10081 3.55.3 - 2018-04-12
10083 This release updates the URL metadata associated with the PyPI package
10084 (again). It has no other user visible effects.
10085 3.55.2 - 2018-04-11
10087 This release updates the URL metadata associated with the PyPI package.
10088 It has no other user visible effects.
10089 3.55.1 - 2018-04-06
10091 This patch relaxes constraints in our tests on the expected values re‐
10092 turned by the standard library function hypot() and the internal helper
10093 function cathetus, to fix near-exact test failures on some 32-bit sys‐
10094 tems used by downstream packagers.
10095 3.55.0 - 2018-04-05
10097 This release includes several improvements to the handling of the data‐
10098 base setting.
10099 • The database_file setting was a historical artefact, and you should
10101 just use database directly.
10102 • The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
10104 favor of load_profile() and the database setting.
10105 • If you have not configured the example database at all and the de‐
10107 fault location is not usable (due to e.g. permissions issues), Hy‐
10108 pothesis will fall back to an in-memory database. This is not per‐
10109 sisted between sessions, but means that the defaults work on
10110 read-only filesystems.
10111 3.54.0 - 2018-04-04
10113 This release improves the complex_numbers() strategy, which now sup‐
10114 ports min_magnitude and max_magnitude arguments, along with allow_nan
10115 and allow_infinity like for floats().
10116 Thanks to J.J. Green for this feature.
10118 3.53.0 - 2018-04-01
10120 This release removes support for Django 1.8, which reached end of life
10121 on 2018-04-01. You can see Django's release and support schedule on
10122 the Django Project website.
10123 3.52.3 - 2018-04-01
10125 This patch fixes the min_satisfying_examples settings documentation, by
10126 explaining that example shrinking is tracked at the level of the under‐
10127 lying bytestream rather than the output value.
10128 The output from find() in verbose mode has also been adjusted - see the
10130 example session - to avoid duplicating lines when the example repr is
10131 constant, even if the underlying representation has been shrunken.
10132 3.52.2 - 2018-03-30
10134 This release improves the output of failures with rule based stateful
10135 testing in two ways:
10136 • The output from it is now usually valid Python code.
10138 • When the same value has two different names because it belongs to two
10140 different bundles, it will now display with the name associated with
10141 the correct bundle for a rule argument where it is used.
10142 3.52.1 - 2018-03-29
10144 This release improves the behaviour of stateful testing in two ways:
10145 • Previously some runs would run no steps (issue #376). This should no
10147 longer happen.
10148 • RuleBasedStateMachine tests which used bundles extensively would of‐
10150 ten shrink terribly. This should now be significantly improved,
10151 though there is likely a lot more room for improvement.
10152 This release also involves a low level change to how ranges of integers
10154 are handles which may result in other improvements to shrink quality in
10155 some cases.
10156 3.52.0 - 2018-03-24
10158 This release deprecates use of @settings(...) as a decorator, on func‐
10159 tions or methods that are not also decorated with @given. You can
10160 still apply these decorators in any order, though you should only do so
10161 once each.
10162 Applying @given twice was already deprecated, and applying @set‐
10164 tings(...) twice is deprecated in this release and will become an error
10165 in a future version. Neither could ever be used twice to good effect.
10166 Using @settings(...) as the sole decorator on a test is completely
10168 pointless, so this common usage error will become an error in a future
10169 version of Hypothesis.
10170 3.51.0 - 2018-03-24
10172 This release deprecates the average_size argument to lists() and other
10173 collection strategies. You should simply delete it wherever it was
10174 used in your tests, as it no longer has any effect.
10175 In early versions of Hypothesis, the average_size argument was treated
10177 as a hint about the distribution of examples from a strategy. Subse‐
10178 quent improvements to the conceptual model and the engine for generat‐
10179 ing and shrinking examples mean it is more effective to simply describe
10180 what constitutes a valid example, and let our internals handle the dis‐
10181 tribution.
10182 3.50.3 - 2018-03-24
10184 This patch contains some internal refactoring so that we can run with
10185 warnings as errors in CI.
10186 3.50.2 - 2018-03-20
10188 This has no user-visible changes except one slight formatting change to
10189 one docstring, to avoid a deprecation warning.
10190 3.50.1 - 2018-03-20
10192 This patch fixes an internal error introduced in 3.48.0, where a check
10193 for the Django test runner would expose import-time errors in Django
10194 configuration (issue #1167).
10195 3.50.0 - 2018-03-19
10197 This release improves validation of numeric bounds for some strategies.
10198 • integers() and floats() now raise InvalidArgument if passed a
10200 min_value or max_value which is not an instance of Real, instead of
10201 various internal errors.
10202 • floats() now converts its bounding values to the nearest float above
10204 or below the min or max bound respectively, instead of just casting
10205 to float. The old behaviour was incorrect in that you could generate
10206 float(min_value), even when this was less than min_value itself (pos‐
10207 sible with eg. fractions).
10208 • When both bounds are provided to floats() but there are no floats in
10210 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
10211 raised.
10212 • decimals() gives a more useful error message if passed a string that
10214 cannot be converted to Decimal in a context where this error is not
10215 trapped.
10216 Code that previously seemed to work may be explicitly broken if there
10218 were no floats between min_value and max_value (only possible with
10219 non-float bounds), or if a bound was not a Real number but still al‐
10220 lowed in python:math.isnan (some custom classes with a __float__
10221 method).
10222 3.49.1 - 2018-03-15
10224 This patch fixes our tests for Numpy dtype strategies on big-endian
10225 platforms, where the strategy behaved correctly but the test assumed
10226 that the native byte order was little-endian.
10227 There is no user impact unless you are running our test suite on
10229 big-endian platforms. Thanks to Graham Inggs for reporting issue
10230 #1164.
10231 3.49.0 - 2018-03-12
10233 This release deprecates passing elements=None to collection strategies,
10234 such as lists().
10235 Requiring lists(nothing()) or builds(list) instead of lists() means
10237 slightly more typing, but also improves the consistency and discover‐
10238 ability of our API - as well as showing how to compose or construct
10239 strategies in ways that still work in more complex situations.
10240 Passing a nonzero max_size to a collection strategy where the elements
10242 strategy contains no values is now deprecated, and will be an error in
10243 a future version. The equivalent with elements=None is already an er‐
10244 ror.
10245 3.48.1 - 2018-03-05
10247 This patch will minimize examples that would come out non-minimal in
10248 previous versions. Thanks to Kyle Reeve for this patch.
10249 3.48.0 - 2018-03-05
10251 This release improves some "unhappy paths" when using Hypothesis with
10252 the standard library python:unittest module:
10253 • Applying @given to a non-test method which is overridden from
10255 python:unittest.TestCase, such as setUp, raises a new health check.
10256 (issue #991)
10257 • Using subTest() within a test decorated with @given would leak inter‐
10259 mediate results when tests were run under the python:unittest test
10260 runner. Individual reporting of failing subtests is now disabled
10261 during a test using @given. (issue #1071)
10262 • @given is still not a class decorator, but the error message if you
10264 try using it on a class has been improved.
10265 As a related improvement, using django:django.test.TestCase with @given
10267 instead of hypothesis.extra.django.TestCase raises an explicit error
10268 instead of running all examples in a single database transaction.
10269 3.47.0 - 2018-03-02
10271 register_profile now accepts keyword arguments for specific settings,
10272 and the parent settings object is now optional. Using a name for a
10273 registered profile which is not a string was never suggested, but it is
10274 now also deprecated and will eventually be an error.
10275 3.46.2 - 2018-03-01
10277 This release removes an unnecessary branch from the code, and has no
10278 user-visible impact.
10279 3.46.1 - 2018-03-01
10281 This changes only the formatting of our docstrings and should have no
10282 user-visible effects.
10283 3.46.0 - 2018-02-26
10285 characters() has improved docs about what arguments are valid, and ad‐
10286 ditional validation logic to raise a clear error early (instead of e.g.
10287 silently ignoring a bad argument). Categories may be specified as the
10288 Unicode 'general category' (eg u'Nd'), or as the 'major category' (eg
10289 [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
10290 In previous versions, general categories were supported and all other
10292 input was silently ignored. Now, major categories are supported in ad‐
10293 dition to general categories (which may change the behaviour of some
10294 existing code), and all other input is deprecated.
10295 3.45.5 - 2018-02-26
10297 This patch improves strategy inference in hypothesis.extra.django to
10298 account for some validators in addition to field type - see issue #1116
10299 for ongoing work in this space.
10300 Specifically, if a CharField or TextField has an attached RegexValida‐
10302 tor, we now use from_regex() instead of text() as the underlying strat‐
10303 egy. This allows us to generate examples of the default User model,
10304 closing issue #1112.
10305 3.45.4 - 2018-02-25
10307 This patch improves some internal debugging information, fixes a typo
10308 in a validation error message, and expands the documentation for new
10309 contributors.
10310 3.45.3 - 2018-02-23
10312 This patch may improve example shrinking slightly for some strategies.
10313 3.45.2 - 2018-02-18
10315 This release makes our docstring style more consistent, thanks to
10316 flake8-docstrings. There are no user-visible changes.
10317 3.45.1 - 2018-02-17
10319 This fixes an indentation issue in docstrings for datetimes(), dates(),
10320 times(), and timedeltas().
10321 3.45.0 - 2018-02-13
10323 This release fixes builds() so that target can be used as a keyword ar‐
10324 gument for passing values to the target. The target itself can still be
10325 specified as a keyword argument, but that behavior is now deprecated.
10326 The target should be provided as the first positional argument.
10327 3.44.26 - 2018-02-06
10329 This release fixes some formatting issues in the Hypothesis source
10330 code. It should have no externally visible effects.
10331 3.44.25 - 2018-02-05
10333 This release changes the way in which Hypothesis tries to shrink the
10334 size of examples. It probably won't have much impact, but might make
10335 shrinking faster in some cases. It is unlikely but not impossible that
10336 it will change the resulting examples.
10337 3.44.24 - 2018-01-27
10339 This release fixes dependency information when installing Hypothesis
10340 from a binary "wheel" distribution.
10341 • The install_requires for enum34 is resolved at install time, rather
10343 than at build time (with potentially different results).
10344 • Django has fixed their python_requires for versions 2.0.0 onward,
10346 simplifying Python2-compatible constraints for downstream projects.
10347 3.44.23 - 2018-01-24
10349 This release improves shrinking in a class of pathological examples
10350 that you are probably never hitting in practice. If you are hitting
10351 them in practice this should be a significant speed up in shrinking. If
10352 you are not, you are very unlikely to notice any difference. You might
10353 see a slight slow down and/or slightly better falsifying examples.
10354 3.44.22 - 2018-01-23
10356 This release fixes a dependency problem. It was possible to install
10357 Hypothesis with an old version of attrs, which would throw a TypeError
10358 as soon as you tried to import hypothesis. Specifically, you need at‐
10359 trs 16.0.0 or newer.
10360 Hypothesis will now require the correct version of attrs when in‐
10362 stalling.
10363 3.44.21 - 2018-01-22
10365 This change adds some additional structural information that Hypothesis
10366 will use to guide its search.
10367 You mostly shouldn't see much difference from this. The two most likely
10369 effects you would notice are:
10370 1. Hypothesis stores slightly more examples in its database for passing
10372 tests.
10373 2. Hypothesis may find new bugs that it was previously missing, but it
10375 probably won't (this is a basic implementation of the feature that
10376 is intended to support future work. Although it is useful on its
10377 own, it's not very useful on its own).
10378 3.44.20 - 2018-01-21
10380 This is a small refactoring release that changes how Hypothesis tracks
10381 some information about the boundary of examples in its internal repre‐
10382 sentation.
10383 You are unlikely to see much difference in behaviour, but memory usage
10385 and run time may both go down slightly during normal test execution,
10386 and when failing Hypothesis might print its failing example slightly
10387 sooner.
10388 3.44.19 - 2018-01-21
10390 This changes how we compute the default average_size for all collection
10391 strategies. Previously setting a max_size without setting an aver‐
10392 age_size would have the seemingly paradoxical effect of making data
10393 generation slower, because it would raise the average size from its de‐
10394 fault. Now setting max_size will either leave the default unchanged or
10395 lower it from its default.
10396 If you are currently experiencing this problem, this may make your
10398 tests substantially faster. If you are not, this will likely have no
10399 effect on you.
10400 3.44.18 - 2018-01-20
10402 This is a small refactoring release that changes how Hypothesis detects
10403 when the structure of data generation depends on earlier values gener‐
10404 ated (e.g. when using flatmap or composite()). It should not have any
10405 observable effect on behaviour.
10406 3.44.17 - 2018-01-15
10408 This release fixes a typo in internal documentation, and has no
10409 user-visible impact.
10410 3.44.16 - 2018-01-13
10412 This release improves test case reduction for recursive data struc‐
10413 tures. Hypothesis now guarantees that whenever a strategy calls itself
10414 recursively (usually this will happen because you are using de‐
10415 ferred()), any recursive call may replace the top level value. e.g.
10416 given a tree structure, Hypothesis will always try replacing it with a
10417 subtree.
10418 Additionally this introduces a new heuristic that may in some circum‐
10420 stances significantly speed up test case reduction - Hypothesis should
10421 be better at immediately replacing elements drawn inside another strat‐
10422 egy with their minimal possible value.
10423 3.44.15 - 2018-01-13
10425 from_type() can now resolve recursive types such as binary trees (issue
10426 #1004). Detection of non-type arguments has also improved, leading to
10427 better error messages in many cases involving forward references.
10428 3.44.14 - 2018-01-08
10430 This release fixes a bug in the shrinker that prevented the optimisa‐
10431 tions in 3.44.6 from working in some cases. It would not have worked
10432 correctly when filtered examples were nested (e.g. with a set of inte‐
10433 gers in some range).
10434 This would not have resulted in any correctness problems, but shrinking
10436 may have been slower than it otherwise could be.
10437 3.44.13 - 2018-01-08
10439 This release changes the average bit length of values drawn from inte‐
10440 gers() to be much smaller. Additionally it changes the shrinking order
10441 so that now size is considered before sign - e.g. -1 will be preferred
10442 to +10.
10443 The new internal format for integers required some changes to the mini‐
10445 mizer to make work well, so you may also see some improvements to exam‐
10446 ple quality in unrelated areas.
10447 3.44.12 - 2018-01-07
10449 This changes Hypothesis's internal implementation of weighted sampling.
10450 This will affect example distribution and quality, but you shouldn't
10451 see any other effects.
10452 3.44.11 - 2018-01-06
10454 This is a change to some internals around how Hypothesis handles avoid‐
10455 ing generating duplicate examples and seeking out novel regions of the
10456 search space.
10457 You are unlikely to see much difference as a result of it, but it fixes
10459 a bug where an internal assertion could theoretically be triggered and
10460 has some minor effects on the distribution of examples so could poten‐
10461 tially find bugs that have previously been missed.
10462 3.44.10 - 2018-01-06
10464 This patch avoids creating debug statements when debugging is disabled.
10465 Profiling suggests this is a 5-10% performance improvement (issue
10466 #1040).
10467 3.44.9 - 2018-01-06
10469 This patch blacklists null characters ('\x00') in automatically created
10470 strategies for Django CharField and TextField, due to a database issue
10471 which was recently fixed upstream (Hypothesis issue #1045).
10472 3.44.8 - 2018-01-06
10474 This release makes the Hypothesis shrinker slightly less greedy in or‐
10475 der to avoid local minima - when it gets stuck, it makes a small at‐
10476 tempt to search around the final example it would previously have re‐
10477 turned to find a new starting point to shrink from. This should improve
10478 example quality in some cases, especially ones where the test data has
10479 dependencies among parts of it that make it difficult for Hypothesis to
10480 proceed.
10481 3.44.7 - 2018-01-04
10483 This release adds support for Django 2 in the hypothesis-django extra.
10484 This release drops support for Django 1.10, as it is no longer sup‐
10486 ported by the Django team.
10487 3.44.6 - 2018-01-02
10489 This release speeds up test case reduction in many examples by being
10490 better at detecting large shrinks it can use to discard redundant parts
10491 of its input. This will be particularly noticeable in examples that
10492 make use of filtering and for some integer ranges.
10493 3.44.5 - 2018-01-02
10495 Happy new year!
10496 This is a no-op release that updates the year range on all of the copy‐
10498 right headers in our source to include 2018.
10499 3.44.4 - 2017-12-23
10501 This release fixes issue #1041, which slowed tests by up to 6% due to
10502 broken caching.
10503 3.44.3 - 2017-12-21
10505 This release improves the shrinker in cases where examples drawn ear‐
10506 lier can affect how much data is drawn later (e.g. when you draw a
10507 length parameter in a composite and then draw that many elements). Ex‐
10508 amples found in cases like this should now be much closer to minimal.
10509 3.44.2 - 2017-12-20
10511 This is a pure refactoring release which changes how Hypothesis manages
10512 its set of examples internally. It should have no externally visible
10513 effects.
10514 3.44.1 - 2017-12-18
10516 This release fixes issue #997, in which under some circumstances the
10517 body of tests run under Hypothesis would not show up when run under
10518 coverage even though the tests were run and the code they called out‐
10519 side of the test file would show up normally.
10520 3.44.0 - 2017-12-17
10522 This release adds a new feature: The @reproduce_failure decorator, de‐
10523 signed to make it easy to use Hypothesis's binary format for examples
10524 to reproduce a problem locally without having to share your example
10525 database between machines.
10526 This also changes when seeds are printed:
10528 • They will no longer be printed for normal falsifying examples, as
10530 there are now adequate ways of reproducing those for all cases, so it
10531 just contributes noise.
10532 • They will once again be printed when reusing examples from the data‐
10534 base, as health check failures should now be more reliable in this
10535 scenario so it will almost always work in this case.
10536 This work was funded by Smarkets.
10538 3.43.1 - 2017-12-17
10540 This release fixes a bug with Hypothesis's database management - exam‐
10541 ples that were found in the course of shrinking were saved in a way
10542 that indicated that they had distinct causes, and so they would all be
10543 retried on the start of the next test. The intended behaviour, which is
10544 now what is implemented, is that only a bounded subset of these exam‐
10545 ples would be retried.
10546 3.43.0 - 2017-12-17
10548 HypothesisDeprecationWarning now inherits from python:FutureWarning in‐
10549 stead of python:DeprecationWarning, as recommended by PEP 565 for
10550 user-facing warnings (issue #618). If you have not changed the default
10551 warnings settings, you will now see each distinct HypothesisDepreca‐
10552 tionWarning instead of only the first.
10553 3.42.2 - 2017-12-12
10555 This patch fixes issue #1017, where instances of a list or tuple sub‐
10556 type used as an argument to a strategy would be coerced to tuple.
10557 3.42.1 - 2017-12-10
10559 This release has some internal cleanup, which makes reading the code
10560 more pleasant and may shrink large examples slightly faster.
10561 3.42.0 - 2017-12-09
10563 This release deprecates faker-extra, which was designed as a transition
10564 strategy but does not support example shrinking or coverage-guided dis‐
10565 covery.
10566 3.41.0 - 2017-12-06
10568 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
10569 pling from multi-dimensional ndarrays still results in a deprecation
10570 warning. Thanks to Charlie Tanksley for this patch.
10571 3.40.1 - 2017-12-04
10573 This release makes two changes:
10574 • It makes the calculation of some of the metadata that Hypothesis uses
10576 for shrinking occur lazily. This should speed up performance of test
10577 case generation a bit because it no longer calculates information it
10578 doesn't need.
10579 • It improves the shrinker for certain classes of nested examples. e.g.
10581 when shrinking lists of lists, the shrinker is now able to concate‐
10582 nate two adjacent lists together into a single list. As a result of
10583 this change, shrinking may get somewhat slower when the minimal exam‐
10584 ple found is large.
10585 3.40.0 - 2017-12-02
10587 This release improves how various ways of seeding Hypothesis interact
10588 with the example database:
10589 • Using the example database with seed() is now deprecated. You should
10591 set database=None if you are doing that. This will only warn if you
10592 actually load examples from the database while using @seed.
10593 • The derandomize will behave the same way as @seed.
10595 • Using --hypothesis-seed will disable use of the database.
10597 • If a test used examples from the database, it will not suggest using
10599 a seed to reproduce it, because that won't work.
10600 This work was funded by Smarkets.
10602 3.39.0 - 2017-12-01
10604 This release adds a new health check that checks if the smallest "natu‐
10605 ral" possible example of your test case is very large - this will tend
10606 to cause Hypothesis to generate bad examples and be quite slow.
10607 This work was funded by Smarkets.
10609 3.38.9 - 2017-11-29
10611 This is a documentation release to improve the documentation of shrink‐
10612 ing behaviour for Hypothesis's strategies.
10613 3.38.8 - 2017-11-29
10615 This release improves the performance of characters() when using black‐
10616 list_characters and from_regex() when using negative character classes.
10617 The problems this fixes were found in the course of work funded by
10619 Smarkets.
10620 3.38.7 - 2017-11-29
10622 This is a patch release for from_regex(), which had a bug in handling
10623 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
10624 rectly when parsing regex.
10625 3.38.6 - 2017-11-28
10627 This patch changes a few byte-string literals from double to single
10628 quotes, thanks to an update in unify. There are no user-visible
10629 changes.
10630 3.38.5 - 2017-11-23
10632 This fixes the repr of strategies using lambda that are defined inside
10633 decorators to include the lambda source.
10634 This would mostly have been visible when using the statistics function‐
10636 ality - lambdas used for e.g. filtering would have shown up with a <un‐
10637 known> as their body. This can still happen, but it should happen less
10638 often now.
10639 3.38.4 - 2017-11-22
10641 This release updates the reported statistics so that they show approxi‐
10642 mately what fraction of your test run time is spent in data generation
10643 (as opposed to test execution).
10644 This work was funded by Smarkets.
10646 3.38.3 - 2017-11-21
10648 This is a documentation release, which ensures code examples are up to
10649 date by running them as doctests in CI (issue #711).
10650 3.38.2 - 2017-11-21
10652 This release changes the behaviour of the deadline setting when used
10653 with data(): Time spent inside calls to data.draw will no longer be
10654 counted towards the deadline time.
10655 As a side effect of some refactoring required for this work, the way
10657 flaky tests are handled has changed slightly. You are unlikely to see
10658 much difference from this, but some error messages will have changed.
10659 This work was funded by Smarkets.
10661 3.38.1 - 2017-11-21
10663 This patch has a variety of non-user-visible refactorings, removing
10664 various minor warts ranging from indirect imports to typos in comments.
10665 3.38.0 - 2017-11-18
10667 This release overhauls the health check system in a variety of small
10668 ways. It adds no new features, but is nevertheless a minor release be‐
10669 cause it changes which tests are likely to fail health checks.
10670 The most noticeable effect is that some tests that used to fail health
10672 checks will now pass, and some that used to pass will fail. These
10673 should all be improvements in accuracy. In particular:
10674 • New failures will usually be because they are now taking into account
10676 things like use of data() and assume() inside the test body.
10677 • New failures may also be because for some classes of example the way
10679 data generation performance was measured was artificially faster than
10680 real data generation (for most examples that are hitting performance
10681 health checks the opposite should be the case).
10682 • Tests that used to fail health checks and now pass do so because the
10684 health check system used to run in a way that was subtly different
10685 than the main Hypothesis data generation and lacked some of its sup‐
10686 port for e.g. large examples.
10687 If your data generation is especially slow, you may also see your tests
10689 get somewhat faster, as there is no longer a separate health check
10690 phase. This will be particularly noticeable when rerunning test fail‐
10691 ures.
10692 This work was funded by Smarkets.
10694 3.37.0 - 2017-11-12
10696 This is a deprecation release for some health check related features.
10697 The following are now deprecated:
10699 • Passing HealthCheck.exception_in_generation to suppress_health_check.
10701 This no longer does anything even when passed - All errors that oc‐
10702 cur during data generation will now be immediately reraised rather
10703 than going through the health check mechanism.
10704 • Passing HealthCheck.random_module to suppress_health_check. This
10706 hasn't done anything for a long time, but was never explicitly depre‐
10707 cated. Hypothesis always seeds the random module when running @given
10708 tests, so this is no longer an error and suppressing it doesn't do
10709 anything.
10710 • Passing non-HealthCheck values in suppress_health_check. This was
10712 previously allowed but never did anything useful.
10713 In addition, passing a non-iterable value as suppress_health_check will
10715 now raise an error immediately (it would never have worked correctly,
10716 but it would previously have failed later). Some validation error mes‐
10717 sages have also been updated.
10718 This work was funded by Smarkets.
10720 3.36.1 - 2017-11-10
10722 This is a yak shaving release, mostly concerned with our own tests.
10723 While getfullargspec() was documented as deprecated in Python 3.5, it
10725 never actually emitted a warning. Our code to silence this (nonexis‐
10726 tent) warning has therefore been removed.
10727 We now run our tests with DeprecationWarning as an error, and made some
10729 minor changes to our own tests as a result. This required similar up‐
10730 stream updates to coverage and execnet (a test-time dependency via
10731 pytest-xdist).
10732 There is no user-visible change in Hypothesis itself, but we encourage
10734 you to consider enabling deprecations as errors in your own tests.
10735 3.36.0 - 2017-11-06
10737 This release adds a setting to the public API, and does some internal
10738 cleanup:
10739 • The derandomize setting is now documented (issue #890)
10741 • Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
10743 #953)
10744 • Documented the strict setting as deprecated, and updated the build so
10746 our docs always match deprecations in the code.
10747 3.35.0 - 2017-11-06
10749 This minor release supports constraining uuids() to generate a particu‐
10750 lar version of UUID (issue #721).
10751 Thanks to Dion Misic for this feature.
10753 3.34.1 - 2017-11-02
10755 This patch updates the documentation to suggest builds(callable) in‐
10756 stead of just(callable()).
10757 3.34.0 - 2017-11-02
10759 Hypothesis now emits deprecation warnings if you apply @given more than
10760 once to a target.
10761 Applying @given repeatedly wraps the target multiple times. Each wrap‐
10763 per will search the space of of possible parameters separately. This
10764 is equivalent but will be much more inefficient than doing it with a
10765 single call to @given.
10766 For example, instead of @given(booleans()) @given(integers()), you
10768 could write @given(booleans(), integers())
10769 3.33.1 - 2017-11-02
10771 This is a bugfix release:
10772 • builds() would try to infer a strategy for required positional argu‐
10774 ments of the target from type hints, even if they had been given to
10775 builds() as positional arguments (issue #946). Now it only infers
10776 missing required arguments.
10777 • An internal introspection function wrongly reported self as a re‐
10779 quired argument for bound methods, which might also have affected
10780 builds(). Now it knows better.
10781 3.33.0 - 2017-10-16
10783 This release supports strategy inference for more Django field types -
10784 you can now omit an argument for Date, Time, Duration, Slug, IP Ad‐
10785 dress, and UUID fields. (issue #642)
10786 Strategy generation for fields with grouped choices now selects choices
10788 from each group, instead of selecting from the group names.
10789 3.32.2 - 2017-10-15
10791 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
10792 an experimental basis. It has never actually worked, and the new Hy‐
10793 pothesis example database is designed to make such a tool unnecessary.
10794 3.32.1 - 2017-10-13
10796 This patch has two improvements for strategies based on enumerations.
10797 • from_type() now handles enumerations correctly, delegating to sam‐
10799 pled_from(). Previously it noted that Enum.__init__ has no required
10800 arguments and therefore delegated to builds(), which would subse‐
10801 quently fail.
10802 • When sampling from an python:enum.Flag, we also generate combinations
10804 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
10805 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
10806 3.32.0 - 2017-10-09
10808 This changes the default value of the use_coverage setting to True when
10809 running on pypy (it was already True on CPython).
10810 It was previously set to False because we expected it to be too slow,
10812 but recent benchmarking shows that actually performance of the feature
10813 on pypy is fairly acceptable - sometimes it's slower than on CPython,
10814 sometimes it's faster, but it's generally within a factor of two either
10815 way.
10816 3.31.6 - 2017-10-08
10818 This patch improves the quality of strategies inferred from Numpy
10819 dtypes:
10820 • Integer dtypes generated examples with the upper half of their
10822 (non-sign) bits set to zero. The inferred strategies can now produce
10823 any representable integer.
10824 • Fixed-width unicode- and byte-string dtypes now cap the internal ex‐
10826 ample length, which should improve example and shrink quality.
10827 • Numpy arrays can only store fixed-size strings internally, and allow
10829 shorter strings by right-padding them with null bytes. Inferred
10830 string strategies no longer generate such values, as they can never
10831 be retrieved from an array. This improves shrinking performance by
10832 skipping useless values.
10833 This has already been useful in Hypothesis - we found an overflow bug
10835 in our Pandas support, and as a result indexes() and range_indexes()
10836 now check that min_size and max_size are at least zero.
10837 3.31.5 - 2017-10-08
10839 This release fixes a performance problem in tests where the use_cover‐
10840 age setting is True.
10841 Tests experience a slow-down proportionate to the amount of code they
10843 cover. This is still the case, but the factor is now low enough that
10844 it should be unnoticeable. Previously it was large and became much
10845 larger in 3.30.4.
10846 3.31.4 - 2017-10-08
10848 from_type() failed with a very confusing error if passed a NewType (‐
10849 issue #901). These pseudo-types are now unwrapped correctly, and
10850 strategy inference works as expected.
10851 3.31.3 - 2017-10-06
10853 This release makes some small optimisations to our use of coverage that
10854 should reduce constant per-example overhead. This is probably only no‐
10855 ticeable on examples where the test itself is quite fast. On no-op
10856 tests that don't test anything you may see up to a fourfold speed in‐
10857 crease (which is still significantly slower than without coverage). On
10858 more realistic tests the speed up is likely to be less than that.
10859 3.31.2 - 2017-09-30
10861 This release fixes some formatting and small typos/grammar issues in
10862 the documentation, specifically the page docs/settings.rst, and the in‐
10863 line docs for the various settings.
10864 3.31.1 - 2017-09-30
10866 This release improves the handling of deadlines so that they act better
10867 with the shrinking process. This fixes issue #892.
10868 This involves two changes:
10870 1. The deadline is raised during the initial generation and shrinking,
10872 and then lowered to the set value for final replay. This restricts
10873 our attention to examples which exceed the deadline by a more sig‐
10874 nificant margin, which increases their reliability.
10875 2. When despite the above a test still becomes flaky because it is sig‐
10877 nificantly faster on rerun than it was on its first run, the error
10878 message is now more explicit about the nature of this problem, and
10879 includes both the initial test run time and the new test run time.
10880 In addition, this release also clarifies the documentation of the dead‐
10882 line setting slightly to be more explicit about where it applies.
10883 This work was funded by Smarkets.
10885 3.31.0 - 2017-09-29
10887 This release blocks installation of Hypothesis on Python 3.3, which
10888 reached its end of life date on 2017-09-29.
10889 This should not be of interest to anyone but downstream maintainers -
10891 if you are affected, migrate to a secure version of Python as soon as
10892 possible or at least seek commercial support.
10893 3.30.4 - 2017-09-27
10895 This release makes several changes:
10896 1. It significantly improves Hypothesis's ability to use coverage in‐
10898 formation to find interesting examples.
10899 2. It reduces the default max_examples setting from 200 to 100. This
10901 takes advantage of the improved algorithm meaning fewer examples are
10902 typically needed to get the same testing and is sufficiently better
10903 at covering interesting behaviour, and offsets some of the perfor‐
10904 mance problems of running under coverage.
10905 3. Hypothesis will always try to start its testing with an example that
10907 is near minimized.
10908 The new algorithm for 1 also makes some changes to Hypothesis's low
10910 level data generation which apply even with coverage turned off. They
10911 generally reduce the total amount of data generated, which should im‐
10912 prove test performance somewhat. Between this and 3 you should see a
10913 noticeable reduction in test runtime (how much so depends on your tests
10914 and how much example size affects their performance. On our benchmarks,
10915 where data generation dominates, we saw up to a factor of two perfor‐
10916 mance improvement, but it's unlikely to be that large.
10917 3.30.3 - 2017-09-25
10919 This release fixes some formatting and small typos/grammar issues in
10920 the documentation, specifically the page docs/details.rst, and some in‐
10921 line docs linked from there.
10922 3.30.2 - 2017-09-24
10924 This release changes Hypothesis's caching approach for functions in hy‐
10925 pothesis.strategies. Previously it would have cached extremely aggres‐
10926 sively and cache entries would never be evicted. Now it adopts a
10927 least-frequently used, least recently used key invalidation policy, and
10928 is somewhat more conservative about which strategies it caches.
10929 Workloads which create strategies based on dynamic values, e.g. by us‐
10931 ing flatmap or composite(), will use significantly less memory.
10932 3.30.1 - 2017-09-22
10934 This release fixes a bug where when running with the use_coverage=True
10935 setting inside an existing running instance of coverage, Hypothesis
10936 would frequently put files that the coveragerc excluded in the report
10937 for the enclosing coverage.
10938 3.30.0 - 2017-09-20
10940 This release introduces two new features:
10941 • When a test fails, either with a health check failure or a falsifying
10943 example, Hypothesis will print out a seed that led to that failure,
10944 if the test is not already running with a fixed seed. You can then
10945 recreate that failure using either the @seed decorator or (if you are
10946 running pytest) with --hypothesis-seed.
10947 • pytest users can specify a seed to use for @given based tests by
10949 passing the --hypothesis-seed command line argument.
10950 This work was funded by Smarkets.
10952 3.29.0 - 2017-09-19
10954 This release makes Hypothesis coverage aware. Hypothesis now runs all
10955 test bodies under coverage, and uses this information to guide its
10956 testing.
10957 The use_coverage setting can be used to disable this behaviour if you
10959 want to test code that is sensitive to coverage being enabled (either
10960 because of performance or interaction with the trace function).
10961 The main benefits of this feature are:
10963 • Hypothesis now observes when examples it discovers cover particular
10965 lines or branches and stores them in the database for later.
10966 • Hypothesis will make some use of this information to guide its explo‐
10968 ration of the search space and improve the examples it finds (this is
10969 currently used only very lightly and will likely improve signifi‐
10970 cantly in future releases).
10971 This also has the following side-effects:
10973 • Hypothesis now has an install time dependency on the coverage pack‐
10975 age.
10976 • Tests that are already running Hypothesis under coverage will likely
10978 get faster.
10979 • Tests that are not running under coverage now run their test bodies
10981 under coverage by default.
10982 This feature is only partially supported under pypy. It is signifi‐
10984 cantly slower than on CPython and is turned off by default as a result,
10985 but it should still work correctly if you want to use it.
10986 3.28.3 - 2017-09-18
10988 This release is an internal change that affects how Hypothesis handles
10989 calculating certain properties of strategies.
10990 The primary effect of this is that it fixes a bug where use of de‐
10992 ferred() could sometimes trigger an internal assertion error. However
10993 the fix for this bug involved some moderately deep changes to how Hy‐
10994 pothesis handles certain constructs so you may notice some additional
10995 knock-on effects.
10996 In particular the way Hypothesis handles drawing data from strategies
10998 that cannot generate any values has changed to bail out sooner than it
10999 previously did. This may speed up certain tests, but it is unlikely to
11000 make much of a difference in practice for tests that were not already
11001 failing with Unsatisfiable.
11002 3.28.2 - 2017-09-18
11004 This is a patch release that fixes a bug in the hypothesis.extra.pandas
11005 documentation where it incorrectly referred to column() instead of col‐
11006 umns().
11007 3.28.1 - 2017-09-16
11009 This is a refactoring release. It moves a number of internal uses of
11010 namedtuple() over to using attrs based classes, and removes a couple of
11011 internal namedtuple classes that were no longer in use.
11012 It should have no user visible impact.
11014 3.28.0 - 2017-09-15
11016 This release adds support for testing pandas via the hypothesis.ex‐
11017 tra.pandas module.
11018 It also adds a dependency on attrs.
11020 This work was funded by Stripe.
11022 3.27.1 - 2017-09-14
11024 This release fixes some formatting and broken cross-references in the
11025 documentation, which includes editing docstrings - and thus a patch re‐
11026 lease.
11027 3.27.0 - 2017-09-13
11029 This release introduces a deadline setting to Hypothesis.
11030 When set this turns slow tests into errors. By default it is unset but
11032 will warn if you exceed 200ms, which will become the default value in a
11033 future release.
11034 This work was funded by Smarkets.
11036 3.26.0 - 2017-09-12
11038 Hypothesis now emits deprecation warnings if you are using the legacy
11039 SQLite example database format, or the tool for merging them. These
11040 were already documented as deprecated, so this doesn't change their
11041 deprecation status, only that we warn about it.
11042 3.25.1 - 2017-09-12
11044 This release fixes a bug with generating numpy datetime and timedelta
11045 types: When inferring the strategy from the dtype, datetime and
11046 timedelta dtypes with sub-second precision would always produce exam‐
11047 ples with one second resolution. Inferring a strategy from a time
11048 dtype will now always produce example with the same precision.
11049 3.25.0 - 2017-09-12
11051 This release changes how Hypothesis shrinks and replays examples to
11052 take into account that it can encounter new bugs while shrinking the
11053 bug it originally found. Previously it would end up replacing the orig‐
11054 inally found bug with the new bug and show you only that one. Now it is
11055 (often) able to recognise when two bugs are distinct and when it finds
11056 more than one will show both.
11057 3.24.2 - 2017-09-11
11059 This release removes the (purely internal and no longer useful) strat‐
11060 egy_test_suite function and the corresponding strategytests module.
11061 3.24.1 - 2017-09-06
11063 This release improves the reduction of examples involving floating
11064 point numbers to produce more human readable examples.
11065 It also has some general purpose changes to the way the minimizer works
11067 internally, which may see some improvement in quality and slow down of
11068 test case reduction in cases that have nothing to do with floating
11069 point numbers.
11070 3.24.0 - 2017-09-05
11072 Hypothesis now emits deprecation warnings if you use some_strategy.ex‐
11073 ample() inside a test function or strategy definition (this was never
11074 intended to be supported, but is sufficiently widespread that it war‐
11075 rants a deprecation path).
11076 3.23.3 - 2017-09-05
11078 This is a bugfix release for decimals() with the places argument.
11079 • No longer fails health checks (issue #725, due to internal filtering)
11081 • Specifying a min_value and max_value without any decimals with places
11083 places between them gives a more useful error message.
11084 • Works for any valid arguments, regardless of the decimal precision
11086 context.
11087 3.23.2 - 2017-09-01
11089 This is a small refactoring release that removes a now-unused parameter
11090 to an internal API. It shouldn't have any user visible effect.
11091 3.23.1 - 2017-09-01
11093 Hypothesis no longer propagates the dynamic scope of settings into
11094 strategy definitions.
11095 This release is a small change to something that was never part of the
11097 public API and you will almost certainly not notice any effect unless
11098 you're doing something surprising, but for example the following code
11099 will now give a different answer in some circumstances:
11100 import hypothesis.strategies as st
11102 from hypothesis import settings
11103 CURRENT_SETTINGS = st.builds(lambda: settings.default)
11105 (We don't actually encourage you writing code like this)
11107 Previously this would have generated the settings that were in effect
11109 at the point of definition of CURRENT_SETTINGS. Now it will generate
11110 the settings that are used for the current test.
11111 It is very unlikely to be significant enough to be visible, but you may
11113 also notice a small performance improvement.
11114 3.23.0 - 2017-08-31
11116 This release adds a unique argument to arrays() which behaves the same
11117 ways as the corresponding one for lists(), requiring all of the ele‐
11118 ments in the generated array to be distinct.
11119 3.22.2 - 2017-08-29
11121 This release fixes an issue where Hypothesis would raise a TypeError
11122 when using the datetime-related strategies if running with PYTHONOPTI‐
11123 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
11124 3.22.1 - 2017-08-28
11126 Hypothesis now transparently handles problems with an internal unicode
11127 cache, including file truncation or read-only filesystems (issue #767).
11128 Thanks to Sam Hames for the patch.
11129 3.22.0 - 2017-08-26
11131 This release provides what should be a substantial performance improve‐
11132 ment to numpy arrays generated using provided numpy support, and adds a
11133 new fill_value argument to arrays() to control this behaviour.
11134 This work was funded by Stripe.
11136 3.21.3 - 2017-08-26
11138 This release fixes some extremely specific circumstances that probably
11139 have never occurred in the wild where users of deferred() might have
11140 seen a python:RuntimeError from too much recursion, usually in cases
11141 where no valid example could have been generated anyway.
11142 3.21.2 - 2017-08-25
11144 This release fixes some minor bugs in argument validation:
11145 • hypothesis.extra.numpy dtype strategies would raise an internal
11147 error instead of an InvalidArgument exception when passed an in‐
11148 valid endianness specification.
11149 • fractions() would raise an internal error instead of an InvalidAr‐
11151 gument if passed float("nan") as one of its bounds.
11152 • The error message for passing float("nan") as a bound to various
11154 strategies has been improved.
11155 • Various bound arguments will now raise InvalidArgument in cases
11157 where they would previously have raised an internal TypeError or
11158 ValueError from the relevant conversion function.
11159 • streaming() would not have emitted a deprecation warning when
11161 called with an invalid argument.
11162 3.21.1 - 2017-08-24
11164 This release fixes a bug where test failures that were the result of an
11165 @example would print an extra stack trace before re-raising the excep‐
11166 tion.
11167 3.21.0 - 2017-08-23
11169 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
11170 sis's deprecation warnings into errors. Similar functionality can be
11171 achieved by using simplefilter('error', HypothesisDeprecationWarning).
11172 3.20.0 - 2017-08-22
11174 This release renames the relevant arguments on the datetimes(),
11175 dates(), times(), and timedeltas() strategies to min_value and
11176 max_value, to make them consistent with the other strategies in the
11177 module.
11178 The old argument names are still supported but will emit a deprecation
11180 warning when used explicitly as keyword arguments. Arguments passed po‐
11181 sitionally will go to the new argument names and are not deprecated.
11182 3.19.3 - 2017-08-22
11184 This release provides a major overhaul to the internals of how Hypothe‐
11185 sis handles shrinking.
11186 This should mostly be visible in terms of getting better examples for
11188 tests which make heavy use of composite(), data() or flatmap where the
11189 data drawn depends a lot on previous choices, especially where size pa‐
11190 rameters are affected. Previously Hypothesis would have struggled to
11191 reliably produce good examples here. Now it should do much better. Per‐
11192 formance should also be better for examples with a non-zero min_size.
11193 You may see slight changes to example generation (e.g. improved example
11195 diversity) as a result of related changes to internals, but they are
11196 unlikely to be significant enough to notice.
11197 3.19.2 - 2017-08-21
11199 This release fixes two bugs in hypothesis.extra.numpy:
11200 • unicode_string_dtypes() didn't work at all due to an incorrect dtype
11202 specifier. Now it does.
11203 • Various impossible conditions would have been accepted but would er‐
11205 ror when they fail to produced any example. Now they raise an ex‐
11206 plicit InvalidArgument error.
11207 3.19.1 - 2017-08-21
11209 This is a bugfix release for issue #739, where bounds for fractions()
11210 or floating-point decimals() were not properly converted to integers
11211 before passing them to the integers strategy. This excluded some val‐
11212 ues that should have been possible, and could trigger internal errors
11213 if the bounds lay between adjacent integers.
11214 You can now bound fractions() with two arbitrarily close fractions.
11216 It is now an explicit error to supply a min_value, max_value, and
11218 max_denominator to fractions() where the value bounds do not include a
11219 fraction with denominator at most max_denominator.
11220 3.19.0 - 2017-08-20
11222 This release adds the from_regex() strategy, which generates strings
11223 that contain a match of a regular expression.
11224 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
11226 then helping to upstream it! (issue #662)
11227 3.18.5 - 2017-08-18
11229 This is a bugfix release for integers(). Previously the strategy would
11230 hit an internal assertion if passed non-integer bounds for min_value
11231 and max_value that had no integers between them. The strategy now
11232 raises InvalidArgument instead.
11233 3.18.4 - 2017-08-18
11235 Release to fix a bug where mocks can be used as test runners under cer‐
11236 tain conditions. Specifically, if a mock is injected into a test via
11237 pytest fixtures or patch decorators, and that mock is the first argu‐
11238 ment in the list, hypothesis will think it represents self and turns
11239 the mock into a test runner. If this happens, the affected test always
11240 passes because the mock is executed instead of the test body. Some‐
11241 times, it will also fail health checks.
11242 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
11244 for this bug fix.
11245 3.18.3 - 2017-08-17
11247 This release should improve the performance of some tests which experi‐
11248 enced a slow down as a result of the 3.13.0 release.
11249 Tests most likely to benefit from this are ones that make extensive use
11251 of min_size parameters, but others may see some improvement as well.
11252 3.18.2 - 2017-08-16
11254 This release fixes a bug introduced in 3.18.0. If the arguments
11255 whitelist_characters and blacklist_characters to characters() contained
11256 overlapping elements, then an InvalidArgument exception would be
11257 raised.
11258 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
11260 3.18.1 - 2017-08-14
11262 This is a bug fix release to fix issue #780, where sets() and similar
11263 would trigger health check errors if their element strategy could only
11264 produce one element (e.g. if it was just()).
11265 3.18.0 - 2017-08-13
11267 This is a feature release:
11268 • characters() now accepts whitelist_characters, particular characters
11270 which will be added to those it produces. (issue #668)
11271 • A bug fix for the internal function _union_interval_lists(), and a
11273 rename to _union_intervals(). It now correctly handles all cases
11274 where intervals overlap, and it always returns the result as a tuple
11275 for tuples.
11276 Thanks to Alex Willmer for these.
11278 3.17.0 - 2017-08-07
11280 This release documents the previously undocumented phases feature, mak‐
11281 ing it part of the public API. It also updates how the example database
11282 is used. Principally:
11283 • A Phases.reuse argument will now correctly control whether examples
11285 from the database are run (it previously did exactly the wrong thing
11286 and controlled whether examples would be saved).
11287 • Hypothesis will no longer try to rerun all previously failing exam‐
11289 ples. Instead it will replay the smallest previously failing example
11290 and a selection of other examples that are likely to trigger any
11291 other bugs that will found. This prevents a previous failure from
11292 dominating your tests unnecessarily.
11293 • As a result of the previous change, Hypothesis will be slower about
11295 clearing out old examples from the database that are no longer fail‐
11296 ing (because it can only clear out ones that it actually runs).
11297 3.16.1 - 2017-08-07
11299 This release makes an implementation change to how Hypothesis handles
11300 certain internal constructs.
11301 The main effect you should see is improvement to the behaviour and per‐
11303 formance of collection types, especially ones with a min_size parame‐
11304 ter. Many cases that would previously fail due to being unable to gen‐
11305 erate enough valid examples will now succeed, and other cases should
11306 run slightly faster.
11307 3.16.0 - 2017-08-04
11309 This release introduces a deprecation of the timeout feature. This re‐
11310 sults in the following changes:
11311 • Creating a settings object with an explicit timeout will emit a dep‐
11313 recation warning.
11314 • If your test stops because it hits the timeout (and has not found a
11316 bug) then it will emit a deprecation warning.
11317 • There is a new value unlimited which you can import from hypothesis.
11319 settings(timeout=unlimited) will not cause a deprecation warning.
11320 • There is a new health check, hung_test, which will trigger after a
11322 test has been running for five minutes if it is not suppressed.
11323 3.15.0 - 2017-08-04
11325 This release deprecates two strategies, choices() and streaming().
11326 Both of these are somewhat confusing to use and are entirely redundant
11328 since the introduction of the data() strategy for interactive drawing
11329 in tests, and their use should be replaced with direct use of data()
11330 instead.
11331 3.14.2 - 2017-08-03
11333 This fixes a bug where Hypothesis would not work correctly on Python
11334 2.7 if you had the python:typing module backport installed.
11335 3.14.1 - 2017-08-02
11337 This raises the maximum depth at which Hypothesis starts cutting off
11338 data generation to a more reasonable value which it is harder to hit by
11339 accident.
11340 This resolves (issue #751), in which some examples which previously
11342 worked would start timing out, but it will also likely improve the data
11343 generation quality for complex data types.
11344 3.14.0 - 2017-07-23
11346 Hypothesis now understands inline type annotations (issue #293):
11347 • If the target of builds() has type annotations, a default strategy
11349 for missing required arguments is selected based on the type.
11350 Type-based strategy selection will only override a default if you
11351 pass hypothesis.infer as a keyword argument.
11352 • If @given wraps a function with type annotations, you can pass infer
11354 as a keyword argument and the appropriate strategy will be substi‐
11355 tuted.
11356 • You can check what strategy will be inferred for a type with the new
11358 from_type() function.
11359 • register_type_strategy() teaches Hypothesis which strategy to infer
11361 for custom or unknown types. You can provide a strategy, or for more
11362 complex cases a function which takes the type and returns a strategy.
11363 3.13.1 - 2017-07-20
11365 This is a bug fix release for issue #514 - Hypothesis would continue
11366 running examples after a SkipTest exception was raised, including
11367 printing a falsifying example. Skip exceptions from the standard
11368 python:unittest module, and pytest, nose, or unittest2 modules now
11369 abort the test immediately without printing output.
11370 3.13.0 - 2017-07-16
11372 This release has two major aspects to it: The first is the introduction
11373 of deferred(), which allows more natural definition of recursive (in‐
11374 cluding mutually recursive) strategies.
11375 The second is a number of engine changes designed to support this sort
11377 of strategy better. These should have a knock-on effect of also improv‐
11378 ing the performance of any existing strategies that currently generate
11379 a lot of data or involve heavy nesting by reducing their typical exam‐
11380 ple size.
11381 3.12.0 - 2017-07-07
11383 This release makes some major internal changes to how Hypothesis repre‐
11384 sents data internally, as a prelude to some major engine changes that
11385 should improve data quality. There are no API changes, but it's a sig‐
11386 nificant enough internal change that a minor version bump seemed war‐
11387 ranted.
11388 User facing impact should be fairly mild, but includes:
11390 • All existing examples in the database will probably be invalidated.
11392 Hypothesis handles this automatically, so you don't need to do any‐
11393 thing, but if you see all your examples disappear that's why.
11394 • Almost all data distributions have changed significantly. Possibly
11396 for the better, possibly for the worse. This may result in new bugs
11397 being found, but it may also result in Hypothesis being unable to
11398 find bugs it previously did.
11399 • Data generation may be somewhat faster if your existing bottleneck
11401 was in draw_bytes (which is often the case for large examples).
11402 • Shrinking will probably be slower, possibly significantly.
11404 If you notice any effects you consider to be a significant regression,
11406 please open an issue about them.
11407 3.11.6 - 2017-06-19
11409 This release involves no functionality changes, but is the first to
11410 ship wheels as well as an sdist.
11411 3.11.5 - 2017-06-18
11413 This release provides a performance improvement to shrinking. For cases
11414 where there is some non-trivial "boundary" value (e.g. the bug happens
11415 for all values greater than some other value), shrinking should now be
11416 substantially faster. Other types of bug will likely see improvements
11417 too.
11418 This may also result in some changes to the quality of the final exam‐
11420 ples - it may sometimes be better, but is more likely to get slightly
11421 worse in some edge cases. If you see any examples where this happens in
11422 practice, please report them.
11423 3.11.4 - 2017-06-17
11425 This is a bugfix release: Hypothesis now prints explicit examples when
11426 running in verbose mode. (issue #313)
11427 3.11.3 - 2017-06-11
11429 This is a bugfix release: Hypothesis no longer emits a warning if you
11430 try to use sampled_from() with python:collections.OrderedDict. (issue
11431 #688)
11432 3.11.2 - 2017-06-10
11434 This is a documentation release. Several outdated snippets have been
11435 updated or removed, and many cross-references are now hyperlinks.
11436 3.11.1 - 2017-05-28
11438 This is a minor ergonomics release. Tracebacks shown by pytest no
11439 longer include Hypothesis internals for test functions decorated with
11440 @given.
11441 3.11.0 - 2017-05-23
11443 This is a feature release, adding datetime-related strategies to the
11444 core strategies.
11445 timezones() allows you to sample pytz timezones from the Olsen data‐
11447 base. Use directly in a recipe for tz-aware datetimes, or compose with
11448 none() to allow a mix of aware and naive output.
11449 The new dates(), times(), datetimes(), and timedeltas() strategies are
11451 all constrained by objects of their type. This means that you can gen‐
11452 erate dates bounded by a single day (i.e. a single date), or datetimes
11453 constrained to the microsecond.
11454 times() and datetimes() take an optional timezones= argument, which de‐
11456 faults to none() for naive times. You can use our extra strategy based
11457 on pytz, or roll your own timezones strategy with dateutil or even the
11458 standard library.
11459 The old dates, times, and datetimes strategies in hypothesis.ex‐
11461 tra.datetimes are deprecated in favor of the new core strategies, which
11462 are more flexible and have no dependencies.
11463 3.10.0 - 2017-05-22
11465 Hypothesis now uses python:inspect.getfullargspec() internally. On
11466 Python 2, there are no visible changes.
11467 On Python 3 @given and @composite now preserve PEP 3107 annotations on
11469 the decorated function. Keyword-only arguments are now either handled
11470 correctly (e.g. @composite), or caught in validation instead of
11471 silently discarded or raising an unrelated error later (e.g. @given).
11472 3.9.1 - 2017-05-22
11474 This is a bugfix release: the default field mapping for a DateTimeField
11475 in the Django extra now respects the USE_TZ setting when choosing a
11476 strategy.
11477 3.9.0 - 2017-05-19
11479 This is feature release, expanding the capabilities of the decimals()
11480 strategy.
11481 • The new (optional) places argument allows you to generate decimals
11483 with a certain number of places (e.g. cents, thousandths, satoshis).
11484 • If allow_infinity is None, setting min_bound no longer excludes posi‐
11486 tive infinity and setting max_value no longer excludes negative in‐
11487 finity.
11488 • All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
11490 True, or if allow_nan is None and min_value or max_value is None.
11491 • min_value and max_value may be given as decimal strings, e.g.
11493 "1.234".
11494 3.8.5 - 2017-05-16
11496 Hypothesis now imports python:sqlite3 when a SQLite database is used,
11497 rather than at module load, improving compatibility with Python imple‐
11498 mentations compiled without SQLite support (such as BSD or Jython).
11499 3.8.4 - 2017-05-16
11501 This is a compatibility bugfix release. sampled_from() no longer
11502 raises a deprecation warning when sampling from an python:enum.Enum, as
11503 all enums have a reliable iteration order.
11504 3.8.3 - 2017-05-09
11506 This release removes a version check for older versions of pytest when
11507 using the Hypothesis pytest plugin. The pytest plugin will now run un‐
11508 conditionally on all versions of pytest. This breaks compatibility with
11509 any version of pytest prior to 2.7.0 (which is more than two years
11510 old).
11511 The primary reason for this change is that the version check was a fre‐
11513 quent source of breakage when pytest change their versioning scheme. If
11514 you are not working on pytest itself and are not running a very old
11515 version of it, this release probably doesn't affect you.
11516 3.8.2 - 2017-04-26
11518 This is a code reorganisation release that moves some internal test
11519 helpers out of the main source tree so as to not have changes to them
11520 trigger releases in future.
11521 3.8.1 - 2017-04-26
11523 This is a documentation release. Almost all code examples are now
11524 doctests checked in CI, eliminating stale examples.
11525 3.8.0 - 2017-04-23
11527 This is a feature release, adding the iterables() strategy, equivalent
11528 to lists(...).map(iter) but with a much more useful repr. You can use
11529 this strategy to check that code doesn't accidentally depend on se‐
11530 quence properties such as indexing support or repeated iteration.
11531 3.7.4 - 2017-04-22
11533 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
11534 ture in the same test could cause the test to fail to fill the argu‐
11535 ments, and throw a TypeError.
11536 3.7.3 - 2017-04-21
11538 This release should include no user visible changes and is purely a
11539 refactoring release. This modularises the behaviour of the core given()
11540 function, breaking it up into smaller and more accessible parts, but
11541 its actual behaviour should remain unchanged.
11542 3.7.2 - 2017-04-21
11544 This reverts an undocumented change in 3.7.1 which broke installation
11545 on debian stable: The specifier for the hypothesis[django] extra_re‐
11546 quires had introduced a wild card, which was not supported on the de‐
11547 fault version of pip.
11548 3.7.1 - 2017-04-21
11550 This is a bug fix and internal improvements release.
11551 • In particular Hypothesis now tracks a tree of where it has already
11553 explored. This allows it to avoid some classes of duplicate exam‐
11554 ples, and significantly improves the performance of shrinking failing
11555 examples by allowing it to skip some shrinks that it can determine
11556 can't possibly work.
11557 • Hypothesis will no longer seed the global random arbitrarily unless
11559 you have asked it to using random_module()
11560 • Shrinking would previously have not worked correctly in some special
11562 cases on Python 2, and would have resulted in suboptimal examples.
11563 3.7.0 - 2017-03-20
11565 This is a feature release.
11566 New features:
11568 • Rule based stateful testing now has an @invariant decorator that
11570 specifies methods that are run after init and after every step, al‐
11571 lowing you to encode properties that should be true at all times.
11572 Thanks to Tom Prince for this feature.
11573 • The decimals() strategy now supports allow_nan and allow_infinity
11575 flags.
11576 • There are significantly more strategies available for numpy, includ‐
11578 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
11579 for this feature.
11580 • When using the data() strategy you can now add a label as an argument
11582 to draw(), which will be printed along with the value when an example
11583 fails. Thanks to Peter Inglesby for this feature.
11584 Bug fixes:
11586 • Bug fix: composite() now preserves functions' docstrings.
11588 • The build is now reproducible and doesn't depend on the path you
11590 build it from. Thanks to Chris Lamb for this feature.
11591 • numpy strategies for the void data type did not work correctly.
11593 Thanks to Zac Hatfield Dodds for this fix.
11594 There have also been a number of performance optimizations:
11596 • The permutations() strategy is now significantly faster to use for
11598 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
11599 • Shrinking of failing test cases should have got significantly faster
11601 in some circumstances where it was previously struggling for a long
11602 time.
11603 • Example generation now involves less indirection, which results in a
11605 small speedup in some cases (small enough that you won't really no‐
11606 tice it except in pathological cases).
11607 3.6.1 - 2016-12-20
11609 This release fixes a dependency problem and makes some small behind the
11610 scenes improvements.
11611 • The fake-factory dependency was renamed to faker. If you were depend‐
11613 ing on it through hypothesis[django] or hypothesis[fake-factory]
11614 without pinning it yourself then it would have failed to install
11615 properly. This release changes it so that hypothesis[fakefactory]
11616 (which can now also be installed as hypothesis[faker]) will install
11617 the renamed faker package instead.
11618 • This release also removed the dependency of hypothesis[django] on hy‐
11620 pothesis[fakefactory] - it was only being used for emails. These now
11621 use a custom strategy that isn't from fakefactory. As a result you
11622 should also see performance improvements of tests which generated
11623 User objects or other things with email fields, as well as better
11624 shrinking of email addresses.
11625 • The distribution of code using nested calls to one_of() or the | op‐
11627 erator for combining strategies has been improved, as branches are
11628 now flattened to give a more uniform distribution.
11629 • Examples using composite() or .flatmap should now shrink better. In
11631 particular this will affect things which work by first generating a
11632 length and then generating that many items, which have historically
11633 not shrunk very well.
11634 3.6.0 - 2016-10-31
11636 This release reverts Hypothesis to its old pretty printing of lambda
11637 functions based on attempting to extract the source code rather than
11638 decompile the bytecode. This is unfortunately slightly inferior in
11639 some cases and may result in you occasionally seeing things like lambda
11640 x: <unknown> in statistics reports and strategy reprs.
11641 This removes the dependencies on uncompyle6, xdis and spark-parser.
11643 The reason for this is that the new functionality was based on un‐
11645 compyle6, which turns out to introduce a hidden GPLed dependency - it
11646 in turn depended on xdis, and although the library was licensed under
11647 the MIT license, it contained some GPL licensed source code and thus
11648 should have been released under the GPL.
11649 My interpretation is that Hypothesis itself was never in violation of
11651 the GPL (because the license it is under, the Mozilla Public License
11652 v2, is fully compatible with being included in a GPL licensed work),
11653 but I have not consulted a lawyer on the subject. Regardless of the an‐
11654 swer to this question, adding a GPLed dependency will likely cause a
11655 lot of users of Hypothesis to inadvertently be in violation of the GPL.
11656 As a result, if you are running Hypothesis 3.5.x you really should up‐
11658 grade to this release immediately.
11659 3.5.3 - 2016-10-05
11661 This is a bug fix release.
11662 Bugs fixed:
11664 • If the same test was running concurrently in two processes and there
11666 were examples already in the test database which no longer failed,
11667 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
11668 Python 2) because an example it was trying to read was deleted before
11669 it was read. (issue #372).
11670 • Drawing from an integers() strategy with both a min_value and a
11672 max_value would reject too many examples needlessly. Now it repeat‐
11673 edly redraws until satisfied. (pull request #366. Thanks to Calen
11674 Pennington for the contribution).
11675 3.5.2 - 2016-09-24
11677 This is a bug fix release.
11678 • The Hypothesis pytest plugin broke pytest support for doctests. Now
11680 it doesn't.
11681 3.5.1 - 2016-09-23
11683 This is a bug fix release.
11684 • Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
11686 bytes and unicode.
11687 • python:unittest.TestCase tests would not have shown up in the new
11689 statistics mode. Now they do.
11690 • Similarly, stateful tests would not have shown up in statistics and
11692 now they do.
11693 • Statistics now print with pytest node IDs (the names you'd get in
11695 pytest verbose mode).
11696 3.5.0 - 2016-09-22
11698 This is a feature release.
11699 • fractions() and decimals() strategies now support min_value and
11701 max_value parameters. Thanks go to Anne Mulhern for the development
11702 of this feature.
11703 • The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
11705 tics parameter that gives detailed statistics about the tests that
11706 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
11707 funding the development of this feature.
11708 • There is a new event() function that can be used to add custom sta‐
11710 tistics.
11711 Additionally there have been some minor bug fixes:
11713 • In some cases Hypothesis should produce fewer duplicate examples
11715 (this will mostly only affect cases with a single parameter).
11716 • pytest command line parameters are now under an option group for Hy‐
11718 pothesis (thanks to David Keijser for fixing this)
11719 • Hypothesis would previously error if you used PEP 3107 function anno‐
11721 tations on your tests under Python 3.4.
11722 • The repr of many strategies using lambdas has been improved to in‐
11724 clude the lambda body (this was previously supported in many but not
11725 all cases).
11726 3.4.2 - 2016-07-13
11728 This is a bug fix release, fixing a number of problems with the set‐
11729 tings system:
11730 • Test functions defined using @given can now be called from other
11732 threads (issue #337)
11733 • Attempting to delete a settings property would previously have
11735 silently done the wrong thing. Now it raises an AttributeError.
11736 • Creating a settings object with a custom database_file parameter was
11738 silently getting ignored and the default was being used instead. Now
11739 it's not.
11740 3.4.1 - 2016-07-07
11742 This is a bug fix release for a single bug:
11743 • On Windows when running two Hypothesis processes in parallel (e.g.
11745 using pytest-xdist) they could race with each other and one would
11746 raise an exception due to the non-atomic nature of file renaming on
11747 Windows and the fact that you can't rename over an existing file.
11748 This is now fixed.
11749 3.4.0 - 2016-05-27
11751 This release is entirely provided by Lucas Wiman:
11752 Strategies constructed by the Django extra will now respect much more
11754 of Django's validations out of the box. Wherever possible,
11755 full_clean() should succeed.
11756 In particular:
11758 • The max_length, blank and choices kwargs are now respected.
11760 • Add support for DecimalField.
11762 • If a field includes validators, the list of validators are used to
11764 filter the field strategy.
11765 3.3.0 - 2016-05-27
11767 This release went wrong and is functionally equivalent to 3.2.0. Ignore
11768 it.
11769 3.2.0 - 2016-05-19
11771 This is a small single-feature release:
11772 • All tests using @given now fix the global random seed. This removes
11774 the health check for that. If a non-zero seed is required for the fi‐
11775 nal falsifying example, it will be reported. Otherwise Hypothesis
11776 will assume randomization was not a significant factor for the test
11777 and be silent on the subject. If you use random_module() this will
11778 continue to work and will always display the seed.
11779 3.1.3 - 2016-05-01
11781 Single bug fix release
11782 • Another charmap problem. In 3.1.2 text() and characters() would break
11784 on systems which had /tmp mounted on a different partition than the
11785 Hypothesis storage directory (usually in home). This fixes that.
11786 3.1.2 - 2016-04-30
11788 Single bug fix release:
11789 • Anything which used a text() or characters() strategy was broken on
11791 Windows and I hadn't updated appveyor to use the new repository loca‐
11792 tion so I didn't notice. This is now fixed and windows support should
11793 work correctly.
11794 3.1.1 - 2016-04-29
11796 Minor bug fix release.
11797 • Fix concurrency issue when running tests that use text() from multi‐
11799 ple processes at once (issue #302, thanks to Alex Chan).
11800 • Improve performance of code using lists() with max_size (thanks to
11802 Cristi Cobzarenco).
11803 • Fix install on Python 2 with ancient versions of pip so that it in‐
11805 stalls the enum34 backport (thanks to Donald Stufft for telling me
11806 how to do this).
11807 • Remove duplicated __all__ exports from hypothesis.strategies (thanks
11809 to Piët Delport).
11810 • Update headers to point to new repository location.
11812 • Allow use of strategies that can't be used in find() (e.g. choices())
11814 in stateful testing.
11815 3.1.0 - 2016-03-06
11817 • Add a nothing() strategy that never successfully generates values.
11818 • sampled_from() and one_of() can both now be called with an empty ar‐
11820 gument list, in which case they also never generate any values.
11821 • one_of() may now be called with a single argument that is a collec‐
11823 tion of strategies as well as as varargs.
11824 • Add a runner() strategy which returns the instance of the current
11826 test object if there is one.
11827 • 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
11829 can be used as such.
11830 • Tests using RuleBasedStateMachine should now shrink significantly
11832 better.
11833 • Hypothesis now uses a pretty-printing library internally, compatible
11835 with IPython's pretty printing protocol (actually using the same
11836 code). This may improve the quality of output in some cases.
11837 • Add a 'phases' setting that allows more fine grained control over
11839 which parts of the process Hypothesis runs
11840 • Add a suppress_health_check setting which allows you to turn off spe‐
11842 cific health checks in a fine grained manner.
11843 • Fix a bug where lists of non fixed size would always draw one more
11845 element than they included. This mostly didn't matter, but if would
11846 cause problems with empty strategies or ones with side effects.
11847 • Add a mechanism to the Django model generator to allow you to explic‐
11849 itly request the default value (thanks to Jeremy Thurgood for this
11850 one).
11851 3.0.5 - 2016-02-25
11853 • Fix a bug where Hypothesis would now error on pytest development ver‐
11854 sions.
11855 3.0.4 - 2016-02-24
11857 • Fix a bug where Hypothesis would error when running on Python 2.7.3
11858 or earlier because it was trying to pass a python:bytearray object to
11859 python:struct.unpack() (which is only supported since 2.7.4).
11860 3.0.3 - 2016-02-23
11862 • Fix version parsing of pytest to work with pytest release candidates
11863 • More general handling of the health check problem where things could
11865 fail because of a cache miss - now one "free" example is generated
11866 before the start of the health check run.
11867 3.0.2 - 2016-02-18
11869 • Under certain circumstances, strategies involving text() buried in‐
11870 side some other strategy (e.g. text().filter(...) or recur‐
11871 sive(text(), ...)) would cause a test to fail its health checks the
11872 first time it ran. This was caused by having to compute some related
11873 data and cache it to disk. On travis or anywhere else where the .hy‐
11874 pothesis directory was recreated this would have caused the tests to
11875 fail their health check on every run. This is now fixed for all the
11876 known cases, although there could be others lurking.
11877 3.0.1 - 2016-02-18
11879 • Fix a case where it was possible to trigger an "Unreachable" asser‐
11880 tion when running certain flaky stateful tests.
11881 • Improve shrinking of large stateful tests by eliminating a case where
11883 it was hard to delete early steps.
11884 • Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
11886 cantly by provide a custom implementation for fixed size blocks that
11887 can bypass a lot of machinery.
11888 • Set default home directory based on the current working directory at
11890 the point Hypothesis is imported, not whenever the function first
11891 happens to be called.
11892 3.0.0 - 2016-02-17
11894 Codename: This really should have been 2.1.
11895 Externally this looks like a very small release. It has one small
11897 breaking change that probably doesn't affect anyone at all (some behav‐
11898 iour that never really worked correctly is now outright forbidden) but
11899 necessitated a major version bump and one visible new feature.
11900 Internally this is a complete rewrite. Almost nothing other than the
11902 public API is the same.
11903 New features:
11905 • Addition of data() strategy which allows you to draw arbitrary data
11907 interactively within the test.
11908 • New "exploded" database format which allows you to more easily check
11910 the example database into a source repository while supporting merg‐
11911 ing.
11912 • Better management of how examples are saved in the database.
11914 • Health checks will now raise as errors when they fail. It was too
11916 easy to have the warnings be swallowed entirely.
11917 New limitations:
11919 • choices() and streaming() strategies may no longer be used with
11921 find(). Neither may data() (this is the change that necessitated a
11922 major version bump).
11923 Feature removal:
11925 • The ForkingTestCase executor has gone away. It may return in some
11927 more working form at a later date.
11928 Performance improvements:
11930 • A new model which allows flatmap, composite strategies and stateful
11932 testing to perform much better. They should also be more reliable.
11933 • Filtering may in some circumstances have improved significantly. This
11935 will help especially in cases where you have lots of values with in‐
11936 dividual filters on them, such as lists(x.filter(...)).
11937 • Modest performance improvements to the general test runner by avoid‐
11939 ing expensive operations
11940 In general your tests should have got faster. If they've instead got
11942 significantly slower, I'm interested in hearing about it.
11943 Data distribution:
11945 The data distribution should have changed significantly. This may un‐
11947 cover bugs the previous version missed. It may also miss bugs the pre‐
11948 vious version could have uncovered. Hypothesis is now producing less
11949 strongly correlated data than it used to, but the correlations are ex‐
11950 tended over more of the structure.
11951 Shrinking:
11953 Shrinking quality should have improved. In particular Hypothesis can
11955 now perform simultaneous shrinking of separate examples within a single
11956 test (previously it was only able to do this for elements of a single
11957 collection). In some cases performance will have improved, in some
11958 cases it will have got worse but generally shouldn't have by much.
11959 Older versions
11961 2.0.0 - 2016-01-10
11962 Codename: A new beginning
11963 This release cleans up all of the legacy that accrued in the course of
11965 Hypothesis 1.0. These are mostly things that were emitting deprecation
11966 warnings in 1.19.0, but there were a few additional changes.
11967 In particular:
11969 • non-strategy values will no longer be converted to strategies when
11971 used in given or find.
11972 • FailedHealthCheck is now an error and not a warning.
11974 • Handling of non-ascii reprs in user types have been simplified by us‐
11976 ing raw strings in more places in Python 2.
11977 • given no longer allows mixing positional and keyword arguments.
11979 • given no longer works with functions with defaults.
11981 • given no longer turns provided arguments into defaults - they will
11983 not appear in the argspec at all.
11984 • the basic() strategy no longer exists.
11986 • the n_ary_tree strategy no longer exists.
11988 • the average_list_length setting no longer exists. Note: If you're us‐
11990 ing using recursive() this will cause you a significant slow down.
11991 You should pass explicit average_size parameters to collections in
11992 recursive calls.
11993 • @rule can no longer be applied to the same method twice.
11995 • Python 2.6 and 3.3 are no longer officially supported, although in
11997 practice they still work fine.
11998 This also includes two non-deprecation changes:
12000 • given's keyword arguments no longer have to be the rightmost argu‐
12002 ments and can appear anywhere in the method signature.
12003 • The max_shrinks setting would sometimes not have been respected.
12005 1.19.0 - 2016-01-09
12007 Codename: IT COMES
12008 This release heralds the beginning of a new and terrible age of Hypoth‐
12010 esis 2.0.
12011 It's primary purpose is some final deprecations prior to said release.
12013 The goal is that if your code emits no warnings under this release then
12014 it will probably run unchanged under Hypothesis 2.0 (there are some
12015 caveats to this: 2.0 will drop support for some Python versions, and if
12016 you're using internal APIs then as usual that may break without warn‐
12017 ing).
12018 It does have two new features:
12020 • New @seed() decorator which allows you to manually seed a test. This
12022 may be harmlessly combined with and overrides the derandomize set‐
12023 ting.
12024 • settings objects may now be used as a decorator to fix those settings
12026 to a particular @given test.
12027 API changes (old usage still works but is deprecated):
12029 • Settings has been renamed to settings (lower casing) in order to make
12031 the decorator usage more natural.
12032 • Functions for the storage directory that were in hypothesis.settings
12034 are now in a new hypothesis.configuration module.
12035 Additional deprecations:
12037 • the average_list_length setting has been deprecated in favour of be‐
12039 ing explicit.
12040 • the basic() strategy has been deprecated as it is impossible to sup‐
12042 port it under a Conjecture based model, which will hopefully be im‐
12043 plemented at some point in the 2.x series.
12044 • the n_ary_tree strategy (which was never actually part of the public
12046 API) has been deprecated.
12047 • Passing settings or random as keyword arguments to given is depre‐
12049 cated (use the new functionality instead)
12050 Bug fixes:
12052 • No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
12054 tion in streaming() values.
12055 • When running in health check mode with non strict, don't print quite
12057 so many errors for an exception in reify.
12058 • When an assumption made in a test or a filter is flaky, tests will
12060 now raise Flaky instead of UnsatisfiedAssumption.
12061 1.18.1 - 2015-12-22
12063 Two behind the scenes changes:
12064 • Hypothesis will no longer write generated code to the file system.
12066 This will improve performance on some systems (e.g. if you're using
12067 PythonAnywhere which is running your code from NFS) and prevent some
12068 annoying interactions with auto-restarting systems.
12069 • Hypothesis will cache the creation of some strategies. This can sig‐
12071 nificantly improve performance for code that uses flatmap or compos‐
12072 ite and thus has to instantiate strategies a lot.
12073 1.18.0 - 2015-12-21
12075 Features:
12076 • Tests and find are now explicitly seeded off the global random mod‐
12078 ule. This means that if you nest one inside the other you will now
12079 get a health check error. It also means that you can control global
12080 randomization by seeding random.
12081 • There is a new random_module() strategy which seeds the global random
12083 module for you and handles things so that you don't get a health
12084 check warning if you use it inside your tests.
12085 • floats() now accepts two new arguments: allow_nan and allow_infinity.
12087 These default to the old behaviour, but when set to False will do
12088 what the names suggest.
12089 Bug fixes:
12091 • Fix a bug where tests that used text() on Python 3.4+ would not actu‐
12093 ally be deterministic even when explicitly seeded or using the deran‐
12094 domize mode, because generation depended on dictionary iteration or‐
12095 der which was affected by hash randomization.
12096 • Fix a bug where with complicated strategies the timing of the initial
12098 health check could affect the seeding of the subsequent test, which
12099 would also render supposedly deterministic tests non-deterministic in
12100 some scenarios.
12101 • In some circumstances flatmap() could get confused by two struc‐
12103 turally similar things it could generate and would produce a flaky
12104 test where the first time it produced an error but the second time it
12105 produced the other value, which was not an error. The same bug was
12106 presumably also possible in composite().
12107 • flatmap() and composite() initial generation should now be moderately
12109 faster. This will be particularly noticeable when you have many val‐
12110 ues drawn from the same strategy in a single run, e.g. constructs
12111 like lists(s.flatmap(f)). Shrinking performance may have suffered,
12112 but this didn't actually produce an interestingly worse result in any
12113 of the standard scenarios tested.
12114 1.17.1 - 2015-12-16
12116 A small bug fix release, which fixes the fact that the 'note' function
12117 could not be used on tests which used the @example decorator to provide
12118 explicit examples.
12119 1.17.0 - 2015-12-15
12121 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
12122 because it contains the following additional change that was not in‐
12123 tended to be in a patch release (it's perfectly stable, but is a
12124 larger change that should have required a minor version bump):
12125 • Hypothesis will now perform a series of "health checks" as part of
12127 running your tests. These detect and warn about some common error
12128 conditions that people often run into which wouldn't necessarily have
12129 caused the test to fail but would cause e.g. degraded performance or
12130 confusing results.
12131 1.16.1 - 2015-12-14
12133 Note: This release has been removed.
12134 A small bugfix release that allows bdists for Hypothesis to be built
12136 under 2.7 - the compat3.py file which had Python 3 syntax wasn't in‐
12137 tended to be loaded under Python 2, but when building a bdist it was.
12138 In particular this would break running setup.py test.
12139 1.16.0 - 2015-12-08
12141 There are no public API changes in this release but it includes a be‐
12142 haviour change that I wasn't comfortable putting in a patch release.
12143 • Functions from hypothesis.strategies will no longer raise InvalidAr‐
12145 gument on bad arguments. Instead the same errors will be raised when
12146 a test using such a strategy is run. This may improve startup time in
12147 some cases, but the main reason for it is so that errors in strate‐
12148 gies won't cause errors in loading, and it can interact correctly
12149 with things like pytest.mark.skipif.
12150 • Errors caused by accidentally invoking the legacy API are now much
12152 less confusing, although still throw NotImplementedError.
12153 • hypothesis.extra.django is 1.9 compatible.
12155 • When tests are run with max_shrinks=0 this will now still rerun the
12157 test on failure and will no longer print "Trying example:" before
12158 each run. Additionally note() will now work correctly when used with
12159 max_shrinks=0.
12160 1.15.0 - 2015-11-24
12162 A release with two new features.
12163 • A 'characters' strategy for more flexible generation of text with
12165 particular character ranges and types, kindly contributed by
12166 Alexander Shorin.
12167 • Add support for preconditions to the rule based stateful testing.
12169 Kindly contributed by Christopher Armstrong
12170 1.14.0 - 2015-11-01
12172 New features:
12173 • Add 'note' function which lets you include additional information in
12175 the final test run's output.
12176 • Add 'choices' strategy which gives you a choice function that emu‐
12178 lates random.choice.
12179 • Add 'uuid' strategy that generates UUIDs'
12181 • Add 'shared' strategy that lets you create a strategy that just gen‐
12183 erates a single shared value for each test run
12184 Bugs:
12186 • Using strategies of the form streaming(x.flatmap(f)) with find or in
12188 stateful testing would have caused InvalidArgument errors when the
12189 resulting values were used (because code that expected to only be
12190 called within a test context would be invoked).
12191 1.13.0 - 2015-10-29
12193 This is quite a small release, but deprecates some public API functions
12194 and removes some internal API functionality so gets a minor version
12195 bump.
12196 • All calls to the 'strategy' function are now deprecated, even ones
12198 which pass just a SearchStrategy instance (which is still a no-op).
12199 • Never documented hypothesis.extra entry_points mechanism has now been
12201 removed ( it was previously how hypothesis.extra packages were loaded
12202 and has been deprecated and unused for some time)
12203 • Some corner cases that could previously have produced an OverflowEr‐
12205 ror when simplifying failing cases using hypothesis.extra.datetimes
12206 (or dates or times) have now been fixed.
12207 • Hypothesis load time for first import has been significantly reduced
12209 - it used to be around 250ms (on my SSD laptop) and now is around
12210 100-150ms. This almost never matters but was slightly annoying when
12211 using it in the console.
12212 • hypothesis.strategies.randoms was previously missing from __all__.
12214 1.12.0 - 2015-10-18
12216 • Significantly improved performance of creating strategies using the
12217 functions from the hypothesis.strategies module by deferring the cal‐
12218 culation of their repr until it was needed. This is unlikely to have
12219 been an performance issue for you unless you were using flatmap, com‐
12220 posite or stateful testing, but for some cases it could be quite a
12221 significant impact.
12222 • A number of cases where the repr of strategies build from lambdas is
12224 improved
12225 • Add dates() and times() strategies to hypothesis.extra.datetimes
12227 • Add new 'profiles' mechanism to the settings system
12229 • Deprecates mutability of Settings, both the Settings.default top
12231 level property and individual settings.
12232 • A Settings object may now be directly initialized from a parent Set‐
12234 tings.
12235 • @given should now give a better error message if you attempt to use
12237 it with a function that uses destructuring arguments (it still won't
12238 work, but it will error more clearly),
12239 • A number of spelling corrections in error messages
12241 • pytest should no longer display the intermediate modules Hypothesis
12243 generates when running in verbose mode
12244 • Hypothesis should now correctly handle printing objects with
12246 non-ascii reprs on python 3 when running in a locale that cannot han‐
12247 dle ascii printing to stdout.
12248 • Add a unique=True argument to lists(). This is equivalent to
12250 unique_by=lambda x: x, but offers a more convenient syntax.
12251 1.11.4 - 2015-09-27
12253 • Hide modifications Hypothesis needs to make to sys.path by undoing
12254 them after we've imported the relevant modules. This is a workaround
12255 for issues cryptography experienced on windows.
12256 • Slightly improved performance of drawing from sampled_from on large
12258 lists of alternatives.
12259 • Significantly improved performance of drawing from one_of or strate‐
12261 gies using | (note this includes a lot of strategies internally -
12262 floats() and integers() both fall into this category). There turned
12263 out to be a massive performance regression introduced in 1.10.0 af‐
12264 fecting these which probably would have made tests using Hypothesis
12265 significantly slower than they should have been.
12266 1.11.3 - 2015-09-23
12268 • Better argument validation for datetimes() strategy - previously set‐
12269 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
12270 would not have raised an InvalidArgument error and instead would have
12271 behaved confusingly.
12272 • Compatibility with being run on pytest < 2.7 (achieved by disabling
12274 the plugin).
12275 1.11.2 - 2015-09-23
12277 Bug fixes:
12278 • Settings(database=my_db) would not be correctly inherited when used
12280 as a default setting, so that newly created settings would use the
12281 database_file setting and create an SQLite example database.
12282 • Settings.default.database = my_db would previously have raised an er‐
12284 ror and now works.
12285 • Timeout could sometimes be significantly exceeded if during simplifi‐
12287 cation there were a lot of examples tried that didn't trigger the
12288 bug.
12289 • When loading a heavily simplified example using a basic() strategy
12291 from the database this could cause Python to trigger a recursion er‐
12292 ror.
12293 • Remove use of deprecated API in pytest plugin so as to not emit warn‐
12295 ing
12296 Misc:
12298 • hypothesis-pytest is now part of hypothesis core. This should have no
12300 externally visible consequences, but you should update your dependen‐
12301 cies to remove hypothesis-pytest and depend on only Hypothesis.
12302 • Better repr for hypothesis.extra.datetimes() strategies.
12304 • Add .close() method to abstract base class for Backend (it was al‐
12306 ready present in the main implementation).
12307 1.11.1 - 2015-09-16
12309 Bug fixes:
12310 • When running Hypothesis tests in parallel (e.g. using pytest-xdist)
12312 there was a race condition caused by code generation.
12313 • Example databases are now cached per thread so as to not use sqlite
12315 connections from multiple threads. This should make Hypothesis now
12316 entirely thread safe.
12317 • floats() with only min_value or max_value set would have had a very
12319 bad distribution.
12320 • Running on 3.5, Hypothesis would have emitted deprecation warnings
12322 because of use of inspect.getargspec
12323 1.11.0 - 2015-08-31
12325 • text() with a non-string alphabet would have used the repr() of the
12326 the alphabet instead of its contexts. This is obviously silly. It now
12327 works with any sequence of things convertible to unicode strings.
12328 • @given will now work on methods whose definitions contains no ex‐
12330 plicit positional arguments, only varargs (issue #118). This may
12331 have some knock on effects because it means that @given no longer
12332 changes the argspec of functions other than by adding defaults.
12333 • Introduction of new @composite feature for more natural definition of
12335 strategies you'd previously have used flatmap for.
12336 1.10.6 - 2015-08-26
12338 Fix support for fixtures on Django 1.7.
12339 1.10.4 - 2015-08-21
12341 Tiny bug fix release:
12342 • If the database_file setting is set to None, this would have resulted
12344 in an error when running tests. Now it does the same as setting data‐
12345 base to None.
12346 1.10.3 - 2015-08-19
12348 Another small bug fix release.
12349 • lists(elements, unique_by=some_function, min_size=n) would have
12351 raised a ValidationError if n > Settings.default.average_list_length
12352 because it would have wanted to use an average list length shorter
12353 than the minimum size of the list, which is impossible. Now it in‐
12354 stead defaults to twice the minimum size in these circumstances.
12355 • basic() strategy would have only ever produced at most ten distinct
12357 values per run of the test (which is bad if you e.g. have it inside a
12358 list). This was obviously silly. It will now produce a much better
12359 distribution of data, both duplicated and non duplicated.
12360 1.10.2 - 2015-08-19
12362 This is a small bug fix release:
12363 • star imports from hypothesis should now work correctly.
12365 • example quality for examples using flatmap will be better, as the way
12367 it had previously been implemented was causing problems where Hypoth‐
12368 esis was erroneously labelling some examples as being duplicates.
12369 1.10.0 - 2015-08-04
12371 This is just a bugfix and performance release, but it changes some
12372 semi-public APIs, hence the minor version bump.
12373 • Significant performance improvements for strategies which are
12375 one_of() many branches. In particular this included recursive()
12376 strategies. This should take the case where you use one recursive()
12377 strategy as the base strategy of another from unusably slow (tens of
12378 seconds per generated example) to reasonably fast.
12379 • Better handling of just() and sampled_from() for values which have an
12381 incorrect __repr__ implementation that returns non-ASCII unicode on
12382 Python 2.
12383 • Better performance for flatmap from changing the internal morpher API
12385 to be significantly less general purpose.
12386 • Introduce a new semi-public BuildContext/cleanup API. This allows
12388 strategies to register cleanup activities that should run once the
12389 example is complete. Note that this will interact somewhat weirdly
12390 with find.
12391 • Better simplification behaviour for streaming strategies.
12393 • Don't error on lambdas which use destructuring arguments in Python 2.
12395 • Add some better reprs for a few strategies that were missing good
12397 ones.
12398 • The Random instances provided by randoms() are now copyable.
12400 • Slightly more debugging information about simplify when using a debug
12402 verbosity level.
12403 • Support using given for functions with varargs, but not passing argu‐
12405 ments to it as positional.
12406 1.9.0 - 2015-07-27
12408 Codename: The great bundling.
12409 This release contains two fairly major changes.
12411 The first is the deprecation of the hypothesis-extra mechanism. From
12413 now on all the packages that were previously bundled under it other
12414 than hypothesis-pytest (which is a different beast and will remain sep‐
12415 arate). The functionality remains unchanged and you can still import
12416 them from exactly the same location, they just are no longer separate
12417 packages.
12418 The second is that this introduces a new way of building strategies
12420 which lets you build up strategies recursively from other strategies.
12421 It also contains the minor change that calling .example() on a strategy
12423 object will give you examples that are more representative of the ac‐
12424 tual data you'll get. There used to be some logic in there to make the
12425 examples artificially simple but this proved to be a bad idea.
12426 1.8.5 - 2015-07-24
12428 This contains no functionality changes but fixes a mistake made with
12429 building the previous package that would have broken installation on
12430 Windows.
12431 1.8.4 - 2015-07-20
12433 Bugs fixed:
12434 • When a call to floats() had endpoints which were not floats but
12436 merely convertible to one (e.g. integers), these would be included in
12437 the generated data which would cause it to generate non-floats.
12438 • Splitting lambdas used in the definition of flatmap, map or filter
12440 over multiple lines would break the repr, which would in turn break
12441 their usage.
12442 1.8.3 - 2015-07-20
12444 "Falsifying example" would not have been printed when the failure came
12445 from an explicit example.
12446 1.8.2 - 2015-07-18
12448 Another small bugfix release:
12449 • When using ForkingTestCase you would usually not get the falsifying
12451 example printed if the process exited abnormally (e.g. due to
12452 os._exit).
12453 • Improvements to the distribution of characters when using text() with
12455 a default alphabet. In particular produces a better distribution of
12456 ascii and whitespace in the alphabet.
12457 1.8.1 - 2015-07-17
12459 This is a small release that contains a workaround for people who have
12460 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
12461 for Hypothesis per se because that's not a thing that is actually sup‐
12462 posed to work, but Hypothesis leans more heavily on repr than is typi‐
12463 cal so it's worth having a workaround for.
12464 1.8.0 - 2015-07-16
12466 New features:
12467 • Much more sensible reprs for strategies, especially ones that come
12469 from hypothesis.strategies. These should now have as reprs python
12470 code that would produce the same strategy.
12471 • lists() accepts a unique_by argument which forces the generated lists
12473 to be only contain elements unique according to some function key
12474 (which must return a hashable value).
12475 • Better error messages from flaky tests to help you debug things.
12477 Mostly invisible implementation details that may result in finding new
12479 bugs in your code:
12480 • Sets and dictionary generation should now produce a better range of
12482 results.
12483 • floats with bounds now focus more on 'critical values', trying to
12485 produce values at edge cases.
12486 • flatmap should now have better simplification for complicated cases,
12488 as well as generally being (I hope) more reliable.
12489 Bug fixes:
12491 • You could not previously use assume() if you were using the forking
12493 executor.
12494 1.7.2 - 2015-07-10
12496 This is purely a bug fix release:
12497 • When using floats() with stale data in the database you could some‐
12499 times get values in your tests that did not respect min_value or
12500 max_value.
12501 • When getting a Flaky error from an unreliable test it would have in‐
12503 correctly displayed the example that caused it.
12504 • 2.6 dependency on backports was incorrectly specified. This would
12506 only have caused you problems if you were building a universal wheel
12507 from Hypothesis, which is not how Hypothesis ships, so unless you're
12508 explicitly building wheels for your dependencies and support Python
12509 2.6 plus a later version of Python this probably would never have af‐
12510 fected you.
12511 • If you use flatmap in a way that the strategy on the right hand side
12513 depends sensitively on the left hand side you may have occasionally
12514 seen Flaky errors caused by producing unreliable examples when mini‐
12515 mizing a bug. This use case may still be somewhat fraught to be hon‐
12516 est. This code is due a major rearchitecture for 1.8, but in the
12517 meantime this release fixes the only source of this error that I'm
12518 aware of.
12519 1.7.1 - 2015-06-29
12521 Codename: There is no 1.7.0.
12522 A slight technical hitch with a premature upload means there's was a
12524 yanked 1.7.0 release. Oops.
12525 The major feature of this release is Python 2.6 support. Thanks to Jeff
12527 Meadows for doing most of the work there.
12528 Other minor features
12530 • strategies now has a permutations() function which returns a strategy
12532 yielding permutations of values from a given collection.
12533 • if you have a flaky test it will print the exception that it last saw
12535 before failing with Flaky, even if you do not have verbose reporting
12536 on.
12537 • Slightly experimental git merge script available as "python -m hy‐
12539 pothesis.tools.mergedbs". Instructions on how to use it in the doc‐
12540 string of that file.
12541 Bug fixes:
12543 • Better performance from use of filter. In particular tests which in‐
12545 volve large numbers of heavily filtered strategies should perform a
12546 lot better.
12547 • floats() with a negative min_value would not have worked correctly
12549 (worryingly, it would have just silently failed to run any examples).
12550 This is now fixed.
12551 • tests using sampled_from would error if the number of sampled ele‐
12553 ments was smaller than min_satisfying_examples.
12554 1.6.2 - 2015-06-08
12556 This is just a few small bug fixes:
12557 • Size bounds were not validated for values for a binary() strategy
12559 when reading examples from the database.
12560 • sampled_from is now in __all__ in hypothesis.strategies
12562 • floats no longer consider negative integers to be simpler than posi‐
12564 tive non-integers
12565 • Small floating point intervals now correctly count members, so if you
12567 have a floating point interval so narrow there are only a handful of
12568 values in it, this will no longer cause an error when Hypothesis runs
12569 out of values.
12570 1.6.1 - 2015-05-21
12572 This is a small patch release that fixes a bug where 1.6.0 broke the
12573 use of flatmap with the deprecated API and assumed the passed in func‐
12574 tion returned a SearchStrategy instance rather than converting it to a
12575 strategy.
12576 1.6.0 - 2015-05-21
12578 This is a smallish release designed to fix a number of bugs and smooth
12579 out some weird behaviours.
12580 • Fix a critical bug in flatmap where it would reuse old strategies. If
12582 all your flatmap code was pure you're fine. If it's not, I'm sur‐
12583 prised it's working at all. In particular if you want to use flatmap
12584 with django models, you desperately need to upgrade to this version.
12585 • flatmap simplification performance should now be better in some cases
12587 where it previously had to redo work.
12588 • Fix for a bug where invalid unicode data with surrogates could be
12590 generated during simplification (it was already filtered out during
12591 actual generation).
12592 • The Hypothesis database is now keyed off the name of the test instead
12594 of the type of data. This makes much more sense now with the new
12595 strategies API and is generally more robust. This means you will lose
12596 old examples on upgrade.
12597 • The database will now not delete values which fail to deserialize
12599 correctly, just skip them. This is to handle cases where multiple in‐
12600 compatible strategies share the same key.
12601 • find now also saves and loads values from the database, keyed off a
12603 hash of the function you're finding from.
12604 • Stateful tests now serialize and load values from the database. They
12606 should have before, really. This was a bug.
12607 • Passing a different verbosity level into a test would not have worked
12609 entirely correctly, leaving off some messages. This is now fixed.
12610 • Fix a bug where derandomized tests with unicode characters in the
12612 function body would error on Python 2.7.
12613 1.5.0 - 2015-05-14
12615 Codename: Strategic withdrawal.
12616 The purpose of this release is a radical simplification of the API for
12618 building strategies. Instead of the old approach of @strategy.extend
12619 and things that get converted to strategies, you just build strategies
12620 directly.
12621 The old method of defining strategies will still work until Hypothesis
12623 2.0, because it's a major breaking change, but will now emit depreca‐
12624 tion warnings.
12625 The new API is also a lot more powerful as the functions for defining
12627 strategies give you a lot of dials to turn. See the updated data sec‐
12628 tion for details.
12629 Other changes:
12631 • Mixing keyword and positional arguments in a call to @given is
12633 deprecated as well.
12634 • There is a new setting called 'strict'. When set to True, Hypothe‐
12636 sis will raise warnings instead of merely printing them. Turning
12637 it on by default is inadvisable because it means that Hypothesis
12638 minor releases can break your code, but it may be useful for mak‐
12639 ing sure you catch all uses of deprecated APIs.
12640 • max_examples in settings is now interpreted as meaning the maximum
12642 number of unique (ish) examples satisfying assumptions. A new set‐
12643 ting max_iterations which defaults to a larger value has the old
12644 interpretation.
12645 • Example generation should be significantly faster due to a new
12647 faster parameter selection algorithm. This will mostly show up for
12648 simple data types - for complex ones the parameter selection is
12649 almost certainly dominated.
12650 • Simplification has some new heuristics that will tend to cut down
12652 on cases where it could previously take a very long time.
12653 • timeout would previously not have been respected in cases where
12655 there were a lot of duplicate examples. You probably wouldn't have
12656 previously noticed this because max_examples counted duplicates,
12657 so this was very hard to hit in a way that mattered.
12658 • A number of internal simplifications to the SearchStrategy API.
12660 • You can now access the current Hypothesis version as hypothe‐
12662 sis.__version__.
12663 • A top level function is provided for running the stateful tests
12665 without the TestCase infrastructure.
12666 1.4.0 - 2015-05-04
12668 Codename: What a state.
12669 The big feature of this release is the new and slightly experimental
12671 stateful testing API. You can read more about that in the appropriate
12672 section.
12673 Two minor features the were driven out in the course of developing
12675 this:
12676 • You can now set settings.max_shrinks to limit the number of times Hy‐
12678 pothesis will try to shrink arguments to your test. If this is set to
12679 <= 0 then Hypothesis will not rerun your test and will just raise the
12680 failure directly. Note that due to technical limitations if
12681 max_shrinks is <= 0 then Hypothesis will print every example it calls
12682 your test with rather than just the failing one. Note also that I
12683 don't consider settings max_shrinks to zero a sensible way to run
12684 your tests and it should really be considered a debug feature.
12685 • There is a new debug level of verbosity which is even more verbose
12687 than verbose. You probably don't want this.
12688 Breakage of semi-public SearchStrategy API:
12690 • It is now a required invariant of SearchStrategy that if u simplifies
12692 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
12693 plifying should not increase the complexity even though it is not re‐
12694 quired to decrease it. Enforcing this invariant lead to finding some
12695 bugs where simplifying of integers, floats and sets was suboptimal.
12696 • Integers in basic data are now required to fit into 64 bits. As a re‐
12698 sult python integer types are now serialized as strings, and some
12699 types have stopped using quite so needlessly large random seeds.
12700 Hypothesis Stateful testing was then turned upon Hypothesis itself,
12702 which lead to an amazing number of minor bugs being found in Hypothesis
12703 itself.
12704 Bugs fixed (most but not all from the result of stateful testing) in‐
12706 clude:
12707 • Serialization of streaming examples was flaky in a way that you would
12709 probably never notice: If you generate a template, simplify it, seri‐
12710 alize it, deserialize it, serialize it again and then deserialize it
12711 you would get the original stream instead of the simplified one.
12712 • If you reduced max_examples below the number of examples already
12714 saved in the database, you would have got a ValueError. Additionally,
12715 if you had more than max_examples in the database all of them would
12716 have been considered.
12717 • @given will no longer count duplicate examples (which it never called
12719 your function with) towards max_examples. This may result in your
12720 tests running slower, but that's probably just because they're trying
12721 more examples.
12722 • General improvements to example search which should result in better
12724 performance and higher quality examples. In particular parameters
12725 which have a history of producing useless results will be more ag‐
12726 gressively culled. This is useful both because it decreases the
12727 chance of useless examples and also because it's much faster to not
12728 check parameters which we were unlikely to ever pick!
12729 • integers_from and lists of types with only one value (e.g. [None])
12731 would previously have had a very high duplication rate so you were
12732 probably only getting a handful of examples. They now have a much
12733 lower duplication rate, as well as the improvements to search making
12734 this less of a problem in the first place.
12735 • You would sometimes see simplification taking significantly longer
12737 than your defined timeout. This would happen because timeout was only
12738 being checked after each successful simplification, so if Hypothesis
12739 was spending a lot of time unsuccessfully simplifying things it
12740 wouldn't stop in time. The timeout is now applied for unsuccessful
12741 simplifications too.
12742 • In Python 2.7, integers_from strategies would have failed during sim‐
12744 plification with an OverflowError if their starting point was at or
12745 near to the maximum size of a 64-bit integer.
12746 • flatmap and map would have failed if called with a function without a
12748 __name__ attribute.
12749 • If max_examples was less than min_satisfying_examples this would al‐
12751 ways error. Now min_satisfying_examples is capped to max_examples.
12752 Note that if you have assumptions to satisfy here this will still
12753 cause an error.
12754 Some minor quality improvements:
12756 • Lists of streams, flatmapped strategies and basic strategies should
12758 now now have slightly better simplification.
12759 1.3.0 - 2015-05-22
12761 New features:
12762 • New verbosity level API for printing intermediate results and excep‐
12764 tions.
12765 • New specifier for strings generated from a specified alphabet.
12767 • Better error messages for tests that are failing because of a lack of
12769 enough examples.
12770 Bug fixes:
12772 • Fix error where use of ForkingTestCase would sometimes result in too
12774 many open files.
12775 • Fix error where saving a failing example that used flatmap could er‐
12777 ror.
12778 • Implement simplification for sampled_from, which apparently never
12780 supported it previously. Oops.
12781 General improvements:
12783 • Better range of examples when using one_of or sampled_from.
12785 • Fix some pathological performance issues when simplifying lists of
12787 complex values.
12788 • Fix some pathological performance issues when simplifying examples
12790 that require unicode strings with high codepoints.
12791 • Random will now simplify to more readable examples.
12793 1.2.1 - 2015-04-16
12795 A small patch release for a bug in the new executors feature. Tests
12796 which require doing something to their result in order to fail would
12797 have instead reported as flaky.
12798 1.2.0 - 2015-04-15
12800 Codename: Finders keepers.
12801 A bunch of new features and improvements.
12803 • Provide a mechanism for customizing how your tests are executed.
12805 • Provide a test runner that forks before running each example. This
12807 allows better support for testing native code which might trigger a
12808 segfault or a C level assertion failure.
12809 • Support for using Hypothesis to find examples directly rather than as
12811 just as a test runner.
12812 • New streaming type which lets you generate infinite lazily loaded
12814 streams of data - perfect for if you need a number of examples but
12815 don't know how many.
12816 • Better support for large integer ranges. You can now use inte‐
12818 gers_in_range with ranges of basically any size. Previously large
12819 ranges would have eaten up all your memory and taken forever.
12820 • Integers produce a wider range of data than before - previously they
12822 would only rarely produce integers which didn't fit into a machine
12823 word. Now it's much more common. This percolates to other numeric
12824 types which build on integers.
12825 • Better validation of arguments to @given. Some situations that would
12827 previously have caused silently wrong behaviour will now raise an er‐
12828 ror.
12829 • Include +/- sys.float_info.max in the set of floating point edge
12831 cases that Hypothesis specifically tries.
12832 • Fix some bugs in floating point ranges which happen when given +/-
12834 sys.float_info.max as one of the endpoints... (really any two floats
12835 that are sufficiently far apart so that x, y are finite but y - x is
12836 infinite). This would have resulted in generating infinite values
12837 instead of ones inside the range.
12838 1.1.1 - 2015-04-07
12840 Codename: Nothing to see here
12841 This is just a patch release put out because it fixed some internal
12843 bugs that would block the Django integration release but did not actu‐
12844 ally affect anything anyone could previously have been using. It also
12845 contained a minor quality fix for floats that I'd happened to have fin‐
12846 ished in time.
12847 • Fix some internal bugs with object lifecycle management that were im‐
12849 possible to hit with the previously released versions but broke hy‐
12850 pothesis-django.
12851 • Bias floating point numbers somewhat less aggressively towards very
12853 small numbers
12854 1.1.0 - 2015-04-06
12856 Codename: No-one mention the M word.
12857 • Unicode strings are more strongly biased towards ascii characters.
12859 Previously they would generate all over the space. This is mostly so
12860 that people who try to shape their unicode strings with assume() have
12861 less of a bad time.
12862 • A number of fixes to data deserialization code that could theoreti‐
12864 cally have caused mysterious bugs when using an old version of a Hy‐
12865 pothesis example database with a newer version. To the best of my
12866 knowledge a change that could have triggered this bug has never actu‐
12867 ally been seen in the wild. Certainly no-one ever reported a bug of
12868 this nature.
12869 • Out of the box support for Decimal and Fraction.
12871 • new dictionary specifier for dictionaries with variable keys.
12873 • Significantly faster and higher quality simplification, especially
12875 for collections of data.
12876 • New filter() and flatmap() methods on Strategy for better ways of
12878 building strategies out of other strategies.
12879 • New BasicStrategy class which allows you to define your own strate‐
12881 gies from scratch without needing an existing matching strategy or
12882 being exposed to the full horror or non-public nature of the Search‐
12883 Strategy interface.
12884 1.0.0 - 2015-03-27
12886 Codename: Blast-off!
12887 There are no code changes in this release. This is precisely the 0.9.2
12889 release with some updated documentation.
12890 0.9.2 - 2015-03-26
12892 Codename: T-1 days.
12893 • floats_in_range would not actually have produced floats_in_range un‐
12895 less that range happened to be (0, 1). Fix this.
12896 0.9.1 - 2015-03-25
12898 Codename: T-2 days.
12899 • Fix a bug where if you defined a strategy using map on a lambda then
12901 the results would not be saved in the database.
12902 • Significant performance improvements when simplifying examples using
12904 lists, strings or bounded integer ranges.
12905 0.9.0 - 2015-03-23
12907 Codename: The final countdown
12908 This release could also be called 1.0-RC1.
12910 It contains a teeny tiny bugfix, but the real point of this release is
12912 to declare feature freeze. There will be zero functionality changes be‐
12913 tween 0.9.0 and 1.0 unless something goes really really wrong. No new
12914 features will be added, no breaking API changes will occur, etc. This
12915 is the final shakedown before I declare Hypothesis stable and ready to
12916 use and throw a party to celebrate.
12917 Bug bounty for any bugs found between now and 1.0: I will buy you a
12919 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
12920 we ever find ourselves in the same city at the same time.
12921 The one tiny bugfix:
12923 • Under pypy, databases would fail to close correctly when garbage col‐
12925 lected, leading to a memory leak and a confusing error message if you
12926 were repeatedly creating databases and not closing them. It is very
12927 unlikely you were doing this and the chances of you ever having no‐
12928 ticed this bug are very low.
12929 0.7.2 - 2015-03-22
12931 Codename: Hygienic macros or bust
12932 • You can now name an argument to @given 'f' and it won't break (issue
12934 #38)
12935 • strategy_test_suite is now named strategy_test_suite as the documen‐
12937 tation claims and not in fact strategy_test_suitee
12938 • Settings objects can now be used as a context manager to temporarily
12940 override the default values inside their context.
12941 0.7.1 - 2015-03-21
12943 Codename: Point releases go faster
12944 • Better string generation by parametrizing by a limited alphabet
12946 • Faster string simplification - previously if simplifying a string
12948 with high range unicode characters it would try every unicode charac‐
12949 ter smaller than that. This was pretty pointless. Now it stops after
12950 it's a short range (it can still reach smaller ones through recursive
12951 calls because of other simplifying operations).
12952 • Faster list simplification by first trying a binary chop down the
12954 middle
12955 • Simultaneous simplification of identical elements in a list. So if a
12957 bug only triggers when you have duplicates but you drew e.g. [-17,
12958 -17], this will now simplify to [0, 0].
12959 0.7.0, - 2015-03-20
12961 Codename: Starting to look suspiciously real
12962 This is probably the last minor release prior to 1.0. It consists of
12964 stability improvements, a few usability things designed to make Hypoth‐
12965 esis easier to try out, and filing off some final rough edges from the
12966 API.
12967 • Significant speed and memory usage improvements
12969 • Add an example() method to strategy objects to give an example of the
12971 sort of data that the strategy generates.
12972 • Remove .descriptor attribute of strategies
12974 • Rename descriptor_test_suite to strategy_test_suite
12976 • Rename the few remaining uses of descriptor to specifier (descriptor
12978 already has a defined meaning in Python)
12979 0.6.0 - 2015-03-13
12981 Codename: I'm sorry, were you using that API?
12982 This is primarily a "simplify all the weird bits of the API" release.
12984 As a result there are a lot of breaking changes. If you just use @given
12985 with core types then you're probably fine.
12986 In particular:
12988 • Stateful testing has been removed from the API
12990 • The way the database is used has been rendered less useful (sorry).
12992 The feature for reassembling values saved from other tests doesn't
12993 currently work. This will probably be brought back in post 1.0.
12994 • SpecificationMapper is no longer a thing. Instead there is an
12996 ExtMethod called strategy which you extend to specify how to convert
12997 other types to strategies.
12998 • Settings are now extensible so you can add your own for configuring a
13000 strategy
13001 • MappedSearchStrategy no longer needs an unpack method
13003 • Basically all the SearchStrategy internals have changed massively. If
13005 you implemented SearchStrategy directly rather than using Mapped‐
13006 SearchStrategy talk to me about fixing it.
13007 • Change to the way extra packages work. You now specify the package.
13009 This must have a load() method. Additionally any modules in the pack‐
13010 age will be loaded in under hypothesis.extra
13011 Bug fixes:
13013 • Fix for a bug where calling falsify on a lambda with a non-ascii
13015 character in its body would error.
13016 Hypothesis Extra:
13018 hypothesis-fakefactory: An extension for using faker data in hypothe‐
13020 sis. Depends
13021 on fake-factory.
13022 0.5.0 - 2015-02-10
13024 Codename: Read all about it.
13025 Core hypothesis:
13027 • Add support back in for pypy and python 3.2
13029 • @given functions can now be invoked with some arguments explicitly
13031 provided. If all arguments that hypothesis would have provided are
13032 passed in then no falsification is run.
13033 • Related to the above, this means that you can now use pytest fixtures
13035 and mark.parametrize with Hypothesis without either interfering with
13036 the other.
13037 • Breaking change: @given no longer works for functions with varargs
13039 (varkwargs are fine). This might be added back in at a later date.
13040 • Windows is now fully supported. A limited version (just the tests
13042 with none of the extras) of the test suite is run on windows with
13043 each commit so it is now a first class citizen of the Hypothesis
13044 world.
13045 • Fix a bug for fuzzy equality of equal complex numbers with different
13047 reprs (this can happen when one coordinate is zero). This shouldn't
13048 affect users - that feature isn't used anywhere public facing.
13049 • Fix generation of floats on windows and 32-bit builds of python. I
13051 was using some struct.pack logic that only worked on certain word
13052 sizes.
13053 • When a test times out and hasn't produced enough examples this now
13055 raises a Timeout subclass of Unfalsifiable.
13056 • Small search spaces are better supported. Previously something like a
13058 @given(bool, bool) would have failed because it couldn't find enough
13059 examples. Hypothesis is now aware of the fact that these are small
13060 search spaces and will not error in this case.
13061 • Improvements to parameter search in the case of hard to satisfy as‐
13063 sume. Hypothesis will now spend less time exploring parameters that
13064 are unlikely to provide anything useful.
13065 • Increase chance of generating "nasty" floats
13067 • Fix a bug that would have caused unicode warnings if you had a sam‐
13069 pled_from that was mixing unicode and byte strings.
13070 • Added a standard test suite that you can use to validate a custom
13072 strategy you've defined is working correctly.
13073 Hypothesis extra:
13075 First off, introducing Hypothesis extra packages!
13077 These are packages that are separated out from core Hypothesis because
13079 they have one or more dependencies. Every hypothesis-extra package is
13080 pinned to a specific point release of Hypothesis and will have some
13081 version requirements on its dependency. They use entry_points so you
13082 will usually not need to explicitly import them, just have them in‐
13083 stalled on the path.
13084 This release introduces two of them:
13086 hypothesis-datetime:
13088 Does what it says on the tin: Generates datetimes for Hypothesis. Just
13090 install the package and datetime support will start working.
13091 Depends on pytz for timezone support
13093 hypothesis-pytest:
13095 A very rudimentary pytest plugin. All it does right now is hook the
13097 display of falsifying examples into pytest reporting.
13098 Depends on pytest.
13100 0.4.3 - 2015-02-05
13102 Codename: TIL narrow Python builds are a thing
13103 This just fixes the one bug.
13105 • Apparently there is such a thing as a "narrow python build" and OS X
13107 ships with these by default for python 2.7. These are builds where
13108 you only have two bytes worth of unicode. As a result, generating
13109 unicode was completely broken on OS X. Fix this by only generating
13110 unicode codepoints in the range supported by the system.
13111 0.4.2 - 2015-02-04
13113 Codename: O(dear)
13114 This is purely a bugfix release:
13116 • Provide sensible external hashing for all core types. This will sig‐
13118 nificantly improve performance of tracking seen examples which hap‐
13119 pens in literally every falsification run. For Hypothesis fixing this
13120 cut 40% off the runtime of the test suite. The behaviour is quadratic
13121 in the number of examples so if you're running the default configura‐
13122 tion this will be less extreme (Hypothesis's test suite runs at a
13123 higher number of examples than default), but you should still see a
13124 significant improvement.
13125 • Fix a bug in formatting of complex numbers where the string could get
13127 incorrectly truncated.
13128 0.4.1 - 2015-02-03
13130 Codename: Cruel and unusual edge cases
13131 This release is mostly about better test case generation.
13133 Enhancements:
13135 • Has a cool release name
13137 • text_type (str in python 3, unicode in python 2) example generation
13139 now actually produces interesting unicode instead of boring ascii
13140 strings.
13141 • floating point numbers are generated over a much wider range, with
13143 particular attention paid to generating nasty numbers - nan, infin‐
13144 ity, large and small values, etc.
13145 • examples can be generated using pieces of examples previously saved
13147 in the database. This allows interesting behaviour that has previ‐
13148 ously been discovered to be propagated to other examples.
13149 • improved parameter exploration algorithm which should allow it to
13151 more reliably hit interesting edge cases.
13152 • Timeout can now be disabled entirely by setting it to any value <= 0.
13154 Bug fixes:
13156 • The descriptor on a OneOfStrategy could be wrong if you had descrip‐
13158 tors which were equal but should not be coalesced. e.g. a strategy
13159 for one_of((frozenset({int}), {int})) would have reported its de‐
13160 scriptor as {int}. This is unlikely to have caused you any problems
13161 • If you had strategies that could produce NaN (which float previously
13163 couldn't but e.g. a Just(float('nan')) could) then this would have
13164 sent hypothesis into an infinite loop that would have only been ter‐
13165 minated when it hit the timeout.
13166 • Given elements that can take a long time to minimize, minimization of
13168 floats or tuples could be quadratic or worse in the that value. You
13169 should now see much better performance for simplification, albeit at
13170 some cost in quality.
13171 Other:
13173 • A lot of internals have been been rewritten. This shouldn't affect
13175 you at all, but it opens the way for certain of hypothesis's oddities
13176 to be a lot more extensible by users. Whether this is a good thing
13177 may be up for debate...
13178 0.4.0 - 2015-01-21
13180 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
13181 stores data in a local SQLite database and will reuse it for all tests
13182 of the same type.
13183 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
13185 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
13186 ther notice. The previous license remains applicable to all code prior
13187 to 0.4.0.
13188 Enhancements:
13190 • Printing of failing examples. I was finding that the pytest runner
13192 was not doing a good job of displaying these, and that Hypothesis it‐
13193 self could do much better.
13194 • Drop dependency on six for cross-version compatibility. It was easy
13196 enough to write the shim for the small set of features that we care
13197 about and this lets us avoid a moderately complex dependency.
13198 • Some improvements to statistical distribution of selecting from small
13200 (<= 3 elements)
13201 • Improvements to parameter selection for finding examples.
13203 Bugs fixed:
13205 • could_have_produced for lists, dicts and other collections would not
13207 have examined the elements and thus when using a union of different
13208 types of list this could result in Hypothesis getting confused and
13209 passing a value to the wrong strategy. This could potentially result
13210 in exceptions being thrown from within simplification.
13211 • sampled_from would not work correctly on a single element list.
13213 • Hypothesis could get very confused by values which are equal despite
13215 having different types being used in descriptors. Hypothesis now has
13216 its own more specific version of equality it uses for descriptors and
13217 tracking. It is always more fine grained than Python equality: Things
13218 considered != are not considered equal by hypothesis, but some things
13219 that are considered == are distinguished. If your test suite uses
13220 both frozenset and set tests this bug is probably affecting you.
13221 0.3.2 - 2015-01-16
13223 • Fix a bug where if you specified floats_in_range with integer argu‐
13224 ments Hypothesis would error in example simplification.
13225 • Improve the statistical distribution of the floats you get for the
13227 floats_in_range strategy. I'm not sure whether this will affect users
13228 in practice but it took my tests for various conditions from flaky to
13229 rock solid so it at the very least improves discovery of the artifi‐
13230 cial cases I'm looking for.
13231 • Improved repr() for strategies and RandomWithSeed instances.
13233 • Add detection for flaky test cases where hypothesis managed to find
13235 an example which breaks it but on the final invocation of the test it
13236 does not raise an error. This will typically happen with too much re‐
13237 cursion errors but could conceivably happen in other circumstances
13238 too.
13239 • Provide a "derandomized" mode. This allows you to run hypothesis with
13241 zero real randomization, making your build nice and deterministic.
13242 The tests run with a seed calculated from the function they're test‐
13243 ing so you should still get a good distribution of test cases.
13244 • Add a mechanism for more conveniently defining tests which just sam‐
13246 ple from some collection.
13247 • Fix for a really subtle bug deep in the internals of the strategy ta‐
13249 ble. In some circumstances if you were to define instance strategies
13250 for both a parent class and one or more of its subclasses you would
13251 under some circumstances get the strategy for the wrong superclass of
13252 an instance. It is very unlikely anyone has ever encountered this in
13253 the wild, but it is conceivably possible given that a mix of namedtu‐
13254 ple and tuple are used fairly extensively inside hypothesis which do
13255 exhibit this pattern of strategy.
13256 0.3.1 - 2015-01-13
13258 • Support for generation of frozenset and Random values
13259 • Correct handling of the case where a called function mutates it argu‐
13261 ment. This involved introducing a notion of a strategies knowing how
13262 to copy their argument. The default method should be entirely accept‐
13263 able and the worst case is that it will continue to have the old be‐
13264 haviour if you don't mark your strategy as mutable, so this shouldn't
13265 break anything.
13266 • Fix for a bug where some strategies did not correctly implement
13268 could_have_produced. It is very unlikely that any of these would have
13269 been seen in the wild, and the consequences if they had been would
13270 have been minor.
13271 • Re-export the @given decorator from the main hypothesis namespace.
13273 It's still available at the old location too.
13274 • Minor performance optimisation for simplifying long lists.
13276 0.3.0 - 2015-01-12
13278 • Complete redesign of the data generation system. Extreme breaking
13279 change for anyone who was previously writing their own SearchStrategy
13280 implementations. These will not work any more and you'll need to mod‐
13281 ify them.
13282 • New settings system allowing more global and modular control of Veri‐
13284 fier behaviour.
13285 • Decouple SearchStrategy from the StrategyTable. This leads to much
13287 more composable code which is a lot easier to understand.
13288 • A significant amount of internal API renaming and moving. This may
13290 also break your code.
13291 • Expanded available descriptors, allowing for generating integers or
13293 floats in a specific range.
13294 • Significantly more robust. A very large number of small bug fixes,
13296 none of which anyone is likely to have ever noticed.
13297 • Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
13299 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
13300 these to remain officially supported for a very long time. I would
13301 not be surprised to add pypy support back in later but I'm not going
13302 to do so until I know someone cares about it. In the meantime it will
13303 probably still work.
13304 0.2.2 - 2015-01-08
13306 • Fix an embarrassing complete failure of the installer caused by my
13307 being bad at version control
13308 0.2.1 - 2015-01-07
13310 • Fix a bug in the new stateful testing feature where you could make
13311 __init__ a @requires method. Simplification would not always work if
13312 the prune method was able to successfully shrink the test.
13313 0.2.0 - 2015-01-07
13315 • It's aliiive.
13316 • Improve python 3 support using six.
13318 • Distinguish between byte and unicode types.
13320 • Fix issues where FloatStrategy could raise.
13322 • Allow stateful testing to request constructor args.
13324 • Fix for issue where test annotations would timeout based on when the
13326 module was loaded instead of when the test started
13327 0.1.4 - 2013-12-14
13329 • Make verification runs time bounded with a configurable timeout
13330 0.1.3 - 2013-05-03
13332 • Bugfix: Stateful testing behaved incorrectly with subclassing.
13333 • Complex number support
13335 • support for recursive strategies
13337 • different error for hypotheses with unsatisfiable assumptions
13339 0.1.2 - 2013-03-24
13341 • Bugfix: Stateful testing was not minimizing correctly and could throw
13342 exceptions.
13343 • Better support for recursive strategies.
13345 • Support for named tuples.
13347 • Much faster integer generation.
13349 0.1.1 - 2013-03-24
13351 • Python 3.x support via 2to3.
13352 • Use new style classes (oops).
13354 0.1.0 - 2013-03-23
13356 • Introduce stateful testing.
13357 • Massive rewrite of internals to add flags and strategies.
13359 0.0.5 - 2013-03-13
13361 • No changes except trying to fix packaging
13362 0.0.4 - 2013-03-13
13364 • No changes except that I checked in a failing test case for 0.0.3 so
13365 had to replace the release. Doh
13366 0.0.3 - 2013-03-13
13368 • Improved a few internals.
13369 • Opened up creating generators from instances as a general API.
13371 • Test integration.
13373 0.0.2 - 2013-03-12
13375 • Starting to tighten up on the internals.
13376 • Change API to allow more flexibility in configuration.
13378 • More testing.
13380 0.0.1 - 2013-03-10
13382 • Initial release.
13383 • Basic working prototype. Demonstrates idea, probably shouldn't be
13385 used.
13386
13388 Hypothesis development is managed by David R. MacIver and Zac Hat‐
13389 field-Dodds, respectively the first author and lead maintainer.
13390 However, these roles don't include unpaid feature development on Hy‐
13392 pothesis. Our roles as leaders of the project are:
13393 1. Helping other people do feature development on Hypothesis
13395 2. Fixing bugs and other code health issues
13397 3. Improving documentation
13399 4. General release management work
13401 5. Planning the general roadmap of the project
13403 6. Doing sponsored development on tasks that are too large or in depth
13405 for other people to take on
13406 So all new features must either be sponsored or implemented by someone
13408 else. That being said, the maintenance team takes an active role in
13409 shepherding pull requests and helping people write a new feature (see
13410 CONTRIBUTING.rst for details and these examples of how the process
13411 goes). This isn't "patches welcome", it's "we will help you write a
13412 patch".
13413 Release policy
13415 Hypothesis releases follow semantic versioning.
13416 We maintain backwards-compatibility wherever possible, and use depreca‐
13418 tion warnings to mark features that have been superseded by a newer al‐
13419 ternative. If you want to detect this, you can upgrade warnings to er‐
13420 rors in the usual ways.
13421 We use continuous deployment to ensure that you can always use our new‐
13423 est and shiniest features - every change to the source tree is automat‐
13424 ically built and published on PyPI as soon as it's merged onto master,
13425 after code review and passing our extensive test suite.
13426 Project roadmap
13428 Hypothesis does not have a long-term release plan. We respond to bug
13429 reports as they are made; new features are released as and when someone
13430 volunteers to write and maintain them.
13431
13433 For questions you are happy to ask in public, the Hypothesis community
13434 is a friendly place where I or others will be more than happy to help
13435 you out. You're also welcome to ask questions on Stack Overflow. If you
13436 do, please tag them with 'python-hypothesis' so someone sees them.
13437 For bugs and enhancements, please file an issue on the GitHub issue
13439 tracker. Note that as per the development policy, enhancements will
13440 probably not get implemented unless you're willing to pay for develop‐
13441 ment or implement them yourself (with assistance from the maintainers).
13442 Bugs will tend to get fixed reasonably promptly, though it is of course
13443 on a best effort basis.
13444 To see the versions of Python, optional dependencies, test runners, and
13446 operating systems Hypothesis supports (meaning incompatibility is
13447 treated as a bug), see supported.
13448 If you need to ask questions privately or want more of a guarantee of
13450 bugs being fixed promptly, please contact me on
13451 hypothesis-support@drmaciver.com to talk about availability of support
13452 contracts.
13453
13455 Downstream packagers often want to package Hypothesis. Here are some
13456 guidelines.
13457 The primary guideline is this: If you are not prepared to keep up with
13459 the Hypothesis release schedule, don't. You will annoy me and are doing
13460 your users a disservice.
13461 Hypothesis has a very frequent release schedule. It's rare that it goes
13463 a week without a release, and there are often multiple releases in a
13464 given week.
13465 If you are prepared to keep up with this schedule, you might find the
13467 rest of this document useful.
13468 Release tarballs
13470 These are available from the GitHub releases page. The tarballs on PyPI
13471 are intended for installation from a Python tool such as pip and should
13472 not be considered complete releases. Requests to include additional
13473 files in them will not be granted. Their absence is not a bug.
13474 Dependencies
13476 Python versions
13477 Hypothesis is designed to work with a range of Python versions - we
13478 support all versions of CPython with upstream support. We also support
13479 the latest versions of PyPy for Python 3.
13480 Other Python libraries
13482 Hypothesis has mandatory dependencies on the following libraries:
13483 • attrs
13485 • sortedcontainers
13487 Hypothesis has optional dependencies on the following libraries:
13489 extras_require = {
13491 "cli": ["click>=7.0", "black>=19.10b0", "rich>=9.0.0"],
13492 "codemods": ["libcst>=0.3.16"],
13493 "ghostwriter": ["black>=19.10b0"],
13494 "pytz": ["pytz>=2014.1"],
13495 "dateutil": ["python-dateutil>=1.4"],
13496 "lark": ["lark-parser>=0.6.5"],
13497 "numpy": ["numpy>=1.9.0"],
13498 "pandas": ["pandas>=0.25"],
13499 "pytest": ["pytest>=4.6"],
13500 "dpcontracts": ["dpcontracts>=0.4"],
13501 "redis": ["redis>=3.0.0"],
13502 # zoneinfo is an odd one: every dependency is conditional, because they're
13503 # only necessary on old versions of Python or Windows systems.
13504 "zoneinfo": [
13505 "tzdata>=2021.5 ; sys_platform == 'win32'",
13506 "backports.zoneinfo>=0.2.1 ; python_version<'3.9'",
13507 ],
13508 # We only support Django versions with upstream support - see
13509 # https://www.djangoproject.com/download/#supported-versions
13510 # We also leave the choice of timezone library to the user, since it
13511 # might be zoneinfo or pytz depending on version and configuration.
13512 "django": ["django>=2.2"],
13513 }
13514 The way this works when installing Hypothesis normally is that these
13517 features become available if the relevant library is installed.
13518 Specifically for pytest, our plugin supports versions of pytest which
13520 have been out of upstream support for some time. Hypothesis tests can
13521 still be executed by even older versions of pytest - you just won't
13522 have the plugin to provide automatic marks, helpful usage warnings, and
13523 per-test statistics.
13524 Testing Hypothesis
13526 If you want to test Hypothesis as part of your packaging you will prob‐
13527 ably not want to use the mechanisms Hypothesis itself uses for running
13528 its tests, because it has a lot of logic for installing and testing
13529 against different versions of Python.
13530 The tests must be run with fairly recent tooling; check the
13532 tree/master/requirements/ directory for details.
13533 The organisation of the tests is described in the
13535 hypothesis-python/tests/README.rst.
13536 Examples
13538 • arch linux
13539 • fedora
13541 • gentoo
13543
13545 One of the things that is often concerning for people using randomized
13546 testing is the question of how to reproduce failing test cases.
13547 NOTE:
13549 It is better to think about the data Hypothesis generates as being
13550 arbitrary, rather than random. We deliberately generate any valid
13551 data that seems likely to cause errors, so you shouldn't rely on any
13552 expected distribution of or relationships between generated data.
13553 You can read about "swarm testing" and "coverage guided fuzzing" if
13554 you're interested, because you don't need to know for Hypothesis!
13555 Fortunately Hypothesis has a number of features to support reproducing
13557 test failures. The one you will use most commonly when developing lo‐
13558 cally is the example database, which means that you shouldn't have to
13559 think about the problem at all for local use - test failures will just
13560 automatically reproduce without you having to do anything.
13561 The example database is perfectly suitable for sharing between ma‐
13563 chines, but there currently aren't very good work flows for that, so
13564 Hypothesis provides a number of ways to make examples reproducible by
13565 adding them to the source code of your tests. This is particularly use‐
13566 ful when e.g. you are trying to run an example that has failed on your
13567 CI, or otherwise share them between machines.
13568 Providing explicit examples
13570 The simplest way to reproduce a failed test is to ask Hypothesis to run
13571 the failing example it printed. For example, if Falsifying example:
13572 test(n=1) was printed you can decorate test with @example(n=1).
13573 @example can also be used to ensure a specific example is always exe‐
13575 cuted as a regression test or to cover some edge case - basically com‐
13576 bining a Hypothesis test and a traditional parametrized test.
13577 hypothesis.example(*args, **kwargs)
13579 A decorator which ensures a specific example is always tested.
13580 Hypothesis will run all examples you've asked for first. If any of them
13582 fail it will not go on to look for more examples.
13583 It doesn't matter whether you put the example decorator before or after
13585 given. Any permutation of the decorators in the above will do the same
13586 thing.
13587 Note that examples can be positional or keyword based. If they're posi‐
13589 tional then they will be filled in from the right when calling, so ei‐
13590 ther of the following styles will work as expected:
13591 @given(text())
13593 @example("Hello world")
13594 @example(x="Some very long string")
13595 def test_some_code(x):
13596 pass
13597 from unittest import TestCase
13600 class TestThings(TestCase):
13603 @given(text())
13604 @example("Hello world")
13605 @example(x="Some very long string")
13606 def test_some_code(self, x):
13607 pass
13608 As with @given, it is not permitted for a single example to be a mix of
13610 positional and keyword arguments. Either are fine, and you can use one
13611 in one example and the other in another example if for some reason you
13612 really want to, but a single example must be consistent.
13613 Reproducing a test run with @seed
13615 hypothesis.seed(seed)
13616 seed: Start the test execution from a specific seed.
13617 May be any hashable object. No exact meaning for seed is pro‐
13619 vided other than that for a fixed seed value Hypothesis will try
13620 the same actions (insofar as it can given external sources of
13621 non- determinism. e.g. timing and hash randomization).
13622 Overrides the derandomize setting, which is designed to enable
13624 deterministic builds rather than reproducing observed failures.
13625 When a test fails unexpectedly, usually due to a health check failure,
13627 Hypothesis will print out a seed that led to that failure, if the test
13628 is not already running with a fixed seed. You can then recreate that
13629 failure using either the @seed decorator or (if you are running pytest)
13630 with --hypothesis-seed. For example, the following test function and
13631 RuleBasedStateMachine will each check the same examples each time they
13632 are executed, thanks to @seed():
13633 @seed(1234)
13635 @given(x=...)
13636 def test(x):
13637 ...
13638 @seed(6789)
13641 class MyModel(RuleBasedStateMachine):
13642 ...
13643 The seed will not be printed if you could simply use @example instead.
13645 Reproducing an example with @reproduce_failure
13647 Hypothesis has an opaque binary representation that it uses for all ex‐
13648 amples it generates. This representation is not intended to be stable
13649 across versions or with respect to changes in the test, but can be used
13650 to to reproduce failures with the @reproduce_failure decorator.
13651 hypothesis.reproduce_failure(version, blob)
13653 Run the example that corresponds to this data blob in order to
13654 reproduce a failure.
13655 A test with this decorator always runs only one example and al‐
13657 ways fails. If the provided example does not cause a failure,
13658 or is in some way invalid for this test, then this will fail
13659 with a DidNotReproduce error.
13660 This decorator is not intended to be a permanent addition to
13662 your test suite. It's simply some code you can add to ease re‐
13663 production of a problem in the event that you don't have access
13664 to the test database. Because of this, no compatibility guaran‐
13665 tees are made between different versions of Hypothesis - its API
13666 may change arbitrarily from version to version.
13667 The intent is that you should never write this decorator by hand, but
13669 it is instead provided by Hypothesis. When a test fails with a falsi‐
13670 fying example, Hypothesis may print out a suggestion to use @repro‐
13671 duce_failure on the test to recreate the problem as follows:
13672 >>> from hypothesis import settings, given, PrintSettings
13674 >>> import hypothesis.strategies as st
13675 >>> @given(st.floats())
13676 ... @settings(print_blob=True)
13677 ... def test(f):
13678 ... assert f == f
13679 ...
13680 >>> try:
13681 ... test()
13682 ... except AssertionError:
13683 ... pass
13684 ...
13685 Falsifying example: test(f=nan)
13686 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
13688 Adding the suggested decorator to the test should reproduce the failure
13690 (as long as everything else is the same - changing the versions of
13691 Python or anything else involved, might of course affect the behaviour
13692 of the test! Note that changing the version of Hypothesis will result
13693 in a different error - each @reproduce_failure invocation is specific
13694 to a Hypothesis version).
13695 By default these messages are not printed. If you want to see these
13697 you must set the print_blob setting to True.
13698
13700 David R. MacIver
13701
13703 2013-2022, David R. MacIver
137046.39.1 Mar 03, 2022 HYPOTHESIS(1)
