1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property-based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the fu‐
41 ture.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 • Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 • If you delete an object, it is no longer visible.
53 • If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ""
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ""
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Let's see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127 Anyway, this test immediately finds a bug in the code:
129 Falsifying example: test_decode_inverts_encode(s='')
131 UnboundLocalError: local variable 'character' referenced before assignment
133 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136 If we fix that by just adding the following code to the beginning of
138 our encode function then Hypothesis tells us the code is correct (by
139 doing nothing as you'd expect a passing test to).
140 if not input_string:
142 return []
143 If we wanted to make sure this example was always checked we could add
145 it in explicitly by using the @example decorator:
146 from hypothesis import example, given, strategies as st
148 @given(st.text())
151 @example("")
152 def test_decode_inverts_encode(s):
153 assert decode(encode(s)) == s
154 This can be useful to show other developers (or your future self) what
156 kinds of data are valid inputs, or to ensure that particular edge cases
157 such as "" are tested every time. It's also great for regression tests
158 because although Hypothesis will remember failing examples, we don't
159 recommend distributing that database.
160 It's also worth noting that both @example and @given support keyword
162 arguments as well as positional. The following would have worked just
163 as well:
164 @given(s=st.text())
166 @example(s="")
167 def test_decode_inverts_encode(s):
168 assert decode(encode(s)) == s
169 Suppose we had a more interesting bug and forgot to reset the count
171 each time. Say we missed a line in our encode method:
172 def encode(input_string):
174 count = 1
175 prev = ""
176 lst = []
177 for character in input_string:
178 if character != prev:
179 if prev:
180 entry = (prev, count)
181 lst.append(entry)
182 # count = 1 # Missing reset operation
183 prev = character
184 else:
185 count += 1
186 entry = (character, count)
187 lst.append(entry)
188 return lst
189 Hypothesis quickly informs us of the following example:
191 Falsifying example: test_decode_inverts_encode(s='001')
193 Note that the example provided is really quite simple. Hypothesis
195 doesn't just find any counter-example to your tests, it knows how to
196 simplify the examples it finds to produce small easy to understand
197 ones. In this case, two identical values are enough to set the count to
198 a number different from one, followed by another distinct value which
199 should have reset the count but in this case didn't.
200 Installing
202 Hypothesis is available on PyPI as "hypothesis". You can install it
203 with:
204 pip install hypothesis
206 You can install the dependencies for optional extensions with e.g. pip
208 install hypothesis[pandas,django].
209 If you want to install directly from the source code (e.g. because you
211 want to make changes and install the changed version), check out the
212 instructions in CONTRIBUTING.rst.
213 Running tests
215 In our example above we just let pytest discover and run our tests, but
216 we could also have run it explicitly ourselves:
217 if __name__ == "__main__":
219 test_decode_inverts_encode()
220 We could also have done this as a python:unittest.TestCase:
222 import unittest
224 class TestEncoding(unittest.TestCase):
227 @given(text())
228 def test_decode_inverts_encode(self, s):
229 self.assertEqual(decode(encode(s)), s)
230 if __name__ == "__main__":
233 unittest.main()
234 A detail: This works because Hypothesis ignores any arguments it hasn't
236 been told to provide (positional arguments start from the right), so
237 the self argument to the test is simply ignored and works as normal.
238 This also means that Hypothesis will play nicely with other ways of pa‐
239 rameterizing tests. e.g it works fine if you use pytest fixtures for
240 some arguments and Hypothesis for others.
241 Writing tests
243 A test in Hypothesis consists of two parts: A function that looks like
244 a normal test in your test framework of choice but with some additional
245 arguments, and a @given decorator that specifies how to provide those
246 arguments.
247 Here are some other examples of how you could use that:
249 from hypothesis import given, strategies as st
251 @given(st.integers(), st.integers())
254 def test_ints_are_commutative(x, y):
255 assert x + y == y + x
256 @given(x=st.integers(), y=st.integers())
259 def test_ints_cancel(x, y):
260 assert (x + y) - y == x
261 @given(st.lists(st.integers()))
264 def test_reversing_twice_gives_same_list(xs):
265 # This will generate lists of arbitrary length (usually between 0 and
266 # 100 elements) whose elements are integers.
267 ys = list(xs)
268 ys.reverse()
269 ys.reverse()
270 assert xs == ys
271 @given(st.tuples(st.booleans(), st.text()))
274 def test_look_tuples_work_too(t):
275 # A tuple is generated as the one you provided, with the corresponding
276 # types in those positions.
277 assert len(t) == 2
278 assert isinstance(t[0], bool)
279 assert isinstance(t[1], str)
280 Note that as we saw in the above example you can pass arguments to
282 @given either as positional or as keywords.
283 Where to start
285 You should now know enough of the basics to write some tests for your
286 code using Hypothesis. The best way to learn is by doing, so go have a
287 try.
288 If you're stuck for ideas for how to use this sort of test for your
290 code, here are some good starting points:
291 1. Try just calling functions with appropriate arbitrary data and see
293 if they crash. You may be surprised how often this works. e.g. note
294 that the first bug we found in the encoding example didn't even get
295 as far as our assertion: It crashed because it couldn't handle the
296 data we gave it, not because it did the wrong thing.
297 2. Look for duplication in your tests. Are there any cases where you're
299 testing the same thing with multiple different examples? Can you
300 generalise that to a single test using Hypothesis?
301 3. This piece is designed for an F# implementation, but is still very
303 good advice which you may find helps give you good ideas for using
304 Hypothesis.
305 If you have any trouble getting started, don't feel shy about asking
307 for help.
308
310 This is an account of slightly less common Hypothesis features that you
311 don't need to get started but will nevertheless make your life easier.
312 Additional test output
314 Normally the output of a failing test will look something like:
315 Falsifying example: test_a_thing(x=1, y="foo")
317 With the repr of each keyword argument being printed.
319 Sometimes this isn't enough, either because you have a value with a
321 __repr__() method that isn't very descriptive or because you need to
322 see the output of some intermediate steps of your test. That's where
323 the note function comes in:
324 hypothesis.note(value)
326 Report this value for the minimal failing example.
327 >>> from hypothesis import given, note, strategies as st
329 >>> @given(st.lists(st.integers()), st.randoms())
330 ... def test_shuffle_is_noop(ls, r):
331 ... ls2 = list(ls)
332 ... r.shuffle(ls2)
333 ... note(f"Shuffle: {ls2!r}")
334 ... assert ls == ls2
335 ...
336 >>> try:
337 ... test_shuffle_is_noop()
338 ... except AssertionError:
339 ... print("ls != ls2")
340 ...
341 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
342 Shuffle: [1, 0]
343 ls != ls2
344 The note is printed for the minimal failing example of the test in or‐
346 der to include any additional information you might need in your test.
347 Test statistics
349 If you are using pytest you can see a number of statistics about the
350 executed tests by passing the command line argument --hypothe‐
351 sis-show-statistics. This will include some general statistics about
352 the test:
353 For example if you ran the following with --hypothesis-show-statistics:
355 from hypothesis import given, strategies as st
357 @given(st.integers())
360 def test_integers(i):
361 pass
362 You would see:
364 - during generate phase (0.06 seconds):
366 - Typical runtimes: < 1ms, ~ 47% in data generation
367 - 100 passing examples, 0 failing examples, 0 invalid examples
368 - Stopped because settings.max_examples=100
369 The final "Stopped because" line is particularly important to note: It
371 tells you the setting value that determined when the test should stop
372 trying new examples. This can be useful for understanding the behaviour
373 of your tests. Ideally you'd always want this to be max_examples.
374 In some cases (such as filtered and recursive strategies) you will see
376 events mentioned which describe some aspect of the data generation:
377 from hypothesis import given, strategies as st
379 @given(st.integers().filter(lambda x: x % 2 == 0))
382 def test_even_integers(i):
383 pass
384 You would see something like:
386 test_even_integers:
388 - during generate phase (0.08 seconds):
390 - Typical runtimes: < 1ms, ~ 57% in data generation
391 - 100 passing examples, 0 failing examples, 12 invalid examples
392 - Events:
393 * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
394 * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
395 - Stopped because settings.max_examples=100
396 You can also mark custom events in a test using the event function:
398 hypothesis.event(value)
400 Record an event that occurred this test. Statistics on number of
401 test runs with each event will be reported at the end if you run
402 Hypothesis in statistics reporting mode.
403 Events should be strings or convertible to them.
405 from hypothesis import event, given, strategies as st
407 @given(st.integers().filter(lambda x: x % 2 == 0))
410 def test_even_integers(i):
411 event(f"i mod 3 = {i%3}")
412 You will then see output like:
414 test_even_integers:
416 - during generate phase (0.09 seconds):
418 - Typical runtimes: < 1ms, ~ 59% in data generation
419 - 100 passing examples, 0 failing examples, 32 invalid examples
420 - Events:
421 * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
422 * 31.06%, i mod 3 = 2
423 * 28.79%, i mod 3 = 0
424 * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
425 * 15.91%, i mod 3 = 1
426 - Stopped because settings.max_examples=100
427 Arguments to event can be any hashable type, but two events will be
429 considered the same if they are the same when converted to a string
430 with python:str.
431 Making assumptions
433 Sometimes Hypothesis doesn't give you exactly the right sort of data
434 you want - it's mostly of the right shape, but some examples won't work
435 and you don't want to care about them. You can just ignore these by
436 aborting the test early, but this runs the risk of accidentally testing
437 a lot less than you think you are. Also it would be nice to spend less
438 time on bad examples - if you're running 100 examples per test (the de‐
439 fault) and it turns out 70 of those examples don't match your needs,
440 that's a lot of wasted time.
441 hypothesis.assume(condition)
443 Calling assume is like an assert that marks the example as bad,
444 rather than failing the test.
445 This allows you to specify properties that you assume will be
447 true, and let Hypothesis try to avoid similar examples in fu‐
448 ture.
449 For example suppose you had the following test:
451 @given(floats())
453 def test_negation_is_self_inverse(x):
454 assert x == -(-x)
455 Running this gives us:
457 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
459 AssertionError
460 This is annoying. We know about NaN and don't really care about it, but
462 as soon as Hypothesis finds a NaN example it will get distracted by
463 that and tell us about it. Also the test will fail and we want it to
464 pass.
465 So let's block off this particular example:
467 from math import isnan
469 @given(floats())
472 def test_negation_is_self_inverse_for_non_nan(x):
473 assume(not isnan(x))
474 assert x == -(-x)
475 And this passes without a problem.
477 In order to avoid the easy trap where you assume a lot more than you
479 intended, Hypothesis will fail a test when it can't find enough exam‐
480 ples passing the assumption.
481 If we'd written:
483 @given(floats())
485 def test_negation_is_self_inverse_for_non_nan(x):
486 assume(False)
487 assert x == -(-x)
488 Then on running we'd have got the exception:
490 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
492 How good is assume?
494 Hypothesis has an adaptive exploration strategy to try to avoid things
495 which falsify assumptions, which should generally result in it still
496 being able to find examples in hard to find situations.
497 Suppose we had the following:
499 @given(lists(integers()))
501 def test_sum_is_positive(xs):
502 assert sum(xs) > 0
503 Unsurprisingly this fails and gives the falsifying example [].
505 Adding assume(xs) to this removes the trivial empty example and gives
507 us [0].
508 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
510 of positive integers is positive.
511 The reason that this should be surprising is not that it doesn't find a
513 counter-example, but that it finds enough examples at all.
514 In order to make sure something interesting is happening, suppose we
516 wanted to try this for long lists. e.g. suppose we added an as‐
517 sume(len(xs) > 10) to it. This should basically never find an example:
518 a naive strategy would find fewer than one in a thousand examples, be‐
519 cause if each element of the list is negative with probability
520 one-half, you'd have to have ten of these go the right way by chance.
521 In the default configuration Hypothesis gives up long before it's tried
522 1000 examples (by default it tries 200).
523 Here's what happens if we try to run this:
525 @given(lists(integers()))
527 def test_sum_is_positive(xs):
528 assume(len(xs) > 10)
529 assume(all(x > 0 for x in xs))
530 print(xs)
531 assert sum(xs) > 0
532 In: test_sum_is_positive()
534 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
536 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
537 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
538 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
539 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
540 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
541 As you can see, Hypothesis doesn't find many examples here, but it
543 finds some - enough to keep it happy.
544 In general if you can shape your strategies better to your tests you
546 should - for example integers(1, 1000) is a lot better than assume(1 <=
547 x <= 1000), but assume will take you a long way if you can't.
548 Defining strategies
550 The type of object that is used to explore the examples given to your
551 test function is called a SearchStrategy. These are created using the
552 functions exposed in the hypothesis.strategies module.
553 Many of these strategies expose a variety of arguments you can use to
555 customize generation. For example for integers you can specify min and
556 max values of integers you want. If you want to see exactly what a
557 strategy produces you can ask for an example:
558 >>> integers(min_value=0, max_value=10).example()
560 1
561 Many strategies are built out of other strategies. For example, if you
563 want to define a tuple you need to say what goes in each element:
564 >>> from hypothesis.strategies import tuples
566 >>> tuples(integers(), integers()).example()
567 (-24597, 12566)
568 Further details are available in a separate document.
570 The gory details of given parameters
572 hypothesis.given(*_given_arguments, **_given_kwargs)
573 A decorator for turning a test function that accepts arguments
574 into a randomized test.
575 This is the main entry point to Hypothesis.
577 The @given decorator may be used to specify which arguments of a func‐
579 tion should be parametrized over. You can use either positional or key‐
580 word arguments, but not a mixture of both.
581 For example all of the following are valid uses:
583 @given(integers(), integers())
585 def a(x, y):
586 pass
587 @given(integers())
590 def b(x, y):
591 pass
592 @given(y=integers())
595 def c(x, y):
596 pass
597 @given(x=integers())
600 def d(x, y):
601 pass
602 @given(x=integers(), y=integers())
605 def e(x, **kwargs):
606 pass
607 @given(x=integers(), y=integers())
610 def f(x, *args, **kwargs):
611 pass
612 class SomeTest(TestCase):
615 @given(integers())
616 def test_a_thing(self, x):
617 pass
618 The following are not:
620 @given(integers(), integers(), integers())
622 def g(x, y):
623 pass
624 @given(integers())
627 def h(x, *args):
628 pass
629 @given(integers(), x=integers())
632 def i(x, y):
633 pass
634 @given()
637 def j(x, y):
638 pass
639 The rules for determining what are valid uses of given are as follows:
641 1. You may pass any keyword argument to given.
643 2. Positional arguments to given are equivalent to the rightmost named
645 arguments for the test function.
646 3. Positional arguments may not be used if the underlying test function
648 has varargs, arbitrary keywords, or keyword-only arguments.
649 4. Functions tested with given may not have any defaults.
651 The reason for the "rightmost named arguments" behaviour is so that us‐
653 ing @given with instance methods works: self will be passed to the
654 function as normal and not be parametrized over.
655 The function returned by given has all the same arguments as the origi‐
657 nal test, minus those that are filled in by @given. Check the notes on
658 framework compatibility to see how this affects other testing libraries
659 you may be using.
660 Targeted example generation
662 Targeted property-based testing combines the advantages of both
663 search-based and property-based testing. Instead of being completely
664 random, T-PBT uses a search-based component to guide the input genera‐
665 tion towards values that have a higher probability of falsifying a
666 property. This explores the input space more effectively and requires
667 fewer tests to find a bug or achieve a high confidence in the system
668 being tested than random PBT. (Löscher and Sagonas)
669 This is not always a good idea - for example calculating the search
671 metric might take time better spent running more uniformly-random test
672 cases, or your target metric might accidentally lead Hypothesis away
673 from bugs - but if there is a natural metric like "floating-point er‐
674 ror", "load factor" or "queue length", we encourage you to experiment
675 with targeted testing.
676 hypothesis.target(observation, *, label='')
678 Calling this function with an int or float observation gives it
679 feedback with which to guide our search for inputs that will
680 cause an error, in addition to all the usual heuristics. Obser‐
681 vations must always be finite.
682 Hypothesis will try to maximize the observed value over several
684 examples; almost any metric will work so long as it makes sense
685 to increase it. For example, -abs(error) is a metric that in‐
686 creases as error approaches zero.
687 Example metrics:
689 • Number of elements in a collection, or tasks in a queue
691 • Mean or maximum runtime of a task (or both, if you use label)
693 • Compression ratio for data (perhaps per-algorithm or
695 per-level)
696 • Number of steps taken by a state machine
698 The optional label argument can be used to distinguish between
700 and therefore separately optimise distinct observations, such as
701 the mean and standard deviation of a dataset. It is an error to
702 call target() with any label more than once per test case.
703 NOTE:
705 The more examples you run, the better this technique works.
706 As a rule of thumb, the targeting effect is noticeable above
708 max_examples=1000, and immediately obvious by around ten
709 thousand examples per label used by your test.
710 Test statistics include the best score seen for each label,
712 which can help avoid the threshold problem when the minimal ex‐
713 ample shrinks right down to the threshold of failure (issue
714 #2180).
715 from hypothesis import given, strategies as st, target
717 @given(st.floats(0, 1e100), st.floats(0, 1e100), st.floats(0, 1e100))
720 def test_associativity_with_target(a, b, c):
721 ab_c = (a + b) + c
722 a_bc = a + (b + c)
723 difference = abs(ab_c - a_bc)
724 target(difference) # Without this, the test almost always passes
725 assert difference < 2.0
726 We recommend that users also skim the papers introducing targeted PBT;
728 from ISSTA 2017 and ICST 2018. For the curious, the initial implemen‐
729 tation in Hypothesis uses hill-climbing search via a mutating fuzzer,
730 with some tactics inspired by simulated annealing to avoid getting
731 stuck and endlessly mutating a local maximum.
732 Custom function execution
734 Hypothesis provides you with a hook that lets you control how it runs
735 examples.
736 This lets you do things like set up and tear down around each example,
738 run examples in a subprocess, transform coroutine tests into normal
739 tests, etc. For example, TransactionTestCase in the Django extra runs
740 each example in a separate database transaction.
741 The way this works is by introducing the concept of an executor. An ex‐
743 ecutor is essentially a function that takes a block of code and run it.
744 The default executor is:
745 def default_executor(function):
747 return function()
748 You define executors by defining a method execute_example on a class.
750 Any test methods on that class with @given used on them will use
751 self.execute_example as an executor with which to run tests. For exam‐
752 ple, the following executor runs all its code twice:
753 from unittest import TestCase
755 class TestTryReallyHard(TestCase):
758 @given(integers())
759 def test_something(self, i):
760 perform_some_unreliable_operation(i)
761 def execute_example(self, f):
763 f()
764 return f()
765 Note: The functions you use in map, etc. will run inside the executor.
767 i.e. they will not be called until you invoke the function passed to
768 execute_example.
769 An executor must be able to handle being passed a function which re‐
771 turns None, otherwise it won't be able to run normal test cases. So for
772 example the following executor is invalid:
773 from unittest import TestCase
775 class TestRunTwice(TestCase):
778 def execute_example(self, f):
779 return f()()
780 and should be rewritten as:
782 from unittest import TestCase
784 class TestRunTwice(TestCase):
787 def execute_example(self, f):
788 result = f()
789 if callable(result):
790 result = result()
791 return result
792 An alternative hook is provided for use by test runner extensions such
794 as pytest-trio, which cannot use the execute_example method. This is
795 not recommended for end-users - it is better to write a complete test
796 function directly, perhaps by using a decorator to perform the same
797 transformation before applying @given.
798 @given(x=integers())
800 @pytest.mark.trio
801 async def test(x):
802 ...
803 # Illustrative code, inside the pytest-trio plugin
806 test.hypothesis.inner_test = lambda x: trio.run(test, x)
807 For authors of test runners however, assigning to the inner_test attri‐
809 bute of the hypothesis attribute of the test will replace the interior
810 test.
811 NOTE:
813 The new inner_test must accept and pass through all the *args and
814 **kwargs expected by the original test.
815 If the end user has also specified a custom executor using the exe‐
817 cute_example method, it - and all other execution-time logic - will be
818 applied to the new inner test assigned by the test runner.
819 Making random code deterministic
821 While Hypothesis' example generation can be used for nondeterministic
822 tests, debugging anything nondeterministic is usually a very frustrat‐
823 ing exercise. To make things worse, our example shrinking relies on
824 the same input causing the same failure each time - though we show the
825 un-shrunk failure and a decent error message if it doesn't.
826 By default, Hypothesis will handle the global random and numpy.random
828 random number generators for you, and you can register others:
829 hypothesis.register_random(r)
831 Register (a weakref to) the given Random-like instance for man‐
832 agement by Hypothesis.
833 You can pass instances of structural subtypes of random.Random
835 (i.e., objects with seed, getstate, and setstate methods) to
836 register_random(r) to have their states seeded and restored in
837 the same way as the global PRNGs from the random and numpy.ran‐
838 dom modules.
839 All global PRNGs, from e.g. simulation or scheduling frameworks,
841 should be registered to prevent flaky tests. Hypothesis will en‐
842 sure that the PRNG state is consistent for all test runs, always
843 seeding them to zero and restoring the previous state after the
844 test, or, reproducibly varied if you choose to use the
845 random_module() strategy.
846 register_random only makes weakrefs to r, thus r will only be
848 managed by Hypothesis as long as it has active references else‐
849 where at runtime. The pattern register_random(MyRandom()) will
850 raise a ReferenceError to help protect users from this issue.
851 This check does not occur for the PyPy interpreter. See the fol‐
852 lowing example for an illustration of this issue
853 def my_BROKEN_hook():
855 r = MyRandomLike()
856 # `r` will be garbage collected after the hook resolved
858 # and Hypothesis will 'forget' that it was registered
859 register_random(r) # Hypothesis will emit a warning
860 rng = MyRandomLike()
863 def my_WORKING_hook():
866 register_random(rng)
867 Inferred strategies
869 In some cases, Hypothesis can work out what to do when you omit argu‐
870 ments. This is based on introspection, not magic, and therefore has
871 well-defined limits.
872 builds() will check the signature of the target (using signature()).
874 If there are required arguments with type annotations and no strategy
875 was passed to builds(), from_type() is used to fill them in. You can
876 also pass the value ... (Ellipsis) as a keyword argument, to force this
877 inference for arguments with a default value.
878 >>> def func(a: int, b: str):
880 ... return [a, b]
881 ...
882 >>> builds(func).example()
883 [-6993, '']
884 hypothesis.infer
886 @given does not perform any implicit inference for required arguments,
888 as this would break compatibility with pytest fixtures. ...
889 (python:Ellipsis), can be used as a keyword argument to explicitly fill
890 in an argument from its type annotation. You can also use the hypothe‐
891 sis.infer alias if writing a literal ... seems too weird.
892 @given(a=...) # or @given(a=infer)
894 def test(a: int):
895 pass
896 # is equivalent to
899 @given(a=from_type(int))
900 def test(a):
901 pass
902 @given(...) can also be specified to fill all arguments from their type
904 annotations.
905 @given(...)
907 def test(a: int, b: str):
908 pass
909 # is equivalent to
912 @given(a=..., b=...)
913 def test(a, b):
914 pass
915 Limitations
917 Hypothesis does not inspect PEP 484 type comments at runtime. While
918 from_type() will work as usual, inference in builds() and @given will
919 only work if you manually create the __annotations__ attribute (e.g. by
920 using @annotations(...) and @returns(...) decorators).
921 The python:typing module changes between different Python releases, in‐
923 cluding at minor versions. These are all supported on a best-effort
924 basis, but you may encounter problems. Please report them to us, and
925 consider updating to a newer version of Python as a workaround.
926 Type annotations in Hypothesis
928 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
929 patible tool, the type checker should automatically pick up our type
930 hints.
931 NOTE:
933 Hypothesis' type hints may make breaking changes between minor re‐
934 leases.
935 Upstream tools and conventions about type hints remain in flux - for
937 example the python:typing module itself is provisional - and we plan
938 to support the latest version of this ecosystem, as well as older
939 versions where practical.
940 We may also find more precise ways to describe the type of various
942 interfaces, or change their type and runtime behaviour together in a
943 way which is otherwise backwards-compatible. We often omit type
944 hints for deprecated features or arguments, as an additional form of
945 warning.
946 There are known issues inferring the type of examples generated by
948 deferred(), recursive(), one_of(), dictionaries(), and
949 fixed_dictionaries(). We will fix these, and require correspondingly
950 newer versions of Mypy for type hinting, as the ecosystem improves.
951 Writing downstream type hints
953 Projects that provide Hypothesis strategies and use type hints may wish
954 to annotate their strategies too. This is a supported use-case, again
955 on a best-effort provisional basis. For example:
956 def foo_strategy() -> SearchStrategy[Foo]:
958 ...
959 class hypothesis.strategies.SearchStrategy
961 SearchStrategy is the type of all strategy objects. It is a generic
963 type, and covariant in the type of the examples it creates. For exam‐
964 ple:
965 • integers() is of type SearchStrategy[int].
967 • lists(integers()) is of type SearchStrategy[List[int]].
969 • SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
971 a subtype of Animal (as seems likely).
972 WARNING:
974 SearchStrategy should only be used in type hints. Please do not in‐
975 herit from, compare to, or otherwise use it in any way outside of
976 type hints. The only supported way to construct objects of this
977 type is to use the functions provided by the hypothesis.strategies
978 module!
979 The Hypothesis pytest plugin
981 Hypothesis includes a tiny plugin to improve integration with pytest,
982 which is activated by default (but does not affect other test runners).
983 It aims to improve the integration between Hypothesis and Pytest by
984 providing extra information and convenient access to config options.
985 • pytest --hypothesis-show-statistics can be used to display test and
987 data generation statistics.
988 • pytest --hypothesis-profile=<profile name> can be used to load a set‐
990 tings profile.
991 • pytest --hypothesis-verbosity=<level name> can be used to override
993 the current verbosity level.
994 • pytest --hypothesis-seed=<an int> can be used to reproduce a failure
996 with a particular seed.
997 • pytest --hypothesis-explain can be used to temporarily enable the ex‐
999 plain phase.
1000 Finally, all tests that are defined with Hypothesis automatically have
1002 @pytest.mark.hypothesis applied to them. See here for information on
1003 working with markers.
1004 NOTE:
1006 Pytest will load the plugin automatically if Hypothesis is in‐
1007 stalled. You don't need to do anything at all to use it.
1008 Use with external fuzzers
1010 TIP:
1011 Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
1012 Use HypoFuzz to fuzz your whole test suite,
1013 and find more bugs without more tests!
1014 Sometimes, you might want to point a traditional fuzzer such as
1017 python-afl, pythonfuzz, or Google's atheris (for Python and native ex‐
1018 tensions) at your code. Wouldn't it be nice if you could use any of
1019 your @given tests as fuzz targets, instead of converting bytestrings
1020 into your objects by hand?
1021 @given(st.text())
1023 def test_foo(s):
1024 ...
1025 # This is a traditional fuzz target - call it with a bytestring,
1028 # or a binary IO object, and it runs the test once.
1029 fuzz_target = test_foo.hypothesis.fuzz_one_input
1030 # For example:
1032 fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
1033 fuzz_target(io.BytesIO(...))
1034 Depending on the input to fuzz_one_input, one of three things will hap‐
1036 pen:
1037 • If the bytestring was invalid, for example because it was too short
1039 or failed a filter or assume() too many times, fuzz_one_input returns
1040 None.
1041 • If the bytestring was valid and the test passed, fuzz_one_input re‐
1043 turns a canonicalised and pruned buffer which will replay that test
1044 case. This is provided as an option to improve the performance of
1045 mutating fuzzers, but can safely be ignored.
1046 • If the test failed, i.e. raised an exception, fuzz_one_input will add
1048 the pruned buffer to the Hypothesis example database and then
1049 re-raise that exception. All you need to do to reproduce, minimize,
1050 and de-duplicate all the failures found via fuzzing is run your test
1051 suite!
1052 Note that the interpretation of both input and output bytestrings is
1054 specific to the exact version of Hypothesis you are using and the
1055 strategies given to the test, just like the example database and
1056 @reproduce_failure decorator.
1057 Interaction with settings
1059 fuzz_one_input uses just enough of Hypothesis' internals to drive your
1060 test function with a fuzzer-provided bytestring, and most settings
1061 therefore have no effect in this mode. We recommend running your tests
1062 the usual way before fuzzing to get the benefits of healthchecks, as
1063 well as afterwards to replay, shrink, deduplicate, and report whatever
1064 errors were discovered.
1065 • The database setting is used by fuzzing mode - adding failures to the
1067 database to be replayed when you next run your tests is our preferred
1068 reporting mechanism and response to the 'fuzzer taming' problem.
1069 • The verbosity and stateful_step_count settings work as usual.
1071 The deadline, derandomize, max_examples, phases, print_blob, re‐
1073 port_multiple_bugs, and suppress_health_check settings do not affect
1074 fuzzing mode.
1075 Thread-Safety Policy
1077 As discussed in issue #2719, Hypothesis is not truly thread-safe and
1078 that's unlikely to change in the future. This policy therefore de‐
1079 scribes what you can expect if you use Hypothesis with multiple
1080 threads.
1081 Running tests in multiple processes, e.g. with pytest -n auto, is fully
1083 supported and we test this regularly in CI - thanks to process isola‐
1084 tion, we only need to ensure that DirectoryBasedExampleDatabase can't
1085 tread on its own toes too badly. If you find a bug here we will fix it
1086 ASAP.
1087 Running separate tests in multiple threads is not something we design
1089 or test for, and is not formally supported. That said, anecdotally it
1090 does mostly work and we would like it to keep working - we accept rea‐
1091 sonable patches and low-priority bug reports. The main risks here are
1092 global state, shared caches, and cached strategies.
1093 Using multiple threads within a single test , or running a single test
1095 simultaneously in multiple threads, makes it pretty easy to trigger in‐
1096 ternal errors. We usually accept patches for such issues unless read‐
1097 ability or single-thread performance suffer.
1098 Hypothesis assumes that tests are single-threaded, or do a suffi‐
1100 ciently-good job of pretending to be single-threaded. Tests that use
1101 helper threads internally should be OK, but the user must be careful to
1102 ensure that test outcomes are still deterministic. In particular it
1103 counts as nondeterministic if helper-thread timing changes the sequence
1104 of dynamic draws using e.g. the data().
1105 Interacting with any Hypothesis APIs from helper threads might do
1107 weird/bad things, so avoid that too - we rely on thread-local variables
1108 in a few places, and haven't explicitly tested/audited how they respond
1109 to cross-thread API calls. While data() and equivalents are the most
1110 obvious danger, other APIs might also be subtly affected.
1111
1113 Hypothesis tries to have good defaults for its behaviour, but sometimes
1114 that's not enough and you need to tweak it.
1115 The mechanism for doing this is the settings object. You can set up a
1117 @given based test to use this using a settings decorator:
1118 @given invocation is as follows:
1120 from hypothesis import given, settings
1122 @given(integers())
1125 @settings(max_examples=500)
1126 def test_this_thoroughly(x):
1127 pass
1128 This uses a settings object which causes the test to receive a much
1130 larger set of examples than normal.
1131 This may be applied either before or after the given and the results
1133 are the same. The following is exactly equivalent:
1134 from hypothesis import given, settings
1136 @settings(max_examples=500)
1139 @given(integers())
1140 def test_this_thoroughly(x):
1141 pass
1142 Available settings
1144 class hypothesis.settings(parent=None, *, max_examples=not_set, deran‐
1145 domize=not_set, database=not_set, verbosity=not_set, phases=not_set,
1146 stateful_step_count=not_set, report_multiple_bugs=not_set, sup‐
1147 press_health_check=not_set, deadline=not_set, print_blob=not_set)
1148 A settings object configures options including verbosity, run‐
1149 time controls, persistence, determinism, and more.
1150 Default values are picked up from the settings.default object
1152 and changes made there will be picked up in newly created set‐
1153 tings.
1154 database
1156 An instance of ExampleDatabase that will be used to save
1157 examples to and load previous examples from. May be None
1158 in which case no storage will be used.
1159 See the example database documentation for a list of
1161 built-in example database implementations, and how to de‐
1162 fine custom implementations.
1163 default value: (dynamically calculated)
1165 deadline
1167 If set, a duration (as timedelta, or integer or float
1168 number of milliseconds) that each individual example
1169 (i.e. each time your test function is called, not the
1170 whole decorated test) within a test is not allowed to ex‐
1171 ceed. Tests which take longer than that may be converted
1172 into errors (but will not necessarily be if close to the
1173 deadline, to allow some variability in test run time).
1174 Set this to None to disable this behaviour entirely.
1176 default value: timedelta(milliseconds=200)
1178 derandomize
1180 If True, seed Hypothesis' random number generator using a
1181 hash of the test function, so that every run will test
1182 the same set of examples until you update Hypothesis,
1183 Python, or the test function.
1184 This allows you to check for regressions and look for
1186 bugs using separate settings profiles - for example run‐
1187 ning quick deterministic tests on every commit, and a
1188 longer non-deterministic nightly testing run.
1189 default value: False
1191 max_examples
1193 Once this many satisfying examples have been considered
1194 without finding any counter-example, Hypothesis will stop
1195 looking.
1196 Note that we might call your test function fewer times if
1198 we find a bug early or can tell that we've exhausted the
1199 search space; or more if we discard some examples due to
1200 use of .filter(), assume(), or a few other things that
1201 can prevent the test case from completing successfully.
1202 The default value is chosen to suit a workflow where the
1204 test will be part of a suite that is regularly executed
1205 locally or on a CI server, balancing total running time
1206 against the chance of missing a bug.
1207 If you are writing one-off tests, running tens of thou‐
1209 sands of examples is quite reasonable as Hypothesis may
1210 miss uncommon bugs with default settings. For very com‐
1211 plex code, we have observed Hypothesis finding novel bugs
1212 after several million examples while testing SymPy. If
1213 you are running more than 100k examples for a test, con‐
1214 sider using our integration for coverage-guided fuzzing -
1215 it really shines when given minutes or hours to run.
1216 default value: 100
1218 phases Control which phases should be run. See the full documen‐
1220 tation for more details
1221 default value: (Phase.explicit, Phase.reuse, Phase.gener‐
1223 ate, Phase.target, Phase.shrink)
1224 print_blob
1226 If set to True, Hypothesis will print code for failing
1227 examples that can be used with @reproduce_failure to re‐
1228 produce the failing example. The default is True if the
1229 CI or TF_BUILD env vars are set, False otherwise.
1230 default value: (dynamically calculated)
1232 report_multiple_bugs
1234 Because Hypothesis runs the test many times, it can some‐
1235 times find multiple bugs in a single run. Reporting all
1236 of them at once is usually very useful, but replacing the
1237 exceptions can occasionally clash with debuggers. If
1238 disabled, only the exception with the smallest minimal
1239 example is raised.
1240 default value: True
1242 stateful_step_count
1244 Number of steps to run a stateful program for before giv‐
1245 ing up on it breaking.
1246 default value: 50
1248 suppress_health_check
1250 A list of HealthCheck items to disable.
1251 default value: ()
1253 verbosity
1255 Control the verbosity level of Hypothesis messages
1256 default value: Verbosity.normal
1258 Controlling what runs
1260 Hypothesis divides tests into logically distinct phases:
1261 1. Running explicit examples provided with the @example decorator.
1263 2. Rerunning a selection of previously failing examples to reproduce a
1265 previously seen error.
1266 3. Generating new examples.
1268 4. Mutating examples for targeted property-based testing (requires gen‐
1270 erate phase).
1271 5. Attempting to shrink an example found in previous phases (other than
1273 phase 1 - explicit examples cannot be shrunk). This turns poten‐
1274 tially large and complicated examples which may be hard to read into
1275 smaller and simpler ones.
1276 6. Attempting to explain why your test failed (requires shrink phase).
1278 NOTE:
1280 The explain phase has two parts, each of which is best-effort - if
1281 Hypothesis can't find a useful explanation, we'll just print the
1282 minimal failing example.
1283 Following the first failure, Hypothesis will (usually) track which
1285 lines of code are always run on failing but never on passing inputs.
1286 This relies on python:sys.settrace(), and is therefore automatically
1287 disabled on PyPy or if you are using coverage or a debugger. If
1288 there are no clearly suspicious lines of code, we refuse the tempta‐
1289 tion to guess.
1290 After shrinking to a minimal failing example, Hypothesis will try to
1292 find parts of the example -- e.g. separate args to @given() -- which
1293 can vary freely without changing the result of that minimal failing
1294 example. If the automated experiments run without finding a passing
1295 variation, we leave a comment in the final report:
1296 test_x_divided_by_y(
1298 x=0, # or any other generated value
1299 y=0,
1300 )
1301 Just remember that the lack of an explanation sometimes just means
1303 that Hypothesis couldn't efficiently find one, not that no explana‐
1304 tion (or simpler failing example) exists.
1305 The phases setting provides you with fine grained control over which of
1307 these run, with each phase corresponding to a value on the Phase enum:
1308 class hypothesis.Phase(value, names=None, *values, module=None, qual‐
1310 name=None, type=None, start=1, boundary=None)
1311 explicit = 0
1313 controls whether explicit examples are run.
1314 reuse = 1
1316 controls whether previous examples will be reused.
1317 generate = 2
1319 controls whether new examples will be generated.
1320 target = 3
1322 controls whether examples will be mutated for targeting.
1323 shrink = 4
1325 controls whether examples will be shrunk.
1326 explain = 5
1328 controls whether Hypothesis attempts to explain test
1329 failures.
1330 The phases argument accepts a collection with any subset of these. e.g.
1332 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1333 ples and shrink them, but will not run explicit examples or reuse pre‐
1334 vious failures, while settings(phases=[Phase.explicit]) will only run
1335 the explicit examples.
1336 Seeing intermediate result
1338 To see what's going on while Hypothesis runs your tests, you can turn
1339 up the verbosity setting.
1340 >>> from hypothesis import find, settings, Verbosity
1342 >>> from hypothesis.strategies import lists, integers
1343 >>> @given(lists(integers()))
1344 ... @settings(verbosity=Verbosity.verbose)
1345 ... def f(x):
1346 ... assert not any(x)
1347 ... f()
1348 Trying example: []
1349 Falsifying example: [-1198601713, -67, 116, -29578]
1350 Shrunk example to [-1198601713]
1351 Shrunk example to [-1198601600]
1352 Shrunk example to [-1191228800]
1353 Shrunk example to [-8421504]
1354 Shrunk example to [-32896]
1355 Shrunk example to [-128]
1356 Shrunk example to [64]
1357 Shrunk example to [32]
1358 Shrunk example to [16]
1359 Shrunk example to [8]
1360 Shrunk example to [4]
1361 Shrunk example to [3]
1362 Shrunk example to [2]
1363 Shrunk example to [1]
1364 [1]
1365 The four levels are quiet, normal, verbose and debug. normal is the de‐
1367 fault, while in quiet mode Hypothesis will not print anything out, not
1368 even the final falsifying example. debug is basically verbose but a bit
1369 more so. You probably don't want it.
1370 If you are using pytest, you may also need to disable output capturing
1372 for passing tests.
1373 Building settings objects
1375 Settings can be created by calling settings with any of the available
1376 settings values. Any absent ones will be set to defaults:
1377 >>> from hypothesis import settings
1379 >>> settings().max_examples
1380 100
1381 >>> settings(max_examples=10).max_examples
1382 10
1383 You can also pass a 'parent' settings object as the first argument, and
1385 any settings you do not specify as keyword arguments will be copied
1386 from the parent settings:
1387 >>> parent = settings(max_examples=10)
1389 >>> child = settings(parent, deadline=None)
1390 >>> parent.max_examples == child.max_examples == 10
1391 True
1392 >>> parent.deadline
1393 200
1394 >>> child.deadline is None
1395 True
1396 Default settings
1398 At any given point in your program there is a current default settings,
1399 available as settings.default. As well as being a settings object in
1400 its own right, all newly created settings objects which are not explic‐
1401 itly based off another settings are based off the default, so will in‐
1402 herit any values that are not explicitly set from it.
1403 You can change the defaults by using profiles.
1405 Settings profiles
1407 Depending on your environment you may want different default settings.
1408 For example: during development you may want to lower the number of ex‐
1409 amples to speed up the tests. However, in a CI environment you may want
1410 more examples so you are more likely to find bugs.
1411 Hypothesis allows you to define different settings profiles. These pro‐
1413 files can be loaded at any time.
1414 static settings.register_profile(name, parent=None, **kwargs)
1416 Registers a collection of values to be used as a settings pro‐
1417 file.
1418 Settings profiles can be loaded by name - for example, you might
1420 create a 'fast' profile which runs fewer examples, keep the 'de‐
1421 fault' profile, and create a 'ci' profile that increases the
1422 number of examples and uses a different database to store fail‐
1423 ures.
1424 The arguments to this method are exactly as for settings: op‐
1426 tional parent settings, and keyword arguments for each setting
1427 that will be set differently to parent (or settings.default, if
1428 parent is None).
1429 static settings.get_profile(name)
1431 Return the profile with the given name.
1432 static settings.load_profile(name)
1434 Loads in the settings defined in the profile provided.
1435 If the profile does not exist, InvalidArgument will be raised.
1437 Any setting not defined in the profile will be the library de‐
1438 fined default for that setting.
1439 Loading a profile changes the default settings but will not change the
1441 behaviour of tests that explicitly change the settings.
1442 >>> from hypothesis import settings
1444 >>> settings.register_profile("ci", max_examples=1000)
1445 >>> settings().max_examples
1446 100
1447 >>> settings.load_profile("ci")
1448 >>> settings().max_examples
1449 1000
1450 Instead of loading the profile and overriding the defaults you can re‐
1452 trieve profiles for specific tests.
1453 >>> settings.get_profile("ci").max_examples
1455 1000
1456 Optionally, you may define the environment variable to load a profile
1458 for you. This is the suggested pattern for running your tests on CI.
1459 The code below should run in a conftest.py or any setup/initialization
1460 section of your test suite. If this variable is not defined the Hy‐
1461 pothesis defined defaults will be loaded.
1462 >>> import os
1464 >>> from hypothesis import settings, Verbosity
1465 >>> settings.register_profile("ci", max_examples=1000)
1466 >>> settings.register_profile("dev", max_examples=10)
1467 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1468 >>> settings.load_profile(os.getenv("HYPOTHESIS_PROFILE", "default"))
1469 If you are using the hypothesis pytest plugin and your profiles are
1471 registered by your conftest you can load one with the command line op‐
1472 tion --hypothesis-profile.
1473 $ pytest tests --hypothesis-profile <profile-name>
1475 Health checks
1477 Hypothesis' health checks are designed to detect and warn you about
1478 performance problems where your tests are slow, inefficient, or gener‐
1479 ating very large examples.
1480 If this is expected, e.g. when generating large arrays or dataframes,
1482 you can selectively disable them with the suppress_health_check set‐
1483 ting. The argument for this parameter is a list with elements drawn
1484 from any of the class-level attributes of the HealthCheck class. Using
1485 a value of list(HealthCheck) will disable all health checks.
1486 class hypothesis.HealthCheck(value, names=None, *values, module=None,
1488 qualname=None, type=None, start=1, boundary=None)
1489 Arguments for suppress_health_check.
1490 Each member of this enum is a type of health check to suppress.
1492 data_too_large = 1
1494 Checks if too many examples are aborted for being too
1495 large.
1496 This is measured by the number of random choices that Hy‐
1498 pothesis makes in order to generate something, not the
1499 size of the generated object. For example, choosing a
1500 100MB object from a predefined list would take only a few
1501 bits, while generating 10KB of JSON from scratch might
1502 trigger this health check.
1503 filter_too_much = 2
1505 Check for when the test is filtering out too many exam‐
1506 ples, either through use of assume() or filter(), or oc‐
1507 casionally for Hypothesis internal reasons.
1508 too_slow = 3
1510 Check for when your data generation is extremely slow and
1511 likely to hurt testing.
1512 return_value = 5
1514 Deprecated; we always error if a test returns a non-None
1515 value.
1516 large_base_example = 7
1518 Checks if the natural example to shrink towards is very
1519 large.
1520 not_a_test_method = 8
1522 Deprecated; we always error if @given is applied to a
1523 method defined by python:unittest.TestCase (i.e. not a
1524 test).
1525 function_scoped_fixture = 9
1527 Checks if @given has been applied to a test with a pytest
1528 function-scoped fixture. Function-scoped fixtures run
1529 once for the whole function, not once per example, and
1530 this is usually not what you want.
1531 Because of this limitation, tests that need to set up or
1533 reset state for every example need to do so manually
1534 within the test itself, typically using an appropriate
1535 context manager.
1536 Suppress this health check only in the rare case that you
1538 are using a function-scoped fixture that does not need to
1539 be reset between individual examples, but for some reason
1540 you cannot use a wider fixture scope (e.g. session scope,
1541 module scope, class scope).
1542 This check requires the Hypothesis pytest plugin, which
1544 is enabled by default when running Hypothesis inside
1545 pytest.
1546
1548 Most things should be easy to generate and everything should be possi‐
1549 ble.
1550 To support this principle Hypothesis provides strategies for most
1552 built-in types with arguments to constrain or adjust the output, as
1553 well as higher-order strategies that can be composed to generate more
1554 complex types.
1555 This document is a guide to what strategies are available for generat‐
1557 ing data and how to build them. Strategies have a variety of other im‐
1558 portant internal features, such as how they simplify, but the data they
1559 can generate is the only public part of their API.
1560 Core strategies
1562 Functions for building strategies are all available in the hypothe‐
1563 sis.strategies module. The salient functions from it are as follows:
1564 hypothesis.strategies.binary(*, min_size=0, max_size=None)
1566 Generates python:bytes.
1567 The generated python:bytes will have a length of at least
1569 min_size and at most max_size. If max_size is None there is no
1570 upper limit.
1571 Examples from this strategy shrink towards smaller strings and
1573 lower byte values.
1574 hypothesis.strategies.booleans()
1576 Returns a strategy which generates instances of python:bool.
1577 Examples from this strategy will shrink towards False (i.e.
1579 shrinking will replace True with False where possible).
1580 hypothesis.strategies.builds(target, /, *args, **kwargs)
1582 Generates values by drawing from args and kwargs and passing
1583 them to the callable (provided as the first positional argument)
1584 in the appropriate argument position.
1585 e.g. builds(target, integers(), flag=booleans()) would draw an
1587 integer i and a boolean b and call target(i, flag=b).
1588 If the callable has type annotations, they will be used to infer
1590 a strategy for required arguments that were not passed to
1591 builds. You can also tell builds to infer a strategy for an op‐
1592 tional argument by passing ... (python:Ellipsis) as a keyword
1593 argument to builds, instead of a strategy for that argument to
1594 the callable.
1595 If the callable is a class defined with attrs, missing required
1597 arguments will be inferred from the attribute on a best-effort
1598 basis, e.g. by checking attrs standard validators. Dataclasses
1599 are handled natively by the inference from type hints.
1600 Examples from this strategy shrink by shrinking the argument
1602 values to the callable.
1603 hypothesis.strategies.characters(*, whitelist_categories=None, black‐
1605 list_categories=None, blacklist_characters=None, min_codepoint=None,
1606 max_codepoint=None, whitelist_characters=None)
1607 Generates characters, length-one python:strings, following spec‐
1608 ified filtering rules.
1609 • When no filtering rules are specified, any character can be
1611 produced.
1612 • If min_codepoint or max_codepoint is specified, then only
1614 characters having a codepoint in that range will be produced.
1615 • If whitelist_categories is specified, then only characters
1617 from those Unicode categories will be produced. This is a fur‐
1618 ther restriction, characters must also satisfy min_codepoint
1619 and max_codepoint.
1620 • If blacklist_categories is specified, then any character from
1622 those categories will not be produced. Any overlap between
1623 whitelist_categories and blacklist_categories will raise an
1624 exception, as each character can only belong to a single
1625 class.
1626 • If whitelist_characters is specified, then any additional
1628 characters in that list will also be produced.
1629 • If blacklist_characters is specified, then any characters in
1631 that list will be not be produced. Any overlap between
1632 whitelist_characters and blacklist_characters will raise an
1633 exception.
1634 The _codepoint arguments must be integers between zero and
1636 python:sys.maxunicode. The _characters arguments must be col‐
1637 lections of length-one unicode strings, such as a unicode
1638 string.
1639 The _categories arguments must be used to specify either the
1641 one-letter Unicode major category or the two-letter Unicode
1642 general category. For example, ('Nd', 'Lu') signifies "Number,
1643 decimal digit" and "Letter, uppercase". A single letter ('major
1644 category') can be given to match all corresponding categories,
1645 for example 'P' for characters in any punctuation category.
1646 Examples from this strategy shrink towards the codepoint for
1648 '0', or the first allowable codepoint after it if '0' is ex‐
1649 cluded.
1650 hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magni‐
1652 tude=None, allow_infinity=None, allow_nan=None, allow_subnormal=True,
1653 width=128)
1654 Returns a strategy that generates python:complex numbers.
1655 This strategy draws complex numbers with constrained magnitudes.
1657 The min_magnitude and max_magnitude parameters should be
1658 non-negative Real numbers; a value of None corresponds an infi‐
1659 nite upper bound.
1660 If min_magnitude is nonzero or max_magnitude is finite, it is an
1662 error to enable allow_nan. If max_magnitude is finite, it is an
1663 error to enable allow_infinity.
1664 allow_infinity, allow_nan, and allow_subnormal are applied to
1666 each part of the complex number separately, as for floats().
1667 The magnitude constraints are respected up to a relative error
1669 of (around) floating-point epsilon, due to implementation via
1670 the system sqrt function.
1671 The width argument specifies the maximum number of bits of pre‐
1673 cision required to represent the entire generated complex num‐
1674 ber. Valid values are 32, 64 or 128, which correspond to the
1675 real and imaginary components each having width 16, 32 or 64,
1676 respectively. Passing width=64 will still use the builtin
1677 128-bit python:complex class, but always for values which can be
1678 exactly represented as two 32-bit floats.
1679 Examples from this strategy shrink by shrinking their real and
1681 imaginary parts, as floats().
1682 If you need to generate complex numbers with particular real and
1684 imaginary parts or relationships between parts, consider using
1685 builds(complex, ...) or @composite respectively.
1686 hypothesis.strategies.composite(f)
1688 Defines a strategy that is built out of potentially arbitrarily
1689 many other strategies.
1690 This is intended to be used as a decorator. See the full docu‐
1692 mentation for more details about how to use this function.
1693 Examples from this strategy shrink by shrinking the output of
1695 each draw call.
1696 hypothesis.strategies.data()
1698 This isn't really a normal strategy, but instead gives you an
1699 object which can be used to draw data interactively from other
1700 strategies.
1701 See the rest of the documentation for more complete information.
1703 Examples from this strategy do not shrink (because there is only
1705 one), but the result of calls to each draw() call shrink as they
1706 normally would.
1707 class hypothesis.strategies.DataObject(data)
1709 This type only exists so that you can write type hints for tests
1710 using the data() strategy. Do not use it directly!
1711 hypothesis.strategies.dates(min_value=datetime.date.min,
1713 max_value=datetime.date.max)
1714 A strategy for dates between min_value and max_value.
1715 Examples from this strategy shrink towards January 1st 2000.
1717 hypothesis.strategies.datetimes(min_value=datetime.datetime.min,
1719 max_value=datetime.datetime.max, *, timezones=none(), allow_imagi‐
1720 nary=True)
1721 A strategy for generating datetimes, which may be time‐
1722 zone-aware.
1723 This strategy works by drawing a naive datetime between
1725 min_value and max_value, which must both be naive (have no time‐
1726 zone).
1727 timezones must be a strategy that generates either None, for
1729 naive datetimes, or tzinfo objects for 'aware' datetimes. You
1730 can construct your own, though we recommend using one of these
1731 built-in strategies:
1732 • with Python 3.9 or newer or backports.zoneinfo:
1734 hypothesis.strategies.timezones();
1735 • with dateutil: hypothesis.extra.dateutil.timezones(); or
1737 • with pytz: hypothesis.extra.pytz.timezones().
1739 You may pass allow_imaginary=False to filter out "imaginary"
1741 datetimes which did not (or will not) occur due to daylight sav‐
1742 ings, leap seconds, timezone and calendar adjustments, etc.
1743 Imaginary datetimes are allowed by default, because malformed
1744 timestamps are a common source of bugs.
1745 Examples from this strategy shrink towards midnight on January
1747 1st 2000, local time.
1748 hypothesis.strategies.decimals(min_value=None, max_value=None, *, al‐
1750 low_nan=None, allow_infinity=None, places=None)
1751 Generates instances of python:decimal.Decimal, which may be:
1752 • A finite rational number, between min_value and max_value.
1754 • Not a Number, if allow_nan is True. None means "allow NaN,
1756 unless min_value and max_value are not None".
1757 • Positive or negative infinity, if max_value and min_value re‐
1759 spectively are None, and allow_infinity is not False. None
1760 means "allow infinity, unless excluded by the min and max val‐
1761 ues".
1762 Note that where floats have one NaN value, Decimals have four:
1764 signed, and either quiet or signalling. See the decimal module
1765 docs for more information on special values.
1766 If places is not None, all finite values drawn from the strategy
1768 will have that number of digits after the decimal place.
1769 Examples from this strategy do not have a well defined shrink
1771 order but try to maximize human readability when shrinking.
1772 hypothesis.strategies.deferred(definition)
1774 A deferred strategy allows you to write a strategy that refer‐
1775 ences other strategies that have not yet been defined. This al‐
1776 lows for the easy definition of recursive and mutually recursive
1777 strategies.
1778 The definition argument should be a zero-argument function that
1780 returns a strategy. It will be evaluated the first time the
1781 strategy is used to produce an example.
1782 Example usage:
1784 >>> import hypothesis.strategies as st
1786 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1787 >>> x.example()
1788 (((False, (True, True)), (False, True)), (True, True))
1789 >>> x.example()
1790 True
1791 Mutual recursion also works fine:
1793 >>> a = st.deferred(lambda: st.booleans() | b)
1795 >>> b = st.deferred(lambda: st.tuples(a, a))
1796 >>> a.example()
1797 True
1798 >>> b.example()
1799 (False, (False, ((False, True), False)))
1800 Examples from this strategy shrink as they normally would from
1802 the strategy returned by the definition.
1803 hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class
1805 'dict'>, min_size=0, max_size=None)
1806 Generates dictionaries of type dict_class with keys drawn from
1807 the keys argument and values drawn from the values argument.
1808 The size parameters have the same interpretation as for lists().
1810 Examples from this strategy shrink by trying to remove keys from
1812 the generated dictionary, and by shrinking each generated key
1813 and value.
1814 class hypothesis.strategies.DrawFn
1816 This type only exists so that you can write type hints for func‐
1817 tions decorated with @composite.
1818 @composite
1820 def list_and_index(draw: DrawFn) -> Tuple[int, str]:
1821 i = draw(integers()) # type inferred as 'int'
1822 s = draw(text()) # type inferred as 'str'
1823 return i, s
1824 hypothesis.strategies.emails(*, domains=domains())
1826 A strategy for generating email addresses as unicode strings.
1827 The address format is specified in RFC 5322#section-3.4.1. Val‐
1828 ues shrink towards shorter local-parts and host domains.
1829 If domains is given then it must be a strategy that generates
1831 domain names for the emails, defaulting to domains().
1832 This strategy is useful for generating "user data" for tests, as
1834 mishandling of email addresses is a common source of bugs.
1835 hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)
1837 Generates a dictionary of the same type as mapping with a fixed
1838 set of keys mapping to strategies. mapping must be a dict sub‐
1839 class.
1840 Generated values have all keys present in mapping, in iteration
1842 order, with the corresponding values drawn from mapping[key].
1843 If optional is passed, the generated value may or may not con‐
1845 tain each key from optional and a value drawn from the corre‐
1846 sponding strategy. Generated values may contain optional keys
1847 in an arbitrary order.
1848 Examples from this strategy shrink by shrinking each individual
1850 value in the generated dictionary, and omitting optional
1851 key-value pairs.
1852 hypothesis.strategies.floats(min_value=None, max_value=None, *, al‐
1854 low_nan=None, allow_infinity=None, allow_subnormal=None, width=64, ex‐
1855 clude_min=False, exclude_max=False)
1856 Returns a strategy which generates floats.
1857 • If min_value is not None, all values will be >= min_value (or
1859 > min_value if exclude_min).
1860 • If max_value is not None, all values will be <= max_value (or
1862 < max_value if exclude_max).
1863 • If min_value or max_value is not None, it is an error to en‐
1865 able allow_nan.
1866 • If both min_value and max_value are not None, it is an error
1868 to enable allow_infinity.
1869 • If inferred values range does not include subnormal values, it
1871 is an error to enable allow_subnormal.
1872 Where not explicitly ruled out by the bounds, subnormals, in‐
1874 finities, and NaNs are possible values generated by this strat‐
1875 egy.
1876 The width argument specifies the maximum number of bits of pre‐
1878 cision required to represent the generated float. Valid values
1879 are 16, 32, or 64. Passing width=32 will still use the builtin
1880 64-bit python:float class, but always for values which can be
1881 exactly represented as a 32-bit float.
1882 The exclude_min and exclude_max argument can be used to generate
1884 numbers from open or half-open intervals, by excluding the re‐
1885 spective endpoints. Excluding either signed zero will also ex‐
1886 clude the other. Attempting to exclude an endpoint which is
1887 None will raise an error; use allow_infinity=False to generate
1888 finite floats. You can however use e.g. min_value=-math.inf,
1889 exclude_min=True to exclude only one infinite endpoint.
1890 Examples from this strategy have a complicated and hard to ex‐
1892 plain shrinking behaviour, but it tries to improve "human read‐
1893 ability". Finite numbers will be preferred to infinity and in‐
1894 finity will be preferred to NaN.
1895 hypothesis.strategies.fractions(min_value=None, max_value=None, *,
1897 max_denominator=None)
1898 Returns a strategy which generates Fractions.
1899 If min_value is not None then all generated values are no less
1901 than min_value. If max_value is not None then all generated
1902 values are no greater than max_value. min_value and max_value
1903 may be anything accepted by the Fraction constructor.
1904 If max_denominator is not None then the denominator of any gen‐
1906 erated values is no greater than max_denominator. Note that
1907 max_denominator must be None or a positive integer.
1908 Examples from this strategy shrink towards smaller denominators,
1910 then closer to zero.
1911 hypothesis.strategies.from_regex(regex, *, fullmatch=False)
1913 Generates strings that contain a match for the given regex (i.e.
1914 ones for which python:re.search() will return a non-None re‐
1915 sult).
1916 regex may be a pattern or compiled regex. Both byte-strings and
1918 unicode strings are supported, and will generate examples of the
1919 same type.
1920 You can use regex flags such as python:re.IGNORECASE or
1922 python:re.DOTALL to control generation. Flags can be passed ei‐
1923 ther in compiled regex or inside the pattern with a (?iLmsux)
1924 group.
1925 Some regular expressions are only partly supported - the under‐
1927 lying strategy checks local matching and relies on filtering to
1928 resolve context-dependent expressions. Using too many of these
1929 constructs may cause health-check errors as too many examples
1930 are filtered out. This mainly includes (positive or negative)
1931 lookahead and lookbehind groups.
1932 If you want the generated string to match the whole regex you
1934 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1935 gle character string, while "." will return any string, and
1936 r"\A.$" will return a single character optionally followed by a
1937 "\n". Alternatively, passing fullmatch=True will ensure that
1938 the whole string is a match, as if you had used the \A and \Z
1939 markers.
1940 Examples from this strategy shrink towards shorter strings and
1942 lower character values, with exact behaviour that may depend on
1943 the pattern.
1944 hypothesis.strategies.from_type(thing)
1946 Looks up the appropriate search strategy for the given type.
1947 from_type is used internally to fill in missing arguments to
1949 builds() and can be used interactively to explore what strate‐
1950 gies are available or to debug type resolution.
1951 You can use register_type_strategy() to handle your custom
1953 types, or to globally redefine certain strategies - for example
1954 excluding NaN from floats, or use timezone-aware instead of
1955 naive time and datetime strategies.
1956 The resolution logic may be changed in a future version, but
1958 currently tries these five options:
1959 1. If thing is in the default lookup mapping or user-registered
1961 lookup, return the corresponding strategy. The default
1962 lookup covers all types with Hypothesis strategies, including
1963 extras where possible.
1964 2. If thing is from the python:typing module, return the corre‐
1966 sponding strategy (special logic).
1967 3. If thing has one or more subtypes in the merged lookup, re‐
1969 turn the union of the strategies for those types that are not
1970 subtypes of other elements in the lookup.
1971 4. Finally, if thing has type annotations for all required argu‐
1973 ments, and is not an abstract class, it is resolved via
1974 builds().
1975 5. Because abstract types cannot be instantiated, we treat ab‐
1977 stract types as the union of their concrete subclasses. Note
1978 that this lookup works via inheritance but not via register,
1979 so you may still need to use register_type_strategy().
1980 There is a valuable recipe for leveraging from_type() to gener‐
1982 ate "everything except" values from a specified type. I.e.
1983 def everything_except(excluded_types):
1985 return (
1986 from_type(type)
1987 .flatmap(from_type)
1988 .filter(lambda x: not isinstance(x, excluded_types))
1989 )
1990 For example, everything_except(int) returns a strategy that can
1992 generate anything that from_type() can ever generate, except for
1993 instances of python:int, and excluding instances of types added
1994 via register_type_strategy().
1995 This is useful when writing tests which check that invalid input
1997 is rejected in a certain way.
1998 hypothesis.strategies.frozensets(elements, *, min_size=0,
2000 max_size=None)
2001 This is identical to the sets function but instead returns
2002 frozensets.
2003 hypothesis.strategies.functions(*, like=lambda : ..., returns=...,
2005 pure=False)
2006 A strategy for functions, which can be used in callbacks.
2007 The generated functions will mimic the interface of like, which
2009 must be a callable (including a class, method, or function).
2010 The return value for the function is drawn from the returns ar‐
2011 gument, which must be a strategy. If returns is not passed, we
2012 attempt to infer a strategy from the return-type annotation if
2013 present, falling back to none().
2014 If pure=True, all arguments passed to the generated function
2016 must be hashable, and if passed identical arguments the original
2017 return value will be returned again - not regenerated, so beware
2018 mutable values.
2019 If pure=False, generated functions do not validate their argu‐
2021 ments, and may return a different value if called again with the
2022 same arguments.
2023 Generated functions can only be called within the scope of the
2025 @given which created them. This strategy does not support .ex‐
2026 ample().
2027 hypothesis.strategies.integers(min_value=None, max_value=None)
2029 Returns a strategy which generates integers.
2030 If min_value is not None then all values will be >= min_value.
2032 If max_value is not None then all values will be <= max_value
2033 Examples from this strategy will shrink towards zero, and nega‐
2035 tive values will also shrink towards positive (i.e. -n may be
2036 replaced by +n).
2037 hypothesis.strategies.ip_addresses(*, v=None, network=None)
2039 Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Ad‐
2040 dresses, or leave unspecified to allow both versions.
2041 network may be an IPv4Network or IPv6Network, or a string repre‐
2043 senting a network such as "127.0.0.0/24" or "2001:db8::/32". As
2044 well as generating addresses within a particular routable net‐
2045 work, this can be used to generate addresses from a reserved
2046 range listed in the IANA registries.
2047 If you pass both v and network, they must be for the same ver‐
2049 sion.
2050 hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None,
2052 unique_by=None, unique=False)
2053 This has the same behaviour as lists, but returns iterables in‐
2054 stead.
2055 Some iterables cannot be indexed (e.g. sets) and some do not
2057 have a fixed length (e.g. generators). This strategy produces
2058 iterators, which cannot be indexed and do not have a fixed
2059 length. This ensures that you do not accidentally depend on se‐
2060 quence behaviour.
2061 hypothesis.strategies.just(value)
2063 Return a strategy which only generates value.
2064 Note: value is not copied. Be wary of using mutable values.
2066 If value is the result of a callable, you can use
2068 builds(callable) instead of just(callable()) to get a fresh
2069 value each time.
2070 Examples from this strategy do not shrink (because there is only
2072 one).
2073 hypothesis.strategies.lists(elements, *, min_size=0, max_size=None,
2075 unique_by=None, unique=False)
2076 Returns a list containing values drawn from elements with length
2077 in the interval [min_size, max_size] (no bounds in that direc‐
2078 tion if these are None). If max_size is 0, only the empty list
2079 will be drawn.
2080 If unique is True (or something that evaluates to True), we com‐
2082 pare direct object equality, as if unique_by was lambda x: x.
2083 This comparison only works for hashable types.
2084 If unique_by is not None it must be a callable or tuple of
2086 callables returning a hashable type when given a value drawn
2087 from elements. The resulting list will satisfy the condition
2088 that for i != j, unique_by(result[i]) != unique_by(result[j]).
2089 If unique_by is a tuple of callables the uniqueness will be re‐
2091 spective to each callable.
2092 For example, the following will produce two columns of integers
2094 with both columns being unique respectively.
2095 >>> twoints = st.tuples(st.integers(), st.integers())
2097 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
2098 Examples from this strategy shrink by trying to remove elements
2100 from the list, and by shrinking each individual element of the
2101 list.
2102 hypothesis.strategies.none()
2104 Return a strategy which only generates None.
2105 Examples from this strategy do not shrink (because there is only
2107 one).
2108 hypothesis.strategies.nothing()
2110 This strategy never successfully draws a value and will always
2111 reject on an attempt to draw.
2112 Examples from this strategy do not shrink (because there are
2114 none).
2115 hypothesis.strategies.one_of(*args)
2117 Return a strategy which generates values from any of the argu‐
2118 ment strategies.
2119 This may be called with one iterable argument instead of multi‐
2121 ple strategy arguments, in which case one_of(x) and one_of(*x)
2122 are equivalent.
2123 Examples from this strategy will generally shrink to ones that
2125 come from strategies earlier in the list, then shrink according
2126 to behaviour of the strategy that produced them. In order to get
2127 good shrinking behaviour, try to put simpler strategies first.
2128 e.g. one_of(none(), text()) is better than one_of(text(),
2129 none()).
2130 This is especially important when using recursive strategies.
2132 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
2133 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
2134 st.none()) will shrink very badly indeed.
2135 hypothesis.strategies.permutations(values)
2137 Return a strategy which returns permutations of the ordered col‐
2138 lection values.
2139 Examples from this strategy shrink by trying to become closer to
2141 the original order of values.
2142 hypothesis.strategies.random_module()
2144 The Hypothesis engine handles PRNG state for the stdlib and
2145 Numpy random modules internally, always seeding them to zero and
2146 restoring the previous state after the test.
2147 If having a fixed seed would unacceptably weaken your tests, and
2149 you cannot use a random.Random instance provided by randoms(),
2150 this strategy calls python:random.seed() with an arbitrary inte‐
2151 ger and passes you an opaque object whose repr displays the seed
2152 value for debugging. If numpy.random is available, that state
2153 is also managed.
2154 Examples from these strategy shrink to seeds closer to zero.
2156 hypothesis.strategies.randoms(*, note_method_calls=False, use_true_ran‐
2158 dom=False)
2159 Generates instances of random.Random. The generated Random in‐
2160 stances are of a special HypothesisRandom subclass.
2161 • If note_method_calls is set to True, Hypothesis will print the
2163 randomly drawn values in any falsifying test case. This can be
2164 helpful for debugging the behaviour of randomized algorithms.
2165 • If use_true_random is set to True then values will be drawn
2167 from their usual distribution, otherwise they will actually be
2168 Hypothesis generated values (and will be shrunk accordingly
2169 for any failing test case). Setting use_true_random=False will
2170 tend to expose bugs that would occur with very low probability
2171 when it is set to True, and this flag should only be set to
2172 True when your code relies on the distribution of values for
2173 correctness.
2174 hypothesis.strategies.recursive(base, extend, *, max_leaves=100)
2176 base: A strategy to start from.
2177 extend: A function which takes a strategy and returns a new
2179 strategy.
2180 max_leaves: The maximum number of elements to be drawn from base
2182 on a given run.
2183 This returns a strategy S such that S = extend(base | S). That
2185 is, values may be drawn from base, or from any strategy reach‐
2186 able by mixing applications of | and extend.
2187 An example may clarify: recursive(booleans(), lists) would re‐
2189 turn a strategy that may return arbitrarily nested and mixed
2190 lists of booleans. So e.g. False, [True], [False, []], and
2191 [[[[True]]]] are all valid values to be drawn from that strat‐
2192 egy.
2193 Examples from this strategy shrink by trying to reduce the
2195 amount of recursion and by shrinking according to the shrinking
2196 behaviour of base and the result of extend.
2197 hypothesis.strategies.register_type_strategy(custom_type, strategy)
2199 Add an entry to the global type-to-strategy lookup.
2200 This lookup is used in builds() and @given.
2202 builds() will be used automatically for classes with type anno‐
2204 tations on __init__ , so you only need to register a strategy if
2205 one or more arguments need to be more tightly defined than their
2206 type-based default, or if you want to supply a strategy for an
2207 argument with a default value.
2208 strategy may be a search strategy, or a function that takes a
2210 type and returns a strategy (useful for generic types).
2211 Note that you may not register a parametrised generic type (such
2213 as MyCollection[int]) directly, because the resolution logic
2214 does not handle this case correctly. Instead, you may register
2215 a function for MyCollection and inspect the type parameters
2216 within that function.
2217 hypothesis.strategies.runner(*, default=not_set)
2219 A strategy for getting "the current test runner", whatever that
2220 may be. The exact meaning depends on the entry point, but it
2221 will usually be the associated 'self' value for it.
2222 If there is no current test runner and a default is provided,
2224 return that default. If no default is provided, raises Invali‐
2225 dArgument.
2226 Examples from this strategy do not shrink (because there is only
2228 one).
2229 hypothesis.strategies.sampled_from(elements)
2231 Returns a strategy which generates any value present in ele‐
2232 ments.
2233 Note that as with just(), values will not be copied and thus you
2235 should be careful of using mutable data.
2236 sampled_from supports ordered collections, as well as Enum ob‐
2238 jects. Flag objects may also generate any combination of their
2239 members.
2240 Examples from this strategy shrink by replacing them with values
2242 earlier in the list. So e.g. sampled_from([10, 1]) will shrink
2243 by trying to replace 1 values with 10, and sampled_from([1, 10])
2244 will shrink by trying to replace 10 values with 1.
2245 It is an error to sample from an empty sequence, because return‐
2247 ing nothing() makes it too easy to silently drop parts of com‐
2248 pound strategies. If you need that behaviour, use sam‐
2249 pled_from(seq) if seq else nothing().
2250 hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)
2252 This has the same behaviour as lists, but returns sets instead.
2253 Note that Hypothesis cannot tell if values are drawn from ele‐
2255 ments are hashable until running the test, so you can define a
2256 strategy for sets of an unhashable type but it will fail at test
2257 time.
2258 Examples from this strategy shrink by trying to remove elements
2260 from the set, and by shrinking each individual element of the
2261 set.
2262 hypothesis.strategies.shared(base, *, key=None)
2264 Returns a strategy that draws a single shared value per run,
2265 drawn from base. Any two shared instances with the same key will
2266 share the same value, otherwise the identity of this strategy
2267 will be used. That is:
2268 >>> s = integers() # or any other strategy
2270 >>> x = shared(s)
2271 >>> y = shared(s)
2272 In the above x and y may draw different (or potentially the
2274 same) values. In the following they will always draw the same:
2275 >>> x = shared(s, key="hi")
2277 >>> y = shared(s, key="hi")
2278 Examples from this strategy shrink as per their base strategy.
2280 hypothesis.strategies.slices(size)
2282 Generates slices that will select indices up to the supplied
2283 size
2284 Generated slices will have start and stop indices that range
2286 from -size to size - 1 and will step in the appropriate direc‐
2287 tion. Slices should only produce an empty selection if the start
2288 and end are the same.
2289 Examples from this strategy shrink toward 0 and smaller values
2291 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
2293 gories=('Cs',)), *, min_size=0, max_size=None)
2294 Generates strings with characters drawn from alphabet, which
2295 should be a collection of length one strings or a strategy gen‐
2296 erating such strings.
2297 The default alphabet strategy can generate the full unicode
2299 range but excludes surrogate characters because they are invalid
2300 in the UTF-8 encoding. You can use characters() without argu‐
2301 ments to find surrogate-related bugs such as bpo-34454.
2302 min_size and max_size have the usual interpretations. Note that
2304 Python measures string length by counting codepoints: U+00C5 Å
2305 is a single character, while U+0041 U+030A Å is two - the A,
2306 and a combining ring above.
2307 Examples from this strategy shrink towards shorter strings, and
2309 with the characters in the text shrinking as per the alphabet
2310 strategy. This strategy does not normalize() examples, so gen‐
2311 erated strings may be in any or none of the 'normal forms'.
2312 hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min,
2314 max_value=datetime.timedelta.max)
2315 A strategy for timedeltas between min_value and max_value.
2316 Examples from this strategy shrink towards zero.
2318 hypothesis.strategies.times(min_value=datetime.time.min,
2320 max_value=datetime.time.max, *, timezones=none())
2321 A strategy for times between min_value and max_value.
2322 The timezones argument is handled as for datetimes().
2324 Examples from this strategy shrink towards midnight, with the
2326 timezone component shrinking as for the strategy that provided
2327 it.
2328 hypothesis.strategies.timezone_keys(*, allow_prefix=True)
2330 A strategy for IANA timezone names.
2331 As well as timezone names like "UTC", "Australia/Sydney", or
2333 "America/New_York", this strategy can generate:
2334 • Aliases such as "Antarctica/McMurdo", which links to "Pa‐
2336 cific/Auckland".
2337 • Deprecated names such as "Antarctica/South_Pole", which also
2339 links to "Pacific/Auckland". Note that most but not all dep‐
2340 recated timezone names are also aliases.
2341 • Timezone names with the "posix/" or "right/" prefixes, unless
2343 allow_prefix=False.
2344 These strings are provided separately from Tzinfo objects - such
2346 as ZoneInfo instances from the timezones() strategy - to facili‐
2347 tate testing of timezone logic without needing workarounds to
2348 access non-canonical names.
2349 NOTE:
2351 The python:zoneinfo module is new in Python 3.9, so you will
2352 need to install the backports.zoneinfo module on earlier ver‐
2353 sions.
2354 On Windows, you will also need to install the tzdata package.
2356 pip install hypothesis[zoneinfo] will install these condi‐
2358 tional dependencies if and only if they are needed.
2359 On Windows, you may need to access IANA timezone data via the
2361 tzdata package. For non-IANA timezones, such as Windows-native
2362 names or GNU TZ strings, we recommend using sampled_from() with
2363 the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().
2364 hypothesis.strategies.timezones(*, no_cache=False)
2366 A strategy for python:zoneinfo.ZoneInfo objects.
2367 If no_cache=True, the generated instances are constructed using
2369 ZoneInfo.no_cache instead of the usual constructor. This may
2370 change the semantics of your datetimes in surprising ways, so
2371 only use it if you know that you need to!
2372 NOTE:
2374 The python:zoneinfo module is new in Python 3.9, so you will
2375 need to install the backports.zoneinfo module on earlier ver‐
2376 sions.
2377 On Windows, you will also need to install the tzdata package.
2379 pip install hypothesis[zoneinfo] will install these condi‐
2381 tional dependencies if and only if they are needed.
2382 hypothesis.strategies.tuples(*args)
2384 Return a strategy which generates a tuple of the same length as
2385 args by generating the value at index i from args[i].
2386 e.g. tuples(integers(), integers()) would generate a tuple of
2388 length two with both values an integer.
2389 Examples from this strategy shrink by shrinking their component
2391 parts.
2392 hypothesis.strategies.uuids(*, version=None, allow_nil=False)
2394 Returns a strategy that generates UUIDs.
2395 If the optional version argument is given, value is passed
2397 through to UUID and only UUIDs of that version will be gener‐
2398 ated.
2399 If allow_nil is True, generate the nil UUID much more often.
2401 Otherwise, all returned values from this will be unique, so e.g.
2402 if you do lists(uuids()) the resulting list will never contain
2403 duplicates.
2404 Examples from this strategy don't have any meaningful shrink or‐
2406 der.
2407 Provisional strategies
2409 This module contains various provisional APIs and strategies.
2410 It is intended for internal use, to ease code reuse, and is not stable.
2412 Point releases may move or break the contents at any time!
2413 Internet strategies should conform to RFC 3986 or the authoritative
2415 definitions it links to. If not, report the bug!
2416 hypothesis.provisional.domains(*, max_length=255, max_ele‐
2418 ment_length=63)
2419 Generate RFC 1035 compliant fully qualified domain names.
2420 hypothesis.provisional.urls()
2422 A strategy for RFC 3986, generating http/https URLs.
2423 Shrinking
2425 When using strategies it is worth thinking about how the data shrinks.
2426 Shrinking is the process by which Hypothesis tries to produce human
2427 readable examples when it finds a failure - it takes a complex example
2428 and turns it into a simpler one.
2429 Each strategy defines an order in which it shrinks - you won't usually
2431 need to care about this much, but it can be worth being aware of as it
2432 can affect what the best way to write your own strategies is.
2433 The exact shrinking behaviour is not a guaranteed part of the API, but
2435 it doesn't change that often and when it does it's usually because we
2436 think the new way produces nicer examples.
2437 Possibly the most important one to be aware of is one_of(), which has a
2439 preference for values produced by strategies earlier in its argument
2440 list. Most of the others should largely "do the right thing" without
2441 you having to think about it.
2442 Adapting strategies
2444 Often it is the case that a strategy doesn't produce exactly what you
2445 want it to and you need to adapt it. Sometimes you can do this in the
2446 test, but this hurts reuse because you then have to repeat the adaption
2447 in every test.
2448 Hypothesis gives you ways to build strategies from other strategies
2450 given functions for transforming the data.
2451 Mapping
2453 map is probably the easiest and most useful of these to use. If you
2454 have a strategy s and a function f, then an example s.map(f).example()
2455 is f(s.example()), i.e. we draw an example from s and then apply f to
2456 it.
2457 e.g.:
2459 >>> lists(integers()).map(sorted).example()
2461 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2462 Note that many things that you might use mapping for can also be done
2464 with builds(), and if you find yourself indexing into a tuple within
2465 .map() it's probably time to use that instead.
2466 Filtering
2468 filter lets you reject some examples. s.filter(f).example() is some ex‐
2469 ample of s such that f(example) is truthy.
2470 >>> integers().filter(lambda x: x > 11).example()
2472 26126
2473 >>> integers().filter(lambda x: x > 11).example()
2474 23324
2475 It's important to note that filter isn't magic and if your condition is
2477 too hard to satisfy then this can fail:
2478 >>> integers().filter(lambda x: False).example()
2480 Traceback (most recent call last):
2481 ...
2482 hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries
2483 In general you should try to use filter only to avoid corner cases that
2485 you don't want rather than attempting to cut out a large chunk of the
2486 search space.
2487 A technique that often works well here is to use map to first transform
2489 the data and then use filter to remove things that didn't work out. So
2490 for example if you wanted pairs of integers (x,y) such that x < y you
2491 could do the following:
2492 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2494 [-8543729478746591815, 3760495307320535691]
2495 Chaining strategies together
2497 Finally there is flatmap. flatmap draws an example, then turns that ex‐
2498 ample into a strategy, then draws an example from that strategy.
2499 It may not be obvious why you want this at first, but it turns out to
2501 be quite useful because it lets you generate different types of data
2502 with relationships to each other.
2503 For example suppose we wanted to generate a list of lists of the same
2505 length:
2506 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2508 ... lambda n: lists(lists(integers(), min_size=n, max_size=n))
2509 ... )
2510 >>> rectangle_lists.example()
2511 []
2512 >>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
2513 [[], [], [], [], [], [], [], [], [], []]
2514 >>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
2515 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2516 >>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
2517 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2518 In this example we first choose a length for our tuples, then we build
2520 a strategy which generates lists containing lists precisely of that
2521 length. The finds show what simple examples for this look like.
2522 Most of the time you probably don't want flatmap, but unlike filter and
2524 map which are just conveniences for things you could just do in your
2525 tests, flatmap allows genuinely new data generation that you wouldn't
2526 otherwise be able to easily do.
2527 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2529 don't know Haskell, ignore everything in these parentheses. You do not
2530 need to understand anything about monads to use this, or anything else
2531 in Hypothesis).
2532 Recursive data
2534 Sometimes the data you want to generate has a recursive definition.
2535 e.g. if you wanted to generate JSON data, valid JSON is:
2536 1. Any float, any boolean, any unicode string.
2538 2. Any list of valid JSON data
2540 3. Any dictionary mapping unicode strings to valid JSON data.
2542 The problem is that you cannot call a strategy recursively and expect
2544 it to not just blow up and eat all your memory. The other problem here
2545 is that not all unicode strings display consistently on different ma‐
2546 chines, so we'll restrict them in our doctest.
2547 The way Hypothesis handles this is with the recursive() strategy which
2549 you pass in a base case and a function that, given a strategy for your
2550 data type, returns a new strategy for it. So for example:
2551 >>> from string import printable
2553 ... from pprint import pprint
2554 >>> json = recursive(
2555 ... none() | booleans() | floats() | text(printable),
2556 ... lambda children: lists(children) | dictionaries(text(printable), children),
2557 ... )
2558 >>> pprint(json.example())
2559 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2560 >>> pprint(json.example())
2561 {'de(l': None,
2562 'nK': {'(Rt)': None,
2563 '+hoZh1YU]gy8': True,
2564 '8z]EIFA06^li^': 'LFE{Q',
2565 '9,': 'l{cA=/'}}
2566 That is, we start with our leaf data and then we augment it by allowing
2568 lists and dictionaries of anything we can generate as JSON data.
2569 The size control of this works by limiting the maximum number of values
2571 that can be drawn from the base strategy. So for example if we wanted
2572 to only generate really small JSON we could do this as:
2573 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2575 >>> small_lists.example()
2576 True
2577 >>> small_lists.example()
2578 [False]
2579 Composite strategies
2581 The @composite decorator lets you combine other strategies in more or
2582 less arbitrary ways. It's probably the main thing you'll want to use
2583 for complicated custom strategies.
2584 The composite decorator works by converting a function that returns one
2586 example into a function that returns a strategy that produces such ex‐
2587 amples - which you can pass to @given, modify with .map or .filter, and
2588 generally use like any other strategy.
2589 It does this by giving you a special function draw as the first argu‐
2591 ment, which can be used just like the corresponding method of the
2592 data() strategy within a test. In fact, the implementation is almost
2593 the same - but defining a strategy with @composite makes code reuse
2594 easier, and usually improves the display of failing examples.
2595 For example, the following gives you a list and an index into it:
2597 >>> @composite
2599 ... def list_and_index(draw, elements=integers()):
2600 ... xs = draw(lists(elements, min_size=1))
2601 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2602 ... return (xs, i)
2603 ...
2604 draw(s) is a function that should be thought of as returning s.exam‐
2606 ple(), except that the result is reproducible and will minimize cor‐
2607 rectly. The decorated function has the initial argument removed from
2608 the list, but will accept all the others in the expected order. De‐
2609 faults are preserved.
2610 >>> list_and_index()
2612 list_and_index()
2613 >>> list_and_index().example()
2614 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2615 >>> list_and_index(booleans())
2617 list_and_index(elements=booleans())
2618 >>> list_and_index(booleans()).example()
2619 ([True, False], 0)
2620 Note that the repr will work exactly like it does for all the built-in
2622 strategies: it will be a function that you can call to get the strategy
2623 in question, with values provided only if they do not match the de‐
2624 faults.
2625 You can use assume inside composite functions:
2627 @composite
2629 def distinct_strings_with_common_characters(draw):
2630 x = draw(text(min_size=1))
2631 y = draw(text(alphabet=x))
2632 assume(x != y)
2633 return (x, y)
2634 This works as assume normally would, filtering out any examples for
2636 which the passed in argument is falsey.
2637 Take care that your function can cope with adversarial draws, or ex‐
2639 plicitly rejects them using the .filter() method or assume() - our mu‐
2640 tation and shrinking logic can do some strange things, and a naive im‐
2641 plementation might lead to serious performance problems. For example:
2642 @composite
2644 def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
2645 # The bad way: if Hypothesis keeps generating e.g. zero,
2646 # we'll keep looping for a very long time.
2647 result = set()
2648 while len(result) < size:
2649 result.add(draw(elements))
2650 # The good way: use a filter, so Hypothesis can tell what's valid!
2651 for _ in range(size):
2652 result.add(draw(elements.filter(lambda x: x not in result)))
2653 return result
2654 If @composite is used to decorate a method or classmethod, the draw ar‐
2656 gument must come before self or cls. While we therefore recommend
2657 writing strategies as standalone functions and using the
2658 register_type_strategy() function to associate them with a class, meth‐
2659 ods are supported and the @composite decorator may be applied either
2660 before or after @classmethod or @staticmethod. See issue #2578 and
2661 pull request #2634 for more details.
2662 Drawing interactively in tests
2664 There is also the data() strategy, which gives you a means of using
2665 strategies interactively. Rather than having to specify everything up
2666 front in @given you can draw from strategies in the body of your test.
2667 This is similar to @composite, but even more powerful as it allows you
2669 to mix test code with example generation. The downside of this power
2670 is that data() is incompatible with explicit @example(...)s - and the
2671 mixed code is often harder to debug when something goes wrong.
2672 If you need values that are affected by previous draws but which don't
2674 depend on the execution of your test, stick to the simpler @composite.
2675 @given(data())
2677 def test_draw_sequentially(data):
2678 x = data.draw(integers())
2679 y = data.draw(integers(min_value=x))
2680 assert x < y
2681 If the test fails, each draw will be printed with the falsifying exam‐
2683 ple. e.g. the above is wrong (it has a boundary condition error), so
2684 will print:
2685 Falsifying example: test_draw_sequentially(data=data(...))
2687 Draw 1: 0
2688 Draw 2: 0
2689 As you can see, data drawn this way is simplified as usual.
2691 Optionally, you can provide a label to identify values generated by
2693 each call to data.draw(). These labels can be used to identify values
2694 in the output of a falsifying example.
2695 For instance:
2697 @given(data())
2699 def test_draw_sequentially(data):
2700 x = data.draw(integers(), label="First number")
2701 y = data.draw(integers(min_value=x), label="Second number")
2702 assert x < y
2703 will produce the output:
2705 Falsifying example: test_draw_sequentially(data=data(...))
2707 Draw 1 (First number): 0
2708 Draw 2 (Second number): 0
2709
2711 Hypothesis has minimal dependencies, to maximise compatibility and make
2712 installing Hypothesis as easy as possible.
2713 Our integrations with specific packages are therefore provided by extra
2715 modules that need their individual dependencies installed in order to
2716 work. You can install these dependencies using the setuptools extra
2717 feature as e.g. pip install hypothesis[django]. This will check in‐
2718 stallation of compatible versions.
2719 You can also just install hypothesis into a project using them, ignore
2721 the version constraints, and hope for the best.
2722 In general "Which version is Hypothesis compatible with?" is a hard
2724 question to answer and even harder to regularly test. Hypothesis is al‐
2725 ways tested against the latest compatible version and each package will
2726 note the expected compatibility range. If you run into a bug with any
2727 of these please specify the dependency version.
2728 There are separate pages for Hypothesis for Django users and Hypothesis
2730 for the scientific stack.
2731 hypothesis[cli]
2733 $ hypothesis --help
2734 Usage: hypothesis [OPTIONS] COMMAND [ARGS]...
2735 Options:
2737 --version Show the version and exit.
2738 -h, --help Show this message and exit.
2739 Commands:
2741 codemod `hypothesis codemod` refactors deprecated or inefficient code.
2742 fuzz [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
2743 write `hypothesis write` writes property-based tests for you!
2744 This module requires the click package, and provides Hypothesis' com‐
2746 mand-line interface, for e.g. 'ghostwriting' tests via the terminal.
2747 It's also where HypoFuzz adds the hypothesis fuzz command (learn more
2748 about that here).
2749 hypothesis[codemods]
2751 This module provides codemods based on the LibCST library, which can
2752 both detect and automatically fix issues with code that uses Hypothe‐
2753 sis, including upgrading from deprecated features to our recommended
2754 style.
2755 You can run the codemods via our CLI:
2757 $ hypothesis codemod --help
2759 Usage: hypothesis codemod [OPTIONS] PATH...
2760 `hypothesis codemod` refactors deprecated or inefficient code.
2762 It adapts `python -m libcst.tool`, removing many features and config
2764 options which are rarely relevant for this purpose. If you need more
2765 control, we encourage you to use the libcst CLI directly; if not this one
2766 is easier.
2767 PATH is the file(s) or directories of files to format in place, or "-" to
2769 read from stdin and write to stdout.
2770 Options:
2772 -h, --help Show this message and exit.
2773 Alternatively you can use python -m libcst.tool, which offers more con‐
2775 trol at the cost of additional configuration (adding 'hypothesis.extra'
2776 to the modules list in .libcst.codemod.yaml) and some issues on Win‐
2777 dows.
2778 hypothesis.extra.codemods.refactor(code)
2780 Update a source code string from deprecated to modern Hypothesis
2781 APIs.
2782 This may not fix all the deprecation warnings in your code, but
2784 we're confident that it will be easier than doing it all by
2785 hand.
2786 We recommend using the CLI, but if you want a Python function
2788 here it is.
2789 TIP:
2791 For new projects, we recommend using either deal or icontract and
2792 icontract-hypothesis over dpcontracts. They're generally more pow‐
2793 erful tools for design-by-contract programming, and have substan‐
2794 tially nicer Hypothesis integration too!
2795 hypothesis[lark]
2797 This extra can be used to generate strings matching any context-free
2798 grammar, using the Lark parser library.
2799 It currently only supports Lark's native EBNF syntax, but we plan to
2801 extend this to support other common syntaxes such as ANTLR and RFC 5234
2802 ABNF. Lark already supports loading grammars from nearley.js, so you
2803 may not have to write your own at all.
2804 hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None)
2806 A strategy for strings accepted by the given context-free gram‐
2807 mar.
2808 grammar must be a Lark object, which wraps an EBNF specifica‐
2810 tion. The Lark EBNF grammar reference can be found here.
2811 from_lark will automatically generate strings matching the non‐
2813 terminal start symbol in the grammar, which was supplied as an
2814 argument to the Lark class. To generate strings matching a dif‐
2815 ferent symbol, including terminals, you can override this by
2816 passing the start argument to from_lark. Note that Lark may re‐
2817 move unreachable productions when the grammar is compiled, so
2818 you should probably pass the same value for start to both.
2819 Currently from_lark does not support grammars that need custom
2821 lexing. Any lexers will be ignored, and any undefined terminals
2822 from the use of %declare will result in generation errors. To
2823 define strategies for such terminals, pass a dictionary mapping
2824 their name to a corresponding strategy as the explicit argument.
2825 The hypothesmith project includes a strategy for Python source,
2827 based on a grammar and careful post-processing.
2828 Example grammars, which may provide a useful starting point for your
2830 tests, can be found in the Lark repository and in this third-party col‐
2831 lection.
2832 hypothesis[pytz]
2834 This module provides pytz timezones.
2835 You can use this strategy to make hypothesis.strategies.datetimes() and
2837 hypothesis.strategies.times() produce timezone-aware values.
2838 hypothesis.extra.pytz.timezones()
2840 Any timezone in the Olsen database, as a pytz tzinfo object.
2841 This strategy minimises to UTC, or the smallest possible fixed
2843 offset, and is designed for use with
2844 hypothesis.strategies.datetimes().
2845 hypothesis[dateutil]
2847 This module provides dateutil timezones.
2848 You can use this strategy to make datetimes() and times() produce time‐
2850 zone-aware values.
2851 hypothesis.extra.dateutil.timezones()
2853 Any timezone from dateutil.
2854 This strategy minimises to UTC, or the timezone with the small‐
2856 est offset from UTC as of 2000-01-01, and is designed for use
2857 with datetimes().
2858 Note that the timezones generated by the strategy may vary de‐
2860 pending on the configuration of your machine. See the dateutil
2861 documentation for more information.
2862
2864 Writing tests with Hypothesis frees you from the tedium of deciding on
2865 and writing out specific inputs to test. Now, the hypothesis.ex‐
2866 tra.ghostwriter module can write your test functions for you too!
2867 The idea is to provide an easy way to start property-based testing, and
2869 a seamless transition to more complex test code - because ghostwritten
2870 tests are source code that you could have written for yourself.
2871 So just pick a function you'd like tested, and feed it to one of the
2873 functions below. They follow imports, use but do not require type an‐
2874 notations, and generally do their best to write you a useful test. You
2875 can also use our command-line interface:
2876 $ hypothesis write --help
2878 Usage: hypothesis write [OPTIONS] FUNC...
2879 `hypothesis write` writes property-based tests for you!
2881 Type annotations are helpful but not required for our advanced
2883 introspection and templating logic. Try running the examples below to see
2884 how it works:
2885 hypothesis write gzip
2887 hypothesis write numpy.matmul
2888 hypothesis write pandas.from_dummies
2889 hypothesis write re.compile --except re.error
2890 hypothesis write --equivalent ast.literal_eval eval
2891 hypothesis write --roundtrip json.dumps json.loads
2892 hypothesis write --style=unittest --idempotent sorted
2893 hypothesis write --binary-op operator.add
2894 Options:
2896 --roundtrip start by testing write/read or encode/decode!
2897 --equivalent very useful when optimising or refactoring code
2898 --errors-equivalent --equivalent, but also allows consistent errors
2899 --idempotent check that f(x) == f(f(x))
2900 --binary-op associativity, commutativity, identity element
2901 --style [pytest|unittest] pytest-style function, or unittest-style method?
2902 -e, --except OBJ_NAME dotted name of exception(s) to ignore
2903 --annotate / --no-annotate force ghostwritten tests to be type-annotated
2904 (or not). By default, match the code to test.
2905 -h, --help Show this message and exit.
2906 TIP:
2908 Using a light theme? Hypothesis respects NO_COLOR and DJANGO_COL‐
2909 ORS=light.
2910 NOTE:
2912 The ghostwriter requires black, but the generated code only requires
2913 Hypothesis itself.
2914 NOTE:
2916 Legal questions? While the ghostwriter fragments and logic is under
2917 the MPL-2.0 license like the rest of Hypothesis, the output from the
2918 ghostwriter is made available under the Creative Commons Zero (CC0)
2919 public domain dedication, so you can use it without any restric‐
2920 tions.
2921 hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(),
2923 style='pytest', annotate=None)
2924 Guess which ghostwriters to use, for a module or collection of
2925 functions.
2926 As for all ghostwriters, the except_ argument should be an
2928 python:Exception or tuple of exceptions, and style may be either
2929 "pytest" to write test functions or "unittest" to write test
2930 methods and TestCase.
2931 After finding the public functions attached to any modules, the
2933 magic ghostwriter looks for pairs of functions to pass to
2934 roundtrip(), then checks for binary_operation() and ufunc()
2935 functions, and any others are passed to fuzz().
2936 For example, try hypothesis write gzip on the command line!
2938 hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest',
2940 annotate=None)
2941 Write source code for a property-based test of func.
2942 The resulting test checks that valid input only leads to ex‐
2944 pected exceptions. For example:
2945 from re import compile, error
2947 from hypothesis.extra import ghostwriter
2949 ghostwriter.fuzz(compile, except_=error)
2951 Gives:
2953 # This test code was written by the `hypothesis.extra.ghostwriter` module
2955 # and is provided under the Creative Commons Zero public domain dedication.
2956 import re
2957 from hypothesis import given, reject, strategies as st
2959 # TODO: replace st.nothing() with an appropriate strategy
2961 @given(pattern=st.nothing(), flags=st.just(0))
2964 def test_fuzz_compile(pattern, flags):
2965 try:
2966 re.compile(pattern=pattern, flags=flags)
2967 except re.error:
2968 reject()
2969 Note that it includes all the required imports. Because the
2971 pattern parameter doesn't have annotations or a default argu‐
2972 ment, you'll need to specify a strategy - for example text() or
2973 binary(). After that, you have a test!
2974 hypothesis.extra.ghostwriter.idempotent(func, *, except_=(),
2976 style='pytest', annotate=None)
2977 Write source code for a property-based test of func.
2978 The resulting test checks that if you call func on it's own out‐
2980 put, the result does not change. For example:
2981 from typing import Sequence
2983 from hypothesis.extra import ghostwriter
2985 def timsort(seq: Sequence[int]) -> Sequence[int]:
2988 return sorted(seq)
2989 ghostwriter.idempotent(timsort)
2992 Gives:
2994 # This test code was written by the `hypothesis.extra.ghostwriter` module
2996 # and is provided under the Creative Commons Zero public domain dedication.
2997 from hypothesis import given, strategies as st
2999 @given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
3002 def test_idempotent_timsort(seq):
3003 result = timsort(seq=seq)
3004 repeat = timsort(seq=result)
3005 assert result == repeat, (result, repeat)
3006 hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(),
3008 style='pytest', annotate=None)
3009 Write source code for a property-based test of funcs.
3010 The resulting test checks that if you call the first function,
3012 pass the result to the second (and so on), the final result is
3013 equal to the first input argument.
3014 This is a very powerful property to test, especially when the
3016 config options are varied along with the object to round-trip.
3017 For example, try ghostwriting a test for python:json.dumps() -
3018 would you have thought of all that?
3019 hypothesis write --roundtrip json.dumps json.loads
3021 hypothesis.extra.ghostwriter.equivalent(*funcs, allow_same_er‐
3023 rors=False, except_=(), style='pytest', annotate=None)
3024 Write source code for a property-based test of funcs.
3025 The resulting test checks that calling each of the functions re‐
3027 turns an equal value. This can be used as a classic 'oracle',
3028 such as testing a fast sorting algorithm against the
3029 python:sorted() builtin, or for differential testing where none
3030 of the compared functions are fully trusted but any difference
3031 indicates a bug (e.g. running a function on different numbers of
3032 threads, or simply multiple times).
3033 The functions should have reasonably similar signatures, as only
3035 the common parameters will be passed the same arguments - any
3036 other parameters will be allowed to vary.
3037 If allow_same_errors is True, then the test will pass if calling
3039 each of the functions returns an equal value, or if the first
3040 function raises an exception and each of the others raises an
3041 exception of the same type. This relaxed mode can be useful for
3042 code synthesis projects.
3043 hypothesis.extra.ghostwriter.binary_operation(func, *, associa‐
3045 tive=True, commutative=True, identity=Ellipsis, distributes_over=None,
3046 except_=(), style='pytest', annotate=None)
3047 Write property tests for the binary operation func.
3048 While binary operations are not particularly common, they have
3050 such nice properties to test that it seems a shame not to demon‐
3051 strate them with a ghostwriter. For an operator f, test that:
3052 • if associative, f(a, f(b, c)) == f(f(a, b), c)
3054 • if commutative, f(a, b) == f(b, a)
3056 • if identity is not None, f(a, identity) == a
3058 • if distributes_over is +, f(a, b) + f(a, c) == f(a, b+c)
3060 For example:
3062 ghostwriter.binary_operation(
3064 operator.mul,
3065 identity=1,
3066 distributes_over=operator.add,
3067 style="unittest",
3068 )
3069 hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest',
3071 annotate=None)
3072 Write a property-based test for the array ufunc func.
3073 The resulting test checks that your ufunc or gufunc has the ex‐
3075 pected broadcasting and dtype casting behaviour. You will prob‐
3076 ably want to add extra assertions, but as with the other ghost‐
3077 writers this gives you a great place to start.
3078 hypothesis write numpy.matmul
3080
3082 Hypothesis offers a number of features specific for Django testing,
3083 available in the hypothesis[django] extra. This is tested against each
3084 supported series with mainstream or extended support - if you're still
3085 getting security patches, you can test with Hypothesis.
3086 class hypothesis.extra.django.TestCase
3088 Using it is quite straightforward: All you need to do is subclass
3090 hypothesis.extra.django.TestCase or
3091 hypothesis.extra.django.TransactionTestCase or LiveServerTestCase or
3092 StaticLiveServerTestCase and you can use @given as normal, and the
3093 transactions will be per example rather than per test function as they
3094 would be if you used @given with a normal django test suite (this is
3095 important because your test function will be called multiple times and
3096 you don't want them to interfere with each other). Test cases on these
3097 classes that do not use @given will be run as normal.
3098 class hypothesis.extra.django.TransactionTestCase
3100 class hypothesis.extra.django.LiveServerTestCase
3102 class hypothesis.extra.django.StaticLiveServerTestCase
3104 We recommend avoiding TransactionTestCase unless you really have to run
3106 each test case in a database transaction. Because Hypothesis runs this
3107 in a loop, the performance problems it normally has are significantly
3108 exacerbated and your tests will be really slow. If you are using
3109 TransactionTestCase, you may need to use @settings(sup‐
3110 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
3111 example generation.
3112 Having set up a test class, you can now pass @given a strategy for
3114 Django models:
3115 hypothesis.extra.django.from_model(model, /, **field_strategies)
3117 Return a strategy for examples of model.
3118 WARNING:
3120 Hypothesis creates saved models. This will run inside your
3121 testing transaction when using the test runner, but if you
3122 use the dev console this will leave debris in your database.
3123 model must be an subclass of Model. Strategies for fields may
3125 be passed as keyword arguments, for example
3126 is_staff=st.just(False). In order to support models with fields
3127 named "model", this is a positional-only parameter.
3128 Hypothesis can often infer a strategy based the field type and
3130 validators, and will attempt to do so for any required fields.
3131 No strategy will be inferred for an AutoField, nullable field,
3132 foreign key, or field for which a keyword argument is passed to
3133 from_model(). For example, a Shop type with a foreign key to
3134 Company could be generated with:
3135 shop_strategy = from_model(Shop, company=from_model(Company))
3137 Like for builds(), you can pass ... (python:Ellipsis) as a key‐
3139 word argument to infer a strategy for a field which has a de‐
3140 fault value instead of using the default.
3141 For example, using the trivial django project we have for testing:
3143 >>> from hypothesis.extra.django import from_model
3145 >>> from toystore.models import Customer
3146 >>> c = from_model(Customer).example()
3147 >>> c
3148 <Customer: Customer object>
3149 >>> c.email
3150 'jaime.urbina@gmail.com'
3151 >>> c.name
3152 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
3153 >>> c.age
3154 -873375803
3155 Hypothesis has just created this with whatever the relevant type of
3157 data is.
3158 Obviously the customer's age is implausible, which is only possible be‐
3160 cause we have not used (eg) MinValueValidator to set the valid range
3161 for this field (or used a PositiveSmallIntegerField, which would only
3162 need a maximum value validator).
3163 If you do have validators attached, Hypothesis will only generate exam‐
3165 ples that pass validation. Sometimes that will mean that we fail a
3166 HealthCheck because of the filtering, so let's explicitly pass a strat‐
3167 egy to skip validation at the strategy level:
3168 >>> from hypothesis.strategies import integers
3170 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
3171 >>> c
3172 <Customer: Customer object>
3173 >>> c.age
3174 5
3175 hypothesis.extra.django.from_form(form, form_kwargs=None,
3177 **field_strategies)
3178 Return a strategy for examples of form.
3179 form must be an subclass of Form. Strategies for fields may be
3181 passed as keyword arguments, for example
3182 is_staff=st.just(False).
3183 Hypothesis can often infer a strategy based the field type and
3185 validators, and will attempt to do so for any required fields.
3186 No strategy will be inferred for a disabled field or field for
3187 which a keyword argument is passed to from_form().
3188 This function uses the fields of an unbound form instance to de‐
3190 termine field strategies, any keyword arguments needed to in‐
3191 stantiate the unbound form instance can be passed into
3192 from_form() as a dict with the keyword form_kwargs. E.g.:
3193 shop_strategy = from_form(Shop, form_kwargs={"company_id": 5})
3195 Like for builds(), you can pass ... (python:Ellipsis) as a key‐
3197 word argument to infer a strategy for a field which has a de‐
3198 fault value instead of using the default.
3199 Tips and tricks
3201 Custom field types
3202 If you have a custom Django field type you can register it with Hypoth‐
3203 esis's model deriving functionality by registering a default strategy
3204 for it:
3205 >>> from toystore.models import CustomishField, Customish
3207 >>> from_model(Customish).example()
3208 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
3209 customish for model Customish
3210 >>> from hypothesis.extra.django import register_field_strategy
3211 >>> from hypothesis.strategies import just
3212 >>> register_field_strategy(CustomishField, just("hi"))
3213 >>> x = from_model(Customish).example()
3214 >>> x.customish
3215 'hi'
3216 Note that this mapping is on exact type. Subtypes will not inherit it.
3218 hypothesis.extra.django.register_field_strategy(field_type, strategy)
3220 Add an entry to the global field-to-strategy lookup used by
3221 from_field().
3222 field_type must be a subtype of django.db.models.Field or
3224 django.forms.Field, which must not already be registered.
3225 strategy must be a SearchStrategy.
3226 hypothesis.extra.django.from_field(field)
3228 Return a strategy for values that fit the given field.
3229 This function is used by from_form() and from_model() for any
3231 fields that require a value, or for which you passed ...
3232 (python:Ellipsis) to infer a strategy from an annotation.
3233 It's pretty similar to the core from_type() function, with a
3235 subtle but important difference: from_field takes a Field in‐
3236 stance, rather than a Field subtype, so that it has access to
3237 instance attributes such as string length and validators.
3238 Generating child models
3240 For the moment there's no explicit support in hypothesis-django for
3241 generating dependent models. i.e. a Company model will generate no
3242 Shops. However if you want to generate some dependent models as well,
3243 you can emulate this by using the flatmap function as follows:
3244 from hypothesis.strategies import just, lists
3246 def generate_with_shops(company):
3249 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
3250 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
3253 Let's unpack what this is doing:
3255 The way flatmap works is that we draw a value from the original strat‐
3257 egy, then apply a function to it which gives us a new strategy. We then
3258 draw a value from that strategy. So in this case we're first drawing a
3259 company, and then we're drawing a list of shops belonging to that com‐
3260 pany: The just strategy is a strategy such that drawing it always pro‐
3261 duces the individual value, so from_model(Shop, company=just(company))
3262 is a strategy that generates a Shop belonging to the original company.
3263 So the following code would give us a list of shops all belonging to
3265 the same company:
3266 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
3268 The only difference from this and the above is that we want the com‐
3270 pany, not the shops. This is where the inner map comes in. We build the
3271 list of shops and then throw it away, instead returning the company we
3272 started for. This works because the models that Hypothesis generates
3273 are saved in the database, so we're essentially running the inner
3274 strategy purely for the side effect of creating those children in the
3275 database.
3276 Generating primary key values
3278 If your model includes a custom primary key that you want to generate
3279 using a strategy (rather than a default auto-increment primary key)
3280 then Hypothesis has to deal with the possibility of a duplicate primary
3281 key.
3282 If a model strategy generates a value for the primary key field, Hy‐
3284 pothesis will create the model instance with update_or_create(), over‐
3285 writing any existing instance in the database for this test case with
3286 the same primary key.
3287 On the subject of MultiValueField
3289 Django forms feature the MultiValueField which allows for several
3290 fields to be combined under a single named field, the default example
3291 of this is the SplitDateTimeField.
3292 class CustomerForm(forms.Form):
3294 name = forms.CharField()
3295 birth_date_time = forms.SplitDateTimeField()
3296 from_form supports MultiValueField subclasses directly, however if you
3298 want to define your own strategy be forewarned that Django binds data
3299 for a MultiValueField in a peculiar way. Specifically each sub-field is
3300 expected to have its own entry in data addressed by the field name
3301 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
3302 ValueField, so form data for the example above might look like this:
3303 {
3305 "name": "Samuel John",
3306 "birth_date_time_0": "2018-05-19", # the date, as the first sub-field
3307 "birth_date_time_1": "15:18:00", # the time, as the second sub-field
3308 }
3309 Thus, if you want to define your own strategies for such a field you
3311 must address your sub-fields appropriately:
3312 from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))
3314
3316 numpy
3317 Hypothesis offers a number of strategies for NumPy testing, available
3318 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
3319 package.
3320 The centerpiece is the arrays() strategy, which generates arrays with
3322 any dtype, shape, and contents you can specify or give a strategy for.
3323 To make this as useful as possible, strategies are provided to generate
3324 array shapes and generate all kinds of fixed-size or compound dtypes.
3325 hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0,
3327 max_size=None, min_value=None, max_value=None, allow_nan=None, al‐
3328 low_infinity=None, allow_subnormal=None, exclude_min=None, ex‐
3329 clude_max=None, min_magnitude=0, max_magnitude=None)
3330 Creates a strategy which can generate any value of the given
3331 dtype.
3332 Compatible parameters are passed to the inferred strategy func‐
3334 tion while inapplicable ones are ignored. This allows you, for
3335 example, to customise the min and max values, control the length
3336 or contents of strings, or exclude non-finite numbers. This is
3337 particularly useful when kwargs are passed through from arrays()
3338 which allow a variety of numeric dtypes, as it seamlessly han‐
3339 dles the width or representable bounds for you.
3340 hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None,
3342 fill=None, unique=False)
3343 Returns a strategy for generating numpy:numpy.ndarrays.
3344 • dtype may be any valid input to dtype (this includes dtype ob‐
3346 jects), or a strategy that generates such values.
3347 • shape may be an integer >= 0, a tuple of such integers, or a
3349 strategy that generates such values.
3350 • elements is a strategy for generating values to put in the ar‐
3352 ray. If it is None a suitable value will be inferred based on
3353 the dtype, which may give any legal value (including eg NaN
3354 for floats). If a mapping, it will be passed as **kwargs to
3355 from_dtype()
3356 • fill is a strategy that may be used to generate a single back‐
3358 ground value for the array. If None, a suitable default will
3359 be inferred based on the other arguments. If set to nothing()
3360 then filling behaviour will be disabled entirely and every el‐
3361 ement will be generated independently.
3362 • unique specifies if the elements of the array should all be
3364 distinct from one another. Note that in this case multiple NaN
3365 values may still be allowed. If fill is also set, the only
3366 valid values for it to return are NaN values (anything for
3367 which numpy:numpy.isnan returns True. So e.g. for complex num‐
3368 bers nan+1j is also a valid fill). Note that if unique is set
3369 to True the generated values must be hashable.
3370 Arrays of specified dtype and shape are generated for example
3372 like this:
3373 >>> import numpy as np
3375 >>> arrays(np.int8, (2, 3)).example()
3376 array([[-8, 6, 3],
3377 [-6, 4, 6]], dtype=int8)
3378 >>> arrays(np.float, 3, elements=st.floats(0, 1)).example()
3379 array([ 0.88974794, 0.77387938, 0.1977879 ])
3380 Array values are generated in two parts:
3382 1. Some subset of the coordinates of the array are populated
3384 with a value drawn from the elements strategy (or its in‐
3385 ferred form).
3386 2. If any coordinates were not assigned in the previous step, a
3388 single value is drawn from the fill strategy and is assigned
3389 to all remaining places.
3390 You can set fill=nothing() to disable this behaviour and draw a
3392 value for every element.
3393 If fill=None, then it will attempt to infer the correct behav‐
3395 iour automatically. If unique is True, no filling will occur by
3396 default. Otherwise, if it looks safe to reuse the values of el‐
3397 ements across multiple coordinates (this will be the case for
3398 any inferred strategy, and for most of the builtins, but is not
3399 the case for mutable values or strategies built with flatmap,
3400 map, composite, etc) then it will use the elements strategy as
3401 the fill, else it will default to having no fill.
3402 Having a fill helps Hypothesis craft high quality examples, but
3404 its main importance is when the array generated is large: Hy‐
3405 pothesis is primarily designed around testing small examples. If
3406 you have arrays with hundreds or more elements, having a fill
3407 value is essential if you want your tests to run in reasonable
3408 time.
3409 hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None,
3411 min_side=1, max_side=None)
3412 Return a strategy for array shapes (tuples of int >= 1).
3413 • min_dims is the smallest length that the generated shape can
3415 possess.
3416 • max_dims is the largest length that the generated shape can
3418 possess, defaulting to min_dims + 2.
3419 • min_side is the smallest size that a dimension can possess.
3421 • max_side is the largest size that a dimension can possess, de‐
3423 faulting to min_side + 5.
3424 hypothesis.extra.numpy.scalar_dtypes()
3426 Return a strategy that can return any non-flexible scalar dtype.
3427 hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?',
3429 sizes=(8, 16, 32, 64))
3430 Return a strategy for unsigned integer dtypes.
3431 endianness may be < for little-endian, > for big-endian, = for
3433 native byte order, or ? to allow either byte order. This argu‐
3434 ment only applies to dtypes of more than one byte.
3435 sizes must be a collection of integer sizes in bits. The de‐
3437 fault (8, 16, 32, 64) covers the full range of sizes.
3438 hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16,
3440 32, 64))
3441 Return a strategy for signed integer dtypes.
3442 endianness and sizes are treated as for
3444 unsigned_integer_dtypes().
3445 hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16,
3447 32, 64))
3448 Return a strategy for floating-point dtypes.
3449 sizes is the size in bits of floating-point number. Some ma‐
3451 chines support 96- or 128-bit floats, but these are not gener‐
3452 ated by default.
3453 Larger floats (96 and 128 bit real parts) are not supported on
3455 all platforms and therefore disabled by default. To generate
3456 these dtypes, include these values in the sizes argument.
3457 hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?',
3459 sizes=(64, 128))
3460 Return a strategy for complex-number dtypes.
3461 sizes is the total size in bits of a complex number, which con‐
3463 sists of two floats. Complex halves (a 16-bit real part) are
3464 not supported by numpy and will not be generated by this strat‐
3465 egy.
3466 hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_pe‐
3468 riod='ns', endianness='?')
3469 Return a strategy for datetime64 dtypes, with various precisions
3470 from year to attosecond.
3471 hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_pe‐
3473 riod='ns', endianness='?')
3474 Return a strategy for timedelta64 dtypes, with various preci‐
3475 sions from year to attosecond.
3476 hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1,
3478 max_len=16)
3479 Return a strategy for generating bytestring dtypes, of various
3480 lengths and byteorder.
3481 While Hypothesis' string strategies can generate empty strings,
3483 string dtypes with length 0 indicate that size is still to be
3484 determined, so the minimum length for string dtypes is 1.
3485 hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?',
3487 min_len=1, max_len=16)
3488 Return a strategy for generating unicode string dtypes, of vari‐
3489 ous lengths and byteorder.
3490 While Hypothesis' string strategies can generate empty strings,
3492 string dtypes with length 0 indicate that size is still to be
3493 determined, so the minimum length for string dtypes is 1.
3494 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
3496 *, min_size=1, max_size=5, allow_subarrays=False)
3497 Return a strategy for generating array (compound) dtypes, with
3498 members drawn from the given subtype strategy.
3499 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
3501 *, max_leaves=10, max_itemsize=None)
3502 Return the most-general dtype strategy.
3503 Elements drawn from this strategy may be simple (from the sub‐
3505 type_strategy), or several such values drawn from array_dtypes()
3506 with allow_subarrays=True. Subdtypes in an array dtype may be
3507 nested to any depth, subject to the max_leaves argument.
3508 hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0,
3510 max_size=None)
3511 Return a strategy for generating permissible tuple-values for
3512 the axis argument for a numpy sequential function (e.g.
3513 numpy:numpy.sum()), given an array of the specified dimensional‐
3514 ity.
3515 All tuples will have a length >= min_size and <= max_size. The
3517 default value for max_size is ndim.
3518 Examples from this strategy shrink towards an empty tuple, which
3520 render most sequential functions as no-ops.
3521 The following are some examples drawn from this strategy.
3523 >>> [valid_tuple_axes(3).example() for i in range(4)]
3525 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3526 valid_tuple_axes can be joined with other strategies to generate
3528 any type of valid axis object, i.e. integers, tuples, and None:
3529 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3531 hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0,
3533 max_dims=None, min_side=1, max_side=None)
3534 Return a strategy for shapes that are broadcast-compatible with
3535 the provided shape.
3536 Examples from this strategy shrink towards a shape with length
3538 min_dims. The size of an aligned dimension shrinks towards size
3539 1. The size of an unaligned dimension shrink towards min_side.
3540 • shape is a tuple of integers.
3542 • min_dims is the smallest length that the generated shape can
3544 possess.
3545 • max_dims is the largest length that the generated shape can
3547 possess, defaulting to max(len(shape), min_dims) + 2.
3548 • min_side is the smallest size that an unaligned dimension can
3550 possess.
3551 • max_side is the largest size that an unaligned dimension can
3553 possess, defaulting to 2 plus the size of the largest aligned
3554 dimension.
3555 The following are some examples drawn from this strategy.
3557 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3559 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3560 hypothesis.extra.numpy.mutually_broadcastable_shapes(*,
3562 num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0,
3563 max_dims=None, min_side=1, max_side=None)
3564 Return a strategy for a specified number of shapes N that are
3565 mutually-broadcastable with one another and with the provided
3566 base shape.
3567 • num_shapes is the number of mutually broadcast-compatible
3569 shapes to generate.
3570 • base_shape is the shape against which all generated shapes can
3572 broadcast. The default shape is empty, which corresponds to a
3573 scalar and thus does not constrain broadcasting at all.
3574 • min_dims is the smallest length that the generated shape can
3576 possess.
3577 • max_dims is the largest length that the generated shape can
3579 possess, defaulting to max(len(shape), min_dims) + 2.
3580 • min_side is the smallest size that an unaligned dimension can
3582 possess.
3583 • max_side is the largest size that an unaligned dimension can
3585 possess, defaulting to 2 plus the size of the largest aligned
3586 dimension.
3587 The strategy will generate a python:typing.NamedTuple contain‐
3589 ing:
3590 • input_shapes as a tuple of the N generated shapes.
3592 • result_shape as the resulting shape produced by broadcasting
3594 the N shapes with the base shape.
3595 The following are some examples drawn from this strategy.
3597 >>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
3599 ... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
3600 >>> for _ in range(5):
3601 ... print(strat.example())
3602 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3603 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3604 BroadcastableShapes(input_shapes=((), (), ()), result_shape=())
3605 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(3,))
3606 BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))
3607 Use with Generalised Universal Function signatures
3609 A universal function (or ufunc for short) is a function that op‐
3611 erates on ndarrays in an element-by-element fashion, supporting
3612 array broadcasting, type casting, and several other standard
3613 features. A generalised ufunc operates on sub-arrays rather
3614 than elements, based on the "signature" of the function. Com‐
3615 pare e.g. numpy.add() (ufunc) to numpy.matmul() (gufunc).
3616 To generate shapes for a gufunc, you can pass the signature ar‐
3618 gument instead of num_shapes. This must be a gufunc signature
3619 string; which you can write by hand or access as e.g. np.mat‐
3620 mul.signature on generalised ufuncs.
3621 In this case, the side arguments are applied to the 'core dimen‐
3623 sions' as well, ignoring any frozen dimensions. base_shape and
3624 the dims arguments are applied to the 'loop dimensions', and if
3625 necessary, the dimensionality of each shape is silently capped
3626 to respect the 32-dimension limit.
3627 The generated result_shape is the real result shape of applying
3629 the gufunc to arrays of the generated input_shapes, even where
3630 this is different to broadcasting the loop dimensions.
3631 gufunc-compatible shapes shrink their loop dimensions as above,
3633 towards omitting optional core dimensions, and smaller-size core
3634 dimensions.
3635 >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
3637 >>> for _ in range(3):
3638 ... mutually_broadcastable_shapes(signature=np.matmul.signature).example()
3639 BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
3640 BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
3641 BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))
3642 hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0,
3644 max_dims=None, allow_newaxis=False, allow_ellipsis=True)
3645 Return a strategy for basic indexes of arrays with the specified
3646 shape, which may include dimensions of size zero.
3647 It generates tuples containing some mix of integers,
3649 python:slice objects, ... (an Ellipsis), and None. When a
3650 length-one tuple would be generated, this strategy may instead
3651 return the element which will index the first axis, e.g. 5 in‐
3652 stead of (5,).
3653 • shape is the shape of the array that will be indexed, as a tu‐
3655 ple of positive integers. This must be at least two-dimen‐
3656 sional for a tuple to be a valid index; for one-dimensional
3657 arrays use slices() instead.
3658 • min_dims is the minimum dimensionality of the resulting array
3660 from use of the generated index. When min_dims == 0, scalars
3661 and zero-dimensional arrays are both allowed.
3662 • max_dims is the the maximum dimensionality of the resulting
3664 array, defaulting to len(shape) if not allow_newaxis else
3665 max(len(shape), min_dims) + 2.
3666 • allow_newaxis specifies whether None is allowed in the index.
3668 • allow_ellipsis specifies whether ... is allowed in the index.
3670 hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=ar‐
3672 ray_shapes(), dtype=<class 'numpy.int32'>)
3673 Return a search strategy for tuples of integer-arrays that, when
3674 used to index into an array of shape shape, given an array whose
3675 shape was drawn from result_shape.
3676 Examples from this strategy shrink towards the tuple of in‐
3678 dex-arrays:
3679 len(shape) * (np.zeros(drawn_result_shape, dtype), )
3681 • shape a tuple of integers that indicates the shape of the ar‐
3683 ray, whose indices are being generated.
3684 • result_shape a strategy for generating tuples of integers,
3686 which describe the shape of the resulting index arrays. The
3687 default is array_shapes(). The shape drawn from this strategy
3688 determines the shape of the array that will be produced when
3689 the corresponding example from integer_array_indices is used
3690 as an index.
3691 • dtype the integer data type of the generated index-arrays.
3693 Negative integer indices can be generated if a signed integer
3694 type is specified.
3695 Recall that an array can be indexed using a tuple of integer-ar‐
3697 rays to access its members in an arbitrary order, producing an
3698 array with an arbitrary shape. For example:
3699 >>> from numpy import array
3701 >>> x = array([-0, -1, -2, -3, -4])
3702 >>> ind = (array([[4, 0], [0, 1]]),) # a tuple containing a 2D integer-array
3703 >>> x[ind] # the resulting array is commensurate with the indexing array(s)
3704 array([[-4, 0],
3705 [0, -1]])
3706 Note that this strategy does not accommodate all variations of
3708 so-called 'advanced indexing', as prescribed by NumPy's nomen‐
3709 clature. Combinations of basic and advanced indexes are too
3710 complex to usefully define in a standard strategy; we leave ap‐
3711 plication-specific strategies to the user. Advanced-boolean in‐
3712 dexing can be defined as arrays(shape=..., dtype=bool), and is
3713 similarly left to the user.
3714 pandas
3716 Hypothesis provides strategies for several of the core pandas data
3717 types: pandas.Index, pandas.Series and pandas.DataFrame.
3718 The general approach taken by the pandas module is that there are mul‐
3720 tiple strategies for generating indexes, and all of the other strate‐
3721 gies take the number of entries they contain from their index strategy
3722 (with sensible defaults). So e.g. a Series is specified by specifying
3723 its numpy.dtype (and/or a strategy for generating elements for it).
3724 hypothesis.extra.pandas.indexes(*, elements=None, dtype=None,
3726 min_size=0, max_size=None, unique=True, name=none())
3727 Provides a strategy for producing a pandas.Index.
3728 Arguments:
3730 • elements is a strategy which will be used to generate the in‐
3732 dividual values of the index. If None, it will be inferred
3733 from the dtype. Note: even if the elements strategy produces
3734 tuples, the generated value will not be a MultiIndex, but in‐
3735 stead be a normal index whose elements are tuples.
3736 • dtype is the dtype of the resulting index. If None, it will be
3738 inferred from the elements strategy. At least one of dtype or
3739 elements must be provided.
3740 • min_size is the minimum number of elements in the index.
3742 • max_size is the maximum number of elements in the index. If
3744 None then it will default to a suitable small size. If you
3745 want larger indexes you should pass a max_size explicitly.
3746 • unique specifies whether all of the elements in the resulting
3748 index should be distinct.
3749 • name is a strategy for strings or None, which will be passed
3751 to the pandas.Index constructor.
3752 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None,
3754 name=none())
3755 Provides a strategy which generates an Index whose values are 0,
3756 1, ..., n for some n.
3757 Arguments:
3759 • min_size is the smallest number of elements the index can
3761 have.
3762 • max_size is the largest number of elements the index can have.
3764 If None it will default to some suitable value based on
3765 min_size.
3766 • name is the name of the index. If st.none(), the index will
3768 have no name.
3769 hypothesis.extra.pandas.series(*, elements=None, dtype=None, in‐
3771 dex=None, fill=None, unique=False, name=none())
3772 Provides a strategy for producing a pandas.Series.
3773 Arguments:
3775 • elements: a strategy that will be used to generate the indi‐
3777 vidual values in the series. If None, we will attempt to infer
3778 a suitable default from the dtype.
3779 • dtype: the dtype of the resulting series and may be any value
3781 that can be passed to numpy.dtype. If None, will use pandas's
3782 standard behaviour to infer it from the type of the elements
3783 values. Note that if the type of values that comes out of your
3784 elements strategy varies, then so will the resulting dtype of
3785 the series.
3786 • index: If not None, a strategy for generating indexes for the
3788 resulting Series. This can generate either pandas.Index ob‐
3789 jects or any sequence of values (which will be passed to the
3790 Index constructor).
3791 You will probably find it most convenient to use the indexes()
3793 or range_indexes() function to produce values for this argu‐
3794 ment.
3795 • name: is a strategy for strings or None, which will be passed
3797 to the pandas.Series constructor.
3798 Usage:
3800 >>> series(dtype=int).example()
3802 0 -2001747478
3803 1 1153062837
3804 class hypothesis.extra.pandas.column(name=None, elements=None,
3806 dtype=None, fill=None, unique=False)
3807 Data object for describing a column in a DataFrame.
3808 Arguments:
3810 • name: the column name, or None to default to the column posi‐
3812 tion. Must be hashable, but can otherwise be any value sup‐
3813 ported as a pandas column name.
3814 • elements: the strategy for generating values in this column,
3816 or None to infer it from the dtype.
3817 • dtype: the dtype of the column, or None to infer it from the
3819 element strategy. At least one of dtype or elements must be
3820 provided.
3821 • fill: A default value for elements of the column. See arrays()
3823 for a full explanation.
3824 • unique: If all values in this column should be distinct.
3826 hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, ele‐
3828 ments=None, fill=None, unique=False)
3829 A convenience function for producing a list of column objects of
3830 the same general shape.
3831 The names_or_number argument is either a sequence of values, the
3833 elements of which will be used as the name for individual column
3834 objects, or a number, in which case that many unnamed columns
3835 will be created. All other arguments are passed through verbatim
3836 to create the columns.
3837 hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, in‐
3839 dex=None)
3840 Provides a strategy for producing a pandas.DataFrame.
3841 Arguments:
3843 • columns: An iterable of column objects describing the shape of
3845 the generated DataFrame.
3846 • rows: A strategy for generating a row object. Should generate
3848 either dicts mapping column names to values or a sequence map‐
3849 ping column position to the value in that position (note that
3850 unlike the pandas.DataFrame constructor, single values are not
3851 allowed here. Passing e.g. an integer is an error, even if
3852 there is only one column).
3853 At least one of rows and columns must be provided. If both are
3855 provided then the generated rows will be validated against the
3856 columns and an error will be raised if they don't match.
3857 Caveats on using rows:
3859 • In general you should prefer using columns to rows, and only
3861 use rows if the columns interface is insufficiently flexible
3862 to describe what you need - you will get better performance
3863 and example quality that way.
3864 • If you provide rows and not columns, then the shape and
3866 dtype of the resulting DataFrame may vary. e.g. if you have
3867 a mix of int and float in the values for one column in your
3868 row entries, the column will sometimes have an integral
3869 dtype and sometimes a float.
3870 • index: If not None, a strategy for generating indexes for the
3872 resulting DataFrame. This can generate either pandas.Index ob‐
3873 jects or any sequence of values (which will be passed to the
3874 Index constructor).
3875 You will probably find it most convenient to use the indexes()
3877 or range_indexes() function to produce values for this argu‐
3878 ment.
3879 Usage:
3881 The expected usage pattern is that you use column and columns()
3883 to specify a fixed shape of the DataFrame you want as follows.
3884 For example the following gives a two column data frame:
3885 >>> from hypothesis.extra.pandas import column, data_frames
3887 >>> data_frames([
3888 ... column('A', dtype=int), column('B', dtype=float)]).example()
3889 A B
3890 0 2021915903 1.793898e+232
3891 1 1146643993 inf
3892 2 -2096165693 1.000000e+07
3893 If you want the values in different columns to interact in some
3895 way you can use the rows argument. For example the following
3896 gives a two column DataFrame where the value in the first column
3897 is always at most the value in the second:
3898 >>> from hypothesis.extra.pandas import column, data_frames
3900 >>> import hypothesis.strategies as st
3901 >>> data_frames(
3902 ... rows=st.tuples(st.floats(allow_nan=False),
3903 ... st.floats(allow_nan=False)).map(sorted)
3904 ... ).example()
3905 0 1
3906 0 -3.402823e+38 9.007199e+15
3907 1 -1.562796e-298 5.000000e-01
3908 You can also combine the two:
3910 >>> from hypothesis.extra.pandas import columns, data_frames
3912 >>> import hypothesis.strategies as st
3913 >>> data_frames(
3914 ... columns=columns(["lo", "hi"], dtype=float),
3915 ... rows=st.tuples(st.floats(allow_nan=False),
3916 ... st.floats(allow_nan=False)).map(sorted)
3917 ... ).example()
3918 lo hi
3919 0 9.314723e-49 4.353037e+45
3920 1 -9.999900e-01 1.000000e+07
3921 2 -2.152861e+134 -1.069317e-73
3922 (Note that the column dtype must still be specified and will not
3924 be inferred from the rows. This restriction may be lifted in fu‐
3925 ture).
3926 Combining rows and columns has the following behaviour:
3928 • The column names and dtypes will be used.
3930 • If the column is required to be unique, this will be enforced.
3932 • Any values missing from the generated rows will be provided
3934 using the column's fill.
3935 • Any values in the row not present in the column specification
3937 (if dicts are passed, if there are keys with no corresponding
3938 column name, if sequences are passed if there are too many
3939 items) will result in InvalidArgument being raised.
3940 Supported versions
3942 There is quite a lot of variation between pandas versions. We only com‐
3943 mit to supporting the latest version of pandas, but older minor ver‐
3944 sions are supported on a "best effort" basis. Hypothesis is currently
3945 tested against and confirmed working with every Pandas minor version
3946 from 1.1 through to 2.0.
3947 Releases that are not the latest patch release of their minor version
3949 are not tested or officially supported, but will probably also work un‐
3950 less you hit a pandas bug.
3951 Array API
3953 Hypothesis offers strategies for Array API adopting libraries in the
3954 hypothesis.extra.array_api package. See issue #3037 for more details.
3955 If you want to test with CuPy, Dask, JAX, MXNet, PyTorch, TensorFlow,
3956 or Xarray - or just numpy.array_api - this is the extension for you!
3957 hypothesis.extra.array_api.make_strategies_namespace(xp, *, api_ver‐
3959 sion=None)
3960 Creates a strategies namespace for the given array module.
3961 • xp is the Array API library to automatically pass to the
3963 namespaced methods.
3964 • api_version is the version of the Array API which the returned
3966 strategies namespace should conform to. If None, the latest
3967 API version which xp supports will be inferred from xp.__ar‐
3968 ray_api_version__. If a version string in the YYYY.MM format,
3969 the strategies namespace will conform to that version if sup‐
3970 ported.
3971 A python:types.SimpleNamespace is returned which contains all
3973 the strategy methods in this module but without requiring the xp
3974 argument. Creating and using a strategies namespace for NumPy's
3975 Array API implementation would go like this:
3976 >>> xp.__array_api_version__ # xp is your desired array library
3978 '2021.12'
3979 >>> xps = make_strategies_namespace(xp)
3980 >>> xps.api_version
3981 '2021.12'
3982 >>> x = xps.arrays(xp.int8, (2, 3)).example()
3983 >>> x
3984 Array([[-8, 6, 3],
3985 [-6, 4, 6]], dtype=int8)
3986 >>> x.__array_namespace__() is xp
3987 True
3988 The resulting namespace contains all our familiar strategies like
3990 arrays() and from_dtype(), but based on the Array API standard seman‐
3991 tics and returning objects from the xp module:
3992 xps.from_dtype(dtype, *, min_value=None, max_value=None, al‐
3994 low_nan=None, allow_infinity=None, allow_subnormal=None, ex‐
3995 clude_min=None, exclude_max=None)
3996 Return a strategy for any value of the given dtype.
3997 Values generated are of the Python scalar which is promotable to
3999 dtype, where the values do not exceed its bounds.
4000 • dtype may be a dtype object or the string name of a valid
4002 dtype.
4003 Compatible **kwargs are passed to the inferred strategy function
4005 for integers and floats. This allows you to customise the min
4006 and max values, and exclude non-finite numbers. This is particu‐
4007 larly useful when kwargs are passed through from arrays(), as it
4008 seamlessly handles the width or other representable bounds for
4009 you.
4010 xps.arrays(dtype, shape, *, elements=None, fill=None, unique=False)
4012 Returns a strategy for arrays.
4013 • dtype may be a valid dtype object or name, or a strategy that
4015 generates such values.
4016 • shape may be an integer >= 0, a tuple of such integers, or a
4018 strategy that generates such values.
4019 • elements is a strategy for values to put in the array. If None
4021 then a suitable value will be inferred based on the dtype,
4022 which may give any legal value (including e.g. NaN for
4023 floats). If a mapping, it will be passed as **kwargs to
4024 from_dtype() when inferring based on the dtype.
4025 • fill is a strategy that may be used to generate a single back‐
4027 ground value for the array. If None, a suitable default will
4028 be inferred based on the other arguments. If set to nothing()
4029 then filling behaviour will be disabled entirely and every el‐
4030 ement will be generated independently.
4031 • unique specifies if the elements of the array should all be
4033 distinct from one another; if fill is also set, the only valid
4034 values for fill to return are NaN values.
4035 Arrays of specified dtype and shape are generated for example
4037 like this:
4038 >>> from numpy import array_api as xp
4040 >>> xps.arrays(xp, xp.int8, (2, 3)).example()
4041 Array([[-8, 6, 3],
4042 [-6, 4, 6]], dtype=int8)
4043 Specifying element boundaries by a python:dict of the kwargs to
4045 pass to from_dtype() will ensure dtype bounds will be respected.
4046 >>> xps.arrays(xp, xp.int8, 3, elements={"min_value": 10}).example()
4048 Array([125, 13, 79], dtype=int8)
4049 Refer to What you can generate and how for passing your own ele‐
4051 ments strategy.
4052 >>> xps.arrays(xp, xp.float32, 3, elements=floats(0, 1, width=32)).example()
4054 Array([ 0.88974794, 0.77387938, 0.1977879 ], dtype=float32)
4055 Array values are generated in two parts:
4057 1. A single value is drawn from the fill strategy and is used to
4059 create a filled array.
4060 2. Some subset of the coordinates of the array are populated
4062 with a value drawn from the elements strategy (or its in‐
4063 ferred form).
4064 You can set fill to nothing() if you want to disable this behav‐
4066 iour and draw a value for every element.
4067 By default arrays will attempt to infer the correct fill behav‐
4069 iour: if unique is also True, no filling will occur. Otherwise,
4070 if it looks safe to reuse the values of elements across multiple
4071 coordinates (this will be the case for any inferred strategy,
4072 and for most of the builtins, but is not the case for mutable
4073 values or strategies built with flatmap, map, composite, etc.)
4074 then it will use the elements strategy as the fill, else it will
4075 default to having no fill.
4076 Having a fill helps Hypothesis craft high quality examples, but
4078 its main importance is when the array generated is large: Hy‐
4079 pothesis is primarily designed around testing small examples. If
4080 you have arrays with hundreds or more elements, having a fill
4081 value is essential if you want your tests to run in reasonable
4082 time.
4083 xps.array_shapes(*, min_dims=1, max_dims=None, min_side=1,
4085 max_side=None)
4086 Return a strategy for array shapes (tuples of int >= 1).
4087 • min_dims is the smallest length that the generated shape can
4089 possess.
4090 • max_dims is the largest length that the generated shape can
4092 possess, defaulting to min_dims + 2.
4093 • min_side is the smallest size that a dimension can possess.
4095 • max_side is the largest size that a dimension can possess, de‐
4097 faulting to min_side + 5.
4098 xps.scalar_dtypes()
4100 Return a strategy for all valid dtype objects.
4101 xps.boolean_dtypes()
4103 Return a strategy for just the boolean dtype object.
4104 xps.numeric_dtypes()
4106 Return a strategy for all numeric dtype objects.
4107 xps.real_dtypes()
4109 Return a strategy for all real-valued dtype objects.
4110 xps.integer_dtypes(*, sizes=(8, 16, 32, 64))
4112 Return a strategy for signed integer dtype objects.
4113 sizes contains the signed integer sizes in bits, defaulting to
4115 (8, 16, 32, 64) which covers all valid sizes.
4116 xps.unsigned_integer_dtypes(*, sizes=(8, 16, 32, 64))
4118 Return a strategy for unsigned integer dtype objects.
4119 sizes contains the unsigned integer sizes in bits, defaulting to
4121 (8, 16, 32, 64) which covers all valid sizes.
4122 xps.floating_dtypes(*, sizes=(32, 64))
4124 Return a strategy for real-valued floating-point dtype objects.
4125 sizes contains the floating-point sizes in bits, defaulting to
4127 (32, 64) which covers all valid sizes.
4128 xps.complex_dtypes(*, sizes=(64, 128))
4130 Return a strategy for complex dtype objects.
4131 sizes contains the complex sizes in bits, defaulting to (64,
4133 128) which covers all valid sizes.
4134 xps.valid_tuple_axes(ndim, *, min_size=0, max_size=None)
4136 Return a strategy for permissible tuple-values for the axis ar‐
4137 gument in Array API sequential methods e.g. sum, given the spec‐
4138 ified dimensionality.
4139 All tuples will have a length >= min_size and <= max_size. The
4141 default value for max_size is ndim.
4142 Examples from this strategy shrink towards an empty tuple, which
4144 render most sequential functions as no-ops.
4145 The following are some examples drawn from this strategy.
4147 >>> [valid_tuple_axes(3).example() for i in range(4)]
4149 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
4150 valid_tuple_axes can be joined with other strategies to generate
4152 any type of valid axis object, i.e. integers, tuples, and None:
4153 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
4155 xps.broadcastable_shapes(shape, *, min_dims=0, max_dims=None,
4157 min_side=1, max_side=None)
4158 Return a strategy for shapes that are broadcast-compatible with
4159 the provided shape.
4160 Examples from this strategy shrink towards a shape with length
4162 min_dims. The size of an aligned dimension shrinks towards size
4163 1. The size of an unaligned dimension shrink towards min_side.
4164 • shape is a tuple of integers.
4166 • min_dims is the smallest length that the generated shape can
4168 possess.
4169 • max_dims is the largest length that the generated shape can
4171 possess, defaulting to max(len(shape), min_dims) + 2.
4172 • min_side is the smallest size that an unaligned dimension can
4174 possess.
4175 • max_side is the largest size that an unaligned dimension can
4177 possess, defaulting to 2 plus the size of the largest aligned
4178 dimension.
4179 The following are some examples drawn from this strategy.
4181 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
4183 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
4184 xps.mutually_broadcastable_shapes(num_shapes, *, base_shape=(),
4186 min_dims=0, max_dims=None, min_side=1, max_side=None)
4187 Return a strategy for a specified number of shapes N that are
4188 mutually-broadcastable with one another and with the provided
4189 base shape.
4190 • num_shapes is the number of mutually broadcast-compatible
4192 shapes to generate.
4193 • base_shape is the shape against which all generated shapes can
4195 broadcast. The default shape is empty, which corresponds to a
4196 scalar and thus does not constrain broadcasting at all.
4197 • min_dims is the smallest length that the generated shape can
4199 possess.
4200 • max_dims is the largest length that the generated shape can
4202 possess, defaulting to max(len(shape), min_dims) + 2.
4203 • min_side is the smallest size that an unaligned dimension can
4205 possess.
4206 • max_side is the largest size that an unaligned dimension can
4208 possess, defaulting to 2 plus the size of the largest aligned
4209 dimension.
4210 The strategy will generate a python:typing.NamedTuple contain‐
4212 ing:
4213 • input_shapes as a tuple of the N generated shapes.
4215 • result_shape as the resulting shape produced by broadcasting
4217 the N shapes with the base shape.
4218 The following are some examples drawn from this strategy.
4220 >>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
4222 ... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
4223 >>> for _ in range(5):
4224 ... print(strat.example())
4225 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
4226 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
4227 BroadcastableShapes(input_shapes=((), (), ()), result_shape=())
4228 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(3,))
4229 BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))
4230 xps.indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False,
4232 allow_ellipsis=True)
4233 Return a strategy for valid indices of arrays with the specified
4234 shape, which may include dimensions of size zero.
4235 It generates tuples containing some mix of integers,
4237 python:slice objects, ... (an Ellipsis), and None. When a
4238 length-one tuple would be generated, this strategy may instead
4239 return the element which will index the first axis, e.g. 5 in‐
4240 stead of (5,).
4241 • shape is the shape of the array that will be indexed, as a tu‐
4243 ple of integers >= 0. This must be at least two-dimensional
4244 for a tuple to be a valid index; for one-dimensional arrays
4245 use slices() instead.
4246 • min_dims is the minimum dimensionality of the resulting array
4248 from use of the generated index.
4249 • max_dims is the the maximum dimensionality of the resulting
4251 array, defaulting to len(shape) if not allow_newaxis else
4252 max(len(shape), min_dims) + 2.
4253 • allow_ellipsis specifies whether None is allowed in the index.
4255 • allow_ellipsis specifies whether ... is allowed in the index.
4257
4259 When Hypothesis finds a bug it stores enough information in its data‐
4260 base to reproduce it. This enables you to have a classic testing work‐
4261 flow of find a bug, fix a bug, and be confident that this is actually
4262 doing the right thing because Hypothesis will start by retrying the ex‐
4263 amples that broke things last time.
4264 Limitations
4266 The database is best thought of as a cache that you never need to in‐
4267 validate: Information may be lost when you upgrade a Hypothesis version
4268 or change your test, so you shouldn't rely on it for correctness - if
4269 there's an example you want to ensure occurs each time then there's a
4270 feature for including them in your source code - but it helps the de‐
4271 velopment workflow considerably by making sure that the examples you've
4272 just found are reproduced.
4273 The database also records examples that exercise less-used parts of
4275 your code, so the database may update even when no failing examples
4276 were found.
4277 Upgrading Hypothesis and changing your tests
4279 The design of the Hypothesis database is such that you can put arbi‐
4280 trary data in the database and not get wrong behaviour. When you up‐
4281 grade Hypothesis, old data might be invalidated, but this should happen
4282 transparently. It can never be the case that e.g. changing the strategy
4283 that generates an argument gives you data from the old strategy.
4284 ExampleDatabase implementations
4286 Hypothesis' default database setting creates a
4287 DirectoryBasedExampleDatabase in your current working directory, under
4288 .hypothesis/examples. If this location is unusable, e.g. because you
4289 do not have read or write permissions, Hypothesis will emit a warning
4290 and fall back to an InMemoryExampleDatabase.
4291 Hypothesis provides the following ExampleDatabase implementations:
4293 class hypothesis.database.InMemoryExampleDatabase
4295 A non-persistent example database, implemented in terms of a
4296 dict of sets.
4297 This can be useful if you call a test function several times in
4299 a single session, or for testing other database implementations,
4300 but because it does not persist between runs we do not recommend
4301 it for general use.
4302 class hypothesis.database.DirectoryBasedExampleDatabase(path)
4304 Use a directory to store Hypothesis examples as files.
4305 Each test corresponds to a directory, and each example to a file
4307 within that directory. While the contents are fairly opaque, a
4308 DirectoryBasedExampleDatabase can be shared by checking the di‐
4309 rectory into version control, for example with the following
4310 .gitignore:
4311 # Ignore files cached by Hypothesis...
4313 .hypothesis/*
4314 # except for the examples directory
4315 !.hypothesis/examples/
4316 Note however that this only makes sense if you also pin to an
4318 exact version of Hypothesis, and we would usually recommend im‐
4319 plementing a shared database with a network datastore - see
4320 ExampleDatabase, and the MultiplexedDatabase helper.
4321 class hypothesis.database.GitHubArtifactDatabase(owner, repo, arti‐
4323 fact_name='hypothesis-example-db', cache_timeout=date‐
4324 time.timedelta(days=1), path=None)
4325 A file-based database loaded from a GitHub Actions artifact.
4326 You can use this for sharing example databases between CI runs
4328 and developers, allowing the latter to get read-only access to
4329 the former. This is particularly useful for continuous fuzzing
4330 (i.e. with HypoFuzz), where the CI system can help find new
4331 failing examples through fuzzing, and developers can reproduce
4332 them locally without any manual effort.
4333 NOTE:
4335 You must provide GITHUB_TOKEN as an environment variable. In
4336 CI, Github Actions provides this automatically, but it needs
4337 to be set manually for local usage. In a developer machine,
4338 this would usually be a Personal Access Token. If the repos‐
4339 itory is private, it's necessary for the token to have repo
4340 scope in the case of a classic token, or actions:read in the
4341 case of a fine-grained token.
4342 In most cases, this will be used through the
4344 MultiplexedDatabase, by combining a local directory-based data‐
4345 base with this one. For example:
4346 local = DirectoryBasedExampleDatabase(".hypothesis/examples")
4348 shared = ReadOnlyDatabase(GitHubArtifactDatabase("user", "repo"))
4349 settings.register_profile("ci", database=local)
4351 settings.register_profile("dev", database=MultiplexedDatabase(local, shared))
4352 # We don't want to use the shared database in CI, only to populate its local one.
4353 # which the workflow should then upload as an artifact.
4354 settings.load_profile("ci" if os.environ.get("CI") else "dev")
4355 NOTE:
4357 Because this database is read-only, you always need to wrap
4358 it with the ReadOnlyDatabase.
4359 A setup like this can be paired with a GitHub Actions workflow
4361 including something like the following:
4362 - name: Download example database
4364 uses: dawidd6/action-download-artifact@v2.24.3
4365 with:
4366 name: hypothesis-example-db
4367 path: .hypothesis/examples
4368 if_no_artifact_found: warn
4369 workflow_conclusion: completed
4370 - name: Run tests
4372 run: pytest
4373 - name: Upload example database
4375 uses: actions/upload-artifact@v3
4376 if: always()
4377 with:
4378 name: hypothesis-example-db
4379 path: .hypothesis/examples
4380 In this workflow, we use dawidd6/action-download-artifact to
4382 download the latest artifact given that the official
4383 actions/download-artifact does not support downloading artifacts
4384 from previous workflow runs.
4385 The database automatically implements a simple file-based cache
4387 with a default expiration period of 1 day. You can adjust this
4388 through the cache_timeout property.
4389 For mono-repo support, you can provide a unique artifact_name
4391 (e.g. hypofuzz-example-db-frontend).
4392 class hypothesis.database.ReadOnlyDatabase(db)
4394 A wrapper to make the given database read-only.
4395 The implementation passes through fetch, and turns save, delete,
4397 and move into silent no-ops.
4398 Note that this disables Hypothesis' automatic discarding of
4400 stale examples. It is designed to allow local machines to ac‐
4401 cess a shared database (e.g. from CI servers), without propagat‐
4402 ing changes back from a local or in-development branch.
4403 class hypothesis.database.MultiplexedDatabase(*dbs)
4405 A wrapper around multiple databases.
4406 Each save, fetch, move, or delete operation will be run against
4408 all of the wrapped databases. fetch does not yield duplicate
4409 values, even if the same value is present in two or more of the
4410 wrapped databases.
4411 This combines well with a ReadOnlyDatabase, as follows:
4413 local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
4415 shared = CustomNetworkDatabase()
4416 settings.register_profile("ci", database=shared)
4418 settings.register_profile(
4419 "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
4420 )
4421 settings.load_profile("ci" if os.environ.get("CI") else "dev")
4422 So your CI system or fuzzing runs can populate a central shared
4424 database; while local runs on development machines can reproduce
4425 any failures from CI but will only cache their own failures lo‐
4426 cally and cannot remove examples from the shared database.
4427 class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_af‐
4429 ter=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:')
4430 Store Hypothesis examples as sets in the given Redis datastore.
4431 This is particularly useful for shared databases, as per the
4433 recipe for a MultiplexedDatabase.
4434 NOTE:
4436 If a test has not been run for expire_after, those examples
4437 will be allowed to expire. The default time-to-live persists
4438 examples between weekly runs.
4439 Defining your own ExampleDatabase
4441 You can define your ExampleDatabase, for example to use a shared datas‐
4442 tore, with just a few methods:
4443 class hypothesis.database.ExampleDatabase(*args, **kwargs)
4445 An abstract base class for storing examples in Hypothesis' in‐
4446 ternal format.
4447 An ExampleDatabase maps each bytes key to many distinct bytes
4449 values, like a Mapping[bytes, AbstractSet[bytes]].
4450 abstract save(key, value)
4452 Save value under key.
4453 If this value is already present for this key, silently
4455 do nothing.
4456 abstract fetch(key)
4458 Return an iterable over all values matching this key.
4459 abstract delete(key, value)
4461 Remove this value from this key.
4462 If this value is not present, silently do nothing.
4464 move(src, dest, value)
4466 Move value from key src to key dest. Equivalent to
4467 delete(src, value) followed by save(src, value), but may
4468 have a more efficient implementation.
4469 Note that value will be inserted at dest regardless of
4471 whether it is currently present at src.
4472
4474 With @given, your tests are still something that you mostly write your‐
4475 self, with Hypothesis providing some data. With Hypothesis's stateful
4476 testing, Hypothesis instead tries to generate not just data but entire
4477 tests. You specify a number of primitive actions that can be combined
4478 together, and then Hypothesis will try to find sequences of those ac‐
4479 tions that result in a failure.
4480 TIP:
4482 Before reading this reference documentation, we recommend reading
4483 How not to Die Hard with Hypothesis and An Introduction to
4484 Rule-Based Stateful Testing, in that order. The implementation de‐
4485 tails will make more sense once you've seen them used in practice,
4486 and know why each method or decorator is available.
4487 NOTE:
4489 This style of testing is often called model-based testing, but in
4490 Hypothesis is called stateful testing (mostly for historical reasons
4491 - the original implementation of this idea in Hypothesis was more
4492 closely based on ScalaCheck's stateful testing where the name is
4493 more apt). Both of these names are somewhat misleading: You don't
4494 really need any sort of formal model of your code to use this, and
4495 it can be just as useful for pure APIs that don't involve any state
4496 as it is for stateful ones.
4497 It's perhaps best to not take the name of this sort of testing too
4499 seriously. Regardless of what you call it, it is a powerful form of
4500 testing which is useful for most non-trivial APIs.
4501 You may not need state machines
4503 The basic idea of stateful testing is to make Hypothesis choose actions
4504 as well as values for your test, and state machines are a great declar‐
4505 ative way to do just that.
4506 For simpler cases though, you might not need them at all - a standard
4508 test with @given might be enough, since you can use data() in branches
4509 or loops. In fact, that's how the state machine explorer works inter‐
4510 nally. For more complex workloads though, where a higher level API
4511 comes into it's own, keep reading!
4512 Rule-based state machines
4514 class hypothesis.stateful.RuleBasedStateMachine
4515 A RuleBasedStateMachine gives you a structured way to define
4516 state machines.
4517 The idea is that a state machine carries a bunch of types of
4519 data divided into Bundles, and has a set of rules which may read
4520 data from bundles (or just from normal strategies) and push data
4521 onto bundles. At any given point a random applicable rule will
4522 be executed.
4523 A rule is very similar to a normal @given based test in that it takes
4525 values drawn from strategies and passes them to a user defined test
4526 function. The key difference is that where @given based tests must be
4527 independent, rules can be chained together - a single test run may in‐
4528 volve multiple rule invocations, which may interact in various ways.
4529 Rules can take normal strategies as arguments, or a specific kind of
4531 strategy called a Bundle. A Bundle is a named collection of generated
4532 values that can be reused by other operations in the test. They are
4533 populated with the results of rules, and may be used as arguments to
4534 rules, allowing data to flow from one rule to another, and rules to
4535 work on the results of previous computations or actions.
4536 You can think of each value that gets added to any Bundle as being as‐
4538 signed to a new variable. Drawing a value from the bundle strategy
4539 means choosing one of the corresponding variables and using that value,
4540 and consumes() as a del statement for that variable. If you can re‐
4541 place use of Bundles with instance attributes of the class that is of‐
4542 ten simpler, but often Bundles are strictly more powerful.
4543 The following rule based state machine example is a simplified version
4545 of a test for Hypothesis's example database implementation. An example
4546 database maps keys to sets of values, and in this test we compare one
4547 implementation of it to a simplified in memory model of its behaviour,
4548 which just stores the same values in a Python dict. The test then runs
4549 operations against both the real database and the in-memory representa‐
4550 tion of it and looks for discrepancies in their behaviour.
4551 import shutil
4553 import tempfile
4554 from collections import defaultdict
4555 import hypothesis.strategies as st
4557 from hypothesis.database import DirectoryBasedExampleDatabase
4558 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
4559 class DatabaseComparison(RuleBasedStateMachine):
4562 def __init__(self):
4563 super().__init__()
4564 self.tempd = tempfile.mkdtemp()
4565 self.database = DirectoryBasedExampleDatabase(self.tempd)
4566 self.model = defaultdict(set)
4567 keys = Bundle("keys")
4569 values = Bundle("values")
4570 @rule(target=keys, k=st.binary())
4572 def add_key(self, k):
4573 return k
4574 @rule(target=values, v=st.binary())
4576 def add_value(self, v):
4577 return v
4578 @rule(k=keys, v=values)
4580 def save(self, k, v):
4581 self.model[k].add(v)
4582 self.database.save(k, v)
4583 @rule(k=keys, v=values)
4585 def delete(self, k, v):
4586 self.model[k].discard(v)
4587 self.database.delete(k, v)
4588 @rule(k=keys)
4590 def values_agree(self, k):
4591 assert set(self.database.fetch(k)) == self.model[k]
4592 def teardown(self):
4594 shutil.rmtree(self.tempd)
4595 TestDBComparison = DatabaseComparison.TestCase
4598 In this we declare two bundles - one for keys, and one for values. We
4600 have two trivial rules which just populate them with data (k and v),
4601 and three non-trivial rules: save saves a value under a key and delete
4602 removes a value from a key, in both cases also updating the model of
4603 what should be in the database. values_agree then checks that the con‐
4604 tents of the database agrees with the model for a particular key.
4605 We can then integrate this into our test suite by getting a unittest
4607 TestCase from it:
4608 TestTrees = DatabaseComparison.TestCase
4610 # Or just run with pytest's unittest support
4612 if __name__ == "__main__":
4613 unittest.main()
4614 This test currently passes, but if we comment out the line where we
4616 call self.model[k].discard(v), we would see the following output when
4617 run under pytest:
4618 AssertionError: assert set() == {b''}
4620 ------------ Hypothesis ------------
4622 state = DatabaseComparison()
4624 var1 = state.add_key(k=b'')
4625 var2 = state.add_value(v=var1)
4626 state.save(k=var1, v=var2)
4627 state.delete(k=var1, v=var2)
4628 state.values_agree(k=var1)
4629 state.teardown()
4630 Note how it's printed out a very short program that will demonstrate
4632 the problem. The output from a rule based state machine should gener‐
4633 ally be pretty close to Python code - if you have custom repr implemen‐
4634 tations that don't return valid Python then it might not be, but most
4635 of the time you should just be able to copy and paste the code into a
4636 test to reproduce it.
4637 You can control the detailed behaviour with a settings object on the
4639 TestCase (this is a normal hypothesis settings object using the de‐
4640 faults at the time the TestCase class was first referenced). For exam‐
4641 ple if you wanted to run fewer examples with larger programs you could
4642 change the settings to:
4643 DatabaseComparison.TestCase.settings = settings(
4645 max_examples=50, stateful_step_count=100
4646 )
4647 Which doubles the number of steps each program runs and halves the num‐
4649 ber of test cases that will be run.
4650 Rules
4652 As said earlier, rules are the most common feature used in RuleBased‐
4653 StateMachine. They are defined by applying the rule() decorator on a
4654 function. Note that RuleBasedStateMachine must have at least one rule
4655 defined and that a single function cannot be used to define multiple
4656 rules (this to avoid having multiple rules doing the same things). Due
4657 to the stateful execution method, rules generally cannot take arguments
4658 from other sources such as fixtures or pytest.mark.parametrize - con‐
4659 sider providing them via a strategy such as sampled_from() instead.
4660 hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)
4662 Decorator for RuleBasedStateMachine. Any Bundle present in tar‐
4663 get or targets will define where the end result of this function
4664 should go. If both are empty then the end result will be dis‐
4665 carded.
4666 target must be a Bundle, or if the result should go to multiple
4668 bundles you can pass a tuple of them as the targets argument.
4669 It is invalid to use both arguments for a single rule. If the
4670 result should go to exactly one of several bundles, define a
4671 separate rule for each case.
4672 kwargs then define the arguments that will be passed to the
4674 function invocation. If their value is a Bundle, or if it is
4675 consumes(b) where b is a Bundle, then values that have previ‐
4676 ously been produced for that bundle will be provided. If con‐
4677 sumes is used, the value will also be removed from the bundle.
4678 Any other kwargs should be strategies and values from them will
4680 be provided.
4681 hypothesis.stateful.consumes(bundle)
4683 When introducing a rule in a RuleBasedStateMachine, this func‐
4684 tion can be used to mark bundles from which each value used in a
4685 step with the given rule should be removed. This function re‐
4686 turns a strategy object that can be manipulated and combined
4687 like any other.
4688 For example, a rule declared with
4690 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
4692 sumes(b3)))
4693 will consume a value from Bundle b2 and several values from Bun‐
4695 dle b3 to populate value2 and value3 each time it is executed.
4696 hypothesis.stateful.multiple(*args)
4698 This function can be used to pass multiple results to the tar‐
4699 get(s) of a rule. Just use return multiple(result1, result2,
4700 ...) in your rule.
4701 It is also possible to use return multiple() with no arguments
4703 in order to end a rule without passing any result.
4704 Initializes
4706 Initializes are a special case of rules that are guaranteed to be run
4707 at most once at the beginning of a run (i.e. before any normal rule is
4708 called). Note if multiple initialize rules are defined, they may be
4709 called in any order, and that order will vary from run to run.
4710 Initializes are typically useful to populate bundles:
4712 hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)
4714 Decorator for RuleBasedStateMachine.
4715 An initialize decorator behaves like a rule, but all @initial‐
4717 ize() decorated methods will be called before any @rule() deco‐
4718 rated methods, in an arbitrary order. Each @initialize() method
4719 will be called exactly once per run, unless one raises an excep‐
4720 tion - after which only the .teardown() method will be run.
4721 @initialize() methods may not have preconditions.
4722 import hypothesis.strategies as st
4724 from hypothesis.stateful import Bundle, RuleBasedStateMachine, initialize, rule
4725 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
4727 class NumberModifier(RuleBasedStateMachine):
4730 folders = Bundle("folders")
4731 files = Bundle("files")
4732 @initialize(target=folders)
4734 def init_folders(self):
4735 return "/"
4736 @rule(target=folders, name=name_strategy)
4738 def create_folder(self, parent, name):
4739 return f"{parent}/{name}"
4740 @rule(target=files, name=name_strategy)
4742 def create_file(self, parent, name):
4743 return f"{parent}/{name}"
4744 Preconditions
4746 While it's possible to use assume() in RuleBasedStateMachine rules, if
4747 you use it in only a few rules you can quickly run into a situation
4748 where few or none of your rules pass their assumptions. Thus, Hypothe‐
4749 sis provides a precondition() decorator to avoid this problem. The
4750 precondition() decorator is used on rule-decorated functions, and must
4751 be given a function that returns True or False based on the RuleBased‐
4752 StateMachine instance.
4753 hypothesis.stateful.precondition(precond)
4755 Decorator to apply a precondition for rules in a RuleBased‐
4756 StateMachine. Specifies a precondition for a rule to be consid‐
4757 ered as a valid step in the state machine, which is more effi‐
4758 cient than using assume() within the rule. The precond function
4759 will be called with the instance of RuleBasedStateMachine and
4760 should return True or False. Usually it will need to look at at‐
4761 tributes on that instance.
4762 For example:
4764 class MyTestMachine(RuleBasedStateMachine):
4766 state = 1
4767 @precondition(lambda self: self.state != 0)
4769 @rule(numerator=integers())
4770 def divide_with(self, numerator):
4771 self.state = numerator / self.state
4772 If multiple preconditions are applied to a single rule, it is
4774 only considered a valid step when all of them return True. Pre‐
4775 conditions may be applied to invariants as well as rules.
4776 from hypothesis.stateful import RuleBasedStateMachine, precondition, rule
4778 class NumberModifier(RuleBasedStateMachine):
4781 num = 0
4782 @rule()
4784 def add_one(self):
4785 self.num += 1
4786 @precondition(lambda self: self.num != 0)
4788 @rule()
4789 def divide_with_one(self):
4790 self.num = 1 / self.num
4791 By using precondition() here instead of assume(), Hypothesis can filter
4793 the inapplicable rules before running them. This makes it much more
4794 likely that a useful sequence of steps will be generated.
4795 Note that currently preconditions can't access bundles; if you need to
4797 use preconditions, you should store relevant data on the instance in‐
4798 stead.
4799 Invariants
4801 Often there are invariants that you want to ensure are met after every
4802 step in a process. It would be possible to add these as rules that are
4803 run, but they would be run zero or multiple times between other rules.
4804 Hypothesis provides a decorator that marks a function to be run after
4805 every step.
4806 hypothesis.stateful.invariant(*, check_during_init=False)
4808 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
4809 chine. The decorated function will be run after every rule and
4810 can raise an exception to indicate failed invariants.
4811 For example:
4813 class MyTestMachine(RuleBasedStateMachine):
4815 state = 1
4816 @invariant()
4818 def is_nonzero(self):
4819 assert self.state != 0
4820 By default, invariants are only checked after all @initialize()
4822 rules have been run. Pass check_during_init=True for invariants
4823 which can also be checked during initialization.
4824 from hypothesis.stateful import RuleBasedStateMachine, invariant, rule
4826 class NumberModifier(RuleBasedStateMachine):
4829 num = 0
4830 @rule()
4832 def add_two(self):
4833 self.num += 2
4834 if self.num > 50:
4835 self.num += 1
4836 @invariant()
4838 def divide_with_one(self):
4839 assert self.num % 2 == 0
4840 NumberTest = NumberModifier.TestCase
4843 Invariants can also have precondition()s applied to them, in which case
4845 they will only be run if the precondition function returns true.
4846 Note that currently invariants can't access bundles; if you need to use
4848 invariants, you should store relevant data on the instance instead.
4849 More fine grained control
4851 If you want to bypass the TestCase infrastructure you can invoke these
4852 manually. The stateful module exposes the function run_state_ma‐
4853 chine_as_test, which takes an arbitrary function returning a RuleBased‐
4854 StateMachine and an optional settings parameter and does the same as
4855 the class based runTest provided.
4856 This is not recommended as it bypasses some important internal func‐
4858 tions, including reporting of statistics such as runtimes and event()
4859 calls. It was originally added to support custom __init__ methods, but
4860 you can now use initialize() rules instead.
4861
4863 Hypothesis does its level best to be compatible with everything you
4864 could possibly need it to be compatible with. Generally you should just
4865 try it and expect it to work. If it doesn't, you can be surprised and
4866 check this document for the details.
4867 Hypothesis versions
4869 Backwards compatibility is better than backporting fixes, so we use
4870 semantic versioning and only support the most recent version of Hypoth‐
4871 esis. See Help and support for more information.
4872 Documented APIs will not break except between major version bumps. All
4874 APIs mentioned in this documentation are public unless explicitly noted
4875 as provisional, in which case they may be changed in minor releases.
4876 Undocumented attributes, modules, and behaviour may include breaking
4877 changes in patch releases.
4878 Deprecations
4880 Deprecated features will emit warnings for at least six months, and
4881 then be removed in the following major release.
4882 Note however that not all warnings are subject to this grace period;
4884 sometimes we strengthen validation by adding a warning and these may
4885 become errors immediately at a major release.
4886 We use custom exception and warning types, so you can see exactly where
4888 an error came from, or turn only our warnings into errors.
4889 class hypothesis.errors.HypothesisDeprecationWarning
4891 A deprecation warning issued by Hypothesis.
4892 Actually inherits from FutureWarning, because DeprecationWarning
4894 is hidden by the default warnings filter.
4895 You can configure the Python python:warnings to handle these
4897 warnings differently to others, either turning them into errors
4898 or suppressing them entirely. Obviously we would prefer the
4899 former!
4900 Python versions
4902 Hypothesis is supported and tested on CPython 3.7+, i.e. all versions
4903 of CPython with upstream support, along with PyPy for the same ver‐
4904 sions. 32-bit builds of CPython also work, though we only test them on
4905 Windows.
4906 In general Hypothesis does not officially support anything except the
4908 latest patch release of any version of Python it supports. Earlier re‐
4909 leases should work and bugs in them will get fixed if reported, but
4910 they're not tested in CI and no guarantees are made.
4911 Operating systems
4913 In theory Hypothesis should work anywhere that Python does. In practice
4914 it is only known to work and regularly tested on OS X, Windows and
4915 Linux, and you may experience issues running it elsewhere.
4916 If you're using something else and it doesn't work, do get in touch and
4918 I'll try to help, but unless you can come up with a way for me to run a
4919 CI server on that operating system it probably won't stay fixed due to
4920 the inevitable march of time.
4921 Testing frameworks
4923 In general Hypothesis goes to quite a lot of effort to generate things
4924 that look like normal Python test functions that behave as closely to
4925 the originals as possible, so it should work sensibly out of the box
4926 with every test framework.
4927 If your testing relies on doing something other than calling a function
4929 and seeing if it raises an exception then it probably won't work out of
4930 the box. In particular things like tests which return generators and
4931 expect you to do something with them (e.g. nose's yield based tests)
4932 will not work. Use a decorator or similar to wrap the test to take this
4933 form, or ask the framework maintainer to support our hooks for insert‐
4934 ing such a wrapper later.
4935 In terms of what's actually known to work:
4937 • Hypothesis integrates as smoothly with pytest and unittest as we
4939 can make it, and this is verified as part of the CI.
4940 • pytest fixtures work in the usual way for tests that have been
4942 decorated with @given - just avoid passing a strategy for each ar‐
4943 gument that will be supplied by a fixture. However, each fixture
4944 will run once for the whole function, not once per example. Deco‐
4945 rating a fixture function with @given is meaningless.
4946 • The python:unittest.mock.patch() decorator works with @given, but
4948 we recommend using it as a context manager within the decorated
4949 test to ensure that the mock is per-test-case and avoid poor in‐
4950 teractions with Pytest fixtures.
4951 • Nose works fine with Hypothesis, and this is tested as part of the
4953 CI. yield based tests simply won't work.
4954 • Integration with Django's testing requires use of the Hypothesis
4956 for Django users extra. The issue is that in Django's tests' nor‐
4957 mal mode of execution it will reset the database once per test
4958 rather than once per example, which is not what you want.
4959 • Coverage works out of the box with Hypothesis; our own test suite
4961 has 100% branch coverage.
4962 Optional packages
4964 The supported versions of optional packages, for strategies in hypothe‐
4965 sis.extra, are listed in the documentation for that extra. Our general
4966 goal is to support all versions that are supported upstream.
4967 Regularly verifying this
4969 Everything mentioned above as explicitly supported is checked on every
4970 commit with GitHub Actions. Our continuous delivery pipeline runs all
4971 of these checks before publishing each release, so when we say they're
4972 supported we really mean it.
4973
4975 This is a collection of examples of how to use Hypothesis in interest‐
4976 ing ways. It's small for now but will grow over time.
4977 All of these examples are designed to be run under pytest, and nose
4979 should work too.
4980 How not to sort by a partial order
4982 The following is an example that's been extracted and simplified from a
4983 real bug that occurred in an earlier version of Hypothesis. The real
4984 bug was a lot harder to find.
4985 Suppose we've got the following type:
4987 class Node:
4989 def __init__(self, label, value):
4990 self.label = label
4991 self.value = tuple(value)
4992 def __repr__(self):
4994 return f"Node({self.label!r}, {self.value!r})"
4995 def sorts_before(self, other):
4997 if len(self.value) >= len(other.value):
4998 return False
4999 return other.value[: len(self.value)] == self.value
5000 Each node is a label and a sequence of some data, and we have the rela‐
5002 tionship sorts_before meaning the data of the left is an initial seg‐
5003 ment of the right. So e.g. a node with value [1, 2] will sort before a
5004 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
5005 before the other.
5006 We have a list of nodes, and we want to topologically sort them with
5008 respect to this ordering. That is, we want to arrange the list so that
5009 if x.sorts_before(y) then x appears earlier in the list than y. We
5010 naively think that the easiest way to do this is to extend the partial
5011 order defined here to a total order by breaking ties arbitrarily and
5012 then using a normal sorting algorithm. So we define the following code:
5013 from functools import total_ordering
5015 @total_ordering
5018 class TopoKey:
5019 def __init__(self, node):
5020 self.value = node
5021 def __lt__(self, other):
5023 if self.value.sorts_before(other.value):
5024 return True
5025 if other.value.sorts_before(self.value):
5026 return False
5027 return self.value.label < other.value.label
5029 def sort_nodes(xs):
5032 xs.sort(key=TopoKey)
5033 This takes the order defined by sorts_before and extends it by breaking
5035 ties by comparing the node labels.
5036 But now we want to test that it works.
5038 First we write a function to verify that our desired outcome holds:
5040 def is_prefix_sorted(xs):
5042 for i in range(len(xs)):
5043 for j in range(i + 1, len(xs)):
5044 if xs[j].sorts_before(xs[i]):
5045 return False
5046 return True
5047 This will return false if it ever finds a pair in the wrong order and
5049 return true otherwise.
5050 Given this function, what we want to do with Hypothesis is assert that
5052 for all sequences of nodes, the result of calling sort_nodes on it is
5053 sorted.
5054 First we need to define a strategy for Node:
5056 import hypothesis.strategies as st
5058 NodeStrategy = st.builds(Node, st.integers(), st.lists(st.booleans(), max_size=10))
5060 We want to generate short lists of values so that there's a decent
5062 chance of one being a prefix of the other (this is also why the choice
5063 of bool as the elements). We then define a strategy which builds a node
5064 out of an integer and one of those short lists of booleans.
5065 We can now write a test:
5067 from hypothesis import given
5069 @given(st.lists(NodeStrategy))
5072 def test_sorting_nodes_is_prefix_sorted(xs):
5073 sort_nodes(xs)
5074 assert is_prefix_sorted(xs)
5075 this immediately fails with the following example:
5077 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
5079 The reason for this is that because False is not a prefix of (True,
5081 True) nor vice versa, sorting things the first two nodes are equal be‐
5082 cause they have equal labels. This makes the whole order non-transi‐
5083 tive and produces basically nonsense results.
5084 But this is pretty unsatisfying. It only works because they have the
5086 same label. Perhaps we actually wanted our labels to be unique. Let's
5087 change the test to do that.
5088 def deduplicate_nodes_by_label(nodes):
5090 table = {node.label: node for node in nodes}
5091 return list(table.values())
5092 We define a function to deduplicate nodes by labels, and can now map
5094 that over a strategy for lists of nodes to give us a strategy for lists
5095 of nodes with unique labels:
5096 @given(st.lists(NodeStrategy).map(deduplicate_nodes_by_label))
5098 def test_sorting_nodes_is_prefix_sorted(xs):
5099 sort_nodes(xs)
5100 assert is_prefix_sorted(xs)
5101 Hypothesis quickly gives us an example of this still being wrong:
5103 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]
5105 Now this is a more interesting example. None of the nodes will sort
5107 equal. What is happening here is that the first node is strictly less
5108 than the last node because (False,) is a prefix of (False, False). This
5109 is in turn strictly less than the middle node because neither is a pre‐
5110 fix of the other and -2 < -1. The middle node is then less than the
5111 first node because -1 < 0.
5112 So, convinced that our implementation is broken, we write a better one:
5114 def sort_nodes(xs):
5116 for i in range(1, len(xs)):
5117 j = i - 1
5118 while j >= 0:
5119 if xs[j].sorts_before(xs[j + 1]):
5120 break
5121 xs[j], xs[j + 1] = xs[j + 1], xs[j]
5122 j -= 1
5123 This is just insertion sort slightly modified - we swap a node back‐
5125 wards until swapping it further would violate the order constraints.
5126 The reason this works is because our order is a partial order already
5127 (this wouldn't produce a valid result for a general topological sorting
5128 - you need the transitivity).
5129 We now run our test again and it passes, telling us that this time
5131 we've successfully managed to sort some nodes without getting it com‐
5132 pletely wrong. Go us.
5133 Time zone arithmetic
5135 This is an example of some tests for pytz which check that various
5136 timezone conversions behave as you would expect them to. These tests
5137 should all pass, and are mostly a demonstration of some useful sorts of
5138 thing to test with Hypothesis, and how the datetimes() strategy works.
5139 from datetime import timedelta
5141 # The datetimes strategy is naive by default, so tell it to use timezones
5143 aware_datetimes = st.datetimes(timezones=st.timezones())
5144 @given(aware_datetimes, st.timezones(), st.timezones())
5147 def test_convert_via_intermediary(dt, tz1, tz2):
5148 """Test that converting between timezones is not affected
5149 by a detour via another timezone.
5150 """
5151 assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
5152 @given(aware_datetimes, st.timezones())
5155 def test_convert_to_and_fro(dt, tz2):
5156 """If we convert to a new timezone and back to the old one
5157 this should leave the result unchanged.
5158 """
5159 tz1 = dt.tzinfo
5160 assert dt == dt.astimezone(tz2).astimezone(tz1)
5161 @given(aware_datetimes, st.timezones())
5164 def test_adding_an_hour_commutes(dt, tz):
5165 """When converting between timezones it shouldn't matter
5166 if we add an hour here or add an hour there.
5167 """
5168 an_hour = timedelta(hours=1)
5169 assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
5170 @given(aware_datetimes, st.timezones())
5173 def test_adding_a_day_commutes(dt, tz):
5174 """When converting between timezones it shouldn't matter
5175 if we add a day here or add a day there.
5176 """
5177 a_day = timedelta(days=1)
5178 assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
5179 Condorcet's paradox
5181 A classic paradox in voting theory, called Condorcet's paradox, is that
5182 majority preferences are not transitive. That is, there is a population
5183 and a set of three candidates A, B and C such that the majority of the
5184 population prefer A to B, B to C and C to A.
5185 Wouldn't it be neat if we could use Hypothesis to provide an example of
5187 this?
5188 Well as you can probably guess from the presence of this section, we
5190 can! The main trick is to decide how we want to represent the result
5191 of an election - for this example, we'll use a list of "votes", where
5192 each vote is a list of candidates in the voters preferred order. With‐
5193 out further ado, here is the code:
5194 from collections import Counter
5196 from hypothesis import given
5198 from hypothesis.strategies import lists, permutations
5199 # We need at least three candidates and at least three voters to have a
5202 # paradox; anything less can only lead to victories or at worst ties.
5203 @given(lists(permutations(["A", "B", "C"]), min_size=3))
5204 def test_elections_are_transitive(election):
5205 all_candidates = {"A", "B", "C"}
5206 # First calculate the pairwise counts of how many prefer each candidate
5208 # to the other
5209 counts = Counter()
5210 for vote in election:
5211 for i in range(len(vote)):
5212 for j in range(i + 1, len(vote)):
5213 counts[(vote[i], vote[j])] += 1
5214 # Now look at which pairs of candidates one has a majority over the
5216 # other and store that.
5217 graph = {}
5218 for i in all_candidates:
5219 for j in all_candidates:
5220 if counts[(i, j)] > counts[(j, i)]:
5221 graph.setdefault(i, set()).add(j)
5222 # Now for each triple assert that it is transitive.
5224 for x in all_candidates:
5225 for y in graph.get(x, ()):
5226 for z in graph.get(y, ()):
5227 assert x not in graph.get(z, ())
5228 The example Hypothesis gives me on my first run (your mileage may of
5230 course vary) is:
5231 [["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]
5233 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
5235 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
5236 and 2) prefer C to A. This is in fact basically the canonical example
5237 of the voting paradox.
5238 Fuzzing an HTTP API
5240 Hypothesis's support for testing HTTP services is somewhat nascent.
5241 There are plans for some fully featured things around this, but right
5242 now they're probably quite far down the line.
5243 But you can do a lot yourself without any explicit support! Here's a
5245 script I wrote to throw arbitrary data against the API for an entirely
5246 fictitious service called Waspfinder (this is only lightly obfuscated
5247 and you can easily figure out who I'm actually talking about, but I
5248 don't want you to run this code and hammer their API without their per‐
5249 mission).
5250 All this does is use Hypothesis to generate arbitrary JSON data match‐
5252 ing the format their API asks for and check for 500 errors. More ad‐
5253 vanced tests which then use the result and go on to do other things are
5254 definitely also possible. The schemathesis package provides an excel‐
5255 lent example of this!
5256 import math
5258 import os
5259 import random
5260 import time
5261 import unittest
5262 from collections import namedtuple
5263 import requests
5265 from hypothesis import assume, given, strategies as st
5267 Goal = namedtuple("Goal", ("slug",))
5269 # We just pass in our API credentials via environment variables.
5272 waspfinder_token = os.getenv("WASPFINDER_TOKEN")
5273 waspfinder_user = os.getenv("WASPFINDER_USER")
5274 assert waspfinder_token is not None
5275 assert waspfinder_user is not None
5276 GoalData = st.fixed_dictionaries(
5278 {
5279 "title": st.text(),
5280 "goal_type": st.sampled_from(
5281 ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
5282 ),
5283 "goaldate": st.one_of(st.none(), st.floats()),
5284 "goalval": st.one_of(st.none(), st.floats()),
5285 "rate": st.one_of(st.none(), st.floats()),
5286 "initval": st.floats(),
5287 "panic": st.floats(),
5288 "secret": st.booleans(),
5289 "datapublic": st.booleans(),
5290 }
5291 )
5292 needs2 = ["goaldate", "goalval", "rate"]
5295 class WaspfinderTest(unittest.TestCase):
5298 @given(GoalData)
5299 def test_create_goal_dry_run(self, data):
5300 # We want slug to be unique for each run so that multiple test runs
5301 # don't interfere with each other. If for some reason some slugs trigger
5302 # an error and others don't we'll get a Flaky error, but that's OK.
5303 slug = hex(random.getrandbits(32))[2:]
5304 # Use assume to guide us through validation we know about, otherwise
5306 # we'll spend a lot of time generating boring examples.
5307 # Title must not be empty
5309 assume(data["title"])
5310 # Exactly two of these values should be not None. The other will be
5312 # inferred by the API.
5313 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
5315 for v in data.values():
5316 if isinstance(v, float):
5317 assume(not math.isnan(v))
5318 data["slug"] = slug
5319 # The API nicely supports a dry run option, which means we don't have
5321 # to worry about the user account being spammed with lots of fake goals
5322 # Otherwise we would have to make sure we cleaned up after ourselves
5323 # in this test.
5324 data["dryrun"] = True
5325 data["auth_token"] = waspfinder_token
5326 for d, v in data.items():
5327 if v is None:
5328 data[d] = "null"
5329 else:
5330 data[d] = str(v)
5331 result = requests.post(
5332 "https://waspfinder.example.com/api/v1/users/"
5333 "%s/goals.json" % (waspfinder_user,),
5334 data=data,
5335 )
5336 # Let's not hammer the API too badly. This will of course make the
5338 # tests even slower than they otherwise would have been, but that's
5339 # life.
5340 time.sleep(1.0)
5341 # For the moment all we're testing is that this doesn't generate an
5343 # internal error. If we didn't use the dry run option we could have
5344 # then tried doing more with the result, but this is a good start.
5345 self.assertNotEqual(result.status_code, 500)
5346 if __name__ == "__main__":
5349 unittest.main()
5350
5352 The Hypothesis community is small for the moment but is full of excel‐
5353 lent people who can answer your questions and help you out. Please do
5354 join us. The major place for community discussion is the mailing list.
5355 Feel free to use it to ask for help, provide feedback, or discuss any‐
5357 thing remotely Hypothesis related at all. If you post a question on
5358 Stack Overflow, please use the python-hypothesis tag!
5359 Please note that the Hypothesis code of conduct applies in all Hypothe‐
5361 sis community spaces.
5362 If you would like to cite Hypothesis, please consider our suggested ci‐
5364 tation.
5365 If you like repo badges, we suggest the following badge, which you can
5367 add with reStructuredText or Markdown, respectively: [image]
5368 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
5370 :alt: Tested with Hypothesis
5371 :target: https://hypothesis.readthedocs.io
5372 [](https://hypothesis.readthedocs.io/)
5374 Finally, we have a beautiful logo which appears online, and often on
5376 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
5377 [image]
5378 As well as being beautiful, dragonflies actively hunt down bugs for a
5380 living! You can find the images and a usage guide in the brand direc‐
5381 tory on GitHub, or find us at conferences where we often have stickers
5382 and sometimes other swag.
5383
5385 What is Hypothesis for?
5386 From the perspective of a user, the purpose of Hypothesis is to make it
5388 easier for you to write better tests.
5389 From my perspective as the author, that is of course also a purpose of
5391 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
5392 mania for a moment), the larger purpose of Hypothesis is to drag the
5393 world kicking and screaming into a new and terrifying age of high qual‐
5394 ity software.
5395 Software is, as they say, eating the world. Software is also terrible.
5397 It's buggy, insecure and generally poorly thought out. This combination
5398 is clearly a recipe for disaster.
5399 And the state of software testing is even worse. Although it's fairly
5401 uncontroversial at this point that you should be testing your code, can
5402 you really say with a straight face that most projects you've worked on
5403 are adequately tested?
5404 A lot of the problem here is that it's too hard to write good tests.
5406 Your tests encode exactly the same assumptions and fallacies that you
5407 had when you wrote the code, so they miss exactly the same bugs that
5408 you missed when you wrote the code.
5409 Meanwhile, there are all sorts of tools for making testing better that
5411 are basically unused. The original Quickcheck is from 1999 and the ma‐
5412 jority of developers have not even heard of it, let alone used it.
5413 There are a bunch of half-baked implementations for most languages, but
5414 very few of them are worth using.
5415 The goal of Hypothesis is to bring advanced testing techniques to the
5417 masses, and to provide an implementation that is so high quality that
5418 it is easier to use them than it is not to use them. Where I can, I
5419 will beg, borrow and steal every good idea I can find that someone has
5420 had to make software testing better. Where I can't, I will invent new
5421 ones.
5422 Quickcheck is the start, but I also plan to integrate ideas from fuzz
5424 testing (a planned future feature is to use coverage information to
5425 drive example selection, and the example saving database is already in‐
5426 spired by the workflows people use for fuzz testing), and am open to
5427 and actively seeking out other suggestions and ideas.
5428 The plan is to treat the social problem of people not using these ideas
5430 as a bug to which there is a technical solution: Does property-based
5431 testing not match your workflow? That's a bug, let's fix it by figur‐
5432 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
5433 tom data for your application? That's a bug. Let's fix it by figuring
5434 out how to make it easier, or how to take something you're already us‐
5435 ing to specify your data and derive a generator from that automati‐
5436 cally. Find the explanations of these advanced ideas hopelessly obtuse
5437 and hard to follow? That's a bug. Let's provide you with an easy API
5438 that lets you test your code better without a PhD in software verifica‐
5439 tion.
5440 Grand ambitions, I know, and I expect ultimately the reality will be
5442 somewhat less grand, but so far in about three months of development,
5443 Hypothesis has become the most solid implementation of Quickcheck ever
5444 seen in a mainstream language (as long as we don't count Scala as main‐
5445 stream yet), and at the same time managed to significantly push forward
5446 the state of the art, so I think there's reason to be optimistic.
5447
5449 This is a page for listing people who are using Hypothesis and how ex‐
5450 cited they are about that. If that's you and your name is not on the
5451 list, this file is in Git and I'd love it if you sent me a pull request
5452 to fix that.
5453 Stripe
5455 At Stripe we use Hypothesis to test every piece of our machine learning
5456 model training pipeline (powered by scikit). Before we migrated, our
5457 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
5458 resentative at all of our actual very complex data. Because we needed
5459 to craft examples for each test, we took the easy way out and lived
5460 with extremely low test coverage.
5461 Hypothesis changed all that. Once we had our strategies for generating
5463 Dataframes of features it became trivial to slightly customize each
5464 strategy for new tests. Our coverage is now close to 90%.
5465 Full-stop, property-based testing is profoundly more powerful - and has
5467 caught or prevented far more bugs - than our old style of example-based
5468 testing.
5469 Kristian Glass - Director of Technology at LaterPay GmbH
5471 Hypothesis has been brilliant for expanding the coverage of our test
5472 cases, and also for making them much easier to read and understand, so
5473 we're sure we're testing the things we want in the way we want.
5474 Seth Morton
5476 When I first heard about Hypothesis, I knew I had to include it in my
5477 two open-source Python libraries, natsort and fastnumbers . Quite
5478 frankly, I was a little appalled at the number of bugs and "holes" I
5479 found in the code. I can now say with confidence that my libraries are
5480 more robust to "the wild." In addition, Hypothesis gave me the confi‐
5481 dence to expand these libraries to fully support Unicode input, which I
5482 never would have had the stomach for without such thorough testing ca‐
5483 pabilities. Thanks!
5484 Sixty North
5486 At Sixty North we use Hypothesis for testing Segpy an open source
5487 Python library for shifting data between Python data structures and SEG
5488 Y files which contain geophysical data from the seismic reflection sur‐
5489 veys used in oil and gas exploration.
5490 This is our first experience of property-based testing – as opposed to
5492 example-based testing. Not only are our tests more powerful, they are
5493 also much better explanations of what we expect of the production code.
5494 In fact, the tests are much closer to being specifications. Hypothesis
5495 has located real defects in our code which went undetected by tradi‐
5496 tional test cases, simply because Hypothesis is more relentlessly devi‐
5497 ous about test case generation than us mere humans! We found Hypothe‐
5498 sis particularly beneficial for Segpy because SEG Y is an antiquated
5499 format that uses legacy text encodings (EBCDIC) and even a legacy
5500 floating point format we implemented from scratch in Python.
5501 Hypothesis is sure to find a place in most of our future Python code‐
5503 bases and many existing ones too.
5504 mulkieran
5506 Just found out about this excellent QuickCheck for Python implementa‐
5507 tion and ran up a few tests for my bytesize package last night. Refuted
5508 a few hypotheses in the process.
5509 Looking forward to using it with a bunch of other projects as well.
5511 Adam Johnson
5513 I have written a small library to serialize dicts to MariaDB's dynamic
5514 columns binary format, mariadb-dyncol. When I first developed it, I
5515 thought I had tested it really well - there were hundreds of test
5516 cases, some of them even taken from MariaDB's test suite itself. I was
5517 ready to release.
5518 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
5520 Wow! It found bug after bug after bug. Even after a first release, I
5521 thought of a way to make the tests do more validation, which revealed a
5522 further round of bugs! Most impressively, Hypothesis found a compli‐
5523 cated off-by-one error in a condition with 4095 versus 4096 bytes of
5524 data - something that I would never have found.
5525 Long live Hypothesis! (Or at least, property-based testing).
5527 Josh Bronson
5529 Adopting Hypothesis improved bidict's test coverage and significantly
5530 increased our ability to make changes to the code with confidence that
5531 correct behavior would be preserved. Thank you, David, for the great
5532 testing tool.
5533 Cory Benfield
5535 Hypothesis is the single most powerful tool in my toolbox for working
5536 with algorithmic code, or any software that produces predictable output
5537 from a wide range of sources. When using it with Priority, Hypothesis
5538 consistently found errors in my assumptions and extremely subtle bugs
5539 that would have taken months of real-world use to locate. In some
5540 cases, Hypothesis found subtle deviations from the correct output of
5541 the algorithm that may never have been noticed at all.
5542 When it comes to validating the correctness of your tools, nothing
5544 comes close to the thoroughness and power of Hypothesis.
5545 Jon Moore
5547 One extremely satisfied user here. Hypothesis is a really solid imple‐
5548 mentation of property-based testing, adapted well to Python, and with
5549 good features such as failure-case shrinkers. I first used it on a
5550 project where we needed to verify that a vendor's Python and non-Python
5551 implementations of an algorithm matched, and it found about a dozen
5552 cases that previous example-based testing and code inspections had not.
5553 Since then I've been evangelizing for it at our firm.
5554 Russel Winder
5556 I am using Hypothesis as an integral part of my Python workshops. Test‐
5557 ing is an integral part of Python programming and whilst unittest and,
5558 better, pytest can handle example-based testing, property-based testing
5559 is increasingly far more important than example-base testing, and Hy‐
5560 pothesis fits the bill.
5561 Wellfire Interactive
5563 We've been using Hypothesis in a variety of client projects, from test‐
5564 ing Django-related functionality to domain-specific calculations. It
5565 both speeds up and simplifies the testing process since there's so much
5566 less tedious and error-prone work to do in identifying edge cases. Test
5567 coverage is nice but test depth is even nicer, and it's much easier to
5568 get meaningful test depth using Hypothesis.
5569 Cody Kochmann
5571 Hypothesis is being used as the engine for random object generation
5572 with my open source function fuzzer battle_tested which maps all behav‐
5573 iors of a function allowing you to minimize the chance of unexpected
5574 crashes when running code in production.
5575 With how efficient Hypothesis is at generating the edge cases that
5577 cause unexpected behavior occur, battle_tested is able to map out the
5578 entire behavior of most functions in less than a few seconds.
5579 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
5581 ing it.
5582 Merchise Autrement
5584 Just minutes after our first use of hypothesis we uncovered a subtle
5585 bug in one of our most used library. Since then, we have increasingly
5586 used hypothesis to improve the quality of our testing in libraries and
5587 applications as well.
5588 Florian Kromer
5590 At Roboception GmbH I use Hypothesis to implement fully automated
5591 stateless and stateful reliability tests for the 3D sensor rc_visard
5592 and robotic software components .
5593 Thank you very much for creating the (probably) most powerful prop‐
5595 erty-based testing framework.
5596 Reposit Power
5598 With a micro-service architecture, testing between services is made
5599 easy using Hypothesis in integration testing. Ensuring everything is
5600 running smoothly is vital to help maintain a secure network of Virtual
5601 Power Plants.
5602 It allows us to find potential bugs and edge cases with relative ease
5604 and minimal overhead. As our architecture relies on services communi‐
5605 cating effectively, Hypothesis allows us to strictly test for the kind
5606 of data which moves around our services, particularly our backend
5607 Python applications.
5608 Your name goes here
5610 I know there are many more, because I keep finding out about new people
5611 I'd never even heard of using Hypothesis. If you're looking to way to
5612 give back to a tool you love, adding your name here only takes a moment
5613 and would really help a lot. As per instructions at the top, just send
5614 me a pull request and I'll add you to the list.
5615
5617 The following is a non-exhaustive list of open source projects I know
5618 are using Hypothesis. If you're aware of any others please add them to
5619 the list! The only inclusion criterion right now is that if it's a
5620 Python library then it should be available on PyPI.
5621 You can find hundreds more from the Hypothesis page at libraries.io,
5623 and thousands on GitHub. Hypothesis has over 100,000 downloads per
5624 week, and was used by more than 4% of Python users surveyed by the PSF
5625 in 2020.
5626 • aur
5628 • argon2_cffi
5630 • array-api-tests
5632 • attrs
5634 • axelrod
5636 • bidict
5638 • binaryornot
5640 • brotlicffi
5642 • chardet
5644 • cmph-cffi
5646 • cryptography
5648 • dbus-signature-pyparsing
5650 • dry-python/returns
5652 • fastnumbers
5654 • flocker
5656 • flownetpy
5658 • funsize
5660 • fusion-index
5662 • hyper-h2
5664 • into-dbus-python
5666 • ivy
5668 • justbases
5670 • justbytes
5672 • loris
5674 • mariadb-dyncol
5676 • MDAnalysis
5678 • mercurial
5680 • napari
5682 • natsort
5684 • numpy
5686 • pandas
5688 • pandera
5690 • poliastro
5692 • pretext
5694 • priority
5696 • PyCEbox
5698 • PyPy
5700 • pyrsistent
5702 • python-humble-utils
5704 • pyudev
5706 • qutebrowser
5708 • RubyMarshal
5710 • Segpy
5712 • sgkit
5714 • simoa
5716 • srt
5718 • tchannel
5720 • vdirsyncer
5722 • wcag-contrast-ratio
5724 • xarray
5726 • yacluster
5728 • yturl
5730 • zenml
5732
5734 Hypothesis has been eagerly used and extended by the open source commu‐
5735 nity. This page lists extensions and applications; you can find more
5736 or newer packages by searching PyPI by keyword or filter by classifier,
5737 or search libraries.io.
5738 If there's something missing which you think should be here, let us
5740 know!
5741 NOTE:
5743 Being listed on this page does not imply that the Hypothesis main‐
5744 tainers endorse a package.
5745 External strategies
5747 Some packages provide strategies directly:
5748 • hypothesis-fspaths - strategy to generate filesystem paths.
5750 • hypothesis-geojson - strategy to generate GeoJson.
5752 • hypothesis-geometry - strategies to generate geometric objects.
5754 • hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
5756 • hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
5758 • hypothesis-ros - strategies to generate messages and parameters for
5760 the Robot Operating System.
5761 • hypothesis-csv - strategy to generate CSV files.
5763 • hypothesis-networkx - strategy to generate networkx graphs.
5765 • hypothesis-bio - strategies for bioinformatics data, such as DNA,
5767 codons, FASTA, and FASTQ formats.
5768 • hypothesmith - strategy to generate syntatically-valid Python code.
5770 Others provide a function to infer a strategy from some other schema:
5772 • hypothesis-jsonschema - infer strategies from JSON schemas.
5774 • lollipop-hypothesis - infer strategies from lollipop schemas.
5776 • hypothesis-drf - infer strategies from a djangorestframework seri‐
5778 aliser.
5779 • hypothesis-graphql - infer strategies from GraphQL schemas.
5781 • hypothesis-mongoengine - infer strategies from a mongoengine model.
5783 • hypothesis-pb - infer strategies from Protocol Buffer schemas.
5785 Or some other custom integration, such as a "hypothesis" entry point:
5787 • deal is a design-by-contract library with built-in Hypothesis sup‐
5789 port.
5790 • icontract-hypothesis infers strategies from icontract code contracts.
5792 • Pandera schemas all have a .strategy() method, which returns a strat‐
5794 egy for matching DataFrames.
5795 • Pydantic automatically registers constrained types - so builds() and
5797 from_type() "just work" regardless of the underlying implementation.
5798 Other cool things
5800 schemathesis is a tool for testing web applications built with Open API
5801 / Swagger specifications. It reads the schema and generates test cases
5802 which will ensure that the application is compliant with its schema.
5803 The application under test could be written in any language, the only
5804 thing you need is a valid API schema in a supported format. Includes
5805 CLI and convenient pytest integration. Powered by Hypothesis and
5806 hypothesis-jsonschema, inspired by the earlier swagger-conformance li‐
5807 brary.
5808 Trio is an async framework with "an obsessive focus on usability and
5810 correctness", so naturally it works with Hypothesis! pytest-trio in‐
5811 cludes a custom hook that allows @given(...) to work with Trio-style
5812 async test functions, and hypothesis-trio includes stateful testing ex‐
5813 tensions to support concurrent programs.
5814 pymtl3 is "an open-source Python-based hardware generation, simulation,
5816 and verification framework with multi-level hardware modeling support",
5817 which ships with Hypothesis integrations to check that all of those
5818 levels are equivalent, from function-level to register-transfer level
5819 and even to hardware.
5820 libarchimedes makes it easy to use Hypothesis in the Hy language, a
5822 Lisp embedded in Python.
5823 battle_tested is a fuzzing tool that will show you how your code can
5825 fail - by trying all kinds of inputs and reporting whatever happens.
5826 pytest-subtesthack functions as a workaround for issue #377.
5828 returns uses Hypothesis to verify that Higher Kinded Types correctly
5830 implement functor, applicative, monad, and other laws; allowing a
5831 declarative approach to be combined with traditional pythonic code.
5832 icontract-hypothesis includes a ghostwriter for test files and IDE in‐
5834 tegrations such as icontract-hypothesis-vim,
5835 icontract-hypothesis-pycharm, and icontract-hypothesis-vscode - you can
5836 run a quick 'smoke test' with only a few keystrokes for any type-anno‐
5837 tated function, even if it doesn't have any contracts!
5838 Writing an extension
5840 See CONTRIBUTING.rst for more information.
5841 New strategies can be added to Hypothesis, or published as an external
5843 package on PyPI - either is fine for most strategies. If in doubt, ask!
5844 It's generally much easier to get things working outside, because
5846 there's more freedom to experiment and fewer requirements in stability
5847 and API style. We're happy to review and help with external packages as
5848 well as pull requests!
5849 If you're thinking about writing an extension, please name it hypothe‐
5851 sis-{something} - a standard prefix makes the community more visible
5852 and searching for extensions easier. And make sure you use the Frame‐
5853 work :: Hypothesis trove classifier!
5854 On the other hand, being inside gets you access to some deeper imple‐
5856 mentation features (if you need them) and better long-term guarantees
5857 about maintenance. We particularly encourage pull requests for new
5858 composable primitives that make implementing other strategies easier,
5859 or for widely used types in the standard library. Strategies for other
5860 things are also welcome; anything with external dependencies just goes
5861 in hypothesis.extra.
5862 Tools such as assertion helpers may also need to check whether the cur‐
5864 rent test is using Hypothesis:
5865 hypothesis.currently_in_test_context()
5867 Return True if the calling code is currently running inside an
5868 @given or stateful test, False otherwise.
5869 This is useful for third-party integrations and assertion
5871 helpers which may be called from traditional or property-based
5872 tests, but can only use assume() or target() in the latter case.
5873 Hypothesis integration via setuptools entry points
5875 If you would like to ship Hypothesis strategies for a custom type - ei‐
5876 ther as part of the upstream library, or as a third-party extension,
5877 there's a catch: from_type() only works after the corresponding call to
5878 register_type_strategy(), and you'll have the same problem with
5879 register_random(). This means that either
5880 • you have to try importing Hypothesis to register the strategy when
5882 your library is imported, though that's only useful at test time, or
5883 • the user has to call a 'register the strategies' helper that you pro‐
5885 vide before running their tests
5886 Entry points are Python's standard way of automating the latter: when
5888 you register a "hypothesis" entry point in your setup.py, we'll import
5889 and run it automatically when hypothesis is imported. Nothing happens
5890 unless Hypothesis is already in use, and it's totally seamless for
5891 downstream users!
5892 Let's look at an example. You start by adding a function somewhere in
5894 your package that does all the Hypothesis-related setup work:
5895 # mymodule.py
5897 class MyCustomType:
5900 def __init__(self, x: int):
5901 assert x >= 0, f"got {x}, but only positive numbers are allowed"
5902 self.x = x
5903 def _hypothesis_setup_hook():
5906 import hypothesis.strategies as st
5907 st.register_type_strategy(MyCustomType, st.integers(min_value=0))
5909 and then tell setuptools that this is your "hypothesis" entry point:
5911 # setup.py
5913 # You can list a module to import by dotted name
5915 entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}
5916 # Or name a specific function too, and Hypothesis will call it for you
5918 entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}
5919 And that's all it takes!
5921 HYPOTHESIS_NO_PLUGINS
5923 If set, disables automatic loading of all hypothesis plugins.
5924 This is probably only useful for our own self-tests, but docu‐
5925 mented in case it might help narrow down any particularly weird
5926 bugs in complex environments.
5927 Interaction with pytest-cov
5929 Because pytest does not load plugins from entrypoints in any particular
5930 order, using the Hypothesis entrypoint may import your module before
5931 pytest-cov starts. This is a known issue, but there are workarounds.
5932 You can use coverage run pytest ... instead of pytest --cov ..., opting
5934 out of the pytest plugin entirely. Alternatively, you can ensure that
5935 Hypothesis is loaded after coverage measurement is started by disabling
5936 the entrypoint, and loading our pytest plugin from your conftest.py in‐
5937 stead:
5938 echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
5940 pytest -p "no:hypothesispytest" ...
5941
5943 This is a record of all past Hypothesis releases and what went into
5944 them, in reverse chronological order. All previous releases should
5945 still be available on PyPI.
5946 Hypothesis 6.x
5948 6.82.0 - 2023-07-20
5949 from_regex() now supports the atomic grouping ((?>...)) and possessive
5950 quantifier (*+, ++, ?+, {m,n}+) syntax added in Python 3.11.
5951 Thanks to Cheuk Ting Ho for implementing this!
5953 6.81.2 - 2023-07-15
5955 If the HYPOTHESIS_NO_PLUGINS environment variable is set, we'll avoid
5956 loading plugins such as the old Pydantic integration or HypoFuzz' CLI
5957 options.
5958 This is probably only useful for our own self-tests, but documented in
5960 case it might help narrow down any particularly weird bugs in complex
5961 environments.
5962 6.81.1 - 2023-07-11
5964 Fixes some lingering issues with inference of recursive types in ~hy‐
5965 pothesis.strategies.from_type. Closes issue #3525.
5966 6.81.0 - 2023-07-10
5968 This release further improves our .patch-file support from version
5969 6.75, skipping duplicates, tests which use data() (and don't support
5970 @example()), and various broken edge-cases.
5971 Because libCST has released version 1.0 which uses the native parser by
5973 default, we no longer set the LIBCST_PARSER_TYPE=native environment
5974 variable. If you are using an older version, you may need to upgrade
5975 or set this envvar for yourself.
5976 6.80.1 - 2023-07-06
5978 This patch updates some internal code for selftests. There is no
5979 user-visible change.
5980 6.80.0 - 2023-06-27
5982 This release drops support for Python 3.7, which reached end of life on
5983 2023-06-27.
5984 6.79.4 - 2023-06-27
5986 Fixes occasional recursion-limit-exceeded errors when validating deeply
5987 nested strategies. Closes: issue #3671
5988 6.79.3 - 2023-06-26
5990 This patch updates our vendored list of top-level domains, which is
5991 used by the provisional domains() strategy.
5992 6.79.2 - 2023-06-22
5994 Improve the type rendered in from_type(), which improves the coverage
5995 of Ghostwriter.
5996 6.79.1 - 2023-06-19
5998 We now test against Python 3.12 beta in CI, and this patch fixes some
5999 new deprecations.
6000 6.79.0 - 2023-06-17
6002 This release changes register_type_strategy() for compatibility with
6003 PEP 585: we now store only a single strategy or resolver function which
6004 is used for both the builtin and the typing module version of each type
6005 (issue #3635).
6006 If you previously relied on registering separate strategies for e.g.
6008 list vs typing.List, you may need to use explicit strategies rather
6009 than inferring them from types.
6010 6.78.3 - 2023-06-15
6012 This release ensures that Ghostwriter does not use the deprecated
6013 aliases for the collections.abc classes in collections.
6014 6.78.2 - 2023-06-13
6016 This patch improves Ghostwriter's use of qualified names for re-ex‐
6017 ported functions and classes, and avoids importing useless TypeVars.
6018 6.78.1 - 2023-06-12
6020 This patch updates our vendored list of top-level domains, which is
6021 used by the provisional domains() strategy.
6022 6.78.0 - 2023-06-11
6024 New input validation for recursive() will raise an error rather than
6025 hanging indefinitely if passed invalid max_leaves= arguments.
6026 6.77.0 - 2023-06-09
6028 from_type() now handles numpy array types: np.typing.ArrayLike, np.typ‐
6029 ing.NDArray, and parameterized versions including np.ndarray[shape,
6030 elem_type].
6031 6.76.0 - 2023-06-04
6033 Warn in from_type() if the inferred strategy has no variation (always
6034 returning default instances). Also handles numpy data types by calling
6035 from_dtype() on the corresponding dtype, thus ensuring proper variation
6036 for these types.
6037 6.75.9 - 2023-05-31
6039 from_type() now works in cases where we use builds() to create an in‐
6040 stance and the constructor has an argument which would lead to recur‐
6041 sion. Previously, this would raise an error if the argument had a de‐
6042 fault value.
6043 Thanks to Joachim B Haga for reporting and fixing this problem.
6045 6.75.8 - 2023-05-31
6047 In preparation for supporting JAX in hypothesis.extra.array_api, this
6048 release supports immutable arrays being generated via xps.arrays(). In
6049 particular, we internally removed an instance of in-place array modifi‐
6050 cation, which isn't possible for an immutable array.
6051 6.75.7 - 2023-05-30
6053 This release fixes some .patch-file bugs from version 6.75, and adds
6054 automatic support for writing @hypothesis.example() or @example() de‐
6055 pending on the current style in your test file - defaulting to the lat‐
6056 ter.
6057 Note that this feature requires libcst to be installed, and black is
6059 strongly recommended. You can ensure you have the dependencies with
6060 pip install "hypothesis[cli,codemods]".
6061 6.75.6 - 2023-05-27
6063 This patch continues the work started in pull request #3651 by adding
6064 ruff linter rules for pyflakes, flake8-comprehensions, and flake8-im‐
6065 plicit-str-concat.
6066 6.75.5 - 2023-05-26
6068 This patch updates our linter stack to use ruff, and fixes some previ‐
6069 ously-ignored lints. Thanks to Christian Clauss for his careful review
6070 and pull request #3651!
6071 6.75.4 - 2023-05-26
6073 Hypothesis will now record an event for more cases where data is marked
6074 invalid, including for exceeding the internal depth limit.
6075 6.75.3 - 2023-05-14
6077 This patch fixes complex_numbers() accidentally invalidating itself
6078 when passed magnitude arguments for 32 and 64-bit widths, i.e. 16- and
6079 32-bit floats, due to not internally down-casting numbers (issue
6080 #3573).
6081 6.75.2 - 2023-05-04
6083 Improved the documentation regarding how to use GitHubArtifactDatabase
6084 and fixed a bug that occurred in repositories with no existing arti‐
6085 facts.
6086 Thanks to Agustín Covarrubias for this contribution.
6088 6.75.1 - 2023-04-30
6090 hypothesis.errors will now raise AttributeError when attempting to ac‐
6091 cess an undefined attribute, rather than returning None.
6092 6.75.0 - 2023-04-30
6094 Sick of adding @example()s by hand? Our Pytest plugin now writes
6095 .patch files to insert them for you, making this workflow easier than
6096 ever before.
6097 Note that you'll need LibCST (via hypothesis[codemods]), and that
6099 @example().via() requires PEP 614 (Python 3.9 or later).
6100 6.74.1 - 2023-04-28
6102 This patch provides better error messages for datetime- and
6103 timedelta-related invalid dtypes in our Pandas extra (issue #3518).
6104 Thanks to Nick Muoh at the PyCon Sprints!
6105 6.74.0 - 2023-04-26
6107 This release adds support for nullable pandas dtypes in pandas() (issue
6108 #3604). Thanks to Cheuk Ting Ho for implementing this at the PyCon
6109 sprints!
6110 6.73.1 - 2023-04-27
6112 This patch updates our minimum Numpy version to 1.16, and restores com‐
6113 patibility with versions before 1.20, which were broken by a mistake in
6114 Hypothesis 6.72.4 (issue #3625).
6115 6.73.0 - 2023-04-25
6117 This release upgrades the explain phase (issue #3411).
6118 • Following the first failure, Hypothesis will (usually) track which
6120 lines of code were executed by passing and failing examples, and re‐
6121 port where they diverged - with some heuristics to drop unhelpful re‐
6122 ports. This is an existing feature, now upgraded and newly enabled
6123 by default.
6124 • After shrinking to a minimal failing example, Hypothesis will try to
6126 find parts of the example -- e.g. separate args to @given() -- which
6127 can vary freely without changing the result of that minimal failing
6128 example. If the automated experiments run without finding a passing
6129 variation, we leave a comment in the final report:
6130 test_x_divided_by_y(
6132 x=0, # or any other generated value
6133 y=0,
6134 )
6135 Just remember that the lack of an explanation sometimes just means that
6137 Hypothesis couldn't efficiently find one, not that no explanation (or
6138 simpler failing example) exists.
6139 6.72.4 - 2023-04-25
6141 This patch fixes type annotations for the arrays() strategy. Thanks to
6142 Francesc Elies for pull request #3602.
6143 6.72.3 - 2023-04-25
6145 This patch fixes a bug with from_type() with dict[tuple[int, int], str]
6146 (issue #3527).
6147 Thanks to Nick Muoh at the PyCon Sprints!
6148 6.72.2 - 2023-04-24
6150 This patch refactors our internals to facilitate an upcoming feature.
6151 6.72.1 - 2023-04-19
6153 This patch fixes some documentation and prepares for future features.
6154 6.72.0 - 2023-04-16
6156 This release deprecates Healthcheck.all(), and adds a codemod to auto‐
6157 matically replace it with list(Healthcheck) (issue #3596).
6158 6.71.0 - 2023-04-07
6160 This release adds GitHubArtifactDatabase, a new database backend that
6161 allows developers to access the examples found by a Github Actions CI
6162 job. This is particularly useful for workflows that involve continuous
6163 fuzzing, like HypoFuzz.
6164 Thanks to Agustín Covarrubias for this feature!
6166 6.70.2 - 2023-04-03
6168 This patch clarifies the reporting of time spent generating data. A
6169 simple arithmetic mean of the percentage of time spent can be mislead‐
6170 ing; reporting the actual time spent avoids misunderstandings.
6171 Thanks to Andrea Reina for reporting and fixing issue #3598!
6173 6.70.1 - 2023-03-27
6175 This patch updates our vendored list of top-level domains, which is
6176 used by the provisional domains() strategy.
6177 6.70.0 - 2023-03-16
6179 This release adds an optional domains= parameter to the emails() strat‐
6180 egy, and excludes the special-use .arpa domain from the default strat‐
6181 egy (issue #3567).
6182 Thanks to Jens Tröger for reporting and fixing this bug!
6184 6.69.0 - 2023-03-15
6186 This release turns HealthCheck.return_value and
6187 HealthCheck.not_a_test_method into unconditional errors. Passing them
6188 to suppress_health_check= is therefore a deprecated no-op. (issue
6189 #3568). Thanks to Reagan Lee for the patch!
6190 Separately, GraalPy can now run and pass most of the hypothesis test
6192 suite (issue #3587).
6193 6.68.3 - 2023-03-15
6195 This patch updates our vendored list of top-level domains, which is
6196 used by the provisional domains() strategy.
6197 6.68.2 - 2023-02-17
6199 This patch fixes missing imports of the re module, when ghostwriting
6200 tests which include compiled patterns or regex flags. Thanks to Jens
6201 Heinrich for reporting and promptly fixing this bug!
6202 6.68.1 - 2023-02-12
6204 This patch adds some private hooks for use in research on Schemathesis
6205 (see our preprint here).
6206 6.68.0 - 2023-02-09
6208 This release adds support for the Array API's 2022.12 release via the
6209 api_version argument in make_strategies_namespace(). Concretely this
6210 involves complex support in its existing strategies, plus an introduced
6211 xps.complex_dtypes() strategy.
6212 Additionally this release now treats hypothesis.extra.array_api as sta‐
6214 ble, meaning breaking changes should only happen with major releases of
6215 Hypothesis.
6216 6.67.1 - 2023-02-05
6218 This patch updates our autoformatting tools, improving our code style
6219 without any API changes.
6220 6.67.0 - 2023-02-05
6222 This release allows for more precise generation of complex numbers us‐
6223 ing from_dtype(), by supporting the width, min_magnitude, and min_mag‐
6224 nitude arguments (issue #3468).
6225 Thanks to Felix Divo for this feature!
6227 6.66.2 - 2023-02-04
6229 This patch fixes a rare RecursionError when pretty-printing a
6230 multi-line object without type-specific printer, which was passed to a
6231 function which returned the same object by .map() or builds() and thus
6232 recursed due to the new pretty reprs in Hypothesis 6.65.0 - 2023-01-24
6233 (issue #3560). Apologies to all those affected.
6234 6.66.1 - 2023-02-03
6236 This makes from_dtype() pass through the parameter allow_subnormal for
6237 complex dtypes.
6238 6.66.0 - 2023-02-02
6240 This release adds a width parameter to complex_numbers(), analogously
6241 to floats().
6242 Thanks to Felix Divo for the new feature!
6244 6.65.2 - 2023-01-27
6246 This patch fixes invalid annotations detected for the tests generated
6247 by Ghostwritter. It will now correctly generate Optional types with
6248 just one type argument and handle union expressions inside of type ar‐
6249 guments correctly. Additionally, it now supports code with the from
6250 __future__ import annotations marker for Python 3.10 and newer.
6251 6.65.1 - 2023-01-26
6253 This release improves the pretty-printing of enums in falsifying exam‐
6254 ples, so that they print as their full identifier rather than their
6255 repr.
6256 6.65.0 - 2023-01-24
6258 Hypothesis now reports some failing inputs by showing the call which
6259 constructed an object, rather than the repr of the object. This can be
6260 helpful when the default repr does not include all relevant details,
6261 and will unlock further improvements in a future version.
6262 For now, we capture calls made via builds(), and via
6264 SearchStrategy.map().
6265 6.64.0 - 2023-01-23
6267 The Ghostwritter will now include type annotations on tests for
6268 type-annotated code. If you want to force this to happen (or not hap‐
6269 pen), pass a boolean to the new annotate= argument to the Python func‐
6270 tions, or the --[no-]annotate CLI flag.
6271 Thanks to Nicolas Ganz for this new feature!
6273 6.63.0 - 2023-01-20
6275 range_indexes() now accepts a name= argument, to generate named pan‐
6276 das.RangeIndex objects.
6277 Thanks to Sam Watts for this new feature!
6279 6.62.1 - 2023-01-14
6281 This patch tweaks xps.arrays() internals to improve PyTorch compatibil‐
6282 ity. Specifically, torch.full() does not accept integers as the shape
6283 argument (n.b. technically "size" in torch), but such behaviour is ex‐
6284 pected in internal code, so we copy the torch module and patch in a
6285 working full() function.
6286 6.62.0 - 2023-01-08
6288 A classic error when testing is to write a test function that can never
6289 fail, even on inputs that aren't allowed or manually provided. By
6290 analogy to the design pattern of:
6291 @pytest.mark.parametrize("arg", [
6293 ..., # passing examples
6294 pytest.param(..., marks=[pytest.mark.xfail]) # expected-failing input
6295 ])
6296 we now support @example(...).xfail(), with the same (optional) condi‐
6298 tion, reason, and raises arguments as pytest.mark.xfail().
6299 Naturally you can also write .via(...).xfail(...), or
6301 .xfail(...).via(...), if you wish to note the provenance of ex‐
6302 pected-failing examples.
6303 6.61.3 - 2023-01-08
6305 This patch teaches our enhanced get_type_hints() function to 'see
6306 through' partial application, allowing inference from type hints to
6307 work in a few more cases which aren't (yet!) supported by the stan‐
6308 dard-library version.
6309 6.61.2 - 2023-01-07
6311 This patch improves our pretty-printing of failing examples, including
6312 some refactoring to prepare for exciting future features.
6313 6.61.1 - 2023-01-06
6315 This patch brings our domains() and emails() strategies into compliance
6316 with RFC 5890 §2.3.1: we no longer generate parts-of-domains where the
6317 third and fourth characters are -- ("R-LDH labels"), though future ver‐
6318 sions may deliberately generate xn-- punycode labels. Thanks to
6319 python-email-validator for the report!
6320 6.61.0 - 2022-12-11
6322 This release improves our treatment of database keys, which based on
6323 (among other things) the source code of your test function. We now
6324 post-process this source to ignore decorators, comments, trailing
6325 whitespace, and blank lines - so that you can add @example()s or make
6326 some small no-op edits to your code without preventing replay of any
6327 known failing or covering examples.
6328 6.60.1 - 2022-12-11
6330 This patch updates our vendored list of top-level domains, which is
6331 used by the provisional domains() strategy.
6332 6.60.0 - 2022-12-04
6334 This release improves Hypothesis' ability to resolve forward references
6335 in type annotations. It fixes a bug that prevented builds() from being
6336 used with pydantic models that possess updated forward references. See
6337 issue #3519.
6338 6.59.0 - 2022-12-02
6340 The @example(...) decorator now has a .via() method, which future tools
6341 will use to track automatically-added covering examples (issue #3506).
6342 6.58.2 - 2022-11-30
6344 This patch updates our vendored list of top-level domains, which is
6345 used by the provisional domains() strategy.
6346 6.58.1 - 2022-11-26
6348 This patch shifts hypothesis[lark] from depending on the old
6349 lark-parser package to the new lark package. There are no code changes
6350 in Hypothesis, it's just that Lark got a new name on PyPI for version
6351 1.0 onwards.
6352 6.58.0 - 2022-11-19
6354 register_random() has used weakref since 6.27.1 - 2021-11-22, allowing
6355 the Random-compatible objects to be garbage-collected when there are no
6356 other references remaining in order to avoid memory leaks. We now
6357 raise an error or emit a warning when this seems likely to happen imme‐
6358 diately.
6359 The type annotation of register_random() was also widened so that
6361 structural subtypes of Random are accepted by static typecheckers.
6362 6.57.1 - 2022-11-14
6364 This patch updates some internal type annotations and fixes a format‐
6365 ting bug in the explain phase reporting.
6366 6.57.0 - 2022-11-14
6368 Hypothesis now raises an error if you passed a strategy as the alpha‐
6369 bet= argument to text(), and it generated something which was not a
6370 length-one string. This has never been supported, we're just adding
6371 explicit validation to catch cases like this StackOverflow question.
6372 6.56.4 - 2022-10-28
6374 This patch updates some docs, and depends on exceptiongroup 1.0.0 final
6375 to avoid a bug in the previous version.
6376 6.56.3 - 2022-10-17
6378 This patch teaches text() to rewrite a few more filter predicates (‐
6379 issue #3134). You're unlikely to notice any change.
6380 6.56.2 - 2022-10-10
6382 This patch updates our vendored list of top-level domains, which is
6383 used by the provisional domains() strategy, and fixes some incorrect
6384 examples in the docs for mutually_broadcastable_shapes().
6385 6.56.1 - 2022-10-05
6387 This patch improves the error message when Hypothesis detects "flush to
6388 zero" mode for floating-point: we now report which package(s) enabled
6389 this, which can make debugging much easier. See issue #3458 for de‐
6390 tails.
6391 6.56.0 - 2022-10-02
6393 This release defines __bool__() on SearchStrategy. It always returns
6394 True, like before, but also emits a warning to help with cases where
6395 you intended to draw a value (issue #3463).
6396 6.55.0 - 2022-09-29
6398 In preparation for future versions of the Array API standard,
6399 make_strategies_namespace() now accepts an optional api_version argu‐
6400 ment, which determines the version conformed to by the returned strate‐
6401 gies namespace. If None, the version of the passed array module xp is
6402 inferred.
6403 This release also introduces xps.real_dtypes(). This is currently
6405 equivalent to the existing xps.numeric_dtypes() strategy, but exists
6406 because the latter is expected to include complex numbers in the next
6407 version of the standard.
6408 6.54.6 - 2022-09-18
6410 If multiple explicit examples (from @example()) raise a Skip exception,
6411 for consistency with generated examples we now re-raise the first in‐
6412 stead of collecting them into an ExceptionGroup (issue #3453).
6413 6.54.5 - 2022-09-05
6415 This patch updates our autoformatting tools, improving our code style
6416 without any API changes.
6417 6.54.4 - 2022-08-20
6419 This patch fixes some type annotations for Python 3.9 and earlier (‐
6420 issue #3397), and teaches explain mode about certain locations it
6421 should not bother reporting (issue #3439).
6422 6.54.3 - 2022-08-12
6424 This patch teaches the Ghostwriter an additional check for function and
6425 class locations that should make it use public APIs more often.
6426 6.54.2 - 2022-08-10
6428 This patch fixes our workaround for a pytest bug where the inner excep‐
6429 tions in an ExceptionGroup are not displayed (issue #3430).
6430 6.54.1 - 2022-08-02
6432 This patch makes FailedHealthCheck and DeadlineExceeded exceptions
6433 picklable, for compatibility with Django's parallel test runner (issue
6434 #3426).
6435 6.54.0 - 2022-08-02
6437 Reporting of multiple failing examples now uses the PEP 654
6438 ExceptionGroup type, which is provided by the exceptiongroup backport
6439 on Python 3.10 and earlier (issue #3175). hypothesis.errors.Multiple‐
6440 Failures is therefore deprecated.
6441 Failing examples and other reports are now stored as PEP 678 exception
6443 notes, which ensures that they will always appear together with the
6444 traceback and other information about their respective error.
6445 6.53.0 - 2022-07-25
6447 from_field() now supports UsernameField from django.contrib.auth.forms.
6448 Thanks to Afonso Silva for reporting and working on issue #3417.
6450 6.52.4 - 2022-07-22
6452 This patch improves the error message when you pass filenames to the
6453 hypothesis write CLI, which takes the name of a module or function
6454 (e.g. hypothesis write gzip or hypothesis write package.some_function
6455 rather than hypothesis write script.py).
6456 Thanks to Ed Rogers for implementing this as part of the SciPy 2022
6458 sprints!
6459 6.52.3 - 2022-07-19
6461 This patch ensures that the warning for non-interactive .example()
6462 points to your code instead of Hypothesis internals (issue #3403).
6463 Thanks to @jameslamb for this fix.
6465 6.52.2 - 2022-07-19
6467 This patch makes integers() more likely to generate boundary values for
6468 large two-sided intervals (issue #2942).
6469 6.52.1 - 2022-07-18
6471 This patch adds filter rewriting for math.isfinite(), math.isinf(), and
6472 math.isnan() on integers() or floats() (issue #2701).
6473 Thanks to Sam Clamons at the SciPy Sprints!
6475 6.52.0 - 2022-07-18
6477 This release adds the allow_subnormal argument to complex_numbers() by
6478 applying it to each of the real and imaginary parts separately. Closes
6479 issue #3390.
6480 Thanks to Evan Tey for this fix.
6482 6.51.0 - 2022-07-17
6484 Issue a deprecation warning if a function decorated with @composite
6485 does not draw any values (issue #3384).
6486 Thanks to Grzegorz Zieba, Rodrigo Girão, and Thomas Ball for working on
6488 this at the EuroPython sprints!
6489 6.50.1 - 2022-07-09
6491 This patch improves the error messages in @example() argument valida‐
6492 tion following the recent release of 6.49.1.
6493 6.50.0 - 2022-07-09
6495 This release allows from_dtype() to generate Unicode strings which can‐
6496 not be encoded in UTF-8, but are valid in Numpy arrays (which use
6497 UTF-32).
6498 This logic will only be used with Numpy >= 1.19, because earlier ver‐
6500 sions have an issue which led us to revert Hypothesis 5.2 last time!
6501 6.49.1 - 2022-07-05
6503 This patch fixes some inconsistency between argument handling for
6504 @example and @given (2706).
6505 6.49.0 - 2022-07-04
6507 This release uses PEP 612 python:typing.ParamSpec (or the
6508 typing_extensions backport) to express the first-argument-removing be‐
6509 haviour of @st.composite and signature-preservation of functions() to
6510 IDEs, editor plugins, and static type checkers such as mypy.
6511 6.48.3 - 2022-07-03
6513 hypothesis.event() now works for hashable objects which do not support
6514 weakrefs, such as integers and tuples.
6515 6.48.2 - 2022-06-29
6517 This patch tidies up some internal introspection logic, which will im‐
6518 prove support for positional-only arguments in a future release (issue
6519 #2706).
6520 6.48.1 - 2022-06-27
6522 This release automatically rewrites some simple filters, such as
6523 floats().filter(lambda x: x >= 10) to the more efficient
6524 floats(min_value=10), based on the AST of the predicate.
6525 We continue to recommend using the efficient form directly wherever
6527 possible, but this should be useful for e.g. pandera "Checks" where you
6528 already have a simple predicate and translating manually is really an‐
6529 noying. See issue #2701 for details.
6530 6.48.0 - 2022-06-27
6532 This release raises SkipTest for which never executed any examples, for
6533 example because the phases setting excluded the explicit, reuse, and
6534 generate phases. This helps to avoid cases where broken tests appear
6535 to pass, because they didn't actually execute (issue #3328).
6536 6.47.5 - 2022-06-25
6538 This patch fixes type annotations that had caused the signature of
6539 @given to be partially-unknown to type-checkers for Python versions be‐
6540 fore 3.10.
6541 6.47.4 - 2022-06-23
6543 This patch fixes from_type() on Python 3.11, following
6544 python/cpython#93754.
6545 6.47.3 - 2022-06-15
6547 This patch makes the too_slow health check more consistent with long
6548 deadline tests (issue #3367) and fixes an install issue under pipenv
6549 which was introduced in Hypothesis 6.47.2 (issue #3374).
6550 6.47.2 - 2022-06-12
6552 We now use the PEP 654 ExceptionGroup type - provided by the
6553 exceptiongroup backport on older Pythons - to ensure that if multiple
6554 errors are raised in teardown, they will all propagate.
6555 6.47.1 - 2022-06-10
6557 Our pretty-printer no longer sorts dictionary keys, since iteration or‐
6558 der is stable in Python 3.7+ and this can affect reproducing examples
6559 (issue #3370). This PR was kindly supported by Ordina Pythoneers.
6560 6.47.0 - 2022-06-07
6562 The Ghostwritter can now write tests for @classmethod or @staticmethod
6563 methods, in addition to the existing support for functions and other
6564 callables (issue #3318). Thanks to Cheuk Ting Ho for the patch.
6565 6.46.11 - 2022-06-02
6567 Mention hypothesis.strategies.timezones() in the documentation of
6568 hypothesis.strategies.datetimes() for completeness.
6569 Thanks to George Macon for this addition.
6571 6.46.10 - 2022-06-01
6573 This release contains some small improvements to our documentation.
6574 Thanks to Felix Divo for his contribution!
6575 6.46.9 - 2022-05-25
6577 This patch by Adrian Garcia Badaracco adds type annotations to some
6578 private internals (issue #3074).
6579 6.46.8 - 2022-05-25
6581 This patch by Phillip Schanely makes changes to the floats() strategy
6582 when min_value or max_value is present. Hypothesis will now be capable
6583 of generating every representable value in the bounds. You may notice
6584 that hypothesis is more likely to test values near boundaries, and val‐
6585 ues that are very close to zero.
6586 These changes also support future integrations with symbolic execution
6588 tools and fuzzers (issue #3086).
6589 6.46.7 - 2022-05-19
6591 This patch updates the type annotations for tuples() and one_of() so
6592 that type-checkers require its arguments to be positional-only, and so
6593 that it no longer fails under pyright-strict mode (see issue #3348).
6594 Additional changes are made to Hypothesis' internals improve pyright
6595 scans.
6596 6.46.6 - 2022-05-18
6598 This patch by Cheuk Ting Ho adds support for PEP 655 Required and
6599 NotRequired as attributes of TypedDict in from_type() (issue #3339).
6600 6.46.5 - 2022-05-15
6602 This patch fixes from_dtype() with long-precision floating-point
6603 datatypes (typecode g; see numpy:numpy.typename()).
6604 6.46.4 - 2022-05-15
6606 This patch improves some error messages for custom signatures contain‐
6607 ing invalid parameter names (issue #3317).
6608 6.46.3 - 2022-05-11
6610 This patch by Cheuk Ting Ho makes it an explicit error to call
6611 from_type() or register_type_strategy() with types that have no runtime
6612 instances (issue #3280).
6613 6.46.2 - 2022-05-03
6615 This patch fixes silently dropping examples when the @example decorator
6616 is applied to itself (issue #3319). This was always a weird pattern,
6617 but now it works. Thanks to Ray Sogata, Keeri Tramm, and Kevin Khuong
6618 for working on this patch!
6619 6.46.1 - 2022-05-01
6621 This patch fixes a rare bug where we could incorrectly treat empty as a
6622 type annotation, if the callable had an explicitly assigned __signa‐
6623 ture__.
6624 6.46.0 - 2022-05-01
6626 This release adds an allow_nil argument to uuids(), which you can use
6627 to... generate the nil UUID. Thanks to Shlok Gandhi for the patch!
6628 6.45.4 - 2022-05-01
6630 This patch fixes some missing imports for certain Ghostwritten tests.
6631 Thanks to Mel Seto for fixing issue #3316.
6632 6.45.3 - 2022-04-30
6634 This patch teaches the Ghostwriter to recognize many more common argu‐
6635 ment names (issue #3311).
6636 6.45.2 - 2022-04-29
6638 This patch fixes issue #3314, where Hypothesis would raise an internal
6639 error from domains() or (only on Windows) from timezones() in some rare
6640 circumstances where the installation was subtly broken.
6641 Thanks to Munir Abdinur for this contribution.
6643 6.45.1 - 2022-04-27
6645 This release fixes deprecation warnings about sre_compile and sre_parse
6646 imports and importlib.resources usage when running Hypothesis on Python
6647 3.11.
6648 Thanks to Florian Bruhin for this contribution.
6650 6.45.0 - 2022-04-22
6652 This release updates xps.indices() by introducing an allow_newaxis ar‐
6653 gument, defaulting to False. If allow_newaxis=True, indices can be gen‐
6654 erated that add dimensions to arrays, which is achieved by the indexer
6655 containing None. This change is to support a specification change that
6656 expand dimensions via indexing (data-apis/array-api#408).
6657 6.44.0 - 2022-04-21
6659 This release adds a names argument to indexes() and series(), so that
6660 you can create Pandas objects with specific or varied names.
6661 Contributed by Sam Watts.
6663 6.43.3 - 2022-04-18
6665 This patch updates the type annotations for @given so that type-check‐
6666 ers will warn on mixed positional and keyword arguments, as well as
6667 fixing issue #3296.
6668 6.43.2 - 2022-04-16
6670 Fixed a type annotation for pyright --strict (issue #3287).
6671 6.43.1 - 2022-04-13
6673 This patch makes it an explicit error to call register_type_strategy()
6674 with a Pydantic GenericModel and a callable, because GenericModel isn't
6675 actually a generic type at runtime and so you have to register each of
6676 the "parametrized versions" (actually subclasses!) manually. See issue
6677 #2940 for more details.
6678 6.43.0 - 2022-04-12
6680 This release makes it an explicit error to apply @pytest.fixture to a
6681 function which has already been decorated with @given(). Previously,
6682 pytest would convert your test to a fixture, and then never run it.
6683 6.42.3 - 2022-04-10
6685 This patch fixes from_type() on a TypedDict with complex annotations,
6686 defined in a file using from __future__ import annotations. Thanks to
6687 Katelyn Gigante for identifying and fixing this bug!
6688 6.42.2 - 2022-04-10
6690 The Hypothesis pytest plugin was not outputting valid xunit2 nodes when
6691 --junit-xml was specified. This has been broken since Pytest 5.4, which
6692 changed the internal API for adding nodes to the junit report.
6693 This also fixes the issue when using hypothesis with --junit-xml and
6695 pytest-xdist where the junit xml report would not be xunit2 compatible.
6696 Now, when using with pytest-xdist, the junit report will just omit the
6697 <properties> node.
6698 For more details, see this pytest issue, this pytest issue, and issue
6700 #1935
6701 Thanks to Brandon Chinn for this bug fix!
6703 6.42.1 - 2022-04-10
6705 This patch fixes pretty-printing of regular expressions in Python
6706 3.11.0a7, and updates our vendored list of top-level domains,.
6707 6.42.0 - 2022-04-09
6709 This release makes st.functions(pure=True) less noisy (issue #3253),
6710 and generally improves pretty-printing of functions.
6711 6.41.0 - 2022-04-01
6713 This release changes the implementation of infer to be an alias for
6714 python:Ellipsis. E.g. @given(a=infer) is now equivalent to
6715 @given(a=...). Furthermore, @given(...) can now be specified so that
6716 @given will infer the strategies for all arguments of the decorated
6717 function based on its annotations.
6718 6.40.3 - 2022-04-01
6720 This patch simplifies the repr of the strategies namespace returned in
6721 make_strategies_namespace(), e.g.
6722 >>> from hypothesis.extra.array_api import make_strategies_namespace
6724 >>> from numpy import array_api as xp
6725 >>> xps = make_strategies_namespace(xp)
6726 >>> xps
6727 make_strategies_namespace(numpy.array_api)
6728 6.40.2 - 2022-04-01
6730 Fixed from_type() support for PEP 604 union types, like int | None (‐
6731 issue #3255).
6732 6.40.1 - 2022-04-01
6734 Fixed an internal error when given() was passed a lambda.
6735 6.40.0 - 2022-03-29
6737 The Ghostwriter can now write tests which check that two or more func‐
6738 tions are equivalent on valid inputs, or raise the same type of excep‐
6739 tion for invalid inputs (issue #3267).
6740 6.39.6 - 2022-03-27
6742 This patch makes some quality-of-life improvements to the Ghostwriter:
6743 we guess the text() strategy for arguments named text (...obvious in
6744 hindsight, eh?); and improved the error message if you accidentally
6745 left in a nothing() or broke your rich install.
6746 6.39.5 - 2022-03-26
6748 This patch improves our error detection and message when Hypothesis is
6749 run on a Python implementation without support for -0.0, which is re‐
6750 quired for the floats() strategy but can be disabled by unsafe compiler
6751 options (issue #3265).
6752 6.39.4 - 2022-03-17
6754 This patch tweaks some internal formatting. There is no user-visible
6755 change.
6756 6.39.3 - 2022-03-07
6758 If the shrink phase is disabled, we now stop the generate phase as soon
6759 as an error is found regardless of the value of the report_multiple_ex‐
6760 amples setting, since that's probably what you wanted (issue #3244).
6761 6.39.2 - 2022-03-07
6763 This patch clarifies rare error messages in builds() (issue #3225) and
6764 floats() (issue #3207).
6765 6.39.1 - 2022-03-03
6767 This patch fixes a regression where the bound inner function
6768 (your_test.hypothesis.inner_test) would be invoked with positional ar‐
6769 guments rather than passing them by name, which broke pytest-asyncio (‐
6770 issue #3245).
6771 6.39.0 - 2022-03-01
6773 This release improves Hypothesis' handling of positional-only argu‐
6774 ments, which are now allowed @st.composite strategies.
6775 On Python 3.8 and later, the first arguments to builds() and
6777 from_model() are now natively positional-only. In cases which were al‐
6778 ready errors, the TypeError from incorrect usage will therefore be
6779 raises immediately when the function is called, rather than when the
6780 strategy object is used.
6781 6.38.0 - 2022-02-26
6783 This release makes floats() error consistently when your floating-point
6784 hardware has been configured to violate IEEE-754 for subnormal numbers,
6785 instead of only when an internal assertion was tripped (issue #3092).
6786 If this happens to you, passing allow_subnormal=False will suppress the
6788 explicit error. However, we strongly recommend fixing the root cause
6789 by disabling global-effect unsafe-math compiler options instead, or at
6790 least consulting e.g. Simon Byrne's Beware of fast-math explainer
6791 first.
6792 6.37.2 - 2022-02-21
6794 This patch fixes a bug in stateful testing, where returning a single
6795 value wrapped in multiple() would be printed such that the assigned
6796 variable was a tuple rather than the single element (issue #3236).
6797 6.37.1 - 2022-02-21
6799 This patch fixes a warning under pytest 7 relating to our rich trace‐
6800 back display logic (issue #3223).
6801 6.37.0 - 2022-02-18
6803 When distinguishing multiple errors, Hypothesis now looks at the inner
6804 exceptions of PEP 654 ExceptionGroups.
6805 6.36.2 - 2022-02-13
6807 This patch updates our vendored list of top-level domains, which is
6808 used by the provisional domains() strategy.
6809 6.36.1 - 2022-01-31
6811 This patch fixes some deprecation warnings from pytest 7.0, along with
6812 some code formatting and docs updates.
6813 6.36.0 - 2022-01-19
6815 This release disallows using python:typing.Final with from_type() and
6816 register_type_strategy().
6817 Why? Because Final can only be used during class definition. We don't
6819 generate class attributes.
6820 It also does not make sense as a runtime type on its own.
6822 6.35.1 - 2022-01-17
6824 This patch fixes hypothesis write output highlighting with rich version
6825 12.0 and later.
6826 6.35.0 - 2022-01-08
6828 This release disallows using python:typing.ClassVar with from_type()
6829 and register_type_strategy().
6830 Why? Because ClassVar can only be used during class definition. We
6832 don't generate class attributes.
6833 It also does not make sense as a runtime type on its own.
6835 6.34.2 - 2022-01-05
6837 This patch updates our vendored list of top-level domains, which is
6838 used by the provisional domains() strategy.
6839 6.34.1 - 2021-12-31
6841 This patch fixes issue #3169, an extremely rare bug which would trigger
6842 if an internal least-recently-reused cache dropped a newly added entry
6843 immediately after it was added.
6844 6.34.0 - 2021-12-31
6846 This release fixes issue #3133 and issue #3144, where attempting to
6847 generate Pandas series of lists or sets would fail with confusing er‐
6848 rors if you did not specify dtype=object.
6849 6.33.0 - 2021-12-30
6851 This release disallows using python:typing.TypeAlias with from_type()
6852 and register_type_strategy().
6853 Why? Because TypeAlias is not really a type, it is a tag for type
6855 checkers that some expression is a type alias, not something else.
6856 It does not make sense for Hypothesis to resolve it as a strategy.
6858 References issue #2978.
6859 6.32.1 - 2021-12-23
6861 This patch updates our autoformatting tools, improving our code style
6862 without any API changes.
6863 6.32.0 - 2021-12-23
6865 This release drops support for Python 3.6, which reached end of life
6866 upstream on 2021-12-23.
6867 6.31.6 - 2021-12-15
6869 This patch adds a temporary hook for a downstream tool, which is not
6870 part of the public API.
6871 6.31.5 - 2021-12-14
6873 This release updates our copyright headers to use a general authorship
6874 statement and omit the year.
6875 6.31.4 - 2021-12-11
6877 This patch makes the .example() method more representative of test-time
6878 data generation, albeit often at a substantial cost to readability (‐
6879 issue #3182).
6880 6.31.3 - 2021-12-10
6882 This patch improves annotations on some of Hypothesis' internal func‐
6883 tions, in order to deobfuscate the signatures of some strategies. In
6884 particular, strategies shared between hypothesis.extra.numpy and the
6885 hypothesis.extra.array_api extra will benefit from this patch.
6886 6.31.2 - 2021-12-10
6888 This patch fix invariants display in stateful falsifying examples (‐
6889 issue #3185).
6890 6.31.1 - 2021-12-10
6892 This patch updates xps.indices() so no flat indices are generated, i.e.
6893 generated indices will now always explicitly cover each axes of an ar‐
6894 ray if no ellipsis is present. This is to be consistent with a specifi‐
6895 cation change that dropped support for flat indexing (#272).
6896 6.31.0 - 2021-12-09
6898 This release makes us compatible with Django 4.0, in particular by
6899 adding support for use of zoneinfo timezones (though we respect the new
6900 USE_DEPRECATED_PYTZ setting if you need it).
6901 6.30.1 - 2021-12-05
6903 This patch updates our vendored list of top-level domains, which is
6904 used by the provisional domains() strategy.
6905 6.30.0 - 2021-12-03
6907 This release adds an allow_subnormal argument to the floats() strategy,
6908 which can explicitly toggle the generation of subnormal floats (issue
6909 #3155). Disabling such generation is useful when testing flush-to-zero
6910 builds of libraries.
6911 nps.from_dtype() and xps.from_dtype() can also accept the allow_subnor‐
6913 mal argument, and xps.from_dtype() or xps.arrays() will disable subnor‐
6914 mals by default if the array module xp is detected to flush-to-zero
6915 (like is typical with CuPy).
6916 6.29.3 - 2021-12-02
6918 This patch fixes a bug in mutually_broadcastable_shapes(), which re‐
6919 stricted the patterns of singleton dimensions that could be generated
6920 for dimensions that extended beyond base_shape (issue #3170).
6921 6.29.2 - 2021-12-02
6923 This patch clarifies our pretty-printing of DataFrames (issue #3114).
6924 6.29.1 - 2021-12-02
6926 This patch documents timezones() Windows-only requirement for the
6927 tzdata package, and ensures that pip install hypothesis[zoneinfo] will
6928 install the latest version.
6929 6.29.0 - 2021-11-29
6931 This release teaches builds() to use deferred() when resolving unrecog‐
6932 nised type hints, so that you can conveniently register strategies for
6933 recursive types with constraints on some arguments (issue #3026):
6934 class RecursiveClass:
6936 def __init__(self, value: int, next_node: typing.Optional["SomeClass"]):
6937 assert value > 0
6938 self.value = value
6939 self.next_node = next_node
6940 st.register_type_strategy(
6943 RecursiveClass, st.builds(RecursiveClass, value=st.integers(min_value=1))
6944 )
6945 6.28.1 - 2021-11-28
6947 This release fixes some internal calculations related to collection
6948 sizes (issue #3143).
6949 6.28.0 - 2021-11-28
6951 This release modifies our pytest plugin, to avoid importing Hypothesis
6952 and therefore triggering Hypothesis' entry points for test suites where
6953 Hypothesis is installed but not actually used (issue #3140).
6954 6.27.3 - 2021-11-28
6956 This release fixes issue #3080, where from_type() failed on unions con‐
6957 taining PEP 585 builtin generic types (like list[int]) in Python 3.9
6958 and later.
6959 6.27.2 - 2021-11-26
6961 This patch makes the hypothesis codemod command somewhat faster.
6962 6.27.1 - 2021-11-22
6964 This patch changes the backing datastructures of register_random() and
6965 a few internal caches to use weakref.WeakValueDictionary. This reduces
6966 memory usage and may improve performance when registered Random in‐
6967 stances are only used for a subset of your tests (issue #3131).
6968 6.27.0 - 2021-11-22
6970 This release teaches Hypothesis' multiple-error reporting to format
6971 tracebacks using pytest or better-exceptions, if they are installed and
6972 enabled (issue #3116).
6973 6.26.0 - 2021-11-21
6975 Did you know that of the 264 possible floating-point numbers, 253 of
6976 them are nan - and Python prints them all the same way?
6977 While nans usually have all zeros in the sign bit and mantissa, this
6979 isn't always true, and 'signaling' nans might trap or error. To help
6980 distinguish such errors in e.g. CI logs, Hypothesis now prints -nan for
6981 negative nans, and adds a comment like # Saw 3 signaling NaNs if appli‐
6982 cable.
6983 6.25.0 - 2021-11-19
6985 This release adds special filtering logic to make a few special cases
6986 like s.map(lambda x: x) and lists().filter(len) more efficient (issue
6987 #2701).
6988 6.24.6 - 2021-11-18
6990 This patch makes floats() generate "subnormal" floating point numbers
6991 more often, as these rare values can have strange interactions with
6992 unsafe compiler optimisations like -ffast-math (issue #2976).
6993 6.24.5 - 2021-11-16
6995 This patch fixes a rare internal error in the datetimes() strategy,
6996 where the implementation of allow_imaginary=False crashed when checking
6997 a time during the skipped hour of a DST transition if the DST offset is
6998 negative - only true of Europe/Dublin, who we presume have their rea‐
6999 sons - and the tzinfo object is a pytz timezone (which predates PEP
7000 495).
7001 6.24.4 - 2021-11-15
7003 This patch gives Hypothesis it's own internal Random instance, ensuring
7004 that test suites which reset the global random state don't induce weird
7005 correlations between property-based tests (issue #2135).
7006 6.24.3 - 2021-11-13
7008 This patch updates documentation of note() (issue #3147).
7009 6.24.2 - 2021-11-05
7011 This patch updates internal testing for the Array API extra to be con‐
7012 sistent with new specification changes: sum() not accepting boolean ar‐
7013 rays (#234), unique() split into separate functions (#275), and treat‐
7014 ing NaNs as distinct (#310). It has no user visible impact.
7015 6.24.1 - 2021-11-01
7017 This patch updates our vendored list of top-level domains, which is
7018 used by the provisional domains() strategy.
7019 6.24.0 - 2021-10-23
7021 This patch updates our vendored list of top-level domains, which is
7022 used by the provisional domains() strategy.
7023 (did you know that gTLDs can be both added and removed?)
7025 6.23.4 - 2021-10-20
7027 This patch adds an error for when shapes in xps.arrays() is not passed
7028 as either a valid shape or strategy.
7029 6.23.3 - 2021-10-18
7031 This patch updates our formatting with shed.
7032 6.23.2 - 2021-10-08
7034 This patch replaces external links to NumPy API docs with
7035 sphinx.ext.intersphinx cross-references. It is purely a documentation
7036 improvement.
7037 6.23.1 - 2021-09-29
7039 This patch cleans up internal logic for xps.arrays(). There is no
7040 user-visible change.
7041 6.23.0 - 2021-09-26
7043 This release follows pytest in considering SystemExit and GeneratorExit
7044 exceptions to be test failures, meaning that we will shink to minimal
7045 examples and check for flakiness even though they subclass BaseExcep‐
7046 tion directly (issue #2223).
7047 KeyboardInterrupt continues to interrupt everything, and will be
7049 re-raised immediately.
7050 6.22.0 - 2021-09-24
7052 This release adds LiveServerTestCase and StaticLiveServerTestCase for
7053 django test. Thanks to Ivan Tham for this feature!
7054 6.21.6 - 2021-09-19
7056 This patch fixes some new linter warnings such as flake8-bugbear's B904
7057 for explicit exception chaining, so tracebacks might be a bit nicer.
7058 6.21.5 - 2021-09-16
7060 This release fixes None being inferred as the float64 dtype in
7061 from_dtype() and arrays() from the Array API extra.
7062 6.21.4 - 2021-09-16
7064 This release fixes the type hint for the @given() decorator when deco‐
7065 rating an async function (issue #3099).
7066 6.21.3 - 2021-09-15
7068 This release improves Ghostwritten tests for builtins (issue #2977).
7069 6.21.2 - 2021-09-15
7071 This release deprecates use of both min_dims > len(shape) and max_dims
7072 > len(shape) when allow_newaxis == False in basic_indices() (issue
7073 #3091).
7074 6.21.1 - 2021-09-13
7076 This release improves the behaviour of builds() and from_type() in cer‐
7077 tain situations involving decorators (issue #2495 and issue #3029).
7078 6.21.0 - 2021-09-11
7080 This release introduces strategies for array/tensor libraries adopting
7081 the Array API standard (issue #3037). They are available in the hy‐
7082 pothesis.extra.array_api extra, and work much like the existing
7083 strategies for NumPy.
7084 6.20.1 - 2021-09-10
7086 This patch fixes issue #961, where calling given() inline on a bound
7087 method would fail to handle the self argument correctly.
7088 6.20.0 - 2021-09-09
7090 This release allows slices() to generate step=None, and fixes an
7091 off-by-one error where the start index could be equal to size. This
7092 works fine for all Python sequences and Numpy arrays, but is undefined
7093 behaviour in the Array API standard (see pull request #3065).
7094 6.19.0 - 2021-09-08
7096 This release makes stateful testing more likely to tell you if you do
7097 something unexpected and unsupported:
7098 • The return_value health check now applies to rule() and initialize()
7100 rules, if they don't have target bundles, as well as invariant().
7101 • Using a consumes() bundle as a target is deprecated, and will be an
7103 error in a future version.
7104 If existing code triggers these new checks, check for related bugs and
7106 misunderstandings - these patterns never had any effect.
7107 6.18.0 - 2021-09-06
7109 This release teaches from_type() a neat trick: when resolving an
7110 python:typing.Annotated type, if one of the annotations is a strategy
7111 object we use that as the inferred strategy. For example:
7112 PositiveInt = Annotated[int, st.integers(min_value=1)]
7114 If there are multiple strategies, we use the last outer-most annota‐
7116 tion. See issue #2978 and pull request #3082 for discussion.
7117 Requires Python 3.9 or later for get_type_hints(..., include_ex‐
7119 tras=False).
7120 6.17.4 - 2021-08-31
7122 This patch makes unique arrays() much more efficient, especially when
7123 there are only a few valid elements - such as for eight-bit integers (‐
7124 issue #3066).
7125 6.17.3 - 2021-08-30
7127 This patch fixes the repr of array_shapes().
7128 6.17.2 - 2021-08-30
7130 This patch wraps some internal helper code in our proxies decorator to
7131 prevent mutations of method docstrings carrying over to other instances
7132 of the respective methods.
7133 6.17.1 - 2021-08-29
7135 This patch moves some internal helper code in preparation for issue
7136 #3065. There is no user-visible change, unless you depended on undocu‐
7137 mented internals.
7138 6.17.0 - 2021-08-27
7140 This release adds type annotations to the stateful testing API.
7141 Thanks to Ruben Opdebeeck for this contribution!
7143 6.16.0 - 2021-08-27
7145 This release adds the DrawFn type as a reusable type hint for the draw
7146 argument of @composite functions.
7147 Thanks to Ruben Opdebeeck for this contribution!
7149 6.15.0 - 2021-08-22
7151 This release emits a more useful error message when @given() is applied
7152 to a coroutine function, i.e. one defined using async def (issue
7153 #3054).
7154 This was previously only handled by the generic return_value health
7156 check, which doesn't direct you to use either a custom executor or a
7157 library such as pytest-trio or pytest-asyncio to handle it for you.
7158 6.14.9 - 2021-08-20
7160 This patch fixes a regression in Hypothesis 6.14.8, where from_type()
7161 failed to resolve types which inherit from multiple parametrised
7162 generic types, affecting the returns package (issue #3060).
7163 6.14.8 - 2021-08-16
7165 This patch ensures that registering a strategy for a subclass of a a
7166 parametrised generic type such as class Lines(Sequence[str]): will not
7167 "leak" into unrelated strategies such as st.from_type(Sequence[int]) (‐
7168 issue #2951). Unfortunately this fix requires PEP 560, meaning Python
7169 3.7 or later.
7170 6.14.7 - 2021-08-14
7172 This patch fixes issue #3050, where attrs classes could cause an inter‐
7173 nal error in the ghostwriter.
7174 6.14.6 - 2021-08-07
7176 This patch improves the error message for issue #3016, where PEP 585
7177 builtin generics with self-referential forward-reference strings cannot
7178 be resolved to a strategy by from_type().
7179 6.14.5 - 2021-07-27
7181 This patch fixes hypothesis.strategies._internal.types.is_a_new_type.
7182 It was failing on Python 3.10.0b4, where NewType is a function.
7183 6.14.4 - 2021-07-26
7185 This patch fixes from_type() and register_type_strategy() for
7186 python:typing.NewType on Python 3.10, which changed the underlying im‐
7187 plementation (see bpo-44353 for details).
7188 6.14.3 - 2021-07-18
7190 This patch updates our autoformatting tools, improving our code style
7191 without any API changes.
7192 6.14.2 - 2021-07-12
7194 This patch ensures that we shorten tracebacks for tests which fail due
7195 to inconsistent data generation between runs (i.e. raise Flaky).
7196 6.14.1 - 2021-07-02
7198 This patch updates some internal type annotations. There is no
7199 user-visible change.
7200 6.14.0 - 2021-06-09
7202 The explain phase now requires shrinking to be enabled, and will be au‐
7203 tomatically skipped for deadline-exceeded errors.
7204 6.13.14 - 2021-06-04
7206 This patch improves the tuples() strategy type annotations, to preserve
7207 the element types for up to length-five tuples (issue #3005).
7208 As for one_of(), this is the best we can do before a planned extension
7210 to PEP 646 is released, hopefully in Python 3.11.
7211 6.13.13 - 2021-06-04
7213 This patch teaches the Ghostwriter how to find custom ufuncs from any
7214 module that defines them, and that yaml.unsafe_load() does not undo
7215 yaml.safe_load().
7216 6.13.12 - 2021-06-03
7218 This patch reduces the amount of internal code excluded from our test
7219 suite's code coverage checks.
7220 There is no user-visible change.
7222 6.13.11 - 2021-06-02
7224 This patch removes some old internal helper code that previously ex‐
7225 isted to make Python 2 compatibility easier.
7226 There is no user-visible change.
7228 6.13.10 - 2021-05-30
7230 This release adjusts some internal code to help make our test suite
7231 more reliable.
7232 There is no user-visible change.
7234 6.13.9 - 2021-05-30
7236 This patch cleans up some internal code related to filtering strate‐
7237 gies.
7238 There is no user-visible change.
7240 6.13.8 - 2021-05-28
7242 This patch slightly improves the performance of some internal code for
7243 generating integers.
7244 6.13.7 - 2021-05-27
7246 This patch fixes a bug in from_regex() that caused from_regex("", full‐
7247 match=True) to unintentionally generate non-empty strings (issue
7248 #4982).
7249 The only strings that completely match an empty regex pattern are empty
7251 strings.
7252 6.13.6 - 2021-05-26
7254 This patch fixes a bug that caused integers() to shrink towards nega‐
7255 tive values instead of positive values in some cases.
7256 6.13.5 - 2021-05-24
7258 This patch fixes rare cases where hypothesis write --binary-op could
7259 print reproducing instructions from the internal search for an identity
7260 element.
7261 6.13.4 - 2021-05-24
7263 This patch removes some unnecessary intermediate list-comprehensions,
7264 using the latest versions of pyupgrade and shed.
7265 6.13.3 - 2021-05-23
7267 This patch adds a .hypothesis property to invalid test functions,
7268 bringing them inline with valid tests and fixing a bug where
7269 pytest-asyncio would swallow the real error message and mistakenly
7270 raise a version incompatibility error.
7271 6.13.2 - 2021-05-23
7273 Some of Hypothesis's numpy/pandas strategies use a fill argument to
7274 speed up generating large arrays, by generating a single fill value and
7275 sharing that value among many array slots instead of filling every sin‐
7276 gle slot individually.
7277 When no fill argument is provided, Hypothesis tries to detect whether
7279 it is OK to automatically use the elements argument as a fill strategy,
7280 so that it can still use the faster approach.
7281 This patch fixes a bug that would cause that optimization to trigger in
7283 some cases where it isn't 100% guaranteed to be OK.
7284 If this makes some of your numpy/pandas tests run more slowly, try
7286 adding an explicit fill argument to the relevant strategies to ensure
7287 that Hypothesis always uses the faster approach.
7288 6.13.1 - 2021-05-20
7290 This patch strengthens some internal import-time consistency checks for
7291 the built-in strategies.
7292 There is no user-visible change.
7294 6.13.0 - 2021-05-18
7296 This release adds URL fragment generation to the urls() strategy (issue
7297 #2908). Thanks to Pax (R. Margret) for contributing this patch at the
7298 PyCon US Mentored Sprints!
7299 6.12.1 - 2021-05-17
7301 This patch fixes issue #2964, where .map() and .filter() methods were
7302 omitted from the repr() of just() and sampled_from() strategies, since
7303 version 5.43.7.
7304 6.12.0 - 2021-05-06
7306 This release automatically rewrites some simple filters, such as inte‐
7307 gers().filter(lambda x: x > 9) to the more efficient inte‐
7308 gers(min_value=10), based on the AST of the predicate.
7309 We continue to recommend using the efficient form directly wherever
7311 possible, but this should be useful for e.g. pandera "Checks" where you
7312 already have a simple predicate and translating manually is really an‐
7313 noying. See issue #2701 for ideas about floats and simple text strate‐
7314 gies.
7315 6.11.0 - 2021-05-06
7317 hypothesis.target() now returns the observation value, allowing it to
7318 be conveniently used inline in expressions such as assert target(abs(a
7319 - b)) < 0.1.
7320 6.10.1 - 2021-04-26
7322 This patch fixes a deprecation warning if you're using recent versions
7323 of importlib-metadata (issue #2934), which we use to load third-party
7324 plugins such as Pydantic's integration. On older versions of
7325 importlib-metadata, there is no change and you don't need to upgrade.
7326 6.10.0 - 2021-04-17
7328 This release teaches the Ghostwriter to read parameter types from
7329 Sphinx, Google, or Numpy-style structured docstrings, and improves some
7330 related heuristics about how to test scientific and numerical programs.
7331 6.9.2 - 2021-04-15
7333 This release improves the Ghostwriter's handling of exceptions, by
7334 reading :raises ...: entries in function docstrings and ensuring that
7335 we don't suppresss the error raised by test assertions.
7336 6.9.1 - 2021-04-12
7338 This patch updates our autoformatting tools, improving our code style
7339 without any API changes.
7340 6.9.0 - 2021-04-11
7342 This release teaches from_type() how to see through python:typing.Anno‐
7343 tated. Thanks to Vytautas Strimaitis for reporting and fixing issue
7344 #2919!
7345 6.8.12 - 2021-04-11
7347 If rich is installed, the hypothesis write command will use it to syn‐
7348 tax-highlight the Ghostwritten code.
7349 6.8.11 - 2021-04-11
7351 This patch improves an error message from builds() when from_type()
7352 would be more suitable (issue #2930).
7353 6.8.10 - 2021-04-11
7355 This patch updates the type annotations for arrays() to reflect that
7356 shape: SearchStrategy[int] is supported.
7357 6.8.9 - 2021-04-07
7359 This patch fixes from_type() with abstract types which have either re‐
7360 quired but non-type-annotated arguments to __init__, or where
7361 from_type() can handle some concrete subclasses but not others.
7362 6.8.8 - 2021-04-07
7364 This patch teaches hypothesis write to check for possible roundtrips in
7365 several more cases, such as by looking for an inverse in the module
7366 which defines the function to test.
7367 6.8.7 - 2021-04-07
7369 This patch adds a more helpful error message if you try to call
7370 sampled_from() on an Enum which has no members, but does have data‐
7371 class()-style annotations (issue #2923).
7372 6.8.6 - 2021-04-06
7374 The fixed_dictionaries() strategy now preserves dict iteration order
7375 instead of sorting the keys. This also affects the pretty-printing of
7376 keyword arguments to @given() (issue #2913).
7377 6.8.5 - 2021-04-05
7379 This patch teaches hypothesis write to default to ghostwriting tests
7380 with --style=pytest only if pytest is installed, or --style=unittest
7381 otherwise.
7382 6.8.4 - 2021-04-01
7384 This patch adds type annotations for the settings decorator, to avoid
7385 an error when running mypy in strict mode.
7386 6.8.3 - 2021-03-28
7388 This patch improves the Ghostwriter's handling of strategies to gener‐
7389 ate various fiddly types including frozensets, keysviews, valuesviews,
7390 regex matches and patterns, and so on.
7391 6.8.2 - 2021-03-27
7393 This patch fixes some internal typos. There is no user-visible change.
7394 6.8.1 - 2021-03-14
7396 This patch lays more groundwork for filter rewriting (issue #2701).
7397 There is no user-visible change... yet.
7398 6.8.0 - 2021-03-11
7400 This release registers the remaining builtin types, and teaches
7401 from_type() to try resolving ForwardRef and Type references to built-in
7402 types.
7403 6.7.0 - 2021-03-10
7405 This release teaches RuleBasedStateMachine to avoid checking
7406 invariant()s until all initialize() rules have been run. You can en‐
7407 able checking of specific invariants for incompletely initialized ma‐
7408 chines by using @invariant(check_during_init=True) (issue #2868).
7409 In previous versions, it was possible if awkward to implement this be‐
7411 haviour using precondition() and an auxiliary variable.
7412 6.6.1 - 2021-03-09
7414 This patch improves the error message when from_type() fails to resolve
7415 a forward-reference inside a python:typing.Type such as Type["int"] (‐
7416 issue #2565).
7417 6.6.0 - 2021-03-07
7419 This release makes it an explicit error to apply invariant() to a
7420 rule() or initialize() rule in stateful testing. Such a combination
7421 had unclear semantics, especially in combination with precondition(),
7422 and was never meant to be allowed (issue #2681).
7423 6.5.0 - 2021-03-07
7425 This release adds the explain phase, in which Hypothesis attempts to
7426 explain why your test failed by pointing to suspicious lines of code
7427 (i.e. those which were always, and only, run on failing inputs). We
7428 plan to include "generalising" failing examples in this phase in a fu‐
7429 ture release (issue #2192).
7430 6.4.3 - 2021-03-04
7432 This patch fixes issue #2794, where nesting deferred() strategies
7433 within recursive() strategies could trigger an internal assertion.
7434 While it was always possible to get the same results from a more sensi‐
7435 ble strategy, the convoluted form now works too.
7436 6.4.2 - 2021-03-04
7438 This patch fixes several problems with mypy when --no-implicit-reexport
7439 was activated in user projects.
7440 Thanks to Nikita Sobolev for fixing issue #2884!
7442 6.4.1 - 2021-03-04
7444 This patch fixes an exception that occurs when using type unions of the
7445 typing_extensions Literal backport on Python 3.6.
7446 Thanks to Ben Anhalt for identifying and fixing this bug.
7448 6.4.0 - 2021-03-02
7450 This release fixes stateful testing methods with multiple
7451 precondition() decorators. Previously, only the outer-most precondi‐
7452 tion was checked (issue #2681).
7453 6.3.4 - 2021-02-28
7455 This patch refactors some internals of RuleBasedStateMachine. There is
7456 no change to the public API or behaviour.
7457 6.3.3 - 2021-02-26
7459 This patch moves some internal code, so that future work can avoid cre‐
7460 ating import cycles. There is no user-visible change.
7461 6.3.2 - 2021-02-25
7463 This patch enables register_type_strategy() for subclasses of
7464 python:typing.TypedDict. Previously, from_type() would ignore the reg‐
7465 istered strategy (issue #2872).
7466 Thanks to Ilya Lebedev for identifying and fixing this bug!
7468 6.3.1 - 2021-02-24
7470 This release lays the groundwork for automatic rewriting of simple fil‐
7471 ters, for example converting integers().filter(lambda x: x > 9) to in‐
7472 tegers(min_value=10).
7473 Note that this is not supported yet, and we will continue to recommend
7475 writing the efficient form directly wherever possible - predicate
7476 rewriting is provided mainly for the benefit of downstream libraries
7477 which would otherwise have to implement it for themselves (e.g. pandera
7478 and icontract-hypothesis). See issue #2701 for details.
7479 6.3.0 - 2021-02-20
7481 The Hypothesis pytest plugin now requires pytest version 4.6 or later.
7482 If the plugin detects an earlier version of pytest, it will automati‐
7483 cally deactivate itself.
7484 (4.6.x is the earliest pytest branch that still accepts community bug‐
7486 fixes.)
7487 Hypothesis-based tests should continue to work in earlier versions of
7489 pytest, but enhanced integrations provided by the plugin (such as --hy‐
7490 pothesis-show-statistics and other command-line flags) will no longer
7491 be available in obsolete pytest versions.
7492 6.2.0 - 2021-02-12
7494 If you use pytest-html, Hypothesis now includes the summary statistics
7495 for each test in the HTML report, whether or not the --hypothe‐
7496 sis-show-statistics argument was passed to show them in the com‐
7497 mand-line output.
7498 6.1.1 - 2021-01-31
7500 This patch updates our automatic code formatting to use shed, which in‐
7501 cludes autoflake, black, isort, and pyupgrade (issue #2780).
7502 6.1.0 - 2021-01-29
7504 This release teaches Hypothesis to distinguish between errors based on
7505 the __cause__ or __context__ of otherwise identical exceptions, which
7506 is particularly useful when internal errors can be wrapped by a li‐
7507 brary-specific or semantically appropriate exception such as:
7508 try:
7510 do_the_thing(foo, timeout=10)
7511 except Exception as err:
7512 raise FooError("Failed to do the thing") from err
7513 Earlier versions of Hypothesis only see the FooError, while we can now
7515 distinguish a FooError raised because of e.g. an internal assertion
7516 from one raised because of a TimeoutExceeded exception.
7517 6.0.4 - 2021-01-27
7519 This release prevents a race condition inside recursive() strategies.
7520 The race condition occurs when the same recursive() strategy is shared
7521 among tests that are running in multiple threads (issue #2717).
7522 6.0.3 - 2021-01-23
7524 This patch improves the type annotations for one_of(), by adding over‐
7525 loads to handle up to five distinct arguments as Union before falling
7526 back to Any, as well as annotating the | (__or__) operator for strate‐
7527 gies (issue #2765).
7528 6.0.2 - 2021-01-14
7530 This release makes some small improvements to how filtered strategies
7531 work. It should improve the performance of shrinking filtered strate‐
7532 gies, and may under some (probably rare) circumstances improve the di‐
7533 versity of generated examples.
7534 6.0.1 - 2021-01-13
7536 This patch fixes an interaction where our test statistics handling made
7537 Pytest's --junit-xml output fail to validate against the strict xunit2
7538 schema (issue #1975).
7539 6.0.0 - 2021-01-08
7541 Welcome to the next major version of Hypothesis!
7542 There are no new features here, as we release those in minor versions.
7544 Instead, 6.0 is a chance for us to remove deprecated features (many al‐
7545 ready converted into no-ops), and turn a variety of warnings into er‐
7546 rors.
7547 If you were running on the last version of Hypothesis 5.x without any
7549 Hypothesis deprecation warnings, this will be a very boring upgrade.
7550 In fact, nothing will change for you at all.
7551 Changes
7553 • Many functions now use PEP 3102 keyword-only arguments where passing
7554 positional arguments was deprecated since 5.5.
7555 • hypothesis.extra.django.from_model() no longer accepts model as a
7557 keyword argument, where it could conflict with fields named "model".
7558 • randoms() now defaults to use_true_random=False.
7560 • complex_numbers() no longer accepts min_magnitude=None; either use
7562 min_magnitude=0 or just omit the argument.
7563 • hypothesis.provisional.ip4_addr_strings and ip6_addr_strings are re‐
7565 moved in favor of ip_addresses(v=...).map(str).
7566 • register_type_strategy() no longer accepts generic types with type
7568 arguments, which were always pretty badly broken.
7569 • Using function-scoped pytest fixtures is now a health-check error,
7571 instead of a warning.
7572 TIP:
7574 The hypothesis codemod command can automatically refactor your code,
7575 particularly to convert positional to keyword arguments where those
7576 are now required.
7577 Hypothesis 5.x
7579 5.49.0 - 2021-01-07
7580 This release adds the function_scoped_fixture health check value, which
7581 can be used to suppress the existing warning that appears when @given
7582 is applied to a test that uses pytest function-scoped fixtures.
7583 (This warning exists because function-scoped fixtures only run once per
7585 function, not once per example, which is usually unexpected and can
7586 cause subtle problems.)
7587 When this warning becomes a health check error in a future release,
7589 suppressing it via Python warning settings will no longer be possible.
7590 In the rare case that once-per-function behaviour is intended, it will
7591 still be possible to use function_scoped_fixture to opt out of the
7592 health check error for specific tests.
7593 5.48.0 - 2021-01-06
7595 This release adds hypothesis.currently_in_test_context(), which can be
7596 used to check whether the calling code is currently running inside an
7597 @given or stateful test.
7598 This is most useful for third-party integrations and assertion helpers
7600 which may wish to use assume() or target(), without also requiring that
7601 the helper only be used from property-based tests (issue #2581).
7602 5.47.0 - 2021-01-05
7604 This release upgrades the import logic for ghostwritten tests, handling
7605 many cases where imports would previously be missing or from unexpected
7606 locations.
7607 5.46.0 - 2021-01-04
7609 This release upgrades from_type(), to infer strategies for type-anno‐
7610 tated arguments even if they have defaults when it otherwise falls back
7611 to builds() (issue #2708).
7612 5.45.0 - 2021-01-04
7614 This release adds the hypothesis[codemods] extra, which you can use to
7615 check for and automatically fix issues such as use of deprecated Hy‐
7616 pothesis APIs (issue #2705).
7617 5.44.0 - 2021-01-03
7619 This patch fixes from_type() with the typing_extensions Literal back‐
7620 port on Python 3.6.
7621 5.43.9 - 2021-01-02
7623 This patch fixes issue #2722, where certain orderings of
7624 register_type_strategy(), ForwardRef, and from_type() could trigger an
7625 internal error.
7626 5.43.8 - 2021-01-02
7628 This patch makes some strategies for collections with a uniqueness con‐
7629 straint much more efficient, including dictionaries(keys=sam‐
7630 pled_from(...), values=..) and lists(tuples(sampled_from(...), ...),
7631 unique_by=lambda x: x[0]). (related to issue #2036)
7632 5.43.7 - 2021-01-02
7634 This patch extends our faster special case for sampled_from() elements
7635 in unique lists() to account for chains of .map(...) and .filter(...)
7636 calls (issue #2036).
7637 5.43.6 - 2021-01-02
7639 This patch improves the type annotations on assume() and
7640 @reproduce_failure().
7641 5.43.5 - 2021-01-01
7643 This patch updates our copyright headers to include 2021. Happy new
7644 year!
7645 5.43.4 - 2020-12-24
7647 This change fixes a documentation error in the database setting.
7648 The previous documentation suggested that callers could specify a data‐
7650 base path string, or the special string ":memory:", but this setting
7651 has never actually allowed string arguments.
7652 Permitted values are None, and instances of ExampleDatabase.
7654 5.43.3 - 2020-12-11
7656 This patch fixes issue #2696, an internal error triggered when the
7657 @example decorator was used and the verbosity setting was quiet.
7658 5.43.2 - 2020-12-10
7660 This patch improves the error message from the data_frames() strategy
7661 when both the rows and columns arguments are given, but there is a
7662 missing entry in rows and the corresponding column has no fill value (‐
7663 issue #2678).
7664 5.43.1 - 2020-12-10
7666 This patch improves the error message if builds() is passed an Enum
7667 which cannot be called without arguments, to suggest using
7668 sampled_from() (issue #2693).
7669 5.43.0 - 2020-12-09
7671 This release adds new timezones() and timezone_keys() strategies (issue
7672 #2630) based on the new python:zoneinfo module in Python 3.9.
7673 pip install hypothesis[zoneinfo] will ensure that you have the appro‐
7675 priate backports installed if you need them.
7676 5.42.3 - 2020-12-09
7678 This patch fixes an internal error in datetimes() with allow_imagi‐
7679 nary=False where the timezones argument can generate tzinfo=None (issue
7680 #2662).
7681 5.42.2 - 2020-12-09
7683 This patch teaches hypothesis.extra.django.from_field() to infer more
7684 efficient strategies by inspecting (not just filtering by) field val‐
7685 idators for numeric and string fields (issue #1116).
7686 5.42.1 - 2020-12-09
7688 This patch refactors hypothesis.settings to use type-annotated keyword
7689 arguments instead of **kwargs, which makes tab-completion much more
7690 useful - as well as type-checkers like mypy.
7691 5.42.0 - 2020-12-09
7693 This patch teaches the magic() ghostwriter to recognise "en/de" func‐
7694 tion roundtrips other than the common encode/decode pattern, such as
7695 encrypt/decrypt or, encipher/decipher.
7696 5.41.5 - 2020-12-05
7698 This patch adds a performance optimisation to avoid saving redundant
7699 seeds when using the .fuzz_one_input hook.
7700 5.41.4 - 2020-11-28
7702 This patch fixes issue #2657, where passing unicode patterns compiled
7703 with python:re.IGNORECASE to from_regex() could trigger an internal er‐
7704 ror when casefolding a character creates a longer string (e.g.
7705 "\u0130".lower() -> "i\u0370").
7706 5.41.3 - 2020-11-18
7708 This patch adds a final fallback clause to our plugin logic to fail
7709 with a warning rather than error on Python < 3.8 when neither the
7710 importlib_metadata (preferred) or setuptools (fallback) packages are
7711 available.
7712 5.41.2 - 2020-11-08
7714 This patch fixes urls() strategy ensuring that ~ (tilde) is treated as
7715 one of the url-safe characters (issue #2658).
7716 5.41.1 - 2020-11-03
7718 This patch improves our CLI help and documentation.
7719 5.41.0 - 2020-10-30
7721 Hypothesis now shrinks examples where the error is raised while drawing
7722 from a strategy. This makes complicated custom strategies much easier
7723 to debug, at the cost of a slowdown for use-cases where you catch and
7724 ignore such errors.
7725 5.40.0 - 2020-10-30
7727 This release teaches from_type() how to handle ChainMap, Counter,
7728 Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).
7729 5.39.0 - 2020-10-30
7731 from_type() now knows how to resolve PEP 585 parameterized standard
7732 collection types, which are new in Python 3.9 (issue #2629).
7733 5.38.1 - 2020-10-26
7735 This patch fixes builds(), so that when passed infer for an argument
7736 with a non-Optional type annotation and a default value of None to
7737 build a class which defines an explicit __signature__ attribute, either
7738 None or that type may be generated.
7739 This is unlikely to happen unless you are using pydantic (issue #2648).
7741 5.38.0 - 2020-10-24
7743 This release improves our support for @st.composite on a python:class‐
7744 method or python:staticmethod (issue #2578).
7745 5.37.5 - 2020-10-24
7747 This patch fixes from_type() with Iterable[T] (issue #2645).
7748 5.37.4 - 2020-10-20
7750 This patch teaches the magic() ghostwriter to recognise that pairs of
7751 functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().
7752 5.37.3 - 2020-10-15
7754 This patch improves builds() and from_type() support for explicitly de‐
7755 fined __signature__ attributes, from version 5.8.3, to support generic
7756 types from the python:typing module.
7757 Thanks to Rónán Carrigan for identifying and fixing this problem!
7759 5.37.2 - 2020-10-14
7761 This patch fixes from_lark() with version 0.10.1+ of the lark-parser
7762 package.
7763 5.37.1 - 2020-10-07
7765 This patch fixes some broken links in the lark extra documentation.
7766 5.37.0 - 2020-10-03
7768 This release adds a new RedisExampleDatabase, along with the
7769 ReadOnlyDatabase and MultiplexedDatabase helpers, to support team work‐
7770 flows where failing examples can be seamlessly shared between everyone
7771 on the team - and your CI servers or buildbots.
7772 5.36.2 - 2020-10-02
7774 This patch ensures that if the "hypothesis" entry point is callable, we
7775 call it after importing it. You can still use non-callable entry
7776 points (like modules), which are only imported.
7777 We also prefer importlib.metadata or the backport over pkg_resources,
7779 which makes import hypothesis around 200 milliseconds faster (issue
7780 #2571).
7781 5.36.1 - 2020-09-25
7783 This patch adds some helpful suggestions to error messages you might
7784 see while learning to use the @example() decorator (issue #2611) or the
7785 one_of() strategy.
7786 5.36.0 - 2020-09-24
7788 This release upgrades the from_dtype() strategy to pass optional
7789 **kwargs to the inferred strategy, and upgrades the arrays() strategy
7790 to accept an elements=kwargs dict to pass through to from_dtype().
7791 arrays(floating_dtypes(), shape, elements={"min_value": -10,
7793 "max_value": 10}) is a particularly useful pattern, as it allows for
7794 any floating dtype without triggering the roundoff warning for smaller
7795 types or sacrificing variety for larger types (issue #2552).
7796 5.35.4 - 2020-09-21
7798 This patch reformats our code with the latest black to take advantage
7799 of the support for magic trailing commas.
7800 5.35.3 - 2020-09-15
7802 This release significantly improves the performance of Hypothesis's in‐
7803 ternal implementation of automaton learning. However this code does not
7804 run as part of the user-accessible API so this has no user-visible im‐
7805 pact.
7806 5.35.2 - 2020-09-14
7808 This patch ensures that, when the generate phases is disabled, we can
7809 replay up to max_examples examples from the database - which is very
7810 useful when using Hypothesis with a fuzzer.
7811 Thanks to Afrida Tabassum for fixing issue #2585!
7813 5.35.1 - 2020-09-14
7815 This patch changes some internal python:struct.Struct.format strings
7816 from bytes to str, to avoid python:BytesWarning when running python
7817 -bb.
7818 Thanks to everyone involved in pytest-xdist issue 596, bpo-16349,
7820 bpo-21071, and bpo-41777 for their work on this - it was a remarkably
7821 subtle issue!
7822 5.35.0 - 2020-09-11
7824 The target() function now accepts integers as well as floats.
7825 5.34.1 - 2020-09-11
7827 This patch adds explicit Optional annotations to our public API, to
7828 better support users who run mypy with --strict or no_implicit_op‐
7829 tional=True.
7830 Thanks to Krzysztof Przybyła for bringing this to our attention and
7832 writing the patch!
7833 5.34.0 - 2020-09-11
7835 This release drops support for Python 3.5, which reached end of life
7836 upstream on 2020-09-13.
7837 5.33.2 - 2020-09-09
7839 This patch fixes a problem with builds() that was not able to generate
7840 valid data for annotated classes with constructors.
7841 Thanks to Nikita Sobolev for fixing issue #2603!
7843 5.33.1 - 2020-09-07
7845 This patch improves the error message from the hypothesis write command
7846 if black (required for the ghostwriter) is not installed.
7847 Thanks to Nikita Sobolev for fixing issue #2604!
7849 5.33.0 - 2020-09-06
7851 When reporting failing examples, or tried examples in verbose mode, Hy‐
7852 pothesis now identifies which were from @example(...) explicit exam‐
7853 ples.
7854 5.32.1 - 2020-09-06
7856 This patch contains some internal refactoring. Thanks to Felix Sheldon
7857 for fixing issue #2516!
7858 5.32.0 - 2020-09-04
7860 An array drawn from arrays() will own its own memory; previously most
7861 arrays returned by this strategy were views.
7862 5.31.0 - 2020-09-04
7864 builds() will use the __signature__ attribute of the target, if it ex‐
7865 ists, to retrieve type hints. Previously python:typ‐
7866 ing.get_type_hints(), was used by default. If argument names varied
7867 between the __annotations__ and __signature__, they would not be sup‐
7868 plied to the target.
7869 This was particularly an issue for pydantic models which use an alias
7871 generator.
7872 5.30.1 - 2020-09-04
7874 This patch makes the ghostwriter much more robust when passed unusual
7875 modules.
7876 • improved support for non-resolvable type annotations
7878 • magic() can now write equivalent() tests
7880 • running magic() on modules where some names in __all__ are undefined
7882 skips such names, instead of raising an error
7883 • magic() now knows to skip mocks
7885 • improved handling of import-time errors found by the ghostwriter CLI
7887 5.30.0 - 2020-08-30
7889 register_type_strategy() now supports python:typing.TypeVar, which was
7890 previously hard-coded, and allows a variety of types to be generated
7891 for an unconstrained TypeVar instead of just text().
7892 Thanks again to Nikita Sobolev for all your work on advanced types!
7894 5.29.4 - 2020-08-28
7896 This release fixes some hard to trigger bugs in Hypothesis's automata
7897 learning code. This code is only run as part of the Hypothesis build
7898 process, and not for user code, so this release has no user visible im‐
7899 pact.
7900 5.29.3 - 2020-08-27
7902 This patch adds type annotations to the hypothesis.database module.
7903 There is no runtime change, but your typechecker might notice.
7904 5.29.2 - 2020-08-27
7906 This patch tracks some additional information in Hypothesis internals,
7907 and has no user-visible impact.
7908 5.29.1 - 2020-08-27
7910 This release fixes a bug in some Hypothesis internal support code for
7911 learning automata. This mostly doesn't have any user visible impact,
7912 although it slightly affects the learned shrink passes so shrinking may
7913 be subtly different.
7914 5.29.0 - 2020-08-24
7916 This release adds support for Hypothesis integration via setuptools en‐
7917 try points, which allows for smoother integration of third-party Hy‐
7918 pothesis extensions and external libraries. Unless you're publishing a
7919 library with Hypothesis integration, you'll probably only ever use this
7920 indirectly!
7921 5.28.0 - 2020-08-24
7923 from_type() can now resolve TypeVar instances when the bound is a For‐
7924 wardRef, so long as that name is in fact defined in the same module as
7925 the typevar (no TYPE_CHECKING tricks, sorry). This feature requires
7926 Python 3.7 or later.
7927 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
7929 5.27.0 - 2020-08-20
7931 This patch adds two new ghostwriters to test binary operations, like
7932 python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().
7933 5.26.1 - 2020-08-19
7935 This release improves the performance of some methods in Hypothesis's
7936 internal automaton library. These are currently only lightly used by
7937 user code, but this may result in slightly faster shrinking.
7938 5.26.0 - 2020-08-17
7940 register_type_strategy() no longer accepts parametrised user-defined
7941 generic types, because the resolution logic was quite badly broken (‐
7942 issue #2537).
7943 Instead of registering a strategy for e.g. MyCollection[int], you
7945 should register a function for MyCollection and inspect the type param‐
7946 eters within that function.
7947 Thanks to Nikita Sobolev for the bug report, design assistance, and
7949 pull request to implement this feature!
7950 5.25.0 - 2020-08-16
7952 Tired of writing tests? Or new to Hypothesis and not sure where to
7953 start?
7954 This release is for you! With our new Ghostwriter functions and hy‐
7956 pothesis write ... command-line interface, you can stop writing tests
7957 entirely... or take the source code Hypothesis writes for you as a
7958 starting point.
7959 This has been in the works for months, from issue #2118 to versions
7961 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who
7962 reviewed pull requests or commented on demos, and to Timothy Crosley's
7963 hypothesis-auto project for inspiration.
7964 5.24.4 - 2020-08-14
7966 This patch adds yet more internal functions to support a new feature
7967 we're working on, like version 5.18.3 and version 5.23.6. We promise
7968 it's worth the wait!
7969 5.24.3 - 2020-08-13
7971 This release fixes a small internal bug in Hypothesis's internal autom‐
7972 aton library. Fortunately this bug was currently impossible to hit in
7973 user facing code, so this has no user visible impact.
7974 5.24.2 - 2020-08-12
7976 This release improves shrink quality by allowing Hypothesis to automat‐
7977 ically learn new shrink passes for difficult to shrink tests.
7978 The automatic learning is not currently accessible in user code (it
7980 still needs significant work on robustness and performance before it is
7981 ready for that), but this release includes learned passes that should
7982 improve shrinking quality for tests which use any of the text(),
7983 floats(), datetimes(), emails(), and complex_numbers() strategies.
7984 5.24.1 - 2020-08-12
7986 This patch updates some docstrings, without changing runtime behaviour.
7987 5.24.0 - 2020-08-10
7989 The functions() strategy has a new argument pure=True, which ensures
7990 that the same return value is generated for identical calls to the gen‐
7991 erated function (issue #2538).
7992 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
7994 5.23.12 - 2020-08-10
7996 This release removes a number of Hypothesis's internal "shrink passes"
7997 - transformations it makes to a generated test case during shrinking -
7998 which appeared to be redundant with other transformations.
7999 It is unlikely that you will see much impact from this. If you do, it
8001 will likely show up as a change in shrinking performance (probably
8002 slower, maybe faster), or possibly in worse shrunk results. If you en‐
8003 counter the latter, please let us know.
8004 5.23.11 - 2020-08-04
8006 This release fixes a bug in some internal Hypothesis support code. It
8007 has no user visible impact.
8008 5.23.10 - 2020-08-04
8010 This release improves the quality of shrunk test cases in some special
8011 cases. Specifically, it should get shrinking unstuck in some scenarios
8012 which require simultaneously changing two parts of the generated test
8013 case.
8014 5.23.9 - 2020-08-03
8016 This release improves the performance of some internal support code. It
8017 has no user visible impact, as that code is not currently run during
8018 normal Hypothesis operation.
8019 5.23.8 - 2020-07-31
8021 This release adds a heuristic to detect when shrinking has finished de‐
8022 spite the fact that there are many more possible transformations to
8023 try. This will be particularly useful for tests where the minimum fail‐
8024 ing test case is very large despite there being many smaller test cases
8025 possible, where it is likely to speed up shrinking dramatically.
8026 In some cases it is likely that this will result in worse shrunk test
8028 cases. In those cases rerunning the test will result in further shrink‐
8029 ing.
8030 5.23.7 - 2020-07-29
8032 This release makes some performance improvements to shrinking. They
8033 should only be noticeable for tests that are currently particularly
8034 slow to shrink.
8035 5.23.6 - 2020-07-29
8037 This patch adds some more internal functions to support a new feature
8038 we're working on, like version 5.18.3. There is still no user-visible
8039 change... yet.
8040 5.23.5 - 2020-07-29
8042 This release makes some changes to internal support code that is not
8043 currently used in production Hypothesis. It has no user visible effect
8044 at present.
8045 5.23.4 - 2020-07-29
8047 This release improves shrinking quality in some special cases.
8048 5.23.3 - 2020-07-27
8050 This release fixes issue #2507, where lazy evaluation meant that the
8051 values drawn from a sampled_from() strategy could depend on mutations
8052 of the sampled sequence that happened after the strategy was construc‐
8053 ted.
8054 5.23.2 - 2020-07-27
8056 This patch fixes issue #2462, a bug in our handling of unittest.Test‐
8057 Case.subTest(). Thanks to Israel Fruchter for fixing this at the Eu‐
8058 roPython sprints!
8059 5.23.1 - 2020-07-26
8061 This release improves the behaviour of the characters() strategy when
8062 shrinking, by changing which characters are considered smallest to pre‐
8063 fer more "normal" ascii characters where available.
8064 5.23.0 - 2020-07-26
8066 The default print_blob setting is now smarter. It defaults to True in
8067 CI and False for local development.
8068 Thanks to Hugo van Kemenade for implementing this feature at the Eu‐
8070 roPython sprints!
8071 5.22.0 - 2020-07-25
8073 The slices() strategy can now generate slices for empty sequences,
8074 slices with negative start and stop indices (from the end of the se‐
8075 quence), and step=None in place of step=1.
8076 Thanks to Sangarshanan for implementing this feature at the EuroPython
8078 sprints!
8079 5.21.0 - 2020-07-23
8081 This release ensures that tests which raise RecursionError are not re‐
8082 ported as flaky simply because we run them from different initial stack
8083 depths (issue #2494).
8084 5.20.4 - 2020-07-23
8086 This release improves the performance of the sample method on objects
8087 obtained from randoms() when use_true_random=False. This should mostly
8088 only be noticeable when the sample size is a large fraction of the pop‐
8089 ulation size, but may also help avoid health check failures in some
8090 other cases.
8091 5.20.3 - 2020-07-21
8093 This release makes some internal changes for testing purposes and
8094 should have no user visible effect.
8095 5.20.2 - 2020-07-18
8097 This release fixes a small caching bug in Hypothesis internals that may
8098 under some circumstances have resulted in a less diverse set of test
8099 cases being generated than was intended.
8100 Fixing this problem revealed some performance problems that could occur
8102 during targeted property based testing, so this release also fixes
8103 those. Targeted property-based testing should now be significantly
8104 faster in some cases, but this may be at the cost of reduced effective‐
8105 ness.
8106 5.20.1 - 2020-07-17
8108 This patch updates our formatting to use isort 5. There is no
8109 user-visible change.
8110 5.20.0 - 2020-07-17
8112 The basic_indices() strategy can now generate bare indexers in place of
8113 length-one tuples. Thanks to Andrea for this patch!
8114 5.19.3 - 2020-07-15
8116 This patch removes an internal use of distutils in order to avoid this
8117 setuptools warning for some users.
8118 5.19.2 - 2020-07-13
8120 This patch contains a small internal refactoring with no user-visible
8121 impact.
8122 Thanks to Andrea for writing this at the SciPy 2020 Sprints!
8124 5.19.1 - 2020-07-12
8126 This release slightly improves shrinking behaviour. This should mainly
8127 only impact stateful tests, but may have some minor positive impact on
8128 shrinking collections (lists, sets, etc).
8129 5.19.0 - 2020-06-30
8131 This release improves the randoms() strategy by adding support for Ran‐
8132 dom instances where Hypothesis generates the random values rather than
8133 having them be "truly" random.
8134 5.18.3 - 2020-06-27
8136 This patch adds some internal functions to support a new feature we're
8137 working on. There is no user-visible change... yet.
8138 5.18.2 - 2020-06-26
8140 This patch improves our docs for the derandomize setting.
8141 5.18.1 - 2020-06-25
8143 This release consists of some internal refactoring to the shrinker in
8144 preparation for future work. It has no user visible impact.
8145 5.18.0 - 2020-06-22
8147 This release teaches Hypothesis to shorten tracebacks for explicit ex‐
8148 amples, as we already do for generated examples, so that you can focus
8149 on your code rather than ours.
8150 If you have multiple failing explicit examples, they will now all be
8152 reported. To report only the first failure, you can use the
8153 report_multiple_bugs=False setting as for generated examples.
8154 5.17.0 - 2020-06-22
8156 This patch adds strategy inference for the Literal, NewType, Type, De‐
8157 faultDict, and TypedDict types from the typing_extensions backport on
8158 PyPI.
8159 5.16.3 - 2020-06-21
8161 This patch precomputes some of the setup logic for our external fuzzer
8162 integration and sets deadline=None in fuzzing mode, saving around 150us
8163 on each iteration.
8164 This is around two-thirds the runtime to fuzz an empty test with
8166 @given(st.none()), and nice to have even as a much smaller fraction of
8167 the runtime for non-trivial tests.
8168 5.16.2 - 2020-06-19
8170 This patch fixes an internal error when warning about the use of func‐
8171 tion-scoped fixtures for parametrised tests where the parametrised
8172 value contained a % character. Thanks to Bryant for reporting and fix‐
8173 ing this bug!
8174 5.16.1 - 2020-06-10
8176 If you pass a python:list or python:tuple where a strategy was ex‐
8177 pected, the error message now mentions sampled_from() as an example
8178 strategy.
8179 Thanks to the enthusiastic participants in the PyCon Mentored Sprints
8181 who suggested adding this hint.
8182 5.16.0 - 2020-05-27
8184 functions() can now infer the appropriate returns strategy if you pass
8185 a like function with a return-type annotation. Before, omitting the
8186 returns argument would generate functions that always returned None.
8187 5.15.1 - 2020-05-21
8189 Fix from_type() with generic types under Python 3.9.
8190 5.15.0 - 2020-05-19
8192 This patch fixes an error that happens when multiple threads create new
8193 strategies.
8194 5.14.0 - 2020-05-13
8196 Passing min_magnitude=None to complex_numbers() is now deprecated - you
8197 can explicitly pass min_magnitude=0, or omit the argument entirely.
8198 5.13.1 - 2020-05-13
8200 This patch fixes an internal error in from_type() for python:typ‐
8201 ing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and
8202 fixing issue #2427!
8203 5.13.0 - 2020-05-12
8205 This release upgrades the test statistics available via the
8206 --hypothesis-show-statistics option to include separate information on
8207 each of the phases (issue #1555).
8208 5.12.2 - 2020-05-12
8210 This patch teaches the from_type() internals to return slightly more
8211 efficient strategies for some generic sets and mappings.
8212 5.12.1 - 2020-05-12
8214 This patch adds a # noqa comment for flake8 3.8.0, which disagrees with
8215 mypy about how to write the type of ....
8216 5.12.0 - 2020-05-10
8218 This release limits the maximum duration of the shrinking phase to five
8219 minutes, so that Hypothesis does not appear to hang when making very
8220 slow progress shrinking a failing example (issue #2340).
8221 If one of your tests triggers this logic, we would really appreciate a
8223 bug report to help us improve the shrinker for difficult but realistic
8224 workloads.
8225 5.11.0 - 2020-05-07
8227 This release improves the interaction between assume() and the
8228 @example() decorator, so that the following test no longer fails with
8229 UnsatisfiedAssumption (issue #2125):
8230 @given(value=floats(0, 1))
8232 @example(value=0.56789) # used to make the test fail!
8233 @pytest.mark.parametrize("threshold", [0.5, 1])
8234 def test_foo(threshold, value):
8235 assume(value < threshold)
8236 ...
8237 5.10.5 - 2020-05-04
8239 If you have django installed but don't use it, this patch will make im‐
8240 port hypothesis a few hundred milliseconds faster (e.g. 0.704s ->
8241 0.271s).
8242 Thanks to importtime-waterfall for highlighting this problem and Jake
8244 Vanderplas for the solution - it's impossible to misuse code from a
8245 module you haven't imported!
8246 5.10.4 - 2020-04-24
8248 This patch improves the internals of builds() type inference, to handle
8249 recursive forward references in certain dataclasses. This is useful
8250 for e.g. hypothesmith's forthcoming LibCST mode.
8251 5.10.3 - 2020-04-22
8253 This release reverses the order in which some operations are tried dur‐
8254 ing shrinking. This should generally be a slight performance improve‐
8255 ment, but most tests are unlikely to notice much difference.
8256 5.10.2 - 2020-04-22
8258 This patch fixes issue #2406, where use of pandas:pandas.Timestamp ob‐
8259 jects as bounds for the datetimes() strategy caused an internal error.
8260 This bug was introduced in version 5.8.1.
8261 5.10.1 - 2020-04-19
8263 This release is a small internal refactoring to how shrinking interacts
8264 with targeted property-based testing that should have no user user vis‐
8265 ible impact.
8266 5.10.0 - 2020-04-18
8268 This release improves our support for datetimes and times around DST
8269 transitions.
8270 times() and datetimes() are now sometimes generated with fold=1, indi‐
8272 cating that they represent the second occurrence of a given wall-time
8273 when clocks are set backwards. This may be set even when there is no
8274 transition, in which case the fold value should be ignored.
8275 For consistency, timezones provided by the pytz package can now gener‐
8277 ate imaginary times (such as the hour skipped over when clocks 'spring
8278 forward' to daylight saving time, or during some historical timezone
8279 transitions). All other timezones have always supported generation of
8280 imaginary times.
8281 If you prefer the previous behaviour, datetimes() now takes an argument
8283 allow_imaginary which defaults to True but can be set to False for any
8284 timezones strategy.
8285 5.9.1 - 2020-04-16
8287 This patch fixes the rendering of binary() docstring by using the
8288 proper backticks syntax.
8289 5.9.0 - 2020-04-15
8291 Failing tests which use target() now report the highest score observed
8292 for each target alongside the failing example(s), even without
8293 explicitly showing test statistics.
8294 This improves the debugging workflow for tests of accuracy, which as‐
8296 sert that the total imprecision is within some error budget - for exam‐
8297 ple, abs(a - b) < 0.5. Previously, shrinking to a minimal failing ex‐
8298 ample could often make errors seem smaller or more subtle than they re‐
8299 ally are (see the threshold problem, and issue #2180).
8300 5.8.6 - 2020-04-15
8302 This patch improves the docstring of binary(), the python:repr() of
8303 sampled_from() on an python:enum.Enum subclass, and a warning in our
8304 pytest plugin. There is no change in runtime behaviour.
8305 5.8.5 - 2020-04-15
8307 This release (potentially very significantly) improves the performance
8308 of failing tests in some rare cases, mostly only relevant when using
8309 targeted property-based testing, by stopping further optimisation of
8310 unrelated test cases once a failing example is found.
8311 5.8.4 - 2020-04-14
8313 This release fixes issue #2395, where under some circumstances targeted
8314 property-based testing could cause Hypothesis to get caught in an infi‐
8315 nite loop.
8316 5.8.3 - 2020-04-12
8318 This patch teaches builds() and from_type() to use the __signature__
8319 attribute of classes where it has been set, improving our support for
8320 Pydantic models (in pydantic >= 1.5).
8321 5.8.2 - 2020-04-12
8323 This release improves the performance of the part of the core engine
8324 that deliberately generates duplicate values.
8325 5.8.1 - 2020-04-12
8327 This patch improves dates() shrinking, to simplify year, month, and day
8328 like datetimes() rather than minimizing the number of days since
8329 2000-01-01.
8330 5.8.0 - 2020-03-24
8332 This release adds a .hypothesis.fuzz_one_input attribute to @given
8333 tests, for easy integration with external fuzzers such as python-afl
8334 (supporting issue #171).
8335 5.7.2 - 2020-03-24
8337 This patch fixes issue #2341, ensuring that the printed output from a
8338 stateful test cannot use variable names before they are defined.
8339 5.7.1 - 2020-03-23
8341 This patch fixes issue #2375, preventing incorrect failure when a func‐
8342 tion scoped fixture is overridden with a higher scoped fixture.
8343 5.7.0 - 2020-03-19
8345 This release allows the array_dtypes() strategy to generate Numpy
8346 dtypes which have field titles in addition to field names. We expect
8347 this to expose latent bugs where code expects that set(dtype.names) ==
8348 set(dtype.fields), though the latter may include titles.
8349 5.6.1 - 2020-03-18
8351 This makes model a positional-only argument to from_model(), to support
8352 models with a field literally named "model" (issue #2369).
8353 5.6.0 - 2020-02-29
8355 This release adds an explicit warning for tests that are both decorated
8356 with @given(...) and request a function-scoped pytest fixture, because
8357 such fixtures are only executed once for all Hypothesis test cases and
8358 that often causes trouble (issue #377).
8359 It's very difficult to fix this on the pytest side, so since 2015 our
8361 advice has been "just don't use function-scoped fixtures with Hypothe‐
8362 sis". Now we detect and warn about the issue at runtime!
8363 5.5.5 - 2020-02-29
8365 This release cleans up the internal machinery for Stateful testing, af‐
8366 ter we dropped the legacy APIs in Hypothesis 5.0 (issue #2218). There
8367 is no user-visible change.
8368 5.5.4 - 2020-02-16
8370 This patch fixes issue #2351, arrays() would raise a confusing error if
8371 we inferred a strategy for datetime64 or timedelta64 values with vary‐
8372 ing time units.
8373 We now infer an internally-consistent strategy for such arrays, and
8375 have a more helpful error message if an inconsistent strategy is ex‐
8376 plicitly specified.
8377 5.5.3 - 2020-02-14
8379 This patch improves the signature of builds() by specifying target as a
8380 positional-only argument on Python 3.8 (see PEP 570). The semantics of
8381 builds() have not changed at all - this just clarifies the documenta‐
8382 tion.
8383 5.5.2 - 2020-02-13
8385 This release makes Hypothesis faster at generating test cases that con‐
8386 tain duplicated values in their inputs.
8387 5.5.1 - 2020-02-07
8389 This patch has some tiny internal code clean-ups, with no user-visible
8390 change.
8391 5.5.0 - 2020-02-07
8393 Our style guide suggests that optional parameters should usually be
8394 keyword-only arguments (see PEP 3102) to prevent confusion based on po‐
8395 sitional arguments - for example, hypothesis.strategies.floats() takes
8396 up to four boolean flags and many of the Numpy strategies have both
8397 dims and side bounds.
8398 This release converts most optional parameters in our API to use key‐
8400 word-only arguments - and adds a compatibility shim so you get warnings
8401 rather than errors everywhere (issue #2130).
8402 5.4.2 - 2020-02-06
8404 This patch fixes compatibility with Python 3.5.2 (issue #2334). Note
8405 that we only test the latest patch of each minor version, though as in
8406 this case we usually accept pull requests for older patch versions.
8407 5.4.1 - 2020-02-01
8409 This patch improves the repr of from_type(), so that in most cases it
8410 will display the strategy it resolves to rather than from_type(...).
8411 The latter form will continue to be used where resolution is not imme‐
8412 diately successful, e.g. invalid arguments or recursive type defini‐
8413 tions involving forward references.
8414 5.4.0 - 2020-01-30
8416 This release removes support for Python 3.5.0 and 3.5.1, where the
8417 python:typing module was quite immature (e.g. missing overload() and
8418 Type).
8419 Note that Python 3.5 will reach its end-of-life in September 2020, and
8421 new releases of Hypothesis may drop support somewhat earlier.
8422 NOTE:
8424 pip install hypothesis should continue to give you the latest com‐
8425 patible version. If you have somehow ended up with an incompatible
8426 version, you need to update your packaging stack to pip >= 9.0 and
8427 setuptools >= 24.2 - see here for details. Then pip uninstall hy‐
8428 pothesis && pip install hypothesis will get you back to a compatible
8429 version.
8430 5.3.1 - 2020-01-26
8432 This patch does some minor internal cleanup; there is no user-visible
8433 change.
8434 5.3.0 - 2020-01-21
8436 The standard library ipaddress module is new in Python 3, and this re‐
8437 lease adds the new ip_addresses() strategy to generate IPv4Addresses
8438 and/or IPv6Addresses (depending on the v and network arguments).
8439 If you use them in type annotations, from_type() now has strategies
8441 registered for ipaddress address, network, and interface types.
8442 The provisional strategies for IP address strings are therefore depre‐
8444 cated.
8445 5.2.1 - 2020-01-21
8447 This patch reverts version 5.2, due to a strange issue where indexing
8448 an array of strings can raise an error instead of returning an item
8449 which contains certain surrogate characters.
8450 5.2.0 - 2020-01-19
8452 This release allows from_dtype() to generate Unicode strings which can‐
8453 not be encoded in UTF-8, but are valid in Numpy arrays (which use
8454 UTF-32).
8455 5.1.6 - 2020-01-19
8457 This patch fixes issue #2320, where from_type(Set[Hashable]) could
8458 raise an internal error because Decimal("snan") is of a hashable type,
8459 but raises an error when hashed. We now ensure that set elements and
8460 dict keys in generic types can actually be hashed.
8461 5.1.5 - 2020-01-12
8463 This patch fixes an internal error when running in an IPython repl or
8464 Jupyter notebook on Windows (issue #2319), and an internal error on
8465 Python 3.5.1 (issue #2318).
8466 5.1.4 - 2020-01-11
8468 This patch fixes a bug where errors in third-party extensions such as
8469 hypothesis-trio or hypothesis-jsonschema were incorrectly considered to
8470 be Hypothesis internal errors, which could result in confusing error
8471 messages.
8472 Thanks to Vincent Michel for reporting and fixing the bug!
8474 5.1.3 - 2020-01-11
8476 This release converts the type hint comments on our public API to PEP
8477 484 type annotations.
8478 Thanks to Ivan Levkivskyi for com2ann - with the refactoring tools from
8480 5.0.1 it made this process remarkably easy!
8481 5.1.2 - 2020-01-09
8483 This patch makes multiple() iterable, so that output like a, b =
8484 state.some_rule() is actually executable and can be used to reproduce
8485 failing examples.
8486 Thanks to Vincent Michel for reporting and fixing issue #2311!
8488 5.1.1 - 2020-01-06
8490 This patch contains many small refactorings to replace our Python 2
8491 compatibility functions with their native Python 3 equivalents. Since
8492 Hypothesis is now Python 3 only, there is no user-visible change.
8493 5.1.0 - 2020-01-03
8495 This release teaches from_type() how to generate python:datetime.time‐
8496 zone. As a result, you can now generate python:datetime.tzinfo objects
8497 without having pytz installed.
8498 If your tests specifically require pytz timezones, you should be using
8500 hypothesis.extra.pytz.timezones() instead of st.from_type(tzinfo).
8501 5.0.1 - 2020-01-01
8503 This patch contains mostly-automated refactorings to remove code that
8504 we only needed to support Python 2. Since Hypothesis is now Python 3
8505 only (hurray!), there is no user-visible change.
8506 Our sincere thanks to the authors of autoflake, black, isort, and
8508 pyupgrade, who have each and collectively made this kind of update
8509 enormously easier.
8510 5.0.0 - 2020-01-01
8512 Welcome to the next major version of Hypothesis!
8513 There are no new features here, as we release those in minor versions.
8515 Instead, 5.0 is a chance for us to remove deprecated features (many al‐
8516 ready converted into no-ops), and turn a variety of warnings into er‐
8517 rors.
8518 If you were running on the last version of Hypothesis 4.x without any
8520 Hypothesis deprecation warnings, this will be a very boring upgrade.
8521 In fact, nothing will change for you at all.
8522 NOTE:
8524 This release drops support for Python 2, which has passed its end of
8525 life date. The Python 3 Statement outlines our reasons, and lists
8526 many other packages that have made the same decision.
8527 pip install hypothesis should continue to give you the latest com‐
8529 patible version. If you have somehow ended up with Hypothesis 5.0
8530 on Python 2, you need to update your packaging stack to pip >= 9.0
8531 and setuptools >= 24.2 - see here for details. Then pip uninstall
8532 hypothesis && pip install hypothesis will get you back to a compati‐
8533 ble version.
8534 Strategies
8536 • integers() bounds must be equal to an integer, though they can still
8537 be other types.
8538 • If fractions() is passed a max_denominator, the bounds must have at
8540 most that denominator.
8541 • floats() bounds must be exactly representable as a floating-point
8543 number with the given width. If not, the error message includes the
8544 nearest such number.
8545 • sampled_from([]) is now an error.
8547 • The values from the elements and fill strategies for
8549 hypothesis.extra.numpy.arrays() must be losslessly representable in
8550 an array of the given dtype.
8551 • The min_size and max_size arguments to all collection strategies must
8553 be of type python:int (or max_size may be None).
8554 Miscellaneous
8556 • The .example() method of strategies (intended for interactive explo‐
8557 ration) no longer takes a random argument.
8558 • It is now an error to apply @example, @seed, or @reproduce_failure
8560 without also applying @given.
8561 • You may pass either the target or targets argument to stateful rules,
8563 but not both.
8564 • deadline must be None (to disable), a timedelta, or an integer or
8566 float number of milliseconds.
8567 • Both of derandomize and print_blob must be either True or False,
8569 where they previously accepted other values.
8570 • stateful_step_count must be at least one.
8572 • max_examples must be at least one. To disable example generation,
8574 use the phases setting.
8575 Removals
8577 • hypothesis.stateful.GenericStateMachine in favor of
8578 hypothesis.stateful.RuleBasedStateMachine
8579 • hypothesis.extra.django.models.models in favor of
8581 hypothesis.extra.django.from_model() and hypothesis.extra.django.mod‐
8582 els.add_default_field_mapping in favor of
8583 hypothesis.extra.django.register_field_strategy()
8584 • hypothesis.HealthCheck.hung_test, without replacement
8586 • hypothesis.settings.buffer, without replacement
8588 • hypothesis.PrintSettings, because hypothesis.settings.print_blob
8590 takes True or False
8591 • hypothesis.settings.timeout, in favor of hypothesis.settings.deadline
8593 • hypothesis.unlimited without replacement (only only useful as argu‐
8595 ment to timeout)
8596 Hypothesis 4.x
8598 4.57.1 - 2019-12-29
8599 This patch improves the type hints and documentation for the django ex‐
8600 tra. There is no runtime change.
8601 4.57.0 - 2019-12-28
8603 This release improves support for the SupportsOp protocols from the
8604 python:typing module when using on from_type() as outlined in issue
8605 #2292. The following types now generate much more varied strategies
8606 when called with from_type():
8607 • python:typing.SupportsAbs
8609 • python:typing.SupportsBytes
8611 • python:typing.SupportsComplex
8613 • python:typing.SupportsInt
8615 • python:typing.SupportsFloat
8617 • python:typing.SupportsRound
8619 Note that using from_type() with one of the above strategies will not
8621 ensure that the the specified function will execute successfully (ie :
8622 the strategy returned for from_type(typing.SupportsAbs) may include
8623 NaNs or things which cause the python:abs() function to error. )
8624 Thanks to Lea Provenzano for this patch.
8626 4.56.3 - 2019-12-22
8628 This release fixes a small internal bug in shrinking which could have
8629 caused it to perform slightly more tests than were necessary. Fixing
8630 this shouldn't have much effect but it will make shrinking slightly
8631 faster.
8632 4.56.2 - 2019-12-21
8634 This release removes an internal heuristic that was no longer providing
8635 much benefit. It is unlikely that there will be any user visible ef‐
8636 fect.
8637 4.56.1 - 2019-12-19
8639 This release further improves the optimisation algorithm for targeted
8640 property-based testing.
8641 4.56.0 - 2019-12-18
8643 This release enables deprecation warnings even when the verbosity set‐
8644 ting is quiet, in preparation for Hypothesis 5.0 (issue #2218).
8645 Warnings can still be filtered by the standard mechanisms provided in
8647 the standard-library python:warnings module.
8648 4.55.4 - 2019-12-18
8650 This release improves Hypothesis's management of the set of test cases
8651 it tracks between runs. It will only do anything if you have the target
8652 phase enabled and an example database set. In those circumstances it
8653 should result in a more thorough and faster set of examples that are
8654 tried on each run.
8655 4.55.3 - 2019-12-18
8657 This release makes Hypothesis better at generating test cases where
8658 generated values are duplicated in different parts of the test case.
8659 This will be especially noticeable with reasonably complex values, as
8660 it was already able to do this for simpler ones such as integers or
8661 floats.
8662 4.55.2 - 2019-12-17
8664 This release expands the set of test cases that Hypothesis saves in its
8665 database for future runs to include a representative set of "struc‐
8666 turally different" test cases - e.g. it might try to save test cases
8667 where a given list is empty or not.
8668 Currently this is unlikely to have much user visible impact except to
8670 produce slightly more consistent behaviour between consecutive runs of
8671 a test suite. It is mostly groundwork for future improvements which
8672 will exploit this functionality more effectively.
8673 4.55.1 - 2019-12-16
8675 This patch fixes issue #2257, where from_type() could incorrectly gen‐
8676 erate bytestrings when passed a generic python:typing.Sequence such as
8677 Sequence[set].
8678 4.55.0 - 2019-12-16
8680 This release adds database support for targeted property-based testing,
8681 so the best examples based on the targeting will be saved and reused
8682 between runs. This is mostly laying groundwork for future features in
8683 this area, but will also make targeted property-based tests more useful
8684 during development, where the same tests tend to get run over and over
8685 again.
8686 If max_examples is large, this may increase memory usage significantly
8688 under some circumstances, but these should be relatively rare.
8689 This release also adds a dependency on the sortedcontainers package.
8691 4.54.2 - 2019-12-16
8693 This release improves the optimisation algorithm for targeted prop‐
8694 erty-based testing, so that it will find higher quality results more
8695 reliably. Specifically, in cases where it would previously have got
8696 near a local optimum, it will now tend to achieve the locally optimal
8697 value.
8698 4.54.1 - 2019-12-16
8700 This release is mostly internal changes in support of better testing of
8701 the core engine. You are unlikely to see much effect, although some in‐
8702 ternal heuristics have changed slightly.
8703 4.54.0 - 2019-12-15
8705 This release adds a dedicated phase for targeted property-based test‐
8706 ing, and (somewhat) improves the targeting algorithm so that it will
8707 find higher quality results more reliably. This comes at a cost of
8708 making it more likely to get stuck in a local optimum.
8709 4.53.3 - 2019-12-15
8711 This patch fixes from_type() with python:typing.Hashable and
8712 python:typing.Sized, which previously failed with an internal error on
8713 Python 3.7 or later.
8714 Thanks to Lea Provenzano for both reporting issue #2272 and writing the
8716 patch!
8717 4.53.2 - 2019-12-11
8719 This release reorganises a number of the Hypothesis internal modules
8720 into a package structure. If you are only depending on the public API
8721 it should have no effect. If you are depending on the internal API
8722 (which you shouldn't be, and which we don't guarantee compatibility on)
8723 you may have to rename some imports.
8724 4.53.1 - 2019-12-09
8726 This release changes the size distribution of the number of steps run
8727 in stateful testing: It will now almost always run the maximum number
8728 of steps permitted.
8729 4.53.0 - 2019-12-09
8731 Test statistics now include the best score seen for each label, which
8732 can help avoid the threshold problem when the minimal example shrinks
8733 right down to the threshold of failure (issue #2180).
8734 4.52.0 - 2019-12-09
8736 This release changes the stateful_step_count setting to raise an error
8737 if set to 0. This is a backwards compatible change because a value of 0
8738 would never have worked and attempting to run it would have resulted in
8739 an internal assertion error.
8740 4.51.1 - 2019-12-09
8742 This release makes a small internal change to the distribution of test
8743 cases. It is unlikely to have much user visible impact.
8744 4.51.0 - 2019-12-07
8746 This release deprecates use of @example, @seed, or @reproduce_failure
8747 without @given.
8748 Thanks to Nick Anyos for the patch!
8750 4.50.8 - 2019-12-05
8752 This patch makes certain uses of Bundles more efficient in stateful
8753 testing (issue #2078).
8754 4.50.7 - 2019-12-05
8756 This release refactors some of Hypothesis's internal interfaces for
8757 representing data generation. It should have no user visible effect.
8758 4.50.6 - 2019-12-02
8760 This patch removes some old debugging helpers in our Numpy extra which
8761 have not been needed since issue #1963 and issue #2245.
8762 4.50.5 - 2019-12-01
8764 This patch fixes issue #2229, where Numpy arrays of unsized strings
8765 would only ever have strings of size one due to an interaction between
8766 our generation logic and Numpy's allocation strategy.
8767 4.50.4 - 2019-12-01
8769 This patch fixes a rare internal error in strategies for a list of
8770 unique items sampled from a short non-unique sequence (issue #2247).
8771 The bug was discovered via hypothesis-jsonschema.
8772 4.50.3 - 2019-12-01
8774 This release improves the error message when @settings tries to inherit
8775 settings from a parent argument that isn't a settings instance.
8776 4.50.2 - 2019-11-29
8778 This release improves Hypothesis's "Falsifying example" output, by
8779 breaking output across multiple lines where necessary, and by removing
8780 irrelevant information from the stateful testing output.
8781 4.50.1 - 2019-11-29
8783 This patch adds flake8-comprehensions to our linter suite. There is no
8784 user-visible change - expect perhaps via some strange microbenchmarks -
8785 but certain parts of the code now have a clear and more consistent
8786 style.
8787 4.50.0 - 2019-11-28
8789 This release fixes some cases where we might previously have failed to
8790 run the validation logic for some strategies. As a result tests which
8791 would previously have been silently testing significantly less than
8792 they should may now start to raise InvalidArgument now that these er‐
8793 rors are caught.
8794 4.49.0 - 2019-11-28
8796 This release significantly improves the data distribution in rule based
8797 stateful testing, by using a technique called Swarm Testing (Groce,
8798 Alex, et al. "Swarm testing." Proceedings of the 2012 International
8799 Symposium on Software Testing and Analysis. ACM, 2012.) to select
8800 which rules are run in any given test case. This should allow it to
8801 find many issues that it would previously have missed.
8802 This change is likely to be especially beneficial for stateful tests
8804 with large numbers of rules.
8805 4.48.1 - 2019-11-28
8807 This release adds some heuristics to test case generation that try to
8808 ensure that test cases generated early on will be relatively small.
8809 This fixes a bug introduced in Hypothesis 4.42.0 which would cause oc‐
8811 casional too_slow failures on some tests.
8812 4.48.0 - 2019-11-28
8814 This release revokes the deprecation of find, as we've now rebuilt it
8815 on top of @given, which means it has minimal maintenance burden and
8816 we're happy to support it.
8817 4.47.5 - 2019-11-28
8819 This release rebuilds find() on top of @given in order to have more
8820 code in common. It should have minimal user visible effect.
8821 4.47.4 - 2019-11-27
8823 This patch removes an internal compatibility shim that we no longer
8824 need.
8825 4.47.3 - 2019-11-26
8827 This patch fixes several typos in our docstrings and comments, with no
8828 change in behaviour. Thanks to Dmitry Dygalo for identifying and fix‐
8829 ing them!
8830 4.47.2 - 2019-11-25
8832 This release fixes an internal issue where Hypothesis would sometimes
8833 generate test cases that were above its intended maximum size. This
8834 would only have happened rarely and probably would not have caused ma‐
8835 jor problems when it did.
8836 Users of the new targeted property-based testing might see minor im‐
8838 pact (possibly slightly faster tests and slightly worse target scores),
8839 but only in the unlikely event that they were hitting this problem.
8840 Other users should not see any effect at all.
8841 4.47.1 - 2019-11-24
8843 This release removes some unused code from the core engine. There is
8844 no user-visible change.
8845 4.47.0 - 2019-11-24
8847 This release commonizes some code between running explicit examples and
8848 normal test execution. The main user visible impact of this is that
8849 deadlines are now enforced when running explicit examples.
8850 4.46.1 - 2019-11-23
8852 This patch ensures that a KeyboardInterrupt received during example
8853 generation is not treated as a mystery test failure but instead propa‐
8854 gates to the top level, not recording the interrupted generation in the
8855 conjecture data tree. Thanks to Anne Archibald for identifying and
8856 fixing the problem.
8857 4.46.0 - 2019-11-22
8859 This release changes the behaviour of floats() when excluding signed
8860 zeros - floats(max_value=0.0, exclude_max=True) can no longer generate
8861 -0.0 nor the much rarer floats(min_value=-0.0, exclude_min=True) gener‐
8862 ate +0.0.
8863 The correct interaction between signed zeros and exclusive endpoints
8865 was unclear; we now enforce the invariant that floats() will never gen‐
8866 erate a value equal to an excluded endpoint (issue #2201).
8867 If you prefer the old behaviour, you can pass floats(max_value=-0.0) or
8869 floats(min_value=0.0) which is exactly equivalent and has not changed.
8870 If you had two endpoints equal to zero, we recommend clarifying your
8871 tests by using just() or sampled_from() instead of floats().
8872 4.45.1 - 2019-11-20
8874 This patch improves the error message when invalid arguments are passed
8875 to rule() or invariant() (issue #2149).
8876 Thanks to Benjamin Palmer for this bugfix!
8878 4.45.0 - 2019-11-20
8880 This release supports python:typing.Final and python:typing.TypedDict
8881 in from_type().
8882 4.44.5 - 2019-11-20
8884 This patch disables our pytest plugin when running on versions of
8885 pytest before 4.3, the oldest our plugin supports. Note that at time
8886 of writing the Pytest developers only support 4.6 and later!
8887 Hypothesis tests using @given() work on any test runner, but our inte‐
8889 grations to e.g. avoid example database collisions when combined with
8890 @pytest.mark.parametrize eventually drop support for obsolete versions.
8891 4.44.4 - 2019-11-20
8893 This patch adds some internal comments and clarifications to the Hy‐
8894 pothesis implementation. There is no user-visible change.
8895 4.44.3 - 2019-11-20
8897 This patch avoids importing test runners such as pytest, unittest2, or
8898 nose solely to access their special "skip test" exception types - if
8899 the module is not in sys.modules, the exception can't be raised anyway.
8900 This fixes a problem where importing an otherwise unused module could
8902 cause spurious errors due to import-time side effects (and possibly
8903 -Werror).
8904 4.44.2 - 2019-11-12
8906 This release fixes @given to only complain about missing keyword-only
8907 arguments if the associated test function is actually called.
8908 This matches the behaviour of other InvalidArgument errors produced by
8910 @given.
8911 4.44.1 - 2019-11-11
8913 This patch allows Hypothesis to run in environments that do not specify
8914 a __file__, such as a python:zipapp (issue #2196).
8915 4.44.0 - 2019-11-11
8917 This release adds a signature argument to
8918 mutually_broadcastable_shapes() (issue #2174), which allows us to gen‐
8919 erate shapes which are valid for functions like np.matmul() that re‐
8920 quire shapes which are not simply broadcastable.
8921 Thanks to everyone who has contributed to this feature over the last
8923 year, and a particular shout-out to Zac Hatfield-Dodds and Ryan Sok‐
8924 laski for mutually_broadcastable_shapes() and to Ryan Turner for the
8925 downstream hypothesis-gufunc project.
8926 4.43.9 - 2019-11-11
8928 This patch fixes issue #2108, where the first test using data() to draw
8929 from characters() or text() would be flaky due to unreliable test tim‐
8930 ings.
8931 Time taken by lazy instantiation of strategies is now counted towards
8933 drawing from the strategy, rather than towards the deadline for the
8934 test function.
8935 4.43.8 - 2019-11-08
8937 This release ensures that the strategies passed to @given are properly
8938 validated when applied to a test method inside a test class.
8939 This should result in clearer error messages when some of those strate‐
8941 gies are invalid.
8942 4.43.7 - 2019-11-08
8944 This release changes how Hypothesis manages its search space in cases
8945 where it generates redundant data. This should cause it to generate
8946 significantly fewer duplicated examples (especially with short integer
8947 ranges), and may cause it to produce more useful examples in some cases
8948 (especially ones where there is a significant amount of filtering).
8949 4.43.6 - 2019-11-07
8951 This patch refactors width handling in floats(); you may notice small
8952 performance improvements but the main purpose is to enable work on
8953 issue #1704 (improving shrinking of bounded floats).
8954 4.43.5 - 2019-11-06
8956 This patch removes an unused internal flag. There is no user-visible
8957 change.
8958 4.43.4 - 2019-11-05
8960 This patch corrects the exception type and error message you get if you
8961 attempt to use data() to draw from something which is not a strategy.
8962 This never worked, but the error is more helpful now.
8963 4.43.3 - 2019-11-05
8965 We've adopted flake8-bugbear to check for a few more style issues, and
8966 this patch implements the minor internal cleanups it suggested. There
8967 is no user-visible change.
8968 4.43.2 - 2019-11-05
8970 This patch fixes the formatting of some documentation, but there is no
8971 change to any executed code.
8972 4.43.1 - 2019-11-04
8974 Python 3.8's new python:typing.Literal type - see PEP 586 for details -
8975 is now supported in from_type().
8976 4.43.0 - 2019-11-04
8978 This release adds the strategy mutually_broadcastable_shapes(), which
8979 generates multiple array shapes that are mutually broadcast-compatible
8980 with an optional user-specified base-shape.
8981 This is a generalisation of broadcastable_shapes(). It relies heavily
8983 on non-public internals for performance when generating and shrinking
8984 examples. We intend to support generating shapes matching a ufunc sig‐
8985 nature in a future version (issue #2174).
8986 Thanks to Ryan Soklaski, Zac Hatfield-Dodds, and @rdturnermtl who con‐
8988 tributed to this new feature.
8989 4.42.10 - 2019-11-03
8991 This release fixes from_type() when used with bounded or constrained
8992 python:typing.TypeVar objects (issue #2094).
8993 Previously, distinct typevars with the same constraints would be
8995 treated as all single typevar, and in cases where a typevar bound was
8996 resolved to a union of subclasses this could result in mixed types be‐
8997 ing generated for that typevar.
8998 4.42.9 - 2019-11-03
9000 This patch ensures that the default value broadcastable_shapes()
9001 chooses for max_dims is always valid (at most 32), even if you pass
9002 min_dims=32.
9003 4.42.8 - 2019-11-02
9005 This patch ensures that we only add profile information to the pytest
9006 header if running either pytest or Hypothesis in verbose mode, matching
9007 the builtin cache plugin (issue #2155).
9008 4.42.7 - 2019-11-02
9010 This patch makes stateful step printing expand the result of a step
9011 into multiple variables when you return multiple() (issue #2139).
9012 Thanks to Joseph Weston for reporting and fixing this bug!
9013 4.42.6 - 2019-11-02
9015 This release fixes a bug (issue #2166) where a Unicode character info
9016 cache file was generated but never used on subsequent test runs, caus‐
9017 ing tests to run more slowly than they should have.
9018 Thanks to Robert Knight for this bugfix!
9020 4.42.5 - 2019-11-01
9022 This patch corrects some internal documentation. There is no user-vis‐
9023 ible change.
9024 4.42.4 - 2019-11-01
9026 This release fixes a bug (issue #2160) where decorators applied after
9027 @settings and before @given were ignored.
9028 Thanks to Tom Milligan for this bugfix!
9030 4.42.3 - 2019-10-30
9032 This release updates Hypothesis's formatting to the new version of
9033 black, and has absolutely no user visible effect.
9034 4.42.2 - 2019-10-30
9036 This release fixes a bug in recursive() which would have meant that in
9037 practice max_leaves was treated as if it was lower than it actually is
9038 - specifically it would be capped at the largest power of two smaller
9039 than it. It is now handled correctly.
9040 4.42.1 - 2019-10-30
9042 Python 3.8's new python:typing.SupportsIndex type - see PEP 357 for de‐
9043 tails - is now supported in from_type().
9044 Thanks to Grigorios Giannakopoulos for the patch!
9046 4.42.0 - 2019-10-27
9048 This release significantly simplifies Hypothesis's internal logic for
9049 data generation, by removing a number of heuristics of questionable or
9050 unproven value.
9051 The results of this change will vary significantly from test to test.
9053 Most test suites will see significantly faster data generation and
9054 lower memory usage. The "quality" of the generated data may go up or
9055 down depending on your particular test suites.
9056 If you see any significant regressions in Hypothesis's ability to find
9058 bugs in your code as a result of this release, please file an issue to
9059 let us know.
9060 Users of the new targeted property-based testing functionality are
9062 reasonably likely to see improvements in data generation, as this re‐
9063 lease changes the search algorithm for targeted property based testing
9064 to one that is more likely to be productive than the existing approach.
9065 4.41.3 - 2019-10-21
9067 This patch is to ensure that our internals remain comprehensible to
9068 mypy 0.740 - there is no user-visible change.
9069 4.41.2 - 2019-10-17
9071 This patch changes some internal hashes to SHA384, to better support
9072 users subject to FIPS-140. There is no user-visible API change.
9073 Thanks to Paul Kehrer for this contribution!
9075 4.41.1 - 2019-10-16
9077 This release makes --hypothesis-show-statistics much more useful for
9078 tests using a RuleBasedStateMachine, by simplifying the reprs so that
9079 events are aggregated correctly.
9080 4.41.0 - 2019-10-16
9082 This release upgrades the fixed_dictionaries() strategy to support op‐
9083 tional keys (issue #1913).
9084 4.40.2 - 2019-10-16
9086 This release makes some minor internal changes in support of improving
9087 the Hypothesis test suite. It should not have any user visible impact.
9088 4.40.1 - 2019-10-14
9090 This release changes how Hypothesis checks if a parameter to a test
9091 function is a mock object. It is unlikely to have any noticeable ef‐
9092 fect, but may result in a small performance improvement, especially for
9093 test functions where a mock object is being passed as the first argu‐
9094 ment.
9095 4.40.0 - 2019-10-09
9097 This release fixes a bug where our example database logic did not dis‐
9098 tinguish between failing examples based on arguments from a
9099 @pytest.mark.parametrize(...). This could in theory cause data loss if
9100 a common failure overwrote a rare one, and in practice caused occa‐
9101 sional file-access collisions in highly concurrent workloads (e.g. dur‐
9102 ing a 300-way parametrize on 16 cores).
9103 For internal reasons this also involves bumping the minimum supported
9105 version of pytest to 4.3
9106 Thanks to Peter C Kroon for the Hacktoberfest patch!
9108 4.39.3 - 2019-10-09
9110 This patch improves our type hints on the emails(), functions(),
9111 integers(), iterables(), and slices() strategies, as well as the .fil‐
9112 ter() method.
9113 There is no runtime change, but if you use mypy or a similar
9115 type-checker on your tests the results will be a bit more precise.
9116 4.39.2 - 2019-10-09
9118 This patch improves the performance of unique collections such as
9119 sets() of just() or booleans() strategies. They were already pretty
9120 good though, so you're unlikely to notice much!
9121 4.39.1 - 2019-10-09
9123 If a value in a dict passed to fixed_dictionaries() is not a strategy,
9124 Hypothesis now tells you which one.
9125 4.39.0 - 2019-10-07
9127 This release adds the basic_indices() strategy, to generate basic in‐
9128 dexes for arrays of the specified shape (issue #1930).
9129 It generates tuples containing some mix of integers, python:slice ob‐
9131 jects, ... (Ellipsis), and numpy:numpy.newaxis; which when used to in‐
9132 dex an array of the specified shape produce either a scalar or a
9133 shared-memory view of the array. Note that the index tuple may be
9134 longer or shorter than the array shape, and may produce a view with an‐
9135 other dimensionality again!
9136 Thanks to Lampros Mountrakis, Ryan Soklaski, and Zac Hatfield-Dodds for
9138 their collaboration on this surprisingly subtle strategy!
9139 4.38.3 - 2019-10-04
9141 This patch defers creation of the .hypothesis directory until we have
9142 something to store in it, meaning that it will appear when Hypothesis
9143 is used rather than simply installed.
9144 Thanks to Peter C Kroon for the Hacktoberfest patch!
9146 4.38.2 - 2019-10-02
9148 This patch bumps our dependency on attrs to >=19.2.0; but there are no
9149 user-visible changes to Hypothesis.
9150 4.38.1 - 2019-10-01
9152 This is a comment-only patch which tells mypy 0.730 to ignore some in‐
9153 ternal compatibility shims we use to support older Pythons.
9154 4.38.0 - 2019-10-01
9156 This release adds the hypothesis.target() function, which implements
9157 targeted property-based testing (issue #1779).
9158 By calling target() in your test function, Hypothesis can do a
9160 hill-climbing search for bugs. If you can calculate a suitable metric
9161 such as the load factor or length of a queue, this can help you find
9162 bugs with inputs that are highly improbably from unguided generation -
9163 however good our heuristics, example diversity, and deduplication logic
9164 might be. After all, those features are at work in targeted PBT too!
9165 4.37.0 - 2019-09-28
9167 This release emits a warning if you use the .example() method of a
9168 strategy in a non-interactive context.
9169 given() is a much better choice for writing tests, whether you care
9171 about performance, minimal examples, reproducing failures, or even just
9172 the variety of inputs that will be tested!
9173 4.36.2 - 2019-09-20
9175 This patch disables part of the typing-based inference for the attrs
9176 package under Python 3.5.0, which has some incompatible internal de‐
9177 tails (issue #2095).
9178 4.36.1 - 2019-09-17
9180 This patch fixes a bug in strategy inference for attrs classes where
9181 Hypothesis would fail to infer a strategy for attributes of a generic
9182 type such as Union[int, str] or List[bool] (issue #2091).
9183 Thanks to Jonathan Gayvallet for the bug report and this patch!
9185 4.36.0 - 2019-09-09
9187 This patch deprecates min_len or max_len of 0 in byte_string_dtypes()
9188 and unicode_string_dtypes(). The lower limit is now 1.
9189 Numpy uses a length of 0 in these dtypes to indicate an undetermined
9191 size, chosen from the data at array creation. However, as the arrays()
9192 strategy creates arrays before filling them, strings were truncated to
9193 1 byte.
9194 4.35.1 - 2019-09-09
9196 This patch improves the messaging that comes from invalid size argu‐
9197 ments to collection strategies such as lists().
9198 4.35.0 - 2019-09-04
9200 This release improves the from_lark() strategy, tightening argument
9201 validation and adding the explicit argument to allow use with terminals
9202 that use @declare instead of a string or regular expression.
9203 This feature is required to handle features such as indent and dedent
9205 tokens in Python code, which can be generated with the hypothesmith
9206 package.
9207 4.34.0 - 2019-08-23
9209 The from_type() strategy now knows to look up the subclasses of ab‐
9210 stract types, which cannot be instantiated directly.
9211 This is very useful for hypothesmith to support libCST.
9213 4.33.1 - 2019-08-21
9215 This patch works around a crash when an incompatible version of Numpy
9216 is installed under PyPy 5.10 (Python 2.7).
9217 If you are still using Python 2, please upgrade to Python 3 as soon as
9219 possible - it will be unsupported at the end of this year.
9220 4.33.0 - 2019-08-20
9222 This release improves the domains() strategy, as well as the urls() and
9223 the emails() strategies which use it. These strategies now use the
9224 full IANA list of Top Level Domains and are correct as per RFC 1035.
9225 Passing tests using these strategies may now fail.
9227 Thanks to TechDragon for this improvement.
9229 4.32.3 - 2019-08-05
9231 This patch tidies up the repr of several settings-related objects, at
9232 runtime and in the documentation, and deprecates the undocumented edge
9233 case that phases=None was treated like phases=tuple(Phase).
9234 It also fixes from_lark() with lark 0.7.2 and later.
9236 4.32.2 - 2019-07-30
9238 This patch updates some internal comments for mypy 0.720. There is no
9239 user-visible impact.
9240 4.32.1 - 2019-07-29
9242 This release changes how the shrinker represents its progress inter‐
9243 nally. For large generated test cases this should result in signifi‐
9244 cantly less memory usage and possibly faster shrinking. Small generated
9245 test cases may be slightly slower to shrink but this shouldn't be very
9246 noticeable.
9247 4.32.0 - 2019-07-28
9249 This release makes arrays() more pedantic about elements strategies
9250 that cannot be exactly represented as array elements.
9251 In practice, you will see new warnings if you were using a float16 or
9253 float32 dtype without passing floats() the width=16 or width=32 argu‐
9254 ments respectively.
9255 The previous behaviour could lead to silent truncation, and thus some
9257 elements being equal to an explicitly excluded bound (issue #1899).
9258 4.31.1 - 2019-07-28
9260 This patch changes an internal use of MD5 to SHA hashes, to better sup‐
9261 port users subject to FIPS-140. There is no user-visible or API
9262 change.
9263 Thanks to Alex Gaynor for this patch.
9265 4.31.0 - 2019-07-24
9267 This release simplifies the logic of the print_blob setting by removing
9268 the option to set it to PrintSettings.INFER. As a result the
9269 print_blob setting now takes a single boolean value, and the use of
9270 PrintSettings is deprecated.
9271 4.28.2 - 2019-07-14
9273 This patch improves the docstrings of several Hypothesis strategies, by
9274 clarifying markup and adding cross-references. There is no runtime
9275 change.
9276 Thanks to Elizabeth Williams and Serah Njambi Rono for their contribu‐
9278 tions at the SciPy 2019 sprints!
9279 4.28.1 - 2019-07-12
9281 This patch improves the behaviour of the text() strategy when passed an
9282 alphabet which is not a strategy. The value is now interpreted as
9283 whitelist_characters to characters() instead of a sequence for
9284 sampled_from(), which standardises the distribution of examples and the
9285 shrinking behaviour.
9286 You can get the previous behaviour by using lists(sampled_from(alpha‐
9288 bet)).map("".map) instead.
9289 4.28.0 - 2019-07-11
9291 This release deprecates find(). The .example() method is a better re‐
9292 placement if you want an example, and for the rare occasions where you
9293 want the minimal example you can get it from @given.
9294 @given has steadily outstripped find() in both features and performance
9296 over recent years, and as we do not have the resources to maintain and
9297 test both we think it is better to focus on just one.
9298 4.27.0 - 2019-07-08
9300 This release refactors the implementation of the .example() method, to
9301 more accurately represent the data which will be generated by @given.
9302 As a result, calling s.example() on an empty strategy s (such as
9304 nothing()) now raises Unsatisfiable instead of the old NoExamples ex‐
9305 ception.
9306 4.26.4 - 2019-07-07
9308 This patch ensures that the Pandas extra will keep working when Python
9309 3.8 removes abstract base classes from the top-level python:collections
9310 namespace. This also fixes the relevant warning in Python 3.7, but
9311 there is no other difference in behaviour and you do not need to do
9312 anything.
9313 4.26.3 - 2019-07-05
9315 This release fixes issue #2027, by changing the way Hypothesis tries
9316 to generate distinct examples to be more efficient.
9317 This may result in slightly different data distribution, and should im‐
9319 prove generation performance in general, but should otherwise have min‐
9320 imal user impact.
9321 4.26.2 - 2019-07-04
9323 This release fixes issue #1864, where some simple tests would perform
9324 very slowly, because they would run many times with each subsequent run
9325 being progressively slower. They will now stop after a more reasonable
9326 number of runs without hitting this problem.
9327 Unless you are hitting exactly this issue, it is unlikely that this re‐
9329 lease will have any effect, but certain classes of custom generators
9330 that are currently very slow may become a bit faster, or start to trig‐
9331 ger health check failures.
9332 4.26.1 - 2019-07-04
9334 This release adds the strategy integer_array_indices(), which generates
9335 tuples of Numpy arrays that can be used for advanced indexing to select
9336 an array of a specified shape.
9337 4.26.0 - 2019-07-04
9339 This release significantly improves the performance of drawing unique
9340 collections whose elements are drawn from sampled_from() strategies.
9341 As a side effect, this detects an error condition that would previously
9343 have passed silently: When the min_size argument on a collection with
9344 distinct elements is greater than the number of elements being sampled,
9345 this will now raise an error.
9346 4.25.1 - 2019-07-03
9348 This release removes some defunct internal functionality that was only
9349 being used for testing. It should have no user visible impact.
9350 4.25.0 - 2019-07-03
9352 This release deprecates and disables the buffer_size setting, which
9353 should have been treated as a private implementation detail all along.
9354 We recommend simply deleting this settings argument.
9355 4.24.6 - 2019-06-26
9357 This patch makes datetimes() more efficient, as it now handles short
9358 months correctly by construction instead of filtering.
9359 4.24.5 - 2019-06-23
9361 This patch improves the development experience by simplifying the
9362 tracebacks you will see when e.g. you have used the .map(...) method of
9363 a strategy and the mapped function raises an exception.
9364 No new exceptions can be raised, nor existing exceptions change any‐
9366 thing but their traceback. We're simply using if-statements rather
9367 than exceptions for control flow in a certain part of the internals!
9368 4.24.4 - 2019-06-21
9370 This patch fixes issue #2014, where our compatibility layer broke with
9371 version 3.7.4 of the typing module backport on PyPI.
9372 This issue only affects Python 2. We remind users that Hypothesis,
9374 like many other packages, will drop Python 2 support on 2020-01-01 and
9375 already has several features that are only available on Python 3.
9376 4.24.3 - 2019-06-07
9378 This patch improves the implementation of an internal wrapper on Python
9379 3.8 beta1 (and will break on the alphas; but they're not meant to be
9380 stable). On other versions, there is no change at all.
9381 Thanks to Daniel Hahler for the patch, and Victor Stinner for his work
9383 on bpo-37032 that made it possible.
9384 4.24.2 - 2019-06-06
9386 Deprecation messages for functions in hypothesis.extra.django.models
9387 now explicitly name the deprecated function to make it easier to track
9388 down usages. Thanks to Kristian Glass for this contribution!
9389 4.24.1 - 2019-06-04
9391 This patch fixes issue #1999, a spurious bug raised when a
9392 @st.composite function was passed a keyword-only argument.
9393 Thanks to Jim Nicholls for his fantastic bug report.
9395 4.24.0 - 2019-05-29
9397 This release deprecates GenericStateMachine, in favor of
9398 RuleBasedStateMachine. Rule-based stateful testing is significantly
9399 faster, especially during shrinking.
9400 If your use-case truly does not fit rule-based stateful testing, we
9402 recommend writing a custom test function which drives your specific
9403 control-flow using data().
9404 4.23.9 - 2019-05-28
9406 This patch fixes a very rare example database issue with file permis‐
9407 sions.
9408 When running a test that uses both @given and pytest.mark.parametrize,
9410 using pytest-xdist on Windows, with failing examples in the database,
9411 two attempts to read a file could overlap and we caught FileNotFound
9412 but not other OSErrors.
9413 4.23.8 - 2019-05-26
9415 This patch has a minor cleanup of the internal engine. There is no
9416 user-visible impact.
9417 4.23.7 - 2019-05-26
9419 This patch clarifies some error messages when the test function signa‐
9420 ture is incompatible with the arguments to @given, especially when the
9421 @settings() decorator is also used (issue #1978).
9422 4.23.6 - 2019-05-19
9424 This release adds the pyupgrade fixer to our code style, for consistent
9425 use of dict and set literals and comprehensions.
9426 4.23.5 - 2019-05-16
9428 This release slightly simplifies a small part of the core engine.
9429 There is no user-visible change.
9430 4.23.4 - 2019-05-09
9432 Fixes a minor formatting issue the docstring of from_type()
9433 4.23.3 - 2019-05-09
9435 Adds a recipe to the docstring of from_type() that describes a means
9436 for drawing values for "everything except" a specified type. This
9437 recipe is especially useful for writing tests that perform input-type
9438 validation.
9439 4.23.2 - 2019-05-08
9441 This patch uses autoflake to remove some pointless pass statements,
9442 which improves our workflow but has no user-visible impact.
9443 4.23.1 - 2019-05-08
9445 This patch fixes an OverflowError in from_type(xrange) on Python 2.
9446 It turns out that not only do the start and stop values have to fit in
9448 a C long, but so does stop - start. We now handle this even on 32bit
9449 platforms, but remind users that Python2 will not be supported after
9450 2019 without specific funding.
9451 4.23.0 - 2019-05-08
9453 This release implements the slices() strategy, to generate slices of a
9454 length-size sequence.
9455 Thanks to Daniel J. West for writing this patch at the PyCon 2019
9457 sprints!
9458 4.22.3 - 2019-05-07
9460 This patch exposes DataObject, solely to support more precise type
9461 hints. Objects of this type are provided by data(), and can be used to
9462 draw examples from strategies intermixed with your test code.
9463 4.22.2 - 2019-05-07
9465 This patch fixes the very rare issue #1798 in array_dtypes(), which
9466 caused an internal error in our tests.
9467 4.22.1 - 2019-05-07
9469 This patch fixes a rare bug in from_type(range).
9470 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
9472 Sprints.
9473 4.22.0 - 2019-05-07
9475 The unique_by argument to lists now accepts a tuple of callables such
9476 that every element of the generated list will be unique with respect to
9477 each callable in the tuple (issue #1916).
9478 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
9480 4.21.1 - 2019-05-06
9482 This patch cleans up the internals of one_of(). You may see a slight
9483 change to the distribution of examples from this strategy but there is
9484 no change to the public API.
9485 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
9487 sprints!
9488 4.21.0 - 2019-05-05
9490 The from_type() strategy now supports python:slice objects.
9491 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
9493 Mentored Sprints.
9494 4.20.0 - 2019-05-05
9496 This release improves the array_shapes() strategy, to choose an appro‐
9497 priate default for max_side based on the min_side, and max_dims based
9498 on the min_dims. An explicit error is raised for dimensions greater
9499 than 32, which are not supported by Numpy, as for other invalid combi‐
9500 nations of arguments.
9501 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
9503 tored Sprints.
9504 4.19.0 - 2019-05-05
9506 The from_type() strategy now supports python:range objects (or xrange
9507 on Python 2).
9508 Thanks to Katrina Durance for writing this feature at the PyCon 2019
9510 Mentored Sprints.
9511 4.18.3 - 2019-04-30
9513 This release fixes a very rare edge case in the test-case mutator,
9514 which could cause an internal error with certain unusual tests.
9515 4.18.2 - 2019-04-30
9517 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
9518 changed the representation of code objects to support positional-only
9519 arguments. Note however that Hypothesis does not (yet) support such
9520 functions as e.g. arguments to builds() or inputs to @given.
9521 Thanks to Paul Ganssle for identifying and fixing this bug.
9523 4.18.1 - 2019-04-29
9525 This patch improves the performance of unique collections such as
9526 sets() when the elements are drawn from a sampled_from() strategy (‐
9527 issue #1115).
9528 4.18.0 - 2019-04-24
9530 This release adds the functions() strategy, which can be used to imi‐
9531 tate your 'real' function for callbacks.
9532 4.17.2 - 2019-04-19
9534 This release refactors stateful rule selection to share the new machin‐
9535 ery with sampled_from() instead of using the original independent im‐
9536 plementation.
9537 4.17.1 - 2019-04-16
9539 This patch allows Hypothesis to try a few more examples after finding
9540 the first bug, in hopes of reporting multiple distinct bugs. The
9541 heuristics described in issue #847 ensure that we avoid wasting time on
9542 fruitless searches, while still surfacing each bug as soon as possible.
9543 4.17.0 - 2019-04-16
9545 This release adds the strategy broadcastable_shapes(), which generates
9546 array shapes that are broadcast-compatible with a provided shape.
9547 4.16.0 - 2019-04-12
9549 This release allows register_type_strategy() to be used with
9550 python:typing.NewType instances. This may be useful to e.g. provide
9551 only positive integers for from_type(UserId) with a UserId = New‐
9552 Type('UserId', int) type.
9553 Thanks to PJCampi for suggesting and writing the patch!
9555 4.15.0 - 2019-04-09
9557 This release supports passing a timedelta as the deadline setting, so
9558 you no longer have to remember that the number is in milliseconds (‐
9559 issue #1900).
9560 Thanks to Damon Francisco for this change!
9562 4.14.7 - 2019-04-09
9564 This patch makes the type annotations on hypothesis.extra.dateutil com‐
9565 patible with mypy 0.700.
9566 4.14.6 - 2019-04-07
9568 This release fixes a bug introduced in Hypothesis 4.14.3 that would
9569 sometimes cause sampled_from(...).filter(...) to encounter an internal
9570 assertion failure when there are three or fewer elements, and every el‐
9571 ement is rejected by the filter.
9572 4.14.5 - 2019-04-05
9574 This patch takes the previous efficiency improvements to
9575 sampled_from(...).filter(...) strategies that reject most elements,
9576 and generalises them to also apply to sampled_from(...).fil‐
9577 ter(...).filter(...) and longer chains of filters.
9578 4.14.4 - 2019-04-05
9580 This release fixes a bug that prevented random_module() from correctly
9581 restoring the previous state of the random module.
9582 The random state was instead being restored to a temporary determinis‐
9584 tic state, which accidentally caused subsequent tests to see the same
9585 random values across multiple test runs.
9586 4.14.3 - 2019-04-03
9588 This patch adds an internal special case to make
9589 sampled_from(...).filter(...) much more efficient when the filter re‐
9590 jects most elements (issue #1885).
9591 4.14.2 - 2019-03-31
9593 This patch improves the error message if the function f in s.flatmap(f)
9594 does not return a strategy.
9595 Thanks to Kai Chen for this change!
9597 4.14.1 - 2019-03-30
9599 This release modifies how Hypothesis selects operations to run during
9600 shrinking, by causing it to deprioritise previously useless classes of
9601 shrink until others have reached a fixed point.
9602 This avoids certain pathological cases where the shrinker gets very
9604 close to finishing and then takes a very long time to finish the last
9605 small changes because it tries many useless shrinks for each useful one
9606 towards the end. It also should cause a more modest improvement (prob‐
9607 ably no more than about 30%) in shrinking performance for most tests.
9608 4.14.0 - 2019-03-19
9610 This release blocks installation of Hypothesis on Python 3.4, which
9611 reached its end of life date on 2019-03-18.
9612 This should not be of interest to anyone but downstream maintainers -
9614 if you are affected, migrate to a secure version of Python as soon as
9615 possible or at least seek commercial support.
9616 4.13.0 - 2019-03-19
9618 This release makes it an explicit error to call floats(min_value=inf,
9619 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
9620 are no possible values that can be generated (issue #1859).
9621 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
9623 -0. and we could thus generate either if comparing by value, violating
9624 the sequence ordering of floats is a special case we don't want or
9625 need.
9626 4.12.1 - 2019-03-18
9628 This release should significantly reduce the amount of memory that Hy‐
9629 pothesis uses for representing large test cases, by storing information
9630 in a more compact representation and only unpacking it lazily when it
9631 is first needed.
9632 4.12.0 - 2019-03-18
9634 This update adds the report_multiple_bugs setting, which you can use to
9635 disable multi-bug reporting and only raise whichever bug had the small‐
9636 est minimal example. This is occasionally useful when using a debugger
9637 or tools that annotate tracebacks via introspection.
9638 4.11.7 - 2019-03-18
9640 This change makes a tiny improvement to the core engine's bookkeeping.
9641 There is no user-visible change.
9642 4.11.6 - 2019-03-15
9644 This release changes some of Hypothesis's internal shrinking behaviour
9645 in order to reduce memory usage and hopefully improve performance.
9646 4.11.5 - 2019-03-13
9648 This release adds a micro-optimisation to how Hypothesis handles debug
9649 reporting internally. Hard to shrink test may see a slight performance
9650 improvement, but in most common scenarios it is unlikely to be notice‐
9651 able.
9652 4.11.4 - 2019-03-13
9654 This release removes some redundant code that was no longer needed but
9655 was still running a significant amount of computation and allocation on
9656 the hot path. This should result in a modest speed improvement for
9657 most tests, especially those with large test cases.
9658 4.11.3 - 2019-03-13
9660 This release adds a micro-optimisation to how Hypothesis caches test
9661 cases. This will cause a small improvement in speed and memory usage
9662 for large test cases, but in most common scenarios it is unlikely to be
9663 noticeable.
9664 4.11.2 - 2019-03-13
9666 This release removes some internal code that populates a field that is
9667 no longer used anywhere. This should result in some modest performance
9668 and speed improvements and no other user visible effects.
9669 4.11.1 - 2019-03-13
9671 This is a formatting-only patch, enabled by a new version of isort.
9672 4.11.0 - 2019-03-12
9674 This release deprecates sampled_from() with empty sequences. This re‐
9675 turns nothing(), which gives a clear error if used directly... but sim‐
9676 ply vanishes if combined with another strategy.
9677 Tests that silently generate less than expected are a serious problem
9679 for anyone relying on them to find bugs, and we think reliability more
9680 important than convenience in this case.
9681 4.10.0 - 2019-03-11
9683 This release improves Hypothesis's to detect flaky tests, by noticing
9684 when the behaviour of the test changes between runs. In particular
9685 this will notice many new cases where data generation depends on exter‐
9686 nal state (e.g. external sources of randomness) and flag those as flaky
9687 sooner and more reliably.
9688 The basis of this feature is a considerable reengineering of how Hy‐
9690 pothesis stores its history of test cases, so on top of this its memory
9691 usage should be considerably reduced.
9692 4.9.0 - 2019-03-09
9694 This release adds the strategy valid_tuple_axes(), which generates tu‐
9695 ples of axis-indices that can be passed to the axis argument in NumPy's
9696 sequential functions (e.g. numpy:numpy.sum()).
9697 Thanks to Ryan Soklaski for this strategy.
9699 4.8.0 - 2019-03-06
9701 This release significantly tightens validation in hypothesis.settings.
9702 max_examples, buffer_size, and stateful_step_count must be positive in‐
9703 tegers; deadline must be a positive number or None; and derandomize
9704 must be either True or False.
9705 As usual, this replaces existing errors with a more helpful error and
9707 starts new validation checks as deprecation warnings.
9708 4.7.19 - 2019-03-04
9710 This release makes some micro-optimisations to certain calculations
9711 performed in the shrinker. These should particularly speed up large
9712 test cases where the shrinker makes many small changes. It will also
9713 reduce the amount allocated, but most of this is garbage that would
9714 have been immediately thrown away, so you probably won't see much ef‐
9715 fect specifically from that.
9716 4.7.18 - 2019-03-03
9718 This patch removes some overhead from arrays() with a constant shape
9719 and dtype. The resulting performance improvement is modest, but worth‐
9720 while for small arrays.
9721 4.7.17 - 2019-03-01
9723 This release makes some micro-optimisations within Hypothesis's inter‐
9724 nal representation of test cases. This should cause heavily nested
9725 test cases to allocate less during generation and shrinking, which
9726 should speed things up slightly.
9727 4.7.16 - 2019-02-28
9729 This changes the order in which Hypothesis runs certain operations dur‐
9730 ing shrinking. This should significantly decrease memory usage and
9731 speed up shrinking of large examples.
9732 4.7.15 - 2019-02-28
9734 This release allows Hypothesis to calculate a number of attributes of
9735 generated test cases lazily. This should significantly reduce memory
9736 usage and modestly improve performance, especially for large test
9737 cases.
9738 4.7.14 - 2019-02-28
9740 This release reduces the number of operations the shrinker will try
9741 when reordering parts of a test case. This should in some circum‐
9742 stances significantly speed up shrinking. It may result in different
9743 final test cases, and if so usually slightly worse ones, but it should
9744 not generally have much impact on the end result as the operations re‐
9745 moved were typically useless.
9746 4.7.13 - 2019-02-27
9748 This release changes how Hypothesis reorders examples within a test
9749 case during shrinking. This should make shrinking considerably faster.
9750 4.7.12 - 2019-02-27
9752 This release slightly improves the shrinker's ability to replace parts
9753 of a test case with their minimal version, by allowing it to do so in
9754 bulk rather than one at a time. Where this is effective, shrinker per‐
9755 formance should be modestly improved.
9756 4.7.11 - 2019-02-25
9758 This release makes some micro-optimisations to common operations per‐
9759 formed during shrinking. Shrinking should now be slightly faster, es‐
9760 pecially for large examples with relatively fast test functions.
9761 4.7.10 - 2019-02-25
9763 This release is a purely internal refactoring of Hypothesis's API for
9764 representing test cases. There should be no user visible effect.
9765 4.7.9 - 2019-02-24
9767 This release changes certain shrink passes to make them more efficient
9768 when they aren't making progress.
9769 4.7.8 - 2019-02-23
9771 This patch removes some unused code, which makes the internals a bit
9772 easier to understand. There is no user-visible impact.
9773 4.7.7 - 2019-02-23
9775 This release reduces the number of operations the shrinker will try
9776 when reordering parts of a test case. This should in some circum‐
9777 stances significantly speed up shrinking. It may result in different
9778 final test cases, and if so usually slightly worse ones, but it should
9779 not generally have much impact on the end result as the operations re‐
9780 moved were typically useless.
9781 4.7.6 - 2019-02-23
9783 This patch removes some unused code from the shrinker. There is no
9784 user-visible change.
9785 4.7.5 - 2019-02-23
9787 This release changes certain shrink passes to make them adaptive - that
9788 is, in cases where they are successfully making progress they may now
9789 do so significantly faster.
9790 4.7.4 - 2019-02-22
9792 This is a docs-only patch, noting that because the lark-parser is under
9793 active development at version 0.x, hypothesis[lark] APIs may break in
9794 minor releases if necessary to keep up with the upstream package.
9795 4.7.3 - 2019-02-22
9797 This changes Hypothesis to no longer import various test frameworks by
9798 default (if they are installed). which will speed up the initial im‐
9799 port hypothesis call.
9800 4.7.2 - 2019-02-22
9802 This release changes Hypothesis's internal representation of a test
9803 case to calculate some expensive structural information on demand
9804 rather than eagerly. This should reduce memory usage a fair bit, and
9805 may make generation somewhat faster.
9806 4.7.1 - 2019-02-21
9808 This release refactors the internal representation of previously run
9809 test cases. The main thing you should see as a result is that Hypothe‐
9810 sis becomes somewhat less memory hungry.
9811 4.7.0 - 2019-02-21
9813 This patch allows array_shapes() to generate shapes with side-length or
9814 even dimension zero, though the minimum still defaults to one. These
9815 shapes are rare and have some odd behavior, but are particularly impor‐
9816 tant to test for just that reason!
9817 In a related bigfix, arrays() now supports generating zero-dimensional
9819 arrays with dtype=object and a strategy for iterable elements. Previ‐
9820 ously, the array element would incorrectly be set to the first item in
9821 the generated iterable.
9822 Thanks to Ryan Turner for continuing to improve our Numpy support.
9824 4.6.1 - 2019-02-19
9826 This release is a trivial micro-optimisation inside Hypothesis which
9827 should result in it using significantly less memory.
9828 4.6.0 - 2019-02-18
9830 This release changes some inconsistent behavior of arrays() from the
9831 Numpy extra when asked for an array of shape=(). arrays() will now al‐
9832 ways return a Numpy ndarray, and the array will always be of the re‐
9833 quested dtype.
9834 Thanks to Ryan Turner for this change.
9836 4.5.12 - 2019-02-18
9838 This release fixes a minor typo in an internal comment. There is no
9839 user-visible change.
9840 4.5.11 - 2019-02-15
9842 This release fixes issue #1813, a bug introduced in 3.59.1, which
9843 caused random_module() to no longer affect the body of the test: Al‐
9844 though Hypothesis would claim to be seeding the random module in fact
9845 tests would always run with a seed of zero.
9846 4.5.10 - 2019-02-14
9848 This patch fixes an off-by-one error in the maximum length of emails().
9849 Thanks to Krzysztof Jurewicz for pull request #1812.
9850 4.5.9 - 2019-02-14
9852 This patch removes some unused code from the shrinker. There is no
9853 user-visible change.
9854 4.5.8 - 2019-02-12
9856 This release fixes an internal IndexError in Hypothesis that could
9857 sometimes be triggered during shrinking.
9858 4.5.7 - 2019-02-11
9860 This release modifies the shrinker to interleave different types of re‐
9861 duction operations, e.g. switching between deleting data and lowering
9862 scalar values rather than trying entirely deletions then entirely low‐
9863 ering.
9864 This may slow things down somewhat in the typical case, but has the ma‐
9866 jor advantage that many previously difficult to shrink examples should
9867 become much faster, because the shrinker will no longer tend to stall
9868 when trying some ineffective changes to the shrink target but will in‐
9869 stead interleave it with other more effective operations.
9870 4.5.6 - 2019-02-11
9872 This release makes a number of internal changes to the implementation
9873 of hypothesis.extra.lark.from_lark(). These are primarily intended as
9874 a refactoring, but you may see some minor improvements to performance
9875 when generating large strings, and possibly to shrink quality.
9876 4.5.5 - 2019-02-10
9878 This patch prints an explanatory note when issue #1798 is triggered,
9879 because the error message from Numpy is too terse to locate the prob‐
9880 lem.
9881 4.5.4 - 2019-02-08
9883 In Python 2, long integers are not allowed in the shape argument to
9884 arrays(). Thanks to Ryan Turner for fixing this.
9885 4.5.3 - 2019-02-08
9887 This release makes a small internal refactoring to clarify how Hypothe‐
9888 sis instructs tests to stop running when appropriate. There is no
9889 user-visible change.
9890 4.5.2 - 2019-02-06
9892 This release standardises all of the shrinker's internal operations on
9893 running in a random order.
9894 The main effect you will see from this that it should now be much less
9896 common for the shrinker to stall for a long time before making further
9897 progress. In some cases this will correspond to shrinking more slowly,
9898 but on average it should result in faster shrinking.
9899 4.5.1 - 2019-02-05
9901 This patch updates some docstrings, but has no runtime changes.
9902 4.5.0 - 2019-02-03
9904 This release adds exclude_min and exclude_max arguments to floats(), so
9905 that you can easily generate values from open or half-open intervals (‐
9906 issue #1622).
9907 4.4.6 - 2019-02-03
9909 This patch fixes a bug where from_regex() could throw an internal error
9910 if the python:re.IGNORECASE flag was used (issue #1786).
9911 4.4.5 - 2019-02-02
9913 This release removes two shrink passes that Hypothesis runs late in the
9914 process. These were very expensive when the test function was slow and
9915 often didn't do anything useful.
9916 Shrinking should get faster for most failing tests. If you see any re‐
9918 gression in example quality as a result of this release, please let us
9919 know.
9920 4.4.4 - 2019-02-02
9922 This release modifies the way that Hypothesis deletes data during
9923 shrinking. It will primarily be noticeable for very large examples,
9924 which should now shrink faster.
9925 The shrinker is now also able to perform some deletions that it could
9927 not previously, but this is unlikely to be very noticeable.
9928 4.4.3 - 2019-01-25
9930 This release fixes an open file leak that used to cause ResourceWarn‐
9931 ings.
9932 4.4.2 - 2019-01-24
9934 This release changes Hypothesis's internal approach to caching the re‐
9935 sults of executing test cases. The result should be that it is now
9936 significantly less memory hungry, especially when shrinking large test
9937 cases.
9938 Some tests may get slower or faster depending on whether the new or old
9940 caching strategy was well suited to them, but any change in speed in
9941 either direction should be minor.
9942 4.4.1 - 2019-01-24
9944 This patch tightens up some of our internal heuristics to deal with
9945 shrinking floating point numbers, which will now run in fewer circum‐
9946 stances.
9947 You are fairly unlikely to see much difference from this, but if you do
9949 you are likely to see shrinking become slightly faster and/or producing
9950 slightly worse results.
9951 4.4.0 - 2019-01-24
9953 This release adds the from_form() function, which allows automatic
9954 testing against Django forms. (issue #35)
9955 Thanks to Paul Stiverson for this feature, which resolves our oldest
9957 open issue!
9958 4.3.0 - 2019-01-24
9960 This release deprecates HealthCheck.hung_test and disables the associ‐
9961 ated runtime check for tests that ran for more than five minutes. Such
9962 a check is redundant now that we enforce the deadline and max_examples
9963 setting, which can be adjusted independently.
9964 4.2.0 - 2019-01-23
9966 This release adds a new module, hypothesis.extra.lark, which you can
9967 use to generate strings matching a context-free grammar.
9968 In this initial version, only lark-parser EBNF grammars are supported,
9970 by the new hypothesis.extra.lark.from_lark() function.
9971 4.1.2 - 2019-01-23
9973 This patch fixes a very rare overflow bug (issue #1748) which could
9974 raise an InvalidArgument error in complex_numbers() even though the ar‐
9975 guments were valid.
9976 4.1.1 - 2019-01-23
9978 This release makes some improvements to internal code organisation and
9979 documentation and has no impact on behaviour.
9980 4.1.0 - 2019-01-22
9982 This release adds register_random(), which registers random.Random in‐
9983 stances or compatible objects to be seeded and reset by Hypothesis to
9984 ensure that test cases are deterministic.
9985 We still recommend explicitly passing a random.Random instance from
9987 randoms() if possible, but registering a framework-global state for Hy‐
9988 pothesis to manage is better than flaky tests!
9989 4.0.2 - 2019-01-22
9991 This patch fixes issue #1387, where bounded integers() with a very
9992 large range would almost always generate very large numbers. Now, we
9993 usually use the same tuned distribution as unbounded integers().
9994 4.0.1 - 2019-01-16
9996 This release randomizes the order in which the shrinker tries some of
9997 its initial normalization operations. You are unlikely to see much
9998 difference as a result unless your generated examples are very large.
9999 In this case you may see some performance improvements in shrinking.
10000 4.0.0 - 2019-01-14
10002 Welcome to the next major version of Hypothesis!
10003 There are no new features here, as we release those in minor versions.
10005 Instead, 4.0 is a chance for us to remove deprecated features (many al‐
10006 ready converted into no-ops), and turn a variety of warnings into er‐
10007 rors.
10008 If you were running on the last version of Hypothesis 3.x without any
10010 Hypothesis deprecation warnings (or using private APIs), this will be a
10011 very boring upgrade. In fact, nothing will change for you at all. Per
10012 our deprecation policy, warnings added in the last six months (after
10013 2018-07-05) have not been converted to errors.
10014 Removals
10016 • hypothesis.extra.datetime has been removed, replaced by the core date
10017 and time strategies.
10018 • hypothesis.extra.fakefactory has been removed, replaced by general
10020 expansion of Hypothesis' strategies and the third-party ecosystem.
10021 • The SQLite example database backend has been removed.
10023 Settings
10025 • The deadline is now enforced by default, rather than just emitting a
10026 warning when the default (200 milliseconds per test case) deadline is
10027 exceeded.
10028 • The database_file setting has been removed; use database.
10030 • The perform_health_check setting has been removed; use
10032 suppress_health_check.
10033 • The max_shrinks setting has been removed; use phases to disable
10035 shrinking.
10036 • The min_satisfying_examples, max_iterations, strict, timeout, and
10038 use_coverage settings have been removed without user-configurable re‐
10039 placements.
10040 Strategies
10042 • The elements argument is now required for collection strategies.
10043 • The average_size argument was a no-op and has been removed.
10045 • Date and time strategies now only accept min_value and max_value for
10047 bounds.
10048 • builds() now requires that the thing to build is passed as the first
10050 positional argument.
10051 • Alphabet validation for text() raises errors, not warnings, as does
10053 category validation for characters().
10054 • The choices() strategy has been removed. Instead, you can use data()
10056 with sampled_from(), so choice(elements) becomes data.draw(sam‐
10057 pled_from(elements)).
10058 • The streaming() strategy has been removed. Instead, you can use
10060 data() and replace iterating over the stream with data.draw() calls.
10061 • sampled_from() and permutations() raise errors instead of warnings if
10063 passed a collection that is not a sequence.
10064 Miscellaneous
10066 • Applying @given to a test function multiple times was really ineffi‐
10067 cient, and now it's also an error.
10068 • Using the .example() method of a strategy (intended for interactive
10070 exploration) within another strategy or a test function always weak‐
10071 ened data generation and broke shrinking, and now it's an error too.
10072 • The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
10074 ported, as the database_file setting has been removed.
10075 • The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
10077 ported. You can use the --hypothesis-verbosity pytest argument in‐
10078 stead, or write your own setup code using the settings profile system
10079 to replace it.
10080 • Using @seed or derandomize=True now forces database=None to ensure
10082 results are in fact reproducible. If database is not None, doing so
10083 also emits a HypothesisWarning.
10084 • Unused exception types have been removed from hypothesis.errors;
10086 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
10087 ple, Timeout, and WrongFormat.
10088 Hypothesis 3.x
10090 3.88.3 - 2019-01-11
10091 This changes the order that the shrinker tries certain operations in
10092 its "emergency" phase which runs late in the process. The new order
10093 should be better at avoiding long stalls where the shrinker is failing
10094 to make progress, which may be helpful if you have difficult to shrink
10095 test cases. However this will not be noticeable in the vast majority
10096 of use cases.
10097 3.88.2 - 2019-01-11
10099 This is a pure refactoring release that extracts some logic from the
10100 core Hypothesis engine into its own class and file. It should have no
10101 user visible impact.
10102 3.88.1 - 2019-01-11
10104 This patch fixes some markup in our documentation.
10105 3.88.0 - 2019-01-10
10107 Introduces hypothesis.stateful.multiple(), which allows rules in rule
10108 based state machines to send multiple results at once to their target
10109 Bundle, or none at all.
10110 3.87.0 - 2019-01-10
10112 This release contains a massive cleanup of the Hypothesis for Django
10113 extra:
10114 • hypothesis.extra.django.models.models() is deprecated in favor of
10116 hypothesis.extra.django.from_model().
10117 • hypothesis.extra.django.models.add_default_field_mapping() is depre‐
10119 cated in favor of hypothesis.extra.django.register_field_strategy().
10120 • from_model() does not infer a strategy for nullable fields or fields
10122 with a default unless passed infer, like builds(). models.models()
10123 would usually but not always infer, and a special default_value
10124 marker object was required to disable inference.
10125 3.86.9 - 2019-01-09
10127 This release improves some internal logic about when a test case in Hy‐
10128 pothesis's internal representation could lead to a valid test case. In
10129 some circumstances this can lead to a significant speed up during
10130 shrinking. It may have some minor negative impact on the quality of
10131 the final result due to certain shrink passes now having access to less
10132 information about test cases in some circumstances, but this should
10133 rarely matter.
10134 3.86.8 - 2019-01-09
10136 This release has no user visible changes but updates our URLs to use
10137 HTTPS.
10138 3.86.7 - 2019-01-08
10140 Hypothesis can now automatically generate values for Django models with
10141 a URLfield, thanks to a new provisional strategy for URLs (issue
10142 #1388).
10143 3.86.6 - 2019-01-07
10145 This release is a pure refactoring that extracts some internal code
10146 into its own file. It should have no user visible effect.
10147 3.86.5 - 2019-01-06
10149 This is a docs-only patch, which fixes some typos and removes a few hy‐
10150 perlinks for deprecated features.
10151 3.86.4 - 2019-01-04
10153 This release changes the order in which the shrinker tries to delete
10154 data. For large and slow tests this may significantly improve the per‐
10155 formance of shrinking.
10156 3.86.3 - 2019-01-04
10158 This release fixes a bug where certain places Hypothesis internal er‐
10159 rors could be raised during shrinking when a user exception occurred
10160 that suppressed an exception Hypothesis uses internally in its genera‐
10161 tion.
10162 The two known ways to trigger this problem were:
10164 • Errors raised in stateful tests' teardown function.
10166 • Errors raised in finally blocks that wrapped a call to data.draw.
10168 These cases will now be handled correctly.
10170 3.86.2 - 2019-01-04
10172 This patch is a docs-only change to fix a broken hyperlink.
10173 3.86.1 - 2019-01-04
10175 This patch fixes issue #1732, where integers() would always return long
10176 values on Python 2.
10177 3.86.0 - 2019-01-03
10179 This release ensures that infinite numbers are never generated by
10180 floats() with allow_infinity=False, which could previously happen in
10181 some cases where one bound was also provided.
10182 The trivially inconsistent min_value=inf, allow_infinity=False now
10184 raises an InvalidArgumentError, as does the inverse with max_value.
10185 You can still use just(inf) to generate inf without violating other
10186 constraints.
10187 3.85.3 - 2019-01-02
10189 Happy new year everyone! This release has no user visible changes but
10190 updates our copyright headers to include 2019.
10191 3.85.2 - 2018-12-31
10193 This release makes a small change to the way the shrinker works. You
10194 may see some improvements to speed of shrinking on especially large and
10195 hard to shrink examples, but most users are unlikely to see much dif‐
10196 ference.
10197 3.85.1 - 2018-12-30
10199 This patch fixes issue #1700, where a line that contained a Unicode
10200 character before a lambda definition would cause an internal exception.
10201 3.85.0 - 2018-12-29
10203 Introduces the hypothesis.stateful.consumes() function. When defining a
10204 rule in stateful testing, it can be used to mark bundles from which
10205 values should be consumed, i. e. removed after use in the rule. This
10206 has been proposed in issue #136.
10207 Thanks to Jochen Müller for this long-awaited feature.
10209 3.84.6 - 2018-12-28
10211 This patch makes a small internal change to fix an issue in Hypothe‐
10212 sis's own coverage tests (issue #1718).
10213 There is no user-visible change.
10215 3.84.5 - 2018-12-21
10217 This patch refactors the hypothesis.strategies module, so that private
10218 names should no longer appear in tab-completion lists. We previously
10219 relied on __all__ for this, but not all editors respect it.
10220 3.84.4 - 2018-12-21
10222 This is a follow-up patch to ensure that the deprecation date is auto‐
10223 matically recorded for any new deprecations. There is no user-visible
10224 effect.
10225 3.84.3 - 2018-12-20
10227 This patch updates the Hypothesis pytest plugin to avoid a recently
10228 deprecated hook interface. There is no user-visible change.
10229 3.84.2 - 2018-12-19
10231 This patch fixes the internals for integers() with one bound. Values
10232 from this strategy now always shrink towards zero instead of towards
10233 the bound, and should shrink much more efficiently too. On Python 2,
10234 providing a bound incorrectly excluded long integers, which can now be
10235 generated.
10236 3.84.1 - 2018-12-18
10238 This patch adds information about when features were deprecated, but
10239 this is only recorded internally and has no user-visible effect.
10240 3.84.0 - 2018-12-18
10242 This release changes the stateful testing backend from find() to use
10243 @given (issue #1300). This doesn't change how you create stateful
10244 tests, but does make them run more like other Hypothesis tests.
10245 @reproduce_failure and @seed now work for stateful tests.
10247 Stateful tests now respect the deadline and suppress_health_check set‐
10249 tings, though they are disabled by default. You can enable them by us‐
10250 ing @settings(...) as a class decorator with whatever arguments you
10251 prefer.
10252 3.83.2 - 2018-12-17
10254 Hypothesis has adopted Black as our code formatter (issue #1686).
10255 There are no functional changes to the source, but it's prettier!
10256 3.83.1 - 2018-12-13
10258 This patch increases the variety of examples generated by from_type().
10259 3.83.0 - 2018-12-12
10261 Our pytest plugin now warns you when strategy functions have been col‐
10262 lected as tests, which may happen when e.g. using the @composite deco‐
10263 rator when you should be using @given(st.data()) for inline draws.
10264 Such functions always pass when treated as tests, because the lazy cre‐
10265 ation of strategies mean that the function body is never actually exe‐
10266 cuted!
10267 3.82.6 - 2018-12-11
10269 Hypothesis can now show statistics when running under pytest-xdist.
10270 Previously, statistics were only reported when all tests were run in a
10271 single process (issue #700).
10272 3.82.5 - 2018-12-08
10274 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
10275 to integers() could cause errors on Python 3 due to internal rounding.
10276 3.82.4 - 2018-12-08
10278 Hypothesis now seeds and resets the global state of np.random for each
10279 test case, to ensure that tests are reproducible.
10280 This matches and complements the existing handling of the python:random
10282 module - Numpy simply maintains an independent PRNG for performance
10283 reasons.
10284 3.82.3 - 2018-12-08
10286 This is a no-op release to add the new Framework :: Hypothesis trove
10287 classifier to hypothesis on PyPI.
10288 You can use it as a filter to find Hypothesis-related packages such as
10290 extensions as they add the tag over the coming weeks, or simply visit
10291 our curated list.
10292 3.82.2 - 2018-12-08
10294 The Hypothesis for Pandas extension is now listed in setup.py, so you
10295 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
10296 tion.
10297 3.82.1 - 2018-10-29
10299 This patch fixes from_type() on Python 2 for classes where cls.__init__
10300 is object.__init__. Thanks to ccxcz for reporting issue #1656.
10301 3.82.0 - 2018-10-29
10303 The alphabet argument for text() now uses its default value of charac‐
10304 ters(blacklist_categories=('Cs',)) directly, instead of hiding that be‐
10305 hind alphabet=None and replacing it within the function. Passing None
10306 is therefore deprecated.
10307 3.81.0 - 2018-10-27
10309 GenericStateMachine and RuleBasedStateMachine now raise an explicit er‐
10310 ror when instances of settings are assigned to the classes' settings
10311 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
10312 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
10313 tor to handle this automatically.
10314 3.80.0 - 2018-10-25
10316 Since version 3.68.0, arrays() checks that values drawn from the ele‐
10317 ments and fill strategies can be safely cast to the dtype of the array,
10318 and emits a warning otherwise.
10319 This release expands the checks to cover overflow for finite complex64
10321 elements and string truncation caused by too-long elements or trailing
10322 null characters (issue #1591).
10323 3.79.4 - 2018-10-25
10325 Tests using @given now shrink errors raised from pytest helper func‐
10326 tions, instead of reporting the first example found.
10327 This was previously fixed in version 3.56.0, but only for stateful
10329 testing.
10330 3.79.3 - 2018-10-23
10332 Traceback elision is now disabled on Python 2, to avoid an import-time
10333 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
10334 3.79.2: issue #1648).
10335 We encourage all users to upgrade to Python 3 before the end of 2019.
10337 3.79.2 - 2018-10-23
10339 This patch shortens tracebacks from Hypothesis, so you can see exactly
10340 happened in your code without having to skip over irrelevant details
10341 about our internals (issue #848).
10342 In the example test (see pull request #1582), this reduces tracebacks
10344 from nine frames to just three - and for a test with multiple errors,
10345 from seven frames per error to just one!
10346 If you do want to see the internal details, you can disable frame eli‐
10348 sion by setting verbosity to debug.
10349 3.79.1 - 2018-10-22
10351 The abstract number classes Number, Complex, Real, Rational, and Inte‐
10352 gral are now supported by the from_type() strategy. Previously, you
10353 would have to use register_type_strategy() before they could be re‐
10354 solved (issue #1636)
10355 3.79.0 - 2018-10-18
10357 This release adds a CLI flag for verbosity --hypothesis-verbosity to
10358 the Hypothesis pytest plugin, applied after loading the profile speci‐
10359 fied by --hypothesis-profile. Valid options are the names of verbosity
10360 settings, quiet, normal, verbose or debug.
10361 Thanks to Bex Dunn for writing this patch at the PyCon Australia
10363 sprints!
10364 The pytest header now correctly reports the current profile if --hy‐
10366 pothesis-profile has been used.
10367 Thanks to Mathieu Paturel for the contribution at the Canberra Python
10369 Hacktoberfest.
10370 3.78.0 - 2018-10-16
10372 This release has deprecated the generation of integers, floats and
10373 fractions when the conversion of the upper and/ or lower bound is not
10374 100% exact, e.g. when an integer gets passed a bound that is not a
10375 whole number. (issue #1625)
10376 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
10378 3.77.0 - 2018-10-16
10380 This minor release adds functionality to settings allowing it to be
10381 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
10382 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
10384 3.76.1 - 2018-10-16
10386 This patch fixes some warnings added by recent releases of pydocstyle
10387 and mypy.
10388 3.76.0 - 2018-10-11
10390 This release deprecates using floats for min_size and max_size.
10391 The type hint for average_size arguments has been changed from Op‐
10393 tional[int] to None, because non-None values are always ignored and
10394 deprecated.
10395 3.75.4 - 2018-10-10
10397 This patch adds more internal comments to the core engine's se‐
10398 quence-length shrinker. There should be no user-visible change.
10399 3.75.3 - 2018-10-09
10401 This patch adds additional comments to some of the core engine's inter‐
10402 nal data structures. There is no user-visible change.
10403 3.75.2 - 2018-10-09
10405 This patch avoids caching a trivial case, fixing issue #493.
10406 3.75.1 - 2018-10-09
10408 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
10409 for this contribution!
10410 3.75.0 - 2018-10-08
10412 This release deprecates the use of min_size=None, setting the default
10413 min_size to 0 (issue #1618).
10414 3.74.3 - 2018-10-08
10416 This patch makes some small internal changes to comply with a new lint
10417 setting in the build. There should be no user-visible change.
10418 3.74.2 - 2018-10-03
10420 This patch fixes issue #1153, where time spent reifying a strategy was
10421 also counted in the time spent generating the first example. Strate‐
10422 gies are now fully constructed and validated before the timer is
10423 started.
10424 3.74.1 - 2018-10-03
10426 This patch fixes some broken formatting and links in the documentation.
10427 3.74.0 - 2018-10-01
10429 This release checks that the value of the print_blob setting is a
10430 PrintSettings instance.
10431 Being able to specify a boolean value was not intended, and is now dep‐
10433 recated. In addition, specifying True will now cause the blob to al‐
10434 ways be printed, instead of causing it to be suppressed.
10435 Specifying any value that is not a PrintSettings or a boolean is now an
10437 error.
10438 3.73.5 - 2018-10-01
10440 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
10441 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
10442 eter names min_value and max_value instead of the deprecated names
10443 3.73.4 - 2018-09-30
10445 This patch ensures that Hypothesis deprecation warnings display the
10446 code that emitted them when you're not running in -Werror mode (issue
10447 #652).
10448 3.73.3 - 2018-09-27
10450 Tracebacks involving @composite are now slightly shorter due to some
10451 internal refactoring.
10452 3.73.2 - 2018-09-26
10454 This patch fixes errors in the internal comments for one of the
10455 shrinker passes. There is no user-visible change.
10456 3.73.1 - 2018-09-25
10458 This patch substantially improves the distribution of data generated
10459 with recursive(), and fixes a rare internal error (issue #1502).
10460 3.73.0 - 2018-09-24
10462 This release adds the fulfill() function, which is designed for testing
10463 code that uses dpcontracts 0.4 or later for input validation. This
10464 provides some syntactic sugar around use of assume(), to automatically
10465 filter out and retry calls that cause a precondition check to fail (‐
10466 issue #1474).
10467 3.72.0 - 2018-09-24
10469 This release makes setting attributes of the hypothesis.settings class
10470 an explicit error. This has never had any effect, but could mislead
10471 users who confused it with the current settings instance hypothe‐
10472 sis.settings.default (which is also immutable). You can change the
10473 global settings with settings profiles.
10474 3.71.11 - 2018-09-24
10476 This patch factors out some common code in the shrinker for iterating
10477 over pairs of data blocks. There should be no user-visible change.
10478 3.71.10 - 2018-09-18
10480 This patch allows from_type() to handle the empty tuple type, typ‐
10481 ing.Tuple[()].
10482 3.71.9 - 2018-09-17
10484 This patch updates some internal comments for mypy. There is no
10485 user-visible effect, even for Mypy users.
10486 3.71.8 - 2018-09-17
10488 This patch fixes a rare bug that would cause a particular shrinker pass
10489 to raise an IndexError, if a shrink improvement changed the underlying
10490 data in an unexpected way.
10491 3.71.7 - 2018-09-17
10493 This release fixes the broken cross-references in our docs, and adds a
10494 CI check so we don't add new ones.
10495 3.71.6 - 2018-09-16
10497 This patch fixes two bugs (issue #944 and issue #1521), where messages
10498 about @seed did not check the current verbosity setting, and the wrong
10499 settings were active while executing explicit examples.
10500 3.71.5 - 2018-09-15
10502 This patch fixes a DeprecationWarning added in Python 3.8 (issue
10503 #1576).
10504 Thanks to tirkarthi for this contribution!
10506 3.71.4 - 2018-09-14
10508 This is a no-op release, which implements automatic DOI minting and
10509 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
10510 zon 2020 programme for providing this service!
10511 Check our CITATION.cff file for details, or head right on over to
10513 doi.org/10.5281/zenodo.1412597
10514 3.71.3 - 2018-09-10
10516 This release adds the test name to some deprecation warnings, for eas‐
10517 ier debugging.
10518 Thanks to Sanyam Khurana for the patch!
10520 3.71.2 - 2018-09-10
10522 This release makes Hypothesis's memory usage substantially smaller for
10523 tests with many examples, by bounding the number of past examples it
10524 keeps around.
10525 You will not see much difference unless you are running tests with
10527 max_examples set to well over 1000, but if you do have such tests then
10528 you should see memory usage mostly plateau where previously it would
10529 have grown linearly with time.
10530 3.71.1 - 2018-09-09
10532 This patch adds internal comments to some tree traversals in the core
10533 engine. There is no user-visible change.
10534 3.71.0 - 2018-09-08
10536 This release deprecates the coverage-guided testing functionality, as
10537 it has proven brittle and does not really pull its weight.
10538 We intend to replace it with something more useful in the future, but
10540 the feature in its current form does not seem to be worth the cost of
10541 using, and whatever replaces it will likely look very different.
10542 3.70.4 - 2018-09-08
10544 This patch changes the behaviour of reproduce_failure() so that blobs
10545 are only printed in quiet mode when the print_blob setting is set to
10546 ALWAYS.
10547 Thanks to Cameron McGill for writing this patch at the PyCon Australia
10549 sprints!
10550 3.70.3 - 2018-09-03
10552 This patch removes some unnecessary code from the internals. There is
10553 no user-visible change.
10554 3.70.2 - 2018-09-03
10556 This patch fixes an internal bug where a corrupted argument to
10557 @reproduce_failure could raise the wrong type of error. Thanks again
10558 to Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
10559 tently provides excellent bug reports including issue #1558.
10560 3.70.1 - 2018-09-03
10562 This patch updates hypothesis to report its version and settings when
10563 run with pytest. (issue #1223).
10564 Thanks to Jack Massey for this feature.
10566 3.70.0 - 2018-09-01
10568 This release adds a fullmatch argument to from_regex(). When full‐
10569 match=True, the whole example will match the regex pattern as for
10570 python:re.fullmatch().
10571 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
10573 sprints!
10574 3.69.12 - 2018-08-30
10576 This release reverts the changes to logging handling in 3.69.11, which
10577 broke test that use the pytest caplog fixture internally because all
10578 logging was disabled (issue #1546).
10579 3.69.11 - 2018-08-29
10581 This patch will hide all logging messages produced by test cases before
10582 the final, minimal, failing test case (issue #356).
10583 Thanks to Gary Donovan for writing this patch at the PyCon Australia
10585 sprints!
10586 3.69.10 - 2018-08-29
10588 This patch fixes a bug that prevents coverage from reporting unexecuted
10589 Python files (issue #1085).
10590 Thanks to Gary Donovan for writing this patch at the PyCon Australia
10592 sprints!
10593 3.69.9 - 2018-08-28
10595 This patch improves the packaging of the Python package by adding LI‐
10596 CENSE.txt to the sdist (issue #1311), clarifying the minimum supported
10597 versions of pytz and dateutil (issue #1383), and adds keywords to the
10598 metadata (issue #1520).
10599 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
10601 tralia sprints!
10602 3.69.8 - 2018-08-28
10604 This is an internal change which replaces pickle with json to prevent
10605 possible security issues.
10606 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
10608 sprints!
10609 3.69.7 - 2018-08-28
10611 This patch ensures that note() prints the note for every test case when
10612 the verbosity setting is Verbosity.verbose. At normal verbosity it
10613 only prints from the final test case.
10614 Thanks to Tom McDermott for writing this patch at the PyCon Australia
10616 sprints!
10617 3.69.6 - 2018-08-27
10619 This patch improves the testing of some internal caching. It should
10620 have no user-visible effect.
10621 3.69.5 - 2018-08-27
10623 This change performs a small rename and refactoring in the core engine.
10624 There is no user-visible change.
10625 3.69.4 - 2018-08-27
10627 This change improves the core engine's ability to avoid unnecessary
10628 work, by consulting its cache of previously-tried inputs in more cases.
10629 3.69.3 - 2018-08-27
10631 This patch handles passing an empty python:enum.Enum to from_type() by
10632 returning nothing(), instead of raising an internal python:Assertion‐
10633 Error.
10634 Thanks to Paul Amazona for writing this patch at the PyCon Australia
10636 sprints!
10637 3.69.2 - 2018-08-23
10639 This patch fixes a small mistake in an internal comment. There is no
10640 user-visible change.
10641 3.69.1 - 2018-08-21
10643 This change fixes a small bug in how the core engine consults its cache
10644 of previously-tried inputs. There is unlikely to be any user-visible
10645 change.
10646 3.69.0 - 2018-08-20
10648 This release improves argument validation for stateful testing.
10649 • If the target or targets of a rule() are invalid, we now raise a use‐
10651 ful validation error rather than an internal exception.
10652 • Passing both the target and targets arguments is deprecated - append
10654 the target bundle to the targets tuple of bundles instead.
10655 • Passing the name of a Bundle rather than the Bundle itself is also
10657 deprecated.
10658 3.68.3 - 2018-08-20
10660 This is a docs-only patch, fixing some typos and formatting issues.
10661 3.68.2 - 2018-08-19
10663 This change fixes a small bug in how the core engine caches the results
10664 of previously-tried inputs. The effect is unlikely to be noticeable,
10665 but it might avoid unnecessary work in some cases.
10666 3.68.1 - 2018-08-18
10668 This patch documents the from_dtype() function, which infers a strategy
10669 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
10670 directly when creating e.g. Pandas objects.
10671 3.68.0 - 2018-08-15
10673 arrays() now checks that integer and float values drawn from elements
10674 and fill strategies can be safely cast to the dtype of the array, and
10675 emits a warning otherwise (issue #1385).
10676 Elements in the resulting array could previously violate constraints on
10678 the elements strategy due to floating-point overflow or truncation of
10679 integers to fit smaller types.
10680 3.67.1 - 2018-08-14
10682 This release contains a tiny refactoring of the internals. There is no
10683 user-visible change.
10684 3.67.0 - 2018-08-10
10686 This release adds a width argument to floats(), to generate lower-pre‐
10687 cision floating point numbers for e.g. Numpy arrays.
10688 The generated examples are always instances of Python's native float
10690 type, which is 64bit, but passing width=32 will ensure that all values
10691 can be exactly represented as 32bit floats. This can be useful to
10692 avoid overflow (to +/- infinity), and for efficiency of generation and
10693 shrinking.
10694 Half-precision floats (width=16) are also supported, but require Numpy
10696 if you are running Python 3.5 or earlier.
10697 3.66.33 - 2018-08-10
10699 This release fixes a bug in floats(), where setting allow_infin‐
10700 ity=False and exactly one of min_value and max_value would allow infi‐
10701 nite values to be generated.
10702 3.66.32 - 2018-08-09
10704 This release adds type hints to the @example() and seed() decorators,
10705 and fixes the type hint on register_type_strategy(). The second argu‐
10706 ment to register_type_strategy() must either be a SearchStrategy, or a
10707 callable which takes a type and returns a SearchStrategy.
10708 3.66.31 - 2018-08-08
10710 Another set of changes designed to improve the performance of shrinking
10711 on large examples. In particular the shrinker should now spend consid‐
10712 erably less time running useless shrinks.
10713 3.66.30 - 2018-08-06
10715 "Bug fixes and performance improvements".
10716 This release is a fairly major overhaul of the shrinker designed to im‐
10718 prove its behaviour on large examples, especially around stateful test‐
10719 ing. You should hopefully see shrinking become much faster, with little
10720 to no quality degradation (in some cases quality may even improve).
10721 3.66.29 - 2018-08-05
10723 This release fixes two very minor bugs in the core engine:
10724 • it fixes a corner case that was missing in 3.66.28, which should
10726 cause shrinking to work slightly better.
10727 • it fixes some logic for how shrinking interacts with the database
10729 that was causing Hypothesis to be insufficiently aggressive about
10730 clearing out old keys.
10731 3.66.28 - 2018-08-05
10733 This release improves how Hypothesis handles reducing the size of inte‐
10734 gers' representation. This change should mostly be invisible as it's
10735 purely about the underlying representation and not the generated value,
10736 but it may result in some improvements to shrink performance.
10737 3.66.27 - 2018-08-05
10739 This release changes the order in which Hypothesis chooses parts of the
10740 test case to shrink. For typical usage this should be a significant
10741 performance improvement on large examples. It is unlikely to have a ma‐
10742 jor impact on example quality, but where it does change the result it
10743 should usually be an improvement.
10744 3.66.26 - 2018-08-05
10746 This release improves the debugging information that the shrinker emits
10747 about the operations it performs, giving better summary statistics
10748 about which passes resulted in test executions and whether they were
10749 successful.
10750 3.66.25 - 2018-08-05
10752 This release fixes several bugs that were introduced to the shrinker in
10753 3.66.24 which would have caused it to behave significantly less well
10754 than advertised. With any luck you should actually see the promised
10755 benefits now.
10756 3.66.24 - 2018-08-03
10758 This release changes how Hypothesis deletes data when shrinking in or‐
10759 der to better handle deletion of large numbers of contiguous sequences.
10760 Most tests should see little change, but this will hopefully provide a
10761 significant speed up for stateful testing.
10762 3.66.23 - 2018-08-02
10764 This release makes some internal changes to enable further improvements
10765 to the shrinker. You may see some changes in the final shrunk examples,
10766 but they are unlikely to be significant.
10767 3.66.22 - 2018-08-01
10769 This release adds some more internal caching to the shrinker. This
10770 should cause a significant speed up for shrinking, especially for
10771 stateful testing and large example sizes.
10772 3.66.21 - 2018-08-01
10774 This patch is for downstream packagers - our tests now pass under
10775 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
10776 of Hypothesis itself.
10777 3.66.20 - 2018-08-01
10779 This release removes some functionality from the shrinker that was tak‐
10780 ing a considerable amount of time and does not appear to be useful any
10781 more due to a number of quality improvements in the shrinker.
10782 You may see some degradation in shrink quality as a result of this, but
10784 mostly shrinking should just get much faster.
10785 3.66.19 - 2018-08-01
10787 This release slightly changes the format of some debugging information
10788 emitted during shrinking, and refactors some of the internal interfaces
10789 around that.
10790 3.66.18 - 2018-07-31
10792 This release is a very small internal refactoring which should have no
10793 user visible impact.
10794 3.66.17 - 2018-07-31
10796 This release fixes a bug that could cause an IndexError to be raised
10797 from inside Hypothesis during shrinking. It is likely that it was im‐
10798 possible to trigger this bug in practice - it was only made visible by
10799 some currently unreleased work.
10800 3.66.16 - 2018-07-31
10802 This release is a very small internal refactoring which should have no
10803 user visible impact.
10804 3.66.15 - 2018-07-31
10806 This release makes Hypothesis's shrinking faster by removing some re‐
10807 dundant work that it does when minimizing values in its internal repre‐
10808 sentation.
10809 3.66.14 - 2018-07-30
10811 This release expands the deprecation of timeout from 3.16.0 to also
10812 emit the deprecation warning in find or stateful testing.
10813 3.66.13 - 2018-07-30
10815 This release adds an additional shrink pass that is able to reduce the
10816 size of examples in some cases where the transformation is non-obvious.
10817 In particular this will improve the quality of some examples which
10818 would have regressed in 3.66.12.
10819 3.66.12 - 2018-07-28
10821 This release changes how we group data together for shrinking. It
10822 should result in improved shrinker performance, especially in stateful
10823 testing.
10824 3.66.11 - 2018-07-28
10826 This patch modifies how which rule to run is selected during rule based
10827 stateful testing. This should result in a slight performance increase
10828 during generation and a significant performance and quality improvement
10829 when shrinking.
10830 As a result of this change, some state machines which would previously
10832 have thrown an InvalidDefinition are no longer detected as invalid.
10833 3.66.10 - 2018-07-28
10835 This release weakens some minor functionality in the shrinker that had
10836 only modest benefit and made its behaviour much harder to reason about.
10837 This is unlikely to have much user visible effect, but it is possible
10839 that in some cases shrinking may get slightly slower. It is primarily
10840 to make it easier to work on the shrinker and pave the way for future
10841 work.
10842 3.66.9 - 2018-07-26
10844 This release improves the information that Hypothesis emits about its
10845 shrinking when verbosity is set to debug.
10846 3.66.8 - 2018-07-24
10848 This patch includes some minor fixes in the documentation, and updates
10849 the minimum version of pytest to 3.0 (released August 2016).
10850 3.66.7 - 2018-07-24
10852 This release fixes a bug where difficult to shrink tests could some‐
10853 times trigger an internal assertion error inside the shrinker.
10854 3.66.6 - 2018-07-23
10856 This patch ensures that Hypothesis fully supports Python 3.7, by up‐
10857 grading from_type() (issue #1264) and fixing some minor issues in our
10858 test suite (issue #1148).
10859 3.66.5 - 2018-07-22
10861 This patch fixes the online docs for various extras, by ensuring that
10862 their dependencies are installed on readthedocs.io (issue #1326).
10863 3.66.4 - 2018-07-20
10865 This release improves the shrinker's ability to reorder examples.
10866 For example, consider the following test:
10868 import hypothesis.strategies as st
10870 from hypothesis import given
10871 @given(st.text(), st.text())
10874 def test_non_equal(x, y):
10875 assert x != y
10876 Previously this could have failed with either of x="", y="0" or x="0",
10878 y="". Now it should always fail with x="", y="0".
10879 This will allow the shrinker to produce more consistent results, espe‐
10881 cially in cases where test cases contain some ordered collection whose
10882 actual order does not matter.
10883 3.66.3 - 2018-07-20
10885 This patch fixes inference in the builds() strategy with subtypes of
10886 python:typing.NamedTuple, where the __init__ method is not useful for
10887 introspection. We now use the field types instead - thanks to James
10888 Uther for identifying this bug.
10889 3.66.2 - 2018-07-19
10891 This release improves the shrinker's ability to handle situations where
10892 there is an additive constraint between two values.
10893 For example, consider the following test:
10895 import hypothesis.strategies as st
10897 from hypothesis import given
10898 @given(st.integers(), st.integers())
10901 def test_does_not_exceed_100(m, n):
10902 assert m + n < 100
10903 Previously this could have failed with almost any pair (m, n) with 0 <=
10905 m <= n and m + n == 100. Now it should almost always fail with m=0,
10906 n=100.
10907 This is a relatively niche specialisation, but can be useful in situa‐
10909 tions where e.g. a bug is triggered by an integer overflow.
10910 3.66.1 - 2018-07-09
10912 This patch fixes a rare bug where an incorrect percentage drawtime
10913 could be displayed for a test, when the system clock was changed during
10914 a test running under Python 2 (we use python:time.monotonic() where it
10915 is available to avoid such problems). It also fixes a possible
10916 zero-division error that can occur when the underlying C library dou‐
10917 ble-rounds an intermediate value in python:math.fsum() and gets the
10918 least significant bit wrong.
10919 3.66.0 - 2018-07-05
10921 This release improves validation of the alphabet argument to the text()
10922 strategy. The following misuses are now deprecated, and will be an er‐
10923 ror in a future version:
10924 • passing an unordered collection (such as set('abc')), which violates
10926 invariants about shrinking and reproducibility
10927 • passing an alphabet sequence with elements that are not strings
10929 • passing an alphabet sequence with elements that are not of length
10931 one, which violates any size constraints that may apply
10932 Thanks to Sushobhit for adding these warnings (issue #1329).
10934 3.65.3 - 2018-07-04
10936 This release fixes a mostly theoretical bug where certain usage of the
10937 internal API could trigger an assertion error inside Hypothesis. It is
10938 unlikely that this problem is even possible to trigger through the pub‐
10939 lic API.
10940 3.65.2 - 2018-07-04
10942 This release fixes dependency information for coverage. Previously Hy‐
10943 pothesis would allow installing coverage with any version, but it only
10944 works with coverage 4.0 or later.
10945 We now specify the correct metadata in our setup.py, so Hypothesis will
10947 only allow installation with compatible versions of coverage.
10948 3.65.1 - 2018-07-03
10950 This patch ensures that stateful tests which raise an error from a
10951 pytest helper still print the sequence of steps taken to reach that
10952 point (issue #1372). This reporting was previously broken because the
10953 helpers inherit directly from python:BaseException, and therefore re‐
10954 quire special handling to catch without breaking e.g. the use of ctrl-C
10955 to quit the test.
10956 3.65.0 - 2018-06-30
10958 This release deprecates the max_shrinks setting in favor of an internal
10959 heuristic. If you need to avoid shrinking examples, use the phases
10960 setting instead. (issue #1235)
10961 3.64.2 - 2018-06-27
10963 This release fixes a bug where an internal assertion error could some‐
10964 times be triggered while shrinking a failing test.
10965 3.64.1 - 2018-06-27
10967 This patch fixes type-checking errors in our vendored pretty-printer,
10968 which were ignored by our mypy config but visible for anyone else
10969 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
10970 3.64.0 - 2018-06-26
10972 This release adds an interface which can be used to insert a wrapper
10973 between the original test function and @given (issue #1257). This will
10974 be particularly useful for test runner extensions such as pytest-trio,
10975 but is not recommended for direct use by other users of Hypothesis.
10976 3.63.0 - 2018-06-26
10978 This release adds a new mechanism to infer strategies for classes de‐
10979 fined using attrs, based on the the type, converter, or validator of
10980 each attribute. This inference is now built in to builds() and
10981 from_type().
10982 On Python 2, from_type() no longer generates instances of int when
10984 passed long, or vice-versa.
10985 3.62.0 - 2018-06-26
10987 This release adds PEP 484 type hints to Hypothesis on a provisional ba‐
10988 sis, using the comment-based syntax for Python 2 compatibility. You
10989 can read more about our type hints here.
10990 It also adds the py.typed marker specified in PEP 561. After you pip
10992 install hypothesis, mypy 0.590 or later will therefore type-check your
10993 use of our public interface!
10994 3.61.0 - 2018-06-24
10996 This release deprecates the use of settings as a context manager, the
10997 use of which is somewhat ambiguous.
10998 Users should define settings with global state or with the
11000 @settings(...) decorator.
11001 3.60.1 - 2018-06-20
11003 Fixed a bug in generating an instance of a Django model from a strategy
11004 where the primary key is generated as part of the strategy. See details
11005 here.
11006 Thanks to Tim Martin for this contribution.
11008 3.60.0 - 2018-06-20
11010 This release adds the @initialize decorator for stateful testing (orig‐
11011 inally discussed in issue #1216). All @initialize rules will be called
11012 once each in an arbitrary order before any normal rule is called.
11013 3.59.3 - 2018-06-19
11015 This is a no-op release to take into account some changes to the re‐
11016 lease process. It should have no user visible effect.
11017 3.59.2 - 2018-06-18
11019 This adds support for partially sorting examples which cannot be fully
11020 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
11021 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
11022 0].
11023 Thanks to Luke for contributing.
11025 3.59.1 - 2018-06-16
11027 This patch uses python:random.getstate() and python:random.setstate()
11028 to restore the PRNG state after @given runs deterministic tests. With‐
11029 out restoring state, you might have noticed problems such as issue
11030 #1266. The fix also applies to stateful testing (issue #702).
11031 3.59.0 - 2018-06-14
11033 This release adds the emails() strategy, which generates unicode
11034 strings representing an email address.
11035 Thanks to Sushobhit for moving this to the public API (issue #162).
11037 3.58.1 - 2018-06-13
11039 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
11040 1 2 3.
11041 Thanks to Luke for contributing.
11043 3.58.0 - 2018-06-13
11045 This adds a new extra timezones() strategy that generates dateutil
11046 timezones.
11047 Thanks to Conrad for contributing.
11049 3.57.0 - 2018-05-20
11051 Using an unordered collection with the permutations() strategy has been
11052 deprecated because the order in which e.g. a set shrinks is arbitrary.
11053 This may cause different results between runs.
11054 3.56.10 - 2018-05-16
11056 This release makes hypothesis.settings.define_setting a private method,
11057 which has the effect of hiding it from the documentation.
11058 3.56.9 - 2018-05-11
11060 This is another release with no functionality changes as part of
11061 changes to Hypothesis's new release tagging scheme.
11062 3.56.8 - 2018-05-10
11064 This is a release with no functionality changes that moves Hypothesis
11065 over to a new release tagging scheme.
11066 3.56.7 - 2018-05-10
11068 This release provides a performance improvement for most tests, but in
11069 particular users of sampled_from() who don't have numpy installed
11070 should see a significant performance improvement.
11071 3.56.6 - 2018-05-09
11073 This patch contains further internal work to support Mypy. There are
11074 no user-visible changes... yet.
11075 3.56.5 - 2018-04-22
11077 This patch contains some internal refactoring to run mypy in CI. There
11078 are no user-visible changes.
11079 3.56.4 - 2018-04-21
11081 This release involves some very minor internal clean up and should have
11082 no user visible effect at all.
11083 3.56.3 - 2018-04-20
11085 This release fixes a problem introduced in 3.56.0 where setting the hy‐
11086 pothesis home directory (through currently undocumented means) would no
11087 longer result in the default database location living in the new home
11088 directory.
11089 3.56.2 - 2018-04-20
11091 This release fixes a problem introduced in 3.56.0 where setting
11092 max_examples to 1 would result in tests failing with Unsatisfiable.
11093 This problem could also occur in other harder to trigger circumstances
11094 (e.g. by setting it to a low value, having a hard to satisfy assump‐
11095 tion, and disabling health checks).
11096 3.56.1 - 2018-04-20
11098 This release fixes a problem that was introduced in 3.56.0: Use of the
11099 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
11100 cated, actually broken due to being read before various setup the dep‐
11101 recation path needed was done. It now works correctly (and emits a dep‐
11102 recation warning).
11103 3.56.0 - 2018-04-17
11105 This release deprecates several redundant or internally oriented
11106 settings, working towards an orthogonal set of configuration options
11107 that are widely useful without requiring any knowledge of our internals
11108 (issue #535).
11109 • Deprecated settings that no longer have any effect are no longer
11111 shown in the __repr__ unless set to a non-default value.
11112 • hypothesis.settings.perform_health_check is deprecated, as it dupli‐
11114 cates suppress_health_check.
11115 • hypothesis.settings.max_iterations is deprecated and disabled, be‐
11117 cause we can usually get better behaviour from an internal heuristic
11118 than a user-controlled setting.
11119 • hypothesis.settings.min_satisfying_examples is deprecated and dis‐
11121 abled, due to overlap with the filter_too_much healthcheck and poor
11122 interaction with max_examples.
11123 • HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
11125 the profile system instead.
11126 • Examples tried by find() are now reported at debug verbosity level
11128 (as well as verbose level).
11129 3.55.6 - 2018-04-14
11131 This release fixes a somewhat obscure condition (issue #1230) under
11132 which you could occasionally see a failing test trigger an assertion
11133 error inside Hypothesis instead of failing normally.
11134 3.55.5 - 2018-04-14
11136 This patch fixes one possible cause of issue #966. When running Python
11137 2 with hash randomisation, passing a python:bytes object to python:ran‐
11138 dom.seed() would use version=1, which broke derandomize (because the
11139 seed depended on a randomised hash). If derandomize is still nondeter‐
11140 ministic for you, please open an issue.
11141 3.55.4 - 2018-04-13
11143 This patch makes a variety of minor improvements to the documentation,
11144 and improves a few validation messages for invalid inputs.
11145 3.55.3 - 2018-04-12
11147 This release updates the URL metadata associated with the PyPI package
11148 (again). It has no other user visible effects.
11149 3.55.2 - 2018-04-11
11151 This release updates the URL metadata associated with the PyPI package.
11152 It has no other user visible effects.
11153 3.55.1 - 2018-04-06
11155 This patch relaxes constraints in our tests on the expected values re‐
11156 turned by the standard library function hypot() and the internal helper
11157 function cathetus, to fix near-exact test failures on some 32-bit sys‐
11158 tems used by downstream packagers.
11159 3.55.0 - 2018-04-05
11161 This release includes several improvements to the handling of the
11162 database setting.
11163 • The database_file setting was a historical artefact, and you should
11165 just use database directly.
11166 • The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
11168 favor of load_profile() and the database setting.
11169 • If you have not configured the example database at all and the de‐
11171 fault location is not usable (due to e.g. permissions issues), Hy‐
11172 pothesis will fall back to an in-memory database. This is not per‐
11173 sisted between sessions, but means that the defaults work on
11174 read-only filesystems.
11175 3.54.0 - 2018-04-04
11177 This release improves the complex_numbers() strategy, which now sup‐
11178 ports min_magnitude and max_magnitude arguments, along with allow_nan
11179 and allow_infinity like for floats().
11180 Thanks to J.J. Green for this feature.
11182 3.53.0 - 2018-04-01
11184 This release removes support for Django 1.8, which reached end of life
11185 on 2018-04-01. You can see Django's release and support schedule on
11186 the Django Project website.
11187 3.52.3 - 2018-04-01
11189 This patch fixes the min_satisfying_examples settings documentation, by
11190 explaining that example shrinking is tracked at the level of the under‐
11191 lying bytestream rather than the output value.
11192 The output from find() in verbose mode has also been adjusted - see the
11194 example session - to avoid duplicating lines when the example repr is
11195 constant, even if the underlying representation has been shrunken.
11196 3.52.2 - 2018-03-30
11198 This release improves the output of failures with rule based stateful
11199 testing in two ways:
11200 • The output from it is now usually valid Python code.
11202 • When the same value has two different names because it belongs to two
11204 different bundles, it will now display with the name associated with
11205 the correct bundle for a rule argument where it is used.
11206 3.52.1 - 2018-03-29
11208 This release improves the behaviour of stateful testing in two ways:
11209 • Previously some runs would run no steps (issue #376). This should no
11211 longer happen.
11212 • RuleBasedStateMachine tests which used bundles extensively would of‐
11214 ten shrink terribly. This should now be significantly improved,
11215 though there is likely a lot more room for improvement.
11216 This release also involves a low level change to how ranges of integers
11218 are handles which may result in other improvements to shrink quality in
11219 some cases.
11220 3.52.0 - 2018-03-24
11222 This release deprecates use of @settings(...) as a decorator, on func‐
11223 tions or methods that are not also decorated with @given. You can
11224 still apply these decorators in any order, though you should only do so
11225 once each.
11226 Applying @given twice was already deprecated, and applying
11228 @settings(...) twice is deprecated in this release and will become an
11229 error in a future version. Neither could ever be used twice to good ef‐
11230 fect.
11231 Using @settings(...) as the sole decorator on a test is completely
11233 pointless, so this common usage error will become an error in a future
11234 version of Hypothesis.
11235 3.51.0 - 2018-03-24
11237 This release deprecates the average_size argument to lists() and other
11238 collection strategies. You should simply delete it wherever it was
11239 used in your tests, as it no longer has any effect.
11240 In early versions of Hypothesis, the average_size argument was treated
11242 as a hint about the distribution of examples from a strategy. Subse‐
11243 quent improvements to the conceptual model and the engine for generat‐
11244 ing and shrinking examples mean it is more effective to simply describe
11245 what constitutes a valid example, and let our internals handle the dis‐
11246 tribution.
11247 3.50.3 - 2018-03-24
11249 This patch contains some internal refactoring so that we can run with
11250 warnings as errors in CI.
11251 3.50.2 - 2018-03-20
11253 This has no user-visible changes except one slight formatting change to
11254 one docstring, to avoid a deprecation warning.
11255 3.50.1 - 2018-03-20
11257 This patch fixes an internal error introduced in 3.48.0, where a check
11258 for the Django test runner would expose import-time errors in Django
11259 configuration (issue #1167).
11260 3.50.0 - 2018-03-19
11262 This release improves validation of numeric bounds for some strategies.
11263 • integers() and floats() now raise InvalidArgument if passed a
11265 min_value or max_value which is not an instance of Real, instead of
11266 various internal errors.
11267 • floats() now converts its bounding values to the nearest float above
11269 or below the min or max bound respectively, instead of just casting
11270 to float. The old behaviour was incorrect in that you could generate
11271 float(min_value), even when this was less than min_value itself (pos‐
11272 sible with eg. fractions).
11273 • When both bounds are provided to floats() but there are no floats in
11275 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
11276 raised.
11277 • decimals() gives a more useful error message if passed a string that
11279 cannot be converted to Decimal in a context where this error is not
11280 trapped.
11281 Code that previously seemed to work may be explicitly broken if there
11283 were no floats between min_value and max_value (only possible with
11284 non-float bounds), or if a bound was not a Real number but still al‐
11285 lowed in python:math.isnan (some custom classes with a __float__
11286 method).
11287 3.49.1 - 2018-03-15
11289 This patch fixes our tests for Numpy dtype strategies on big-endian
11290 platforms, where the strategy behaved correctly but the test assumed
11291 that the native byte order was little-endian.
11292 There is no user impact unless you are running our test suite on
11294 big-endian platforms. Thanks to Graham Inggs for reporting issue
11295 #1164.
11296 3.49.0 - 2018-03-12
11298 This release deprecates passing elements=None to collection strategies,
11299 such as lists().
11300 Requiring lists(nothing()) or builds(list) instead of lists() means
11302 slightly more typing, but also improves the consistency and discover‐
11303 ability of our API - as well as showing how to compose or construct
11304 strategies in ways that still work in more complex situations.
11305 Passing a nonzero max_size to a collection strategy where the elements
11307 strategy contains no values is now deprecated, and will be an error in
11308 a future version. The equivalent with elements=None is already an er‐
11309 ror.
11310 3.48.1 - 2018-03-05
11312 This patch will minimize examples that would come out non-minimal in
11313 previous versions. Thanks to Kyle Reeve for this patch.
11314 3.48.0 - 2018-03-05
11316 This release improves some "unhappy paths" when using Hypothesis with
11317 the standard library python:unittest module:
11318 • Applying @given to a non-test method which is overridden from
11320 python:unittest.TestCase, such as setUp, raises a new health check.
11321 (issue #991)
11322 • Using subTest() within a test decorated with @given would leak inter‐
11324 mediate results when tests were run under the python:unittest test
11325 runner. Individual reporting of failing subtests is now disabled
11326 during a test using @given. (issue #1071)
11327 • @given is still not a class decorator, but the error message if you
11329 try using it on a class has been improved.
11330 As a related improvement, using django:django.test.TestCase with @given
11332 instead of hypothesis.extra.django.TestCase raises an explicit error
11333 instead of running all examples in a single database transaction.
11334 3.47.0 - 2018-03-02
11336 register_profile now accepts keyword arguments for specific settings,
11337 and the parent settings object is now optional. Using a name for a
11338 registered profile which is not a string was never suggested, but it is
11339 now also deprecated and will eventually be an error.
11340 3.46.2 - 2018-03-01
11342 This release removes an unnecessary branch from the code, and has no
11343 user-visible impact.
11344 3.46.1 - 2018-03-01
11346 This changes only the formatting of our docstrings and should have no
11347 user-visible effects.
11348 3.46.0 - 2018-02-26
11350 characters() has improved docs about what arguments are valid, and ad‐
11351 ditional validation logic to raise a clear error early (instead of e.g.
11352 silently ignoring a bad argument). Categories may be specified as the
11353 Unicode 'general category' (eg 'Nd'), or as the 'major category' (eg
11354 ['N', 'Lu'] is equivalent to ['Nd', 'Nl', 'No', 'Lu']).
11355 In previous versions, general categories were supported and all other
11357 input was silently ignored. Now, major categories are supported in ad‐
11358 dition to general categories (which may change the behaviour of some
11359 existing code), and all other input is deprecated.
11360 3.45.5 - 2018-02-26
11362 This patch improves strategy inference in hypothesis.extra.django to
11363 account for some validators in addition to field type - see issue #1116
11364 for ongoing work in this space.
11365 Specifically, if a CharField or TextField has an attached RegexValida‐
11367 tor, we now use from_regex() instead of text() as the underlying strat‐
11368 egy. This allows us to generate examples of the default User model,
11369 closing issue #1112.
11370 3.45.4 - 2018-02-25
11372 This patch improves some internal debugging information, fixes a typo
11373 in a validation error message, and expands the documentation for new
11374 contributors.
11375 3.45.3 - 2018-02-23
11377 This patch may improve example shrinking slightly for some strategies.
11378 3.45.2 - 2018-02-18
11380 This release makes our docstring style more consistent, thanks to
11381 flake8-docstrings. There are no user-visible changes.
11382 3.45.1 - 2018-02-17
11384 This fixes an indentation issue in docstrings for datetimes(), dates(),
11385 times(), and timedeltas().
11386 3.45.0 - 2018-02-13
11388 This release fixes builds() so that target can be used as a keyword ar‐
11389 gument for passing values to the target. The target itself can still be
11390 specified as a keyword argument, but that behavior is now deprecated.
11391 The target should be provided as the first positional argument.
11392 3.44.26 - 2018-02-06
11394 This release fixes some formatting issues in the Hypothesis source
11395 code. It should have no externally visible effects.
11396 3.44.25 - 2018-02-05
11398 This release changes the way in which Hypothesis tries to shrink the
11399 size of examples. It probably won't have much impact, but might make
11400 shrinking faster in some cases. It is unlikely but not impossible that
11401 it will change the resulting examples.
11402 3.44.24 - 2018-01-27
11404 This release fixes dependency information when installing Hypothesis
11405 from a binary "wheel" distribution.
11406 • The install_requires for enum34 is resolved at install time, rather
11408 than at build time (with potentially different results).
11409 • Django has fixed their python_requires for versions 2.0.0 onward,
11411 simplifying Python2-compatible constraints for downstream projects.
11412 3.44.23 - 2018-01-24
11414 This release improves shrinking in a class of pathological examples
11415 that you are probably never hitting in practice. If you are hitting
11416 them in practice this should be a significant speed up in shrinking. If
11417 you are not, you are very unlikely to notice any difference. You might
11418 see a slight slow down and/or slightly better falsifying examples.
11419 3.44.22 - 2018-01-23
11421 This release fixes a dependency problem. It was possible to install
11422 Hypothesis with an old version of attrs, which would throw a TypeError
11423 as soon as you tried to import hypothesis. Specifically, you need at‐
11424 trs 16.0.0 or newer.
11425 Hypothesis will now require the correct version of attrs when in‐
11427 stalling.
11428 3.44.21 - 2018-01-22
11430 This change adds some additional structural information that Hypothesis
11431 will use to guide its search.
11432 You mostly shouldn't see much difference from this. The two most likely
11434 effects you would notice are:
11435 1. Hypothesis stores slightly more examples in its database for passing
11437 tests.
11438 2. Hypothesis may find new bugs that it was previously missing, but it
11440 probably won't (this is a basic implementation of the feature that
11441 is intended to support future work. Although it is useful on its
11442 own, it's not very useful on its own).
11443 3.44.20 - 2018-01-21
11445 This is a small refactoring release that changes how Hypothesis tracks
11446 some information about the boundary of examples in its internal repre‐
11447 sentation.
11448 You are unlikely to see much difference in behaviour, but memory usage
11450 and run time may both go down slightly during normal test execution,
11451 and when failing Hypothesis might print its failing example slightly
11452 sooner.
11453 3.44.19 - 2018-01-21
11455 This changes how we compute the default average_size for all collection
11456 strategies. Previously setting a max_size without setting an aver‐
11457 age_size would have the seemingly paradoxical effect of making data
11458 generation slower, because it would raise the average size from its de‐
11459 fault. Now setting max_size will either leave the default unchanged or
11460 lower it from its default.
11461 If you are currently experiencing this problem, this may make your
11463 tests substantially faster. If you are not, this will likely have no
11464 effect on you.
11465 3.44.18 - 2018-01-20
11467 This is a small refactoring release that changes how Hypothesis detects
11468 when the structure of data generation depends on earlier values gener‐
11469 ated (e.g. when using flatmap or composite()). It should not have any
11470 observable effect on behaviour.
11471 3.44.17 - 2018-01-15
11473 This release fixes a typo in internal documentation, and has no
11474 user-visible impact.
11475 3.44.16 - 2018-01-13
11477 This release improves test case reduction for recursive data struc‐
11478 tures. Hypothesis now guarantees that whenever a strategy calls itself
11479 recursively (usually this will happen because you are using
11480 deferred()), any recursive call may replace the top level value. e.g.
11481 given a tree structure, Hypothesis will always try replacing it with a
11482 subtree.
11483 Additionally this introduces a new heuristic that may in some circum‐
11485 stances significantly speed up test case reduction - Hypothesis should
11486 be better at immediately replacing elements drawn inside another strat‐
11487 egy with their minimal possible value.
11488 3.44.15 - 2018-01-13
11490 from_type() can now resolve recursive types such as binary trees (issue
11491 #1004). Detection of non-type arguments has also improved, leading to
11492 better error messages in many cases involving forward references.
11493 3.44.14 - 2018-01-08
11495 This release fixes a bug in the shrinker that prevented the optimisa‐
11496 tions in 3.44.6 from working in some cases. It would not have worked
11497 correctly when filtered examples were nested (e.g. with a set of inte‐
11498 gers in some range).
11499 This would not have resulted in any correctness problems, but shrinking
11501 may have been slower than it otherwise could be.
11502 3.44.13 - 2018-01-08
11504 This release changes the average bit length of values drawn from
11505 integers() to be much smaller. Additionally it changes the shrinking
11506 order so that now size is considered before sign - e.g. -1 will be
11507 preferred to +10.
11508 The new internal format for integers required some changes to the mini‐
11510 mizer to make work well, so you may also see some improvements to exam‐
11511 ple quality in unrelated areas.
11512 3.44.12 - 2018-01-07
11514 This changes Hypothesis's internal implementation of weighted sampling.
11515 This will affect example distribution and quality, but you shouldn't
11516 see any other effects.
11517 3.44.11 - 2018-01-06
11519 This is a change to some internals around how Hypothesis handles avoid‐
11520 ing generating duplicate examples and seeking out novel regions of the
11521 search space.
11522 You are unlikely to see much difference as a result of it, but it fixes
11524 a bug where an internal assertion could theoretically be triggered and
11525 has some minor effects on the distribution of examples so could poten‐
11526 tially find bugs that have previously been missed.
11527 3.44.10 - 2018-01-06
11529 This patch avoids creating debug statements when debugging is disabled.
11530 Profiling suggests this is a 5-10% performance improvement (issue
11531 #1040).
11532 3.44.9 - 2018-01-06
11534 This patch blacklists null characters ('\x00') in automatically created
11535 strategies for Django CharField and TextField, due to a database issue
11536 which was recently fixed upstream (Hypothesis issue #1045).
11537 3.44.8 - 2018-01-06
11539 This release makes the Hypothesis shrinker slightly less greedy in or‐
11540 der to avoid local minima - when it gets stuck, it makes a small at‐
11541 tempt to search around the final example it would previously have re‐
11542 turned to find a new starting point to shrink from. This should improve
11543 example quality in some cases, especially ones where the test data has
11544 dependencies among parts of it that make it difficult for Hypothesis to
11545 proceed.
11546 3.44.7 - 2018-01-04
11548 This release adds support for Django 2 in the hypothesis-django extra.
11549 This release drops support for Django 1.10, as it is no longer sup‐
11551 ported by the Django team.
11552 3.44.6 - 2018-01-02
11554 This release speeds up test case reduction in many examples by being
11555 better at detecting large shrinks it can use to discard redundant parts
11556 of its input. This will be particularly noticeable in examples that
11557 make use of filtering and for some integer ranges.
11558 3.44.5 - 2018-01-02
11560 Happy new year!
11561 This is a no-op release that updates the year range on all of the copy‐
11563 right headers in our source to include 2018.
11564 3.44.4 - 2017-12-23
11566 This release fixes issue #1041, which slowed tests by up to 6% due to
11567 broken caching.
11568 3.44.3 - 2017-12-21
11570 This release improves the shrinker in cases where examples drawn ear‐
11571 lier can affect how much data is drawn later (e.g. when you draw a
11572 length parameter in a composite and then draw that many elements). Ex‐
11573 amples found in cases like this should now be much closer to minimal.
11574 3.44.2 - 2017-12-20
11576 This is a pure refactoring release which changes how Hypothesis manages
11577 its set of examples internally. It should have no externally visible
11578 effects.
11579 3.44.1 - 2017-12-18
11581 This release fixes issue #997, in which under some circumstances the
11582 body of tests run under Hypothesis would not show up when run under
11583 coverage even though the tests were run and the code they called out‐
11584 side of the test file would show up normally.
11585 3.44.0 - 2017-12-17
11587 This release adds a new feature: The @reproduce_failure decorator, de‐
11588 signed to make it easy to use Hypothesis's binary format for examples
11589 to reproduce a problem locally without having to share your example
11590 database between machines.
11591 This also changes when seeds are printed:
11593 • They will no longer be printed for normal falsifying examples, as
11595 there are now adequate ways of reproducing those for all cases, so it
11596 just contributes noise.
11597 • They will once again be printed when reusing examples from the data‐
11599 base, as health check failures should now be more reliable in this
11600 scenario so it will almost always work in this case.
11601 This work was funded by Smarkets.
11603 3.43.1 - 2017-12-17
11605 This release fixes a bug with Hypothesis's database management - exam‐
11606 ples that were found in the course of shrinking were saved in a way
11607 that indicated that they had distinct causes, and so they would all be
11608 retried on the start of the next test. The intended behaviour, which is
11609 now what is implemented, is that only a bounded subset of these exam‐
11610 ples would be retried.
11611 3.43.0 - 2017-12-17
11613 HypothesisDeprecationWarning now inherits from python:FutureWarning in‐
11614 stead of python:DeprecationWarning, as recommended by PEP 565 for
11615 user-facing warnings (issue #618). If you have not changed the default
11616 warnings settings, you will now see each distinct
11617 HypothesisDeprecationWarning instead of only the first.
11618 3.42.2 - 2017-12-12
11620 This patch fixes issue #1017, where instances of a list or tuple sub‐
11621 type used as an argument to a strategy would be coerced to tuple.
11622 3.42.1 - 2017-12-10
11624 This release has some internal cleanup, which makes reading the code
11625 more pleasant and may shrink large examples slightly faster.
11626 3.42.0 - 2017-12-09
11628 This release deprecates faker-extra, which was designed as a transition
11629 strategy but does not support example shrinking or coverage-guided dis‐
11630 covery.
11631 3.41.0 - 2017-12-06
11633 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
11634 pling from multi-dimensional ndarrays still results in a deprecation
11635 warning. Thanks to Charlie Tanksley for this patch.
11636 3.40.1 - 2017-12-04
11638 This release makes two changes:
11639 • It makes the calculation of some of the metadata that Hypothesis uses
11641 for shrinking occur lazily. This should speed up performance of test
11642 case generation a bit because it no longer calculates information it
11643 doesn't need.
11644 • It improves the shrinker for certain classes of nested examples. e.g.
11646 when shrinking lists of lists, the shrinker is now able to concate‐
11647 nate two adjacent lists together into a single list. As a result of
11648 this change, shrinking may get somewhat slower when the minimal exam‐
11649 ple found is large.
11650 3.40.0 - 2017-12-02
11652 This release improves how various ways of seeding Hypothesis interact
11653 with the example database:
11654 • Using the example database with seed() is now deprecated. You should
11656 set database=None if you are doing that. This will only warn if you
11657 actually load examples from the database while using @seed.
11658 • The derandomize will behave the same way as @seed.
11660 • Using --hypothesis-seed will disable use of the database.
11662 • If a test used examples from the database, it will not suggest using
11664 a seed to reproduce it, because that won't work.
11665 This work was funded by Smarkets.
11667 3.39.0 - 2017-12-01
11669 This release adds a new health check that checks if the smallest "natu‐
11670 ral" possible example of your test case is very large - this will tend
11671 to cause Hypothesis to generate bad examples and be quite slow.
11672 This work was funded by Smarkets.
11674 3.38.9 - 2017-11-29
11676 This is a documentation release to improve the documentation of shrink‐
11677 ing behaviour for Hypothesis's strategies.
11678 3.38.8 - 2017-11-29
11680 This release improves the performance of characters() when using black‐
11681 list_characters and from_regex() when using negative character classes.
11682 The problems this fixes were found in the course of work funded by
11684 Smarkets.
11685 3.38.7 - 2017-11-29
11687 This is a patch release for from_regex(), which had a bug in handling
11688 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
11689 rectly when parsing regex.
11690 3.38.6 - 2017-11-28
11692 This patch changes a few byte-string literals from double to single
11693 quotes, thanks to an update in unify. There are no user-visible
11694 changes.
11695 3.38.5 - 2017-11-23
11697 This fixes the repr of strategies using lambda that are defined inside
11698 decorators to include the lambda source.
11699 This would mostly have been visible when using the statistics function‐
11701 ality - lambdas used for e.g. filtering would have shown up with a <un‐
11702 known> as their body. This can still happen, but it should happen less
11703 often now.
11704 3.38.4 - 2017-11-22
11706 This release updates the reported statistics so that they show approxi‐
11707 mately what fraction of your test run time is spent in data generation
11708 (as opposed to test execution).
11709 This work was funded by Smarkets.
11711 3.38.3 - 2017-11-21
11713 This is a documentation release, which ensures code examples are up to
11714 date by running them as doctests in CI (issue #711).
11715 3.38.2 - 2017-11-21
11717 This release changes the behaviour of the deadline setting when used
11718 with data(): Time spent inside calls to data.draw will no longer be
11719 counted towards the deadline time.
11720 As a side effect of some refactoring required for this work, the way
11722 flaky tests are handled has changed slightly. You are unlikely to see
11723 much difference from this, but some error messages will have changed.
11724 This work was funded by Smarkets.
11726 3.38.1 - 2017-11-21
11728 This patch has a variety of non-user-visible refactorings, removing
11729 various minor warts ranging from indirect imports to typos in comments.
11730 3.38.0 - 2017-11-18
11732 This release overhauls the health check system in a variety of small
11733 ways. It adds no new features, but is nevertheless a minor release be‐
11734 cause it changes which tests are likely to fail health checks.
11735 The most noticeable effect is that some tests that used to fail health
11737 checks will now pass, and some that used to pass will fail. These
11738 should all be improvements in accuracy. In particular:
11739 • New failures will usually be because they are now taking into account
11741 things like use of data() and assume() inside the test body.
11742 • New failures may also be because for some classes of example the way
11744 data generation performance was measured was artificially faster than
11745 real data generation (for most examples that are hitting performance
11746 health checks the opposite should be the case).
11747 • Tests that used to fail health checks and now pass do so because the
11749 health check system used to run in a way that was subtly different
11750 than the main Hypothesis data generation and lacked some of its sup‐
11751 port for e.g. large examples.
11752 If your data generation is especially slow, you may also see your tests
11754 get somewhat faster, as there is no longer a separate health check
11755 phase. This will be particularly noticeable when rerunning test fail‐
11756 ures.
11757 This work was funded by Smarkets.
11759 3.37.0 - 2017-11-12
11761 This is a deprecation release for some health check related features.
11762 The following are now deprecated:
11764 • Passing HealthCheck.exception_in_generation to suppress_health_check.
11766 This no longer does anything even when passed - All errors that oc‐
11767 cur during data generation will now be immediately reraised rather
11768 than going through the health check mechanism.
11769 • Passing HealthCheck.random_module to suppress_health_check. This
11771 hasn't done anything for a long time, but was never explicitly depre‐
11772 cated. Hypothesis always seeds the random module when running @given
11773 tests, so this is no longer an error and suppressing it doesn't do
11774 anything.
11775 • Passing non-HealthCheck values in suppress_health_check. This was
11777 previously allowed but never did anything useful.
11778 In addition, passing a non-iterable value as suppress_health_check will
11780 now raise an error immediately (it would never have worked correctly,
11781 but it would previously have failed later). Some validation error mes‐
11782 sages have also been updated.
11783 This work was funded by Smarkets.
11785 3.36.1 - 2017-11-10
11787 This is a yak shaving release, mostly concerned with our own tests.
11788 While getfullargspec() was documented as deprecated in Python 3.5, it
11790 never actually emitted a warning. Our code to silence this (nonexis‐
11791 tent) warning has therefore been removed.
11792 We now run our tests with DeprecationWarning as an error, and made some
11794 minor changes to our own tests as a result. This required similar up‐
11795 stream updates to coverage and execnet (a test-time dependency via
11796 pytest-xdist).
11797 There is no user-visible change in Hypothesis itself, but we encourage
11799 you to consider enabling deprecations as errors in your own tests.
11800 3.36.0 - 2017-11-06
11802 This release adds a setting to the public API, and does some internal
11803 cleanup:
11804 • The derandomize setting is now documented (issue #890)
11806 • Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
11808 #953)
11809 • Documented the strict setting as deprecated, and updated the build so
11811 our docs always match deprecations in the code.
11812 3.35.0 - 2017-11-06
11814 This minor release supports constraining uuids() to generate a particu‐
11815 lar version of UUID (issue #721).
11816 Thanks to Dion Misic for this feature.
11818 3.34.1 - 2017-11-02
11820 This patch updates the documentation to suggest builds(callable) in‐
11821 stead of just(callable()).
11822 3.34.0 - 2017-11-02
11824 Hypothesis now emits deprecation warnings if you apply @given more than
11825 once to a target.
11826 Applying @given repeatedly wraps the target multiple times. Each wrap‐
11828 per will search the space of of possible parameters separately. This
11829 is equivalent but will be much more inefficient than doing it with a
11830 single call to @given.
11831 For example, instead of @given(booleans()) @given(integers()), you
11833 could write @given(booleans(), integers())
11834 3.33.1 - 2017-11-02
11836 This is a bugfix release:
11837 • builds() would try to infer a strategy for required positional argu‐
11839 ments of the target from type hints, even if they had been given to
11840 builds() as positional arguments (issue #946). Now it only infers
11841 missing required arguments.
11842 • An internal introspection function wrongly reported self as a re‐
11844 quired argument for bound methods, which might also have affected
11845 builds(). Now it knows better.
11846 3.33.0 - 2017-10-16
11848 This release supports strategy inference for more Django field types -
11849 you can now omit an argument for Date, Time, Duration, Slug, IP Ad‐
11850 dress, and UUID fields. (issue #642)
11851 Strategy generation for fields with grouped choices now selects choices
11853 from each group, instead of selecting from the group names.
11854 3.32.2 - 2017-10-15
11856 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
11857 an experimental basis. It has never actually worked, and the new
11858 Hypothesis example database is designed to make such a tool unneces‐
11859 sary.
11860 3.32.1 - 2017-10-13
11862 This patch has two improvements for strategies based on enumerations.
11863 • from_type() now handles enumerations correctly, delegating to
11865 sampled_from(). Previously it noted that Enum.__init__ has no re‐
11866 quired arguments and therefore delegated to builds(), which would
11867 subsequently fail.
11868 • When sampling from an python:enum.Flag, we also generate combinations
11870 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
11871 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
11872 3.32.0 - 2017-10-09
11874 This changes the default value of the use_coverage setting to True when
11875 running on pypy (it was already True on CPython).
11876 It was previously set to False because we expected it to be too slow,
11878 but recent benchmarking shows that actually performance of the feature
11879 on pypy is fairly acceptable - sometimes it's slower than on CPython,
11880 sometimes it's faster, but it's generally within a factor of two either
11881 way.
11882 3.31.6 - 2017-10-08
11884 This patch improves the quality of strategies inferred from Numpy
11885 dtypes:
11886 • Integer dtypes generated examples with the upper half of their
11888 (non-sign) bits set to zero. The inferred strategies can now produce
11889 any representable integer.
11890 • Fixed-width unicode- and byte-string dtypes now cap the internal ex‐
11892 ample length, which should improve example and shrink quality.
11893 • Numpy arrays can only store fixed-size strings internally, and allow
11895 shorter strings by right-padding them with null bytes. Inferred
11896 string strategies no longer generate such values, as they can never
11897 be retrieved from an array. This improves shrinking performance by
11898 skipping useless values.
11899 This has already been useful in Hypothesis - we found an overflow bug
11901 in our Pandas support, and as a result indexes() and range_indexes()
11902 now check that min_size and max_size are at least zero.
11903 3.31.5 - 2017-10-08
11905 This release fixes a performance problem in tests where the use_cover‐
11906 age setting is True.
11907 Tests experience a slow-down proportionate to the amount of code they
11909 cover. This is still the case, but the factor is now low enough that
11910 it should be unnoticeable. Previously it was large and became much
11911 larger in 3.30.4.
11912 3.31.4 - 2017-10-08
11914 from_type() failed with a very confusing error if passed a NewType (‐
11915 issue #901). These pseudo-types are now unwrapped correctly, and
11916 strategy inference works as expected.
11917 3.31.3 - 2017-10-06
11919 This release makes some small optimisations to our use of coverage that
11920 should reduce constant per-example overhead. This is probably only no‐
11921 ticeable on examples where the test itself is quite fast. On no-op
11922 tests that don't test anything you may see up to a fourfold speed in‐
11923 crease (which is still significantly slower than without coverage). On
11924 more realistic tests the speed up is likely to be less than that.
11925 3.31.2 - 2017-09-30
11927 This release fixes some formatting and small typos/grammar issues in
11928 the documentation, specifically the page docs/settings.rst, and the in‐
11929 line docs for the various settings.
11930 3.31.1 - 2017-09-30
11932 This release improves the handling of deadlines so that they act better
11933 with the shrinking process. This fixes issue #892.
11934 This involves two changes:
11936 1. The deadline is raised during the initial generation and shrinking,
11938 and then lowered to the set value for final replay. This restricts
11939 our attention to examples which exceed the deadline by a more sig‐
11940 nificant margin, which increases their reliability.
11941 2. When despite the above a test still becomes flaky because it is sig‐
11943 nificantly faster on rerun than it was on its first run, the error
11944 message is now more explicit about the nature of this problem, and
11945 includes both the initial test run time and the new test run time.
11946 In addition, this release also clarifies the documentation of the dead‐
11948 line setting slightly to be more explicit about where it applies.
11949 This work was funded by Smarkets.
11951 3.31.0 - 2017-09-29
11953 This release blocks installation of Hypothesis on Python 3.3, which
11954 reached its end of life date on 2017-09-29.
11955 This should not be of interest to anyone but downstream maintainers -
11957 if you are affected, migrate to a secure version of Python as soon as
11958 possible or at least seek commercial support.
11959 3.30.4 - 2017-09-27
11961 This release makes several changes:
11962 1. It significantly improves Hypothesis's ability to use coverage in‐
11964 formation to find interesting examples.
11965 2. It reduces the default max_examples setting from 200 to 100. This
11967 takes advantage of the improved algorithm meaning fewer examples are
11968 typically needed to get the same testing and is sufficiently better
11969 at covering interesting behaviour, and offsets some of the perfor‐
11970 mance problems of running under coverage.
11971 3. Hypothesis will always try to start its testing with an example that
11973 is near minimized.
11974 The new algorithm for 1 also makes some changes to Hypothesis's low
11976 level data generation which apply even with coverage turned off. They
11977 generally reduce the total amount of data generated, which should im‐
11978 prove test performance somewhat. Between this and 3 you should see a
11979 noticeable reduction in test runtime (how much so depends on your tests
11980 and how much example size affects their performance. On our benchmarks,
11981 where data generation dominates, we saw up to a factor of two perfor‐
11982 mance improvement, but it's unlikely to be that large.
11983 3.30.3 - 2017-09-25
11985 This release fixes some formatting and small typos/grammar issues in
11986 the documentation, specifically the page docs/details.rst, and some in‐
11987 line docs linked from there.
11988 3.30.2 - 2017-09-24
11990 This release changes Hypothesis's caching approach for functions in hy‐
11991 pothesis.strategies. Previously it would have cached extremely aggres‐
11992 sively and cache entries would never be evicted. Now it adopts a
11993 least-frequently used, least recently used key invalidation policy, and
11994 is somewhat more conservative about which strategies it caches.
11995 Workloads which create strategies based on dynamic values, e.g. by us‐
11997 ing flatmap or composite(), will use significantly less memory.
11998 3.30.1 - 2017-09-22
12000 This release fixes a bug where when running with the use_coverage=True
12001 setting inside an existing running instance of coverage, Hypothesis
12002 would frequently put files that the coveragerc excluded in the report
12003 for the enclosing coverage.
12004 3.30.0 - 2017-09-20
12006 This release introduces two new features:
12007 • When a test fails, either with a health check failure or a falsifying
12009 example, Hypothesis will print out a seed that led to that failure,
12010 if the test is not already running with a fixed seed. You can then
12011 recreate that failure using either the @seed decorator or (if you are
12012 running pytest) with --hypothesis-seed.
12013 • pytest users can specify a seed to use for @given based tests by
12015 passing the --hypothesis-seed command line argument.
12016 This work was funded by Smarkets.
12018 3.29.0 - 2017-09-19
12020 This release makes Hypothesis coverage aware. Hypothesis now runs all
12021 test bodies under coverage, and uses this information to guide its
12022 testing.
12023 The use_coverage setting can be used to disable this behaviour if you
12025 want to test code that is sensitive to coverage being enabled (either
12026 because of performance or interaction with the trace function).
12027 The main benefits of this feature are:
12029 • Hypothesis now observes when examples it discovers cover particular
12031 lines or branches and stores them in the database for later.
12032 • Hypothesis will make some use of this information to guide its explo‐
12034 ration of the search space and improve the examples it finds (this is
12035 currently used only very lightly and will likely improve signifi‐
12036 cantly in future releases).
12037 This also has the following side-effects:
12039 • Hypothesis now has an install time dependency on the coverage pack‐
12041 age.
12042 • Tests that are already running Hypothesis under coverage will likely
12044 get faster.
12045 • Tests that are not running under coverage now run their test bodies
12047 under coverage by default.
12048 This feature is only partially supported under pypy. It is signifi‐
12050 cantly slower than on CPython and is turned off by default as a result,
12051 but it should still work correctly if you want to use it.
12052 3.28.3 - 2017-09-18
12054 This release is an internal change that affects how Hypothesis handles
12055 calculating certain properties of strategies.
12056 The primary effect of this is that it fixes a bug where use of
12058 deferred() could sometimes trigger an internal assertion error. However
12059 the fix for this bug involved some moderately deep changes to how Hy‐
12060 pothesis handles certain constructs so you may notice some additional
12061 knock-on effects.
12062 In particular the way Hypothesis handles drawing data from strategies
12064 that cannot generate any values has changed to bail out sooner than it
12065 previously did. This may speed up certain tests, but it is unlikely to
12066 make much of a difference in practice for tests that were not already
12067 failing with Unsatisfiable.
12068 3.28.2 - 2017-09-18
12070 This is a patch release that fixes a bug in the hypothesis.extra.pandas
12071 documentation where it incorrectly referred to column() instead of
12072 columns().
12073 3.28.1 - 2017-09-16
12075 This is a refactoring release. It moves a number of internal uses of
12076 namedtuple() over to using attrs based classes, and removes a couple of
12077 internal namedtuple classes that were no longer in use.
12078 It should have no user visible impact.
12080 3.28.0 - 2017-09-15
12082 This release adds support for testing pandas via the
12083 hypothesis.extra.pandas module.
12084 It also adds a dependency on attrs.
12086 This work was funded by Stripe.
12088 3.27.1 - 2017-09-14
12090 This release fixes some formatting and broken cross-references in the
12091 documentation, which includes editing docstrings - and thus a patch re‐
12092 lease.
12093 3.27.0 - 2017-09-13
12095 This release introduces a deadline setting to Hypothesis.
12096 When set this turns slow tests into errors. By default it is unset but
12098 will warn if you exceed 200ms, which will become the default value in a
12099 future release.
12100 This work was funded by Smarkets.
12102 3.26.0 - 2017-09-12
12104 Hypothesis now emits deprecation warnings if you are using the legacy
12105 SQLite example database format, or the tool for merging them. These
12106 were already documented as deprecated, so this doesn't change their
12107 deprecation status, only that we warn about it.
12108 3.25.1 - 2017-09-12
12110 This release fixes a bug with generating numpy datetime and timedelta
12111 types: When inferring the strategy from the dtype, datetime and
12112 timedelta dtypes with sub-second precision would always produce exam‐
12113 ples with one second resolution. Inferring a strategy from a time
12114 dtype will now always produce example with the same precision.
12115 3.25.0 - 2017-09-12
12117 This release changes how Hypothesis shrinks and replays examples to
12118 take into account that it can encounter new bugs while shrinking the
12119 bug it originally found. Previously it would end up replacing the orig‐
12120 inally found bug with the new bug and show you only that one. Now it is
12121 (often) able to recognise when two bugs are distinct and when it finds
12122 more than one will show both.
12123 3.24.2 - 2017-09-11
12125 This release removes the (purely internal and no longer useful) strat‐
12126 egy_test_suite function and the corresponding strategytests module.
12127 3.24.1 - 2017-09-06
12129 This release improves the reduction of examples involving floating
12130 point numbers to produce more human readable examples.
12131 It also has some general purpose changes to the way the minimizer works
12133 internally, which may see some improvement in quality and slow down of
12134 test case reduction in cases that have nothing to do with floating
12135 point numbers.
12136 3.24.0 - 2017-09-05
12138 Hypothesis now emits deprecation warnings if you use some_strategy.ex‐
12139 ample() inside a test function or strategy definition (this was never
12140 intended to be supported, but is sufficiently widespread that it war‐
12141 rants a deprecation path).
12142 3.23.3 - 2017-09-05
12144 This is a bugfix release for decimals() with the places argument.
12145 • No longer fails health checks (issue #725, due to internal filtering)
12147 • Specifying a min_value and max_value without any decimals with places
12149 places between them gives a more useful error message.
12150 • Works for any valid arguments, regardless of the decimal precision
12152 context.
12153 3.23.2 - 2017-09-01
12155 This is a small refactoring release that removes a now-unused parameter
12156 to an internal API. It shouldn't have any user visible effect.
12157 3.23.1 - 2017-09-01
12159 Hypothesis no longer propagates the dynamic scope of settings into
12160 strategy definitions.
12161 This release is a small change to something that was never part of the
12163 public API and you will almost certainly not notice any effect unless
12164 you're doing something surprising, but for example the following code
12165 will now give a different answer in some circumstances:
12166 import hypothesis.strategies as st
12168 from hypothesis import settings
12169 CURRENT_SETTINGS = st.builds(lambda: settings.default)
12171 (We don't actually encourage you writing code like this)
12173 Previously this would have generated the settings that were in effect
12175 at the point of definition of CURRENT_SETTINGS. Now it will generate
12176 the settings that are used for the current test.
12177 It is very unlikely to be significant enough to be visible, but you may
12179 also notice a small performance improvement.
12180 3.23.0 - 2017-08-31
12182 This release adds a unique argument to arrays() which behaves the same
12183 ways as the corresponding one for lists(), requiring all of the ele‐
12184 ments in the generated array to be distinct.
12185 3.22.2 - 2017-08-29
12187 This release fixes an issue where Hypothesis would raise a TypeError
12188 when using the datetime-related strategies if running with PYTHONOPTI‐
12189 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
12190 3.22.1 - 2017-08-28
12192 Hypothesis now transparently handles problems with an internal unicode
12193 cache, including file truncation or read-only filesystems (issue #767).
12194 Thanks to Sam Hames for the patch.
12195 3.22.0 - 2017-08-26
12197 This release provides what should be a substantial performance improve‐
12198 ment to numpy arrays generated using provided numpy support, and adds a
12199 new fill_value argument to arrays() to control this behaviour.
12200 This work was funded by Stripe.
12202 3.21.3 - 2017-08-26
12204 This release fixes some extremely specific circumstances that probably
12205 have never occurred in the wild where users of deferred() might have
12206 seen a python:RuntimeError from too much recursion, usually in cases
12207 where no valid example could have been generated anyway.
12208 3.21.2 - 2017-08-25
12210 This release fixes some minor bugs in argument validation:
12211 • hypothesis.extra.numpy dtype strategies would raise an internal
12213 error instead of an InvalidArgument exception when passed an in‐
12214 valid endianness specification.
12215 • fractions() would raise an internal error instead of an InvalidAr‐
12217 gument if passed float("nan") as one of its bounds.
12218 • The error message for passing float("nan") as a bound to various
12220 strategies has been improved.
12221 • Various bound arguments will now raise InvalidArgument in cases
12223 where they would previously have raised an internal TypeError or
12224 ValueError from the relevant conversion function.
12225 • streaming() would not have emitted a deprecation warning when
12227 called with an invalid argument.
12228 3.21.1 - 2017-08-24
12230 This release fixes a bug where test failures that were the result of an
12231 @example would print an extra stack trace before re-raising the excep‐
12232 tion.
12233 3.21.0 - 2017-08-23
12235 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
12236 sis's deprecation warnings into errors. Similar functionality can be
12237 achieved by using simplefilter('error', HypothesisDeprecationWarning).
12238 3.20.0 - 2017-08-22
12240 This release renames the relevant arguments on the datetimes(),
12241 dates(), times(), and timedeltas() strategies to min_value and
12242 max_value, to make them consistent with the other strategies in the
12243 module.
12244 The old argument names are still supported but will emit a deprecation
12246 warning when used explicitly as keyword arguments. Arguments passed po‐
12247 sitionally will go to the new argument names and are not deprecated.
12248 3.19.3 - 2017-08-22
12250 This release provides a major overhaul to the internals of how Hypothe‐
12251 sis handles shrinking.
12252 This should mostly be visible in terms of getting better examples for
12254 tests which make heavy use of composite(), data() or flatmap where the
12255 data drawn depends a lot on previous choices, especially where size pa‐
12256 rameters are affected. Previously Hypothesis would have struggled to
12257 reliably produce good examples here. Now it should do much better. Per‐
12258 formance should also be better for examples with a non-zero min_size.
12259 You may see slight changes to example generation (e.g. improved example
12261 diversity) as a result of related changes to internals, but they are
12262 unlikely to be significant enough to notice.
12263 3.19.2 - 2017-08-21
12265 This release fixes two bugs in hypothesis.extra.numpy:
12266 • unicode_string_dtypes() didn't work at all due to an incorrect dtype
12268 specifier. Now it does.
12269 • Various impossible conditions would have been accepted but would er‐
12271 ror when they fail to produced any example. Now they raise an ex‐
12272 plicit InvalidArgument error.
12273 3.19.1 - 2017-08-21
12275 This is a bugfix release for issue #739, where bounds for fractions()
12276 or floating-point decimals() were not properly converted to integers
12277 before passing them to the integers strategy. This excluded some val‐
12278 ues that should have been possible, and could trigger internal errors
12279 if the bounds lay between adjacent integers.
12280 You can now bound fractions() with two arbitrarily close fractions.
12282 It is now an explicit error to supply a min_value, max_value, and
12284 max_denominator to fractions() where the value bounds do not include a
12285 fraction with denominator at most max_denominator.
12286 3.19.0 - 2017-08-20
12288 This release adds the from_regex() strategy, which generates strings
12289 that contain a match of a regular expression.
12290 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
12292 then helping to upstream it! (issue #662)
12293 3.18.5 - 2017-08-18
12295 This is a bugfix release for integers(). Previously the strategy would
12296 hit an internal assertion if passed non-integer bounds for min_value
12297 and max_value that had no integers between them. The strategy now
12298 raises InvalidArgument instead.
12299 3.18.4 - 2017-08-18
12301 Release to fix a bug where mocks can be used as test runners under cer‐
12302 tain conditions. Specifically, if a mock is injected into a test via
12303 pytest fixtures or patch decorators, and that mock is the first argu‐
12304 ment in the list, hypothesis will think it represents self and turns
12305 the mock into a test runner. If this happens, the affected test always
12306 passes because the mock is executed instead of the test body. Some‐
12307 times, it will also fail health checks.
12308 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
12310 for this bug fix.
12311 3.18.3 - 2017-08-17
12313 This release should improve the performance of some tests which experi‐
12314 enced a slow down as a result of the 3.13.0 release.
12315 Tests most likely to benefit from this are ones that make extensive use
12317 of min_size parameters, but others may see some improvement as well.
12318 3.18.2 - 2017-08-16
12320 This release fixes a bug introduced in 3.18.0. If the arguments
12321 whitelist_characters and blacklist_characters to characters() contained
12322 overlapping elements, then an InvalidArgument exception would be
12323 raised.
12324 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
12326 3.18.1 - 2017-08-14
12328 This is a bug fix release to fix issue #780, where sets() and similar
12329 would trigger health check errors if their element strategy could only
12330 produce one element (e.g. if it was just()).
12331 3.18.0 - 2017-08-13
12333 This is a feature release:
12334 • characters() now accepts whitelist_characters, particular characters
12336 which will be added to those it produces. (issue #668)
12337 • A bug fix for the internal function _union_interval_lists(), and a
12339 rename to _union_intervals(). It now correctly handles all cases
12340 where intervals overlap, and it always returns the result as a tuple
12341 for tuples.
12342 Thanks to Alex Willmer for these.
12344 3.17.0 - 2017-08-07
12346 This release documents the previously undocumented phases feature, mak‐
12347 ing it part of the public API. It also updates how the example database
12348 is used. Principally:
12349 • The reuse phase will now correctly control whether examples from the
12351 database are run (it previously did exactly the wrong thing and con‐
12352 trolled whether examples would be saved).
12353 • Hypothesis will no longer try to rerun all previously failing exam‐
12355 ples. Instead it will replay the smallest previously failing example
12356 and a selection of other examples that are likely to trigger any
12357 other bugs that will found. This prevents a previous failure from
12358 dominating your tests unnecessarily.
12359 • As a result of the previous change, Hypothesis will be slower about
12361 clearing out old examples from the database that are no longer fail‐
12362 ing (because it can only clear out ones that it actually runs).
12363 3.16.1 - 2017-08-07
12365 This release makes an implementation change to how Hypothesis handles
12366 certain internal constructs.
12367 The main effect you should see is improvement to the behaviour and per‐
12369 formance of collection types, especially ones with a min_size parame‐
12370 ter. Many cases that would previously fail due to being unable to gen‐
12371 erate enough valid examples will now succeed, and other cases should
12372 run slightly faster.
12373 3.16.0 - 2017-08-04
12375 This release introduces a deprecation of the timeout feature. This re‐
12376 sults in the following changes:
12377 • Creating a settings object with an explicit timeout will emit a dep‐
12379 recation warning.
12380 • If your test stops because it hits the timeout (and has not found a
12382 bug) then it will emit a deprecation warning.
12383 • There is a new value unlimited which you can import from hypothesis.
12385 settings(timeout=unlimited) will not cause a deprecation warning.
12386 • There is a new health check, hung_test, which will trigger after a
12388 test has been running for five minutes if it is not suppressed.
12389 3.15.0 - 2017-08-04
12391 This release deprecates two strategies, choices() and streaming().
12392 Both of these are somewhat confusing to use and are entirely redundant
12394 since the introduction of the data() strategy for interactive drawing
12395 in tests, and their use should be replaced with direct use of data()
12396 instead.
12397 3.14.2 - 2017-08-03
12399 This fixes a bug where Hypothesis would not work correctly on Python
12400 2.7 if you had the python:typing module backport installed.
12401 3.14.1 - 2017-08-02
12403 This raises the maximum depth at which Hypothesis starts cutting off
12404 data generation to a more reasonable value which it is harder to hit by
12405 accident.
12406 This resolves (issue #751), in which some examples which previously
12408 worked would start timing out, but it will also likely improve the data
12409 generation quality for complex data types.
12410 3.14.0 - 2017-07-23
12412 Hypothesis now understands inline type annotations (issue #293):
12413 • If the target of builds() has type annotations, a default strategy
12415 for missing required arguments is selected based on the type.
12416 Type-based strategy selection will only override a default if you
12417 pass hypothesis.infer as a keyword argument.
12418 • If @given wraps a function with type annotations, you can pass infer
12420 as a keyword argument and the appropriate strategy will be substi‐
12421 tuted.
12422 • You can check what strategy will be inferred for a type with the new
12424 from_type() function.
12425 • register_type_strategy() teaches Hypothesis which strategy to infer
12427 for custom or unknown types. You can provide a strategy, or for more
12428 complex cases a function which takes the type and returns a strategy.
12429 3.13.1 - 2017-07-20
12431 This is a bug fix release for issue #514 - Hypothesis would continue
12432 running examples after a SkipTest exception was raised, including
12433 printing a falsifying example. Skip exceptions from the standard
12434 python:unittest module, and pytest, nose, or unittest2 modules now
12435 abort the test immediately without printing output.
12436 3.13.0 - 2017-07-16
12438 This release has two major aspects to it: The first is the introduction
12439 of deferred(), which allows more natural definition of recursive (in‐
12440 cluding mutually recursive) strategies.
12441 The second is a number of engine changes designed to support this sort
12443 of strategy better. These should have a knock-on effect of also improv‐
12444 ing the performance of any existing strategies that currently generate
12445 a lot of data or involve heavy nesting by reducing their typical exam‐
12446 ple size.
12447 3.12.0 - 2017-07-07
12449 This release makes some major internal changes to how Hypothesis repre‐
12450 sents data internally, as a prelude to some major engine changes that
12451 should improve data quality. There are no API changes, but it's a sig‐
12452 nificant enough internal change that a minor version bump seemed war‐
12453 ranted.
12454 User facing impact should be fairly mild, but includes:
12456 • All existing examples in the database will probably be invalidated.
12458 Hypothesis handles this automatically, so you don't need to do any‐
12459 thing, but if you see all your examples disappear that's why.
12460 • Almost all data distributions have changed significantly. Possibly
12462 for the better, possibly for the worse. This may result in new bugs
12463 being found, but it may also result in Hypothesis being unable to
12464 find bugs it previously did.
12465 • Data generation may be somewhat faster if your existing bottleneck
12467 was in draw_bytes (which is often the case for large examples).
12468 • Shrinking will probably be slower, possibly significantly.
12470 If you notice any effects you consider to be a significant regression,
12472 please open an issue about them.
12473 3.11.6 - 2017-06-19
12475 This release involves no functionality changes, but is the first to
12476 ship wheels as well as an sdist.
12477 3.11.5 - 2017-06-18
12479 This release provides a performance improvement to shrinking. For cases
12480 where there is some non-trivial "boundary" value (e.g. the bug happens
12481 for all values greater than some other value), shrinking should now be
12482 substantially faster. Other types of bug will likely see improvements
12483 too.
12484 This may also result in some changes to the quality of the final exam‐
12486 ples - it may sometimes be better, but is more likely to get slightly
12487 worse in some edge cases. If you see any examples where this happens in
12488 practice, please report them.
12489 3.11.4 - 2017-06-17
12491 This is a bugfix release: Hypothesis now prints explicit examples when
12492 running in verbose mode. (issue #313)
12493 3.11.3 - 2017-06-11
12495 This is a bugfix release: Hypothesis no longer emits a warning if you
12496 try to use sampled_from() with python:collections.OrderedDict. (issue
12497 #688)
12498 3.11.2 - 2017-06-10
12500 This is a documentation release. Several outdated snippets have been
12501 updated or removed, and many cross-references are now hyperlinks.
12502 3.11.1 - 2017-05-28
12504 This is a minor ergonomics release. Tracebacks shown by pytest no
12505 longer include Hypothesis internals for test functions decorated with
12506 @given.
12507 3.11.0 - 2017-05-23
12509 This is a feature release, adding datetime-related strategies to the
12510 core strategies.
12511 timezones() allows you to sample pytz timezones from the Olsen data‐
12513 base. Use directly in a recipe for tz-aware datetimes, or compose with
12514 none() to allow a mix of aware and naive output.
12515 The new dates(), times(), datetimes(), and timedeltas() strategies are
12517 all constrained by objects of their type. This means that you can gen‐
12518 erate dates bounded by a single day (i.e. a single date), or datetimes
12519 constrained to the microsecond.
12520 times() and datetimes() take an optional timezones= argument, which de‐
12522 faults to none() for naive times. You can use our extra strategy based
12523 on pytz, or roll your own timezones strategy with dateutil or even the
12524 standard library.
12525 The old dates, times, and datetimes strategies in hypothesis.ex‐
12527 tra.datetimes are deprecated in favor of the new core strategies, which
12528 are more flexible and have no dependencies.
12529 3.10.0 - 2017-05-22
12531 Hypothesis now uses python:inspect.getfullargspec() internally. On
12532 Python 2, there are no visible changes.
12533 On Python 3 @given and @composite now preserve PEP 3107 annotations on
12535 the decorated function. Keyword-only arguments are now either handled
12536 correctly (e.g. @composite), or caught in validation instead of
12537 silently discarded or raising an unrelated error later (e.g. @given).
12538 3.9.1 - 2017-05-22
12540 This is a bugfix release: the default field mapping for a DateTimeField
12541 in the Django extra now respects the USE_TZ setting when choosing a
12542 strategy.
12543 3.9.0 - 2017-05-19
12545 This is feature release, expanding the capabilities of the decimals()
12546 strategy.
12547 • The new (optional) places argument allows you to generate decimals
12549 with a certain number of places (e.g. cents, thousandths, satoshis).
12550 • If allow_infinity is None, setting min_bound no longer excludes posi‐
12552 tive infinity and setting max_value no longer excludes negative in‐
12553 finity.
12554 • All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
12556 True, or if allow_nan is None and min_value or max_value is None.
12557 • min_value and max_value may be given as decimal strings, e.g.
12559 "1.234".
12560 3.8.5 - 2017-05-16
12562 Hypothesis now imports python:sqlite3 when a SQLite database is used,
12563 rather than at module load, improving compatibility with Python imple‐
12564 mentations compiled without SQLite support (such as BSD or Jython).
12565 3.8.4 - 2017-05-16
12567 This is a compatibility bugfix release. sampled_from() no longer
12568 raises a deprecation warning when sampling from an python:enum.Enum, as
12569 all enums have a reliable iteration order.
12570 3.8.3 - 2017-05-09
12572 This release removes a version check for older versions of pytest when
12573 using the Hypothesis pytest plugin. The pytest plugin will now run un‐
12574 conditionally on all versions of pytest. This breaks compatibility with
12575 any version of pytest prior to 2.7.0 (which is more than two years
12576 old).
12577 The primary reason for this change is that the version check was a fre‐
12579 quent source of breakage when pytest change their versioning scheme. If
12580 you are not working on pytest itself and are not running a very old
12581 version of it, this release probably doesn't affect you.
12582 3.8.2 - 2017-04-26
12584 This is a code reorganisation release that moves some internal test
12585 helpers out of the main source tree so as to not have changes to them
12586 trigger releases in future.
12587 3.8.1 - 2017-04-26
12589 This is a documentation release. Almost all code examples are now
12590 doctests checked in CI, eliminating stale examples.
12591 3.8.0 - 2017-04-23
12593 This is a feature release, adding the iterables() strategy, equivalent
12594 to lists(...).map(iter) but with a much more useful repr. You can use
12595 this strategy to check that code doesn't accidentally depend on se‐
12596 quence properties such as indexing support or repeated iteration.
12597 3.7.4 - 2017-04-22
12599 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
12600 ture in the same test could cause the test to fail to fill the argu‐
12601 ments, and throw a TypeError.
12602 3.7.3 - 2017-04-21
12604 This release should include no user visible changes and is purely a
12605 refactoring release. This modularises the behaviour of the core given()
12606 function, breaking it up into smaller and more accessible parts, but
12607 its actual behaviour should remain unchanged.
12608 3.7.2 - 2017-04-21
12610 This reverts an undocumented change in 3.7.1 which broke installation
12611 on debian stable: The specifier for the hypothesis[django] extra_re‐
12612 quires had introduced a wild card, which was not supported on the de‐
12613 fault version of pip.
12614 3.7.1 - 2017-04-21
12616 This is a bug fix and internal improvements release.
12617 • In particular Hypothesis now tracks a tree of where it has already
12619 explored. This allows it to avoid some classes of duplicate exam‐
12620 ples, and significantly improves the performance of shrinking failing
12621 examples by allowing it to skip some shrinks that it can determine
12622 can't possibly work.
12623 • Hypothesis will no longer seed the global random arbitrarily unless
12625 you have asked it to using random_module()
12626 • Shrinking would previously have not worked correctly in some special
12628 cases on Python 2, and would have resulted in suboptimal examples.
12629 3.7.0 - 2017-03-20
12631 This is a feature release.
12632 New features:
12634 • Rule based stateful testing now has an @invariant decorator that
12636 specifies methods that are run after init and after every step, al‐
12637 lowing you to encode properties that should be true at all times.
12638 Thanks to Tom Prince for this feature.
12639 • The decimals() strategy now supports allow_nan and allow_infinity
12641 flags.
12642 • There are significantly more strategies available for numpy, includ‐
12644 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
12645 for this feature.
12646 • When using the data() strategy you can now add a label as an argument
12648 to draw(), which will be printed along with the value when an example
12649 fails. Thanks to Peter Inglesby for this feature.
12650 Bug fixes:
12652 • Bug fix: composite() now preserves functions' docstrings.
12654 • The build is now reproducible and doesn't depend on the path you
12656 build it from. Thanks to Chris Lamb for this feature.
12657 • numpy strategies for the void data type did not work correctly.
12659 Thanks to Zac Hatfield Dodds for this fix.
12660 There have also been a number of performance optimizations:
12662 • The permutations() strategy is now significantly faster to use for
12664 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
12665 • Shrinking of failing test cases should have got significantly faster
12667 in some circumstances where it was previously struggling for a long
12668 time.
12669 • Example generation now involves less indirection, which results in a
12671 small speedup in some cases (small enough that you won't really no‐
12672 tice it except in pathological cases).
12673 3.6.1 - 2016-12-20
12675 This release fixes a dependency problem and makes some small behind the
12676 scenes improvements.
12677 • The fake-factory dependency was renamed to faker. If you were depend‐
12679 ing on it through hypothesis[django] or hypothesis[fake-factory]
12680 without pinning it yourself then it would have failed to install
12681 properly. This release changes it so that hypothesis[fakefactory]
12682 (which can now also be installed as hypothesis[faker]) will install
12683 the renamed faker package instead.
12684 • This release also removed the dependency of hypothesis[django] on hy‐
12686 pothesis[fakefactory] - it was only being used for emails. These now
12687 use a custom strategy that isn't from fakefactory. As a result you
12688 should also see performance improvements of tests which generated
12689 User objects or other things with email fields, as well as better
12690 shrinking of email addresses.
12691 • The distribution of code using nested calls to one_of() or the | op‐
12693 erator for combining strategies has been improved, as branches are
12694 now flattened to give a more uniform distribution.
12695 • Examples using composite() or .flatmap should now shrink better. In
12697 particular this will affect things which work by first generating a
12698 length and then generating that many items, which have historically
12699 not shrunk very well.
12700 3.6.0 - 2016-10-31
12702 This release reverts Hypothesis to its old pretty printing of lambda
12703 functions based on attempting to extract the source code rather than
12704 decompile the bytecode. This is unfortunately slightly inferior in
12705 some cases and may result in you occasionally seeing things like lambda
12706 x: <unknown> in statistics reports and strategy reprs.
12707 This removes the dependencies on uncompyle6, xdis and spark-parser.
12709 The reason for this is that the new functionality was based on un‐
12711 compyle6, which turns out to introduce a hidden GPLed dependency - it
12712 in turn depended on xdis, and although the library was licensed under
12713 the MIT license, it contained some GPL licensed source code and thus
12714 should have been released under the GPL.
12715 My interpretation is that Hypothesis itself was never in violation of
12717 the GPL (because the license it is under, the Mozilla Public License
12718 v2, is fully compatible with being included in a GPL licensed work),
12719 but I have not consulted a lawyer on the subject. Regardless of the an‐
12720 swer to this question, adding a GPLed dependency will likely cause a
12721 lot of users of Hypothesis to inadvertently be in violation of the GPL.
12722 As a result, if you are running Hypothesis 3.5.x you really should up‐
12724 grade to this release immediately.
12725 3.5.3 - 2016-10-05
12727 This is a bug fix release.
12728 Bugs fixed:
12730 • If the same test was running concurrently in two processes and there
12732 were examples already in the test database which no longer failed,
12733 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
12734 Python 2) because an example it was trying to read was deleted before
12735 it was read. (issue #372).
12736 • Drawing from an integers() strategy with both a min_value and a
12738 max_value would reject too many examples needlessly. Now it repeat‐
12739 edly redraws until satisfied. (pull request #366. Thanks to Calen
12740 Pennington for the contribution).
12741 3.5.2 - 2016-09-24
12743 This is a bug fix release.
12744 • The Hypothesis pytest plugin broke pytest support for doctests. Now
12746 it doesn't.
12747 3.5.1 - 2016-09-23
12749 This is a bug fix release.
12750 • Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
12752 bytes and unicode.
12753 • python:unittest.TestCase tests would not have shown up in the new
12755 statistics mode. Now they do.
12756 • Similarly, stateful tests would not have shown up in statistics and
12758 now they do.
12759 • Statistics now print with pytest node IDs (the names you'd get in
12761 pytest verbose mode).
12762 3.5.0 - 2016-09-22
12764 This is a feature release.
12765 • fractions() and decimals() strategies now support min_value and
12767 max_value parameters. Thanks go to Anne Mulhern for the development
12768 of this feature.
12769 • The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
12771 tics parameter that gives detailed statistics about the tests that
12772 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
12773 funding the development of this feature.
12774 • There is a new event() function that can be used to add custom sta‐
12776 tistics.
12777 Additionally there have been some minor bug fixes:
12779 • In some cases Hypothesis should produce fewer duplicate examples
12781 (this will mostly only affect cases with a single parameter).
12782 • pytest command line parameters are now under an option group for Hy‐
12784 pothesis (thanks to David Keijser for fixing this)
12785 • Hypothesis would previously error if you used PEP 3107 function anno‐
12787 tations on your tests under Python 3.4.
12788 • The repr of many strategies using lambdas has been improved to in‐
12790 clude the lambda body (this was previously supported in many but not
12791 all cases).
12792 3.4.2 - 2016-07-13
12794 This is a bug fix release, fixing a number of problems with the set‐
12795 tings system:
12796 • Test functions defined using @given can now be called from other
12798 threads (issue #337)
12799 • Attempting to delete a settings property would previously have
12801 silently done the wrong thing. Now it raises an AttributeError.
12802 • Creating a settings object with a custom database_file parameter was
12804 silently getting ignored and the default was being used instead. Now
12805 it's not.
12806 3.4.1 - 2016-07-07
12808 This is a bug fix release for a single bug:
12809 • On Windows when running two Hypothesis processes in parallel (e.g.
12811 using pytest-xdist) they could race with each other and one would
12812 raise an exception due to the non-atomic nature of file renaming on
12813 Windows and the fact that you can't rename over an existing file.
12814 This is now fixed.
12815 3.4.0 - 2016-05-27
12817 This release is entirely provided by Lucas Wiman:
12818 Strategies constructed by the Django extra will now respect much more
12820 of Django's validations out of the box. Wherever possible,
12821 full_clean() should succeed.
12822 In particular:
12824 • The max_length, blank and choices kwargs are now respected.
12826 • Add support for DecimalField.
12828 • If a field includes validators, the list of validators are used to
12830 filter the field strategy.
12831 3.3.0 - 2016-05-27
12833 This release went wrong and is functionally equivalent to 3.2.0. Ignore
12834 it.
12835 3.2.0 - 2016-05-19
12837 This is a small single-feature release:
12838 • All tests using @given now fix the global random seed. This removes
12840 the health check for that. If a non-zero seed is required for the fi‐
12841 nal falsifying example, it will be reported. Otherwise Hypothesis
12842 will assume randomization was not a significant factor for the test
12843 and be silent on the subject. If you use random_module() this will
12844 continue to work and will always display the seed.
12845 3.1.3 - 2016-05-01
12847 Single bug fix release
12848 • Another charmap problem. In 3.1.2 text() and characters() would break
12850 on systems which had /tmp mounted on a different partition than the
12851 Hypothesis storage directory (usually in home). This fixes that.
12852 3.1.2 - 2016-04-30
12854 Single bug fix release:
12855 • Anything which used a text() or characters() strategy was broken on
12857 Windows and I hadn't updated appveyor to use the new repository loca‐
12858 tion so I didn't notice. This is now fixed and windows support should
12859 work correctly.
12860 3.1.1 - 2016-04-29
12862 Minor bug fix release.
12863 • Fix concurrency issue when running tests that use text() from multi‐
12865 ple processes at once (issue #302, thanks to Alex Chan).
12866 • Improve performance of code using lists() with max_size (thanks to
12868 Cristi Cobzarenco).
12869 • Fix install on Python 2 with ancient versions of pip so that it in‐
12871 stalls the enum34 backport (thanks to Donald Stufft for telling me
12872 how to do this).
12873 • Remove duplicated __all__ exports from hypothesis.strategies (thanks
12875 to Piët Delport).
12876 • Update headers to point to new repository location.
12878 • Allow use of strategies that can't be used in find() (e.g. choices())
12880 in stateful testing.
12881 3.1.0 - 2016-03-06
12883 • Add a nothing() strategy that never successfully generates values.
12884 • sampled_from() and one_of() can both now be called with an empty ar‐
12886 gument list, in which case they also never generate any values.
12887 • one_of() may now be called with a single argument that is a collec‐
12889 tion of strategies as well as as varargs.
12890 • Add a runner() strategy which returns the instance of the current
12892 test object if there is one.
12893 • 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
12895 can be used as such.
12896 • Tests using RuleBasedStateMachine should now shrink significantly
12898 better.
12899 • Hypothesis now uses a pretty-printing library internally, compatible
12901 with IPython's pretty printing protocol (actually using the same
12902 code). This may improve the quality of output in some cases.
12903 • Add a 'phases' setting that allows more fine grained control over
12905 which parts of the process Hypothesis runs
12906 • Add a suppress_health_check setting which allows you to turn off spe‐
12908 cific health checks in a fine grained manner.
12909 • Fix a bug where lists of non fixed size would always draw one more
12911 element than they included. This mostly didn't matter, but if would
12912 cause problems with empty strategies or ones with side effects.
12913 • Add a mechanism to the Django model generator to allow you to explic‐
12915 itly request the default value (thanks to Jeremy Thurgood for this
12916 one).
12917 3.0.5 - 2016-02-25
12919 • Fix a bug where Hypothesis would now error on pytest development ver‐
12920 sions.
12921 3.0.4 - 2016-02-24
12923 • Fix a bug where Hypothesis would error when running on Python 2.7.3
12924 or earlier because it was trying to pass a python:bytearray object to
12925 python:struct.unpack() (which is only supported since 2.7.4).
12926 3.0.3 - 2016-02-23
12928 • Fix version parsing of pytest to work with pytest release candidates
12929 • More general handling of the health check problem where things could
12931 fail because of a cache miss - now one "free" example is generated
12932 before the start of the health check run.
12933 3.0.2 - 2016-02-18
12935 • Under certain circumstances, strategies involving text() buried in‐
12936 side some other strategy (e.g. text().filter(...) or recur‐
12937 sive(text(), ...)) would cause a test to fail its health checks the
12938 first time it ran. This was caused by having to compute some related
12939 data and cache it to disk. On travis or anywhere else where the .hy‐
12940 pothesis directory was recreated this would have caused the tests to
12941 fail their health check on every run. This is now fixed for all the
12942 known cases, although there could be others lurking.
12943 3.0.1 - 2016-02-18
12945 • Fix a case where it was possible to trigger an "Unreachable" asser‐
12946 tion when running certain flaky stateful tests.
12947 • Improve shrinking of large stateful tests by eliminating a case where
12949 it was hard to delete early steps.
12950 • Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
12952 cantly by provide a custom implementation for fixed size blocks that
12953 can bypass a lot of machinery.
12954 • Set default home directory based on the current working directory at
12956 the point Hypothesis is imported, not whenever the function first
12957 happens to be called.
12958 3.0.0 - 2016-02-17
12960 Codename: This really should have been 2.1.
12961 Externally this looks like a very small release. It has one small
12963 breaking change that probably doesn't affect anyone at all (some behav‐
12964 iour that never really worked correctly is now outright forbidden) but
12965 necessitated a major version bump and one visible new feature.
12966 Internally this is a complete rewrite. Almost nothing other than the
12968 public API is the same.
12969 New features:
12971 • Addition of data() strategy which allows you to draw arbitrary data
12973 interactively within the test.
12974 • New "exploded" database format which allows you to more easily check
12976 the example database into a source repository while supporting merg‐
12977 ing.
12978 • Better management of how examples are saved in the database.
12980 • Health checks will now raise as errors when they fail. It was too
12982 easy to have the warnings be swallowed entirely.
12983 New limitations:
12985 • choices() and streaming() strategies may no longer be used with
12987 find(). Neither may data() (this is the change that necessitated a
12988 major version bump).
12989 Feature removal:
12991 • The ForkingTestCase executor has gone away. It may return in some
12993 more working form at a later date.
12994 Performance improvements:
12996 • A new model which allows flatmap, composite strategies and stateful
12998 testing to perform much better. They should also be more reliable.
12999 • Filtering may in some circumstances have improved significantly. This
13001 will help especially in cases where you have lots of values with in‐
13002 dividual filters on them, such as lists(x.filter(...)).
13003 • Modest performance improvements to the general test runner by avoid‐
13005 ing expensive operations
13006 In general your tests should have got faster. If they've instead got
13008 significantly slower, I'm interested in hearing about it.
13009 Data distribution:
13011 The data distribution should have changed significantly. This may un‐
13013 cover bugs the previous version missed. It may also miss bugs the pre‐
13014 vious version could have uncovered. Hypothesis is now producing less
13015 strongly correlated data than it used to, but the correlations are ex‐
13016 tended over more of the structure.
13017 Shrinking:
13019 Shrinking quality should have improved. In particular Hypothesis can
13021 now perform simultaneous shrinking of separate examples within a single
13022 test (previously it was only able to do this for elements of a single
13023 collection). In some cases performance will have improved, in some
13024 cases it will have got worse but generally shouldn't have by much.
13025 Older versions
13027 2.0.0 - 2016-01-10
13028 Codename: A new beginning
13029 This release cleans up all of the legacy that accrued in the course of
13031 Hypothesis 1.0. These are mostly things that were emitting deprecation
13032 warnings in 1.19.0, but there were a few additional changes.
13033 In particular:
13035 • non-strategy values will no longer be converted to strategies when
13037 used in given or find.
13038 • FailedHealthCheck is now an error and not a warning.
13040 • Handling of non-ascii reprs in user types have been simplified by us‐
13042 ing raw strings in more places in Python 2.
13043 • given no longer allows mixing positional and keyword arguments.
13045 • given no longer works with functions with defaults.
13047 • given no longer turns provided arguments into defaults - they will
13049 not appear in the argspec at all.
13050 • the basic() strategy no longer exists.
13052 • the n_ary_tree strategy no longer exists.
13054 • the average_list_length setting no longer exists. Note: If you're us‐
13056 ing using recursive() this will cause you a significant slow down.
13057 You should pass explicit average_size parameters to collections in
13058 recursive calls.
13059 • @rule can no longer be applied to the same method twice.
13061 • Python 2.6 and 3.3 are no longer officially supported, although in
13063 practice they still work fine.
13064 This also includes two non-deprecation changes:
13066 • given's keyword arguments no longer have to be the rightmost argu‐
13068 ments and can appear anywhere in the method signature.
13069 • The max_shrinks setting would sometimes not have been respected.
13071 1.19.0 - 2016-01-09
13073 Codename: IT COMES
13074 This release heralds the beginning of a new and terrible age of Hypoth‐
13076 esis 2.0.
13077 It's primary purpose is some final deprecations prior to said release.
13079 The goal is that if your code emits no warnings under this release then
13080 it will probably run unchanged under Hypothesis 2.0 (there are some
13081 caveats to this: 2.0 will drop support for some Python versions, and if
13082 you're using internal APIs then as usual that may break without warn‐
13083 ing).
13084 It does have two new features:
13086 • New @seed() decorator which allows you to manually seed a test. This
13088 may be harmlessly combined with and overrides the derandomize set‐
13089 ting.
13090 • settings objects may now be used as a decorator to fix those settings
13092 to a particular @given test.
13093 API changes (old usage still works but is deprecated):
13095 • Settings has been renamed to settings (lower casing) in order to make
13097 the decorator usage more natural.
13098 • Functions for the storage directory that were in hypothesis.settings
13100 are now in a new hypothesis.configuration module.
13101 Additional deprecations:
13103 • the average_list_length setting has been deprecated in favour of be‐
13105 ing explicit.
13106 • the basic() strategy has been deprecated as it is impossible to sup‐
13108 port it under a Conjecture based model, which will hopefully be im‐
13109 plemented at some point in the 2.x series.
13110 • the n_ary_tree strategy (which was never actually part of the public
13112 API) has been deprecated.
13113 • Passing settings or random as keyword arguments to given is depre‐
13115 cated (use the new functionality instead)
13116 Bug fixes:
13118 • No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
13120 tion in streaming() values.
13121 • When running in health check mode with non strict, don't print quite
13123 so many errors for an exception in reify.
13124 • When an assumption made in a test or a filter is flaky, tests will
13126 now raise Flaky instead of UnsatisfiedAssumption.
13127 1.18.1 - 2015-12-22
13129 Two behind the scenes changes:
13130 • Hypothesis will no longer write generated code to the file system.
13132 This will improve performance on some systems (e.g. if you're using
13133 PythonAnywhere which is running your code from NFS) and prevent some
13134 annoying interactions with auto-restarting systems.
13135 • Hypothesis will cache the creation of some strategies. This can sig‐
13137 nificantly improve performance for code that uses flatmap or compos‐
13138 ite and thus has to instantiate strategies a lot.
13139 1.18.0 - 2015-12-21
13141 Features:
13142 • Tests and find are now explicitly seeded off the global random mod‐
13144 ule. This means that if you nest one inside the other you will now
13145 get a health check error. It also means that you can control global
13146 randomization by seeding random.
13147 • There is a new random_module() strategy which seeds the global random
13149 module for you and handles things so that you don't get a health
13150 check warning if you use it inside your tests.
13151 • floats() now accepts two new arguments: allow_nan and allow_infinity.
13153 These default to the old behaviour, but when set to False will do
13154 what the names suggest.
13155 Bug fixes:
13157 • Fix a bug where tests that used text() on Python 3.4+ would not actu‐
13159 ally be deterministic even when explicitly seeded or using the deran‐
13160 domize mode, because generation depended on dictionary iteration or‐
13161 der which was affected by hash randomization.
13162 • Fix a bug where with complicated strategies the timing of the initial
13164 health check could affect the seeding of the subsequent test, which
13165 would also render supposedly deterministic tests non-deterministic in
13166 some scenarios.
13167 • In some circumstances flatmap() could get confused by two struc‐
13169 turally similar things it could generate and would produce a flaky
13170 test where the first time it produced an error but the second time it
13171 produced the other value, which was not an error. The same bug was
13172 presumably also possible in composite().
13173 • flatmap() and composite() initial generation should now be moderately
13175 faster. This will be particularly noticeable when you have many val‐
13176 ues drawn from the same strategy in a single run, e.g. constructs
13177 like lists(s.flatmap(f)). Shrinking performance may have suffered,
13178 but this didn't actually produce an interestingly worse result in any
13179 of the standard scenarios tested.
13180 1.17.1 - 2015-12-16
13182 A small bug fix release, which fixes the fact that the 'note' function
13183 could not be used on tests which used the @example decorator to provide
13184 explicit examples.
13185 1.17.0 - 2015-12-15
13187 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
13188 because it contains the following additional change that was not in‐
13189 tended to be in a patch release (it's perfectly stable, but is a
13190 larger change that should have required a minor version bump):
13191 • Hypothesis will now perform a series of "health checks" as part of
13193 running your tests. These detect and warn about some common error
13194 conditions that people often run into which wouldn't necessarily have
13195 caused the test to fail but would cause e.g. degraded performance or
13196 confusing results.
13197 1.16.1 - 2015-12-14
13199 Note: This release has been removed.
13200 A small bugfix release that allows bdists for Hypothesis to be built
13202 under 2.7 - the compat3.py file which had Python 3 syntax wasn't in‐
13203 tended to be loaded under Python 2, but when building a bdist it was.
13204 In particular this would break running setup.py test.
13205 1.16.0 - 2015-12-08
13207 There are no public API changes in this release but it includes a be‐
13208 haviour change that I wasn't comfortable putting in a patch release.
13209 • Functions from hypothesis.strategies will no longer raise InvalidAr‐
13211 gument on bad arguments. Instead the same errors will be raised when
13212 a test using such a strategy is run. This may improve startup time in
13213 some cases, but the main reason for it is so that errors in strate‐
13214 gies won't cause errors in loading, and it can interact correctly
13215 with things like pytest.mark.skipif.
13216 • Errors caused by accidentally invoking the legacy API are now much
13218 less confusing, although still throw NotImplementedError.
13219 • hypothesis.extra.django is 1.9 compatible.
13221 • When tests are run with max_shrinks=0 this will now still rerun the
13223 test on failure and will no longer print "Trying example:" before
13224 each run. Additionally note() will now work correctly when used with
13225 max_shrinks=0.
13226 1.15.0 - 2015-11-24
13228 A release with two new features.
13229 • A 'characters' strategy for more flexible generation of text with
13231 particular character ranges and types, kindly contributed by
13232 Alexander Shorin.
13233 • Add support for preconditions to the rule based stateful testing.
13235 Kindly contributed by Christopher Armstrong
13236 1.14.0 - 2015-11-01
13238 New features:
13239 • Add 'note' function which lets you include additional information in
13241 the final test run's output.
13242 • Add 'choices' strategy which gives you a choice function that emu‐
13244 lates random.choice.
13245 • Add 'uuid' strategy that generates UUIDs'
13247 • Add 'shared' strategy that lets you create a strategy that just gen‐
13249 erates a single shared value for each test run
13250 Bugs:
13252 • Using strategies of the form streaming(x.flatmap(f)) with find or in
13254 stateful testing would have caused InvalidArgument errors when the
13255 resulting values were used (because code that expected to only be
13256 called within a test context would be invoked).
13257 1.13.0 - 2015-10-29
13259 This is quite a small release, but deprecates some public API functions
13260 and removes some internal API functionality so gets a minor version
13261 bump.
13262 • All calls to the 'strategy' function are now deprecated, even ones
13264 which pass just a SearchStrategy instance (which is still a no-op).
13265 • Never documented hypothesis.extra entry_points mechanism has now been
13267 removed ( it was previously how hypothesis.extra packages were loaded
13268 and has been deprecated and unused for some time)
13269 • Some corner cases that could previously have produced an OverflowEr‐
13271 ror when simplifying failing cases using hypothesis.extra.datetimes
13272 (or dates or times) have now been fixed.
13273 • Hypothesis load time for first import has been significantly reduced
13275 - it used to be around 250ms (on my SSD laptop) and now is around
13276 100-150ms. This almost never matters but was slightly annoying when
13277 using it in the console.
13278 • hypothesis.strategies.randoms was previously missing from __all__.
13280 1.12.0 - 2015-10-18
13282 • Significantly improved performance of creating strategies using the
13283 functions from the hypothesis.strategies module by deferring the cal‐
13284 culation of their repr until it was needed. This is unlikely to have
13285 been an performance issue for you unless you were using flatmap, com‐
13286 posite or stateful testing, but for some cases it could be quite a
13287 significant impact.
13288 • A number of cases where the repr of strategies build from lambdas is
13290 improved
13291 • Add dates() and times() strategies to hypothesis.extra.datetimes
13293 • Add new 'profiles' mechanism to the settings system
13295 • Deprecates mutability of Settings, both the Settings.default top
13297 level property and individual settings.
13298 • A Settings object may now be directly initialized from a parent Set‐
13300 tings.
13301 • @given should now give a better error message if you attempt to use
13303 it with a function that uses destructuring arguments (it still won't
13304 work, but it will error more clearly),
13305 • A number of spelling corrections in error messages
13307 • pytest should no longer display the intermediate modules Hypothesis
13309 generates when running in verbose mode
13310 • Hypothesis should now correctly handle printing objects with
13312 non-ascii reprs on python 3 when running in a locale that cannot han‐
13313 dle ascii printing to stdout.
13314 • Add a unique=True argument to lists(). This is equivalent to
13316 unique_by=lambda x: x, but offers a more convenient syntax.
13317 1.11.4 - 2015-09-27
13319 • Hide modifications Hypothesis needs to make to sys.path by undoing
13320 them after we've imported the relevant modules. This is a workaround
13321 for issues cryptography experienced on windows.
13322 • Slightly improved performance of drawing from sampled_from on large
13324 lists of alternatives.
13325 • Significantly improved performance of drawing from one_of or strate‐
13327 gies using | (note this includes a lot of strategies internally -
13328 floats() and integers() both fall into this category). There turned
13329 out to be a massive performance regression introduced in 1.10.0 af‐
13330 fecting these which probably would have made tests using Hypothesis
13331 significantly slower than they should have been.
13332 1.11.3 - 2015-09-23
13334 • Better argument validation for datetimes() strategy - previously set‐
13335 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
13336 would not have raised an InvalidArgument error and instead would have
13337 behaved confusingly.
13338 • Compatibility with being run on pytest < 2.7 (achieved by disabling
13340 the plugin).
13341 1.11.2 - 2015-09-23
13343 Bug fixes:
13344 • Settings(database=my_db) would not be correctly inherited when used
13346 as a default setting, so that newly created settings would use the
13347 database_file setting and create an SQLite example database.
13348 • Settings.default.database = my_db would previously have raised an er‐
13350 ror and now works.
13351 • Timeout could sometimes be significantly exceeded if during simplifi‐
13353 cation there were a lot of examples tried that didn't trigger the
13354 bug.
13355 • When loading a heavily simplified example using a basic() strategy
13357 from the database this could cause Python to trigger a recursion er‐
13358 ror.
13359 • Remove use of deprecated API in pytest plugin so as to not emit warn‐
13361 ing
13362 Misc:
13364 • hypothesis-pytest is now part of hypothesis core. This should have no
13366 externally visible consequences, but you should update your dependen‐
13367 cies to remove hypothesis-pytest and depend on only Hypothesis.
13368 • Better repr for hypothesis.extra.datetimes() strategies.
13370 • Add .close() method to abstract base class for Backend (it was al‐
13372 ready present in the main implementation).
13373 1.11.1 - 2015-09-16
13375 Bug fixes:
13376 • When running Hypothesis tests in parallel (e.g. using pytest-xdist)
13378 there was a race condition caused by code generation.
13379 • Example databases are now cached per thread so as to not use sqlite
13381 connections from multiple threads. This should make Hypothesis now
13382 entirely thread safe.
13383 • floats() with only min_value or max_value set would have had a very
13385 bad distribution.
13386 • Running on 3.5, Hypothesis would have emitted deprecation warnings
13388 because of use of inspect.getargspec
13389 1.11.0 - 2015-08-31
13391 • text() with a non-string alphabet would have used the repr() of the
13392 the alphabet instead of its contexts. This is obviously silly. It now
13393 works with any sequence of things convertible to unicode strings.
13394 • @given will now work on methods whose definitions contains no ex‐
13396 plicit positional arguments, only varargs (issue #118). This may
13397 have some knock on effects because it means that @given no longer
13398 changes the argspec of functions other than by adding defaults.
13399 • Introduction of new @composite feature for more natural definition of
13401 strategies you'd previously have used flatmap for.
13402 1.10.6 - 2015-08-26
13404 Fix support for fixtures on Django 1.7.
13405 1.10.4 - 2015-08-21
13407 Tiny bug fix release:
13408 • If the database_file setting is set to None, this would have resulted
13410 in an error when running tests. Now it does the same as setting data‐
13411 base to None.
13412 1.10.3 - 2015-08-19
13414 Another small bug fix release.
13415 • lists(elements, unique_by=some_function, min_size=n) would have
13417 raised a ValidationError if n > Settings.default.average_list_length
13418 because it would have wanted to use an average list length shorter
13419 than the minimum size of the list, which is impossible. Now it in‐
13420 stead defaults to twice the minimum size in these circumstances.
13421 • basic() strategy would have only ever produced at most ten distinct
13423 values per run of the test (which is bad if you e.g. have it inside a
13424 list). This was obviously silly. It will now produce a much better
13425 distribution of data, both duplicated and non duplicated.
13426 1.10.2 - 2015-08-19
13428 This is a small bug fix release:
13429 • star imports from hypothesis should now work correctly.
13431 • example quality for examples using flatmap will be better, as the way
13433 it had previously been implemented was causing problems where Hypoth‐
13434 esis was erroneously labelling some examples as being duplicates.
13435 1.10.0 - 2015-08-04
13437 This is just a bugfix and performance release, but it changes some
13438 semi-public APIs, hence the minor version bump.
13439 • Significant performance improvements for strategies which are
13441 one_of() many branches. In particular this included recursive()
13442 strategies. This should take the case where you use one recursive()
13443 strategy as the base strategy of another from unusably slow (tens of
13444 seconds per generated example) to reasonably fast.
13445 • Better handling of just() and sampled_from() for values which have an
13447 incorrect __repr__ implementation that returns non-ASCII unicode on
13448 Python 2.
13449 • Better performance for flatmap from changing the internal morpher API
13451 to be significantly less general purpose.
13452 • Introduce a new semi-public BuildContext/cleanup API. This allows
13454 strategies to register cleanup activities that should run once the
13455 example is complete. Note that this will interact somewhat weirdly
13456 with find.
13457 • Better simplification behaviour for streaming strategies.
13459 • Don't error on lambdas which use destructuring arguments in Python 2.
13461 • Add some better reprs for a few strategies that were missing good
13463 ones.
13464 • The Random instances provided by randoms() are now copyable.
13466 • Slightly more debugging information about simplify when using a debug
13468 verbosity level.
13469 • Support using given for functions with varargs, but not passing argu‐
13471 ments to it as positional.
13472 1.9.0 - 2015-07-27
13474 Codename: The great bundling.
13475 This release contains two fairly major changes.
13477 The first is the deprecation of the hypothesis-extra mechanism. From
13479 now on all the packages that were previously bundled under it other
13480 than hypothesis-pytest (which is a different beast and will remain sep‐
13481 arate). The functionality remains unchanged and you can still import
13482 them from exactly the same location, they just are no longer separate
13483 packages.
13484 The second is that this introduces a new way of building strategies
13486 which lets you build up strategies recursively from other strategies.
13487 It also contains the minor change that calling .example() on a strategy
13489 object will give you examples that are more representative of the ac‐
13490 tual data you'll get. There used to be some logic in there to make the
13491 examples artificially simple but this proved to be a bad idea.
13492 1.8.5 - 2015-07-24
13494 This contains no functionality changes but fixes a mistake made with
13495 building the previous package that would have broken installation on
13496 Windows.
13497 1.8.4 - 2015-07-20
13499 Bugs fixed:
13500 • When a call to floats() had endpoints which were not floats but
13502 merely convertible to one (e.g. integers), these would be included in
13503 the generated data which would cause it to generate non-floats.
13504 • Splitting lambdas used in the definition of flatmap, map or filter
13506 over multiple lines would break the repr, which would in turn break
13507 their usage.
13508 1.8.3 - 2015-07-20
13510 "Falsifying example" would not have been printed when the failure came
13511 from an explicit example.
13512 1.8.2 - 2015-07-18
13514 Another small bugfix release:
13515 • When using ForkingTestCase you would usually not get the falsifying
13517 example printed if the process exited abnormally (e.g. due to
13518 os._exit).
13519 • Improvements to the distribution of characters when using text() with
13521 a default alphabet. In particular produces a better distribution of
13522 ascii and whitespace in the alphabet.
13523 1.8.1 - 2015-07-17
13525 This is a small release that contains a workaround for people who have
13526 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
13527 for Hypothesis per se because that's not a thing that is actually sup‐
13528 posed to work, but Hypothesis leans more heavily on repr than is typi‐
13529 cal so it's worth having a workaround for.
13530 1.8.0 - 2015-07-16
13532 New features:
13533 • Much more sensible reprs for strategies, especially ones that come
13535 from hypothesis.strategies. These should now have as reprs python
13536 code that would produce the same strategy.
13537 • lists() accepts a unique_by argument which forces the generated lists
13539 to be only contain elements unique according to some function key
13540 (which must return a hashable value).
13541 • Better error messages from flaky tests to help you debug things.
13543 Mostly invisible implementation details that may result in finding new
13545 bugs in your code:
13546 • Sets and dictionary generation should now produce a better range of
13548 results.
13549 • floats with bounds now focus more on 'critical values', trying to
13551 produce values at edge cases.
13552 • flatmap should now have better simplification for complicated cases,
13554 as well as generally being (I hope) more reliable.
13555 Bug fixes:
13557 • You could not previously use assume() if you were using the forking
13559 executor.
13560 1.7.2 - 2015-07-10
13562 This is purely a bug fix release:
13563 • When using floats() with stale data in the database you could some‐
13565 times get values in your tests that did not respect min_value or
13566 max_value.
13567 • When getting a Flaky error from an unreliable test it would have in‐
13569 correctly displayed the example that caused it.
13570 • 2.6 dependency on backports was incorrectly specified. This would
13572 only have caused you problems if you were building a universal wheel
13573 from Hypothesis, which is not how Hypothesis ships, so unless you're
13574 explicitly building wheels for your dependencies and support Python
13575 2.6 plus a later version of Python this probably would never have af‐
13576 fected you.
13577 • If you use flatmap in a way that the strategy on the right hand side
13579 depends sensitively on the left hand side you may have occasionally
13580 seen Flaky errors caused by producing unreliable examples when mini‐
13581 mizing a bug. This use case may still be somewhat fraught to be hon‐
13582 est. This code is due a major rearchitecture for 1.8, but in the
13583 meantime this release fixes the only source of this error that I'm
13584 aware of.
13585 1.7.1 - 2015-06-29
13587 Codename: There is no 1.7.0.
13588 A slight technical hitch with a premature upload means there's was a
13590 yanked 1.7.0 release. Oops.
13591 The major feature of this release is Python 2.6 support. Thanks to Jeff
13593 Meadows for doing most of the work there.
13594 Other minor features
13596 • strategies now has a permutations() function which returns a strategy
13598 yielding permutations of values from a given collection.
13599 • if you have a flaky test it will print the exception that it last saw
13601 before failing with Flaky, even if you do not have verbose reporting
13602 on.
13603 • Slightly experimental git merge script available as "python -m hy‐
13605 pothesis.tools.mergedbs". Instructions on how to use it in the doc‐
13606 string of that file.
13607 Bug fixes:
13609 • Better performance from use of filter. In particular tests which in‐
13611 volve large numbers of heavily filtered strategies should perform a
13612 lot better.
13613 • floats() with a negative min_value would not have worked correctly
13615 (worryingly, it would have just silently failed to run any examples).
13616 This is now fixed.
13617 • tests using sampled_from would error if the number of sampled ele‐
13619 ments was smaller than min_satisfying_examples.
13620 1.6.2 - 2015-06-08
13622 This is just a few small bug fixes:
13623 • Size bounds were not validated for values for a binary() strategy
13625 when reading examples from the database.
13626 • sampled_from is now in __all__ in hypothesis.strategies
13628 • floats no longer consider negative integers to be simpler than posi‐
13630 tive non-integers
13631 • Small floating point intervals now correctly count members, so if you
13633 have a floating point interval so narrow there are only a handful of
13634 values in it, this will no longer cause an error when Hypothesis runs
13635 out of values.
13636 1.6.1 - 2015-05-21
13638 This is a small patch release that fixes a bug where 1.6.0 broke the
13639 use of flatmap with the deprecated API and assumed the passed in func‐
13640 tion returned a SearchStrategy instance rather than converting it to a
13641 strategy.
13642 1.6.0 - 2015-05-21
13644 This is a smallish release designed to fix a number of bugs and smooth
13645 out some weird behaviours.
13646 • Fix a critical bug in flatmap where it would reuse old strategies. If
13648 all your flatmap code was pure you're fine. If it's not, I'm sur‐
13649 prised it's working at all. In particular if you want to use flatmap
13650 with django models, you desperately need to upgrade to this version.
13651 • flatmap simplification performance should now be better in some cases
13653 where it previously had to redo work.
13654 • Fix for a bug where invalid unicode data with surrogates could be
13656 generated during simplification (it was already filtered out during
13657 actual generation).
13658 • The Hypothesis database is now keyed off the name of the test instead
13660 of the type of data. This makes much more sense now with the new
13661 strategies API and is generally more robust. This means you will lose
13662 old examples on upgrade.
13663 • The database will now not delete values which fail to deserialize
13665 correctly, just skip them. This is to handle cases where multiple in‐
13666 compatible strategies share the same key.
13667 • find now also saves and loads values from the database, keyed off a
13669 hash of the function you're finding from.
13670 • Stateful tests now serialize and load values from the database. They
13672 should have before, really. This was a bug.
13673 • Passing a different verbosity level into a test would not have worked
13675 entirely correctly, leaving off some messages. This is now fixed.
13676 • Fix a bug where derandomized tests with unicode characters in the
13678 function body would error on Python 2.7.
13679 1.5.0 - 2015-05-14
13681 Codename: Strategic withdrawal.
13682 The purpose of this release is a radical simplification of the API for
13684 building strategies. Instead of the old approach of @strategy.extend
13685 and things that get converted to strategies, you just build strategies
13686 directly.
13687 The old method of defining strategies will still work until Hypothesis
13689 2.0, because it's a major breaking change, but will now emit depreca‐
13690 tion warnings.
13691 The new API is also a lot more powerful as the functions for defining
13693 strategies give you a lot of dials to turn. See the updated data sec‐
13694 tion for details.
13695 Other changes:
13697 • Mixing keyword and positional arguments in a call to @given is
13699 deprecated as well.
13700 • There is a new setting called 'strict'. When set to True, Hypothe‐
13702 sis will raise warnings instead of merely printing them. Turning
13703 it on by default is inadvisable because it means that Hypothesis
13704 minor releases can break your code, but it may be useful for mak‐
13705 ing sure you catch all uses of deprecated APIs.
13706 • max_examples in settings is now interpreted as meaning the maximum
13708 number of unique (ish) examples satisfying assumptions. A new set‐
13709 ting max_iterations which defaults to a larger value has the old
13710 interpretation.
13711 • Example generation should be significantly faster due to a new
13713 faster parameter selection algorithm. This will mostly show up for
13714 simple data types - for complex ones the parameter selection is
13715 almost certainly dominated.
13716 • Simplification has some new heuristics that will tend to cut down
13718 on cases where it could previously take a very long time.
13719 • timeout would previously not have been respected in cases where
13721 there were a lot of duplicate examples. You probably wouldn't have
13722 previously noticed this because max_examples counted duplicates,
13723 so this was very hard to hit in a way that mattered.
13724 • A number of internal simplifications to the SearchStrategy API.
13726 • You can now access the current Hypothesis version as hypothe‐
13728 sis.__version__.
13729 • A top level function is provided for running the stateful tests
13731 without the TestCase infrastructure.
13732 1.4.0 - 2015-05-04
13734 Codename: What a state.
13735 The big feature of this release is the new and slightly experimental
13737 stateful testing API. You can read more about that in the appropriate
13738 section.
13739 Two minor features the were driven out in the course of developing
13741 this:
13742 • You can now set settings.max_shrinks to limit the number of times Hy‐
13744 pothesis will try to shrink arguments to your test. If this is set to
13745 <= 0 then Hypothesis will not rerun your test and will just raise the
13746 failure directly. Note that due to technical limitations if
13747 max_shrinks is <= 0 then Hypothesis will print every example it calls
13748 your test with rather than just the failing one. Note also that I
13749 don't consider settings max_shrinks to zero a sensible way to run
13750 your tests and it should really be considered a debug feature.
13751 • There is a new debug level of verbosity which is even more verbose
13753 than verbose. You probably don't want this.
13754 Breakage of semi-public SearchStrategy API:
13756 • It is now a required invariant of SearchStrategy that if u simplifies
13758 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
13759 plifying should not increase the complexity even though it is not re‐
13760 quired to decrease it. Enforcing this invariant lead to finding some
13761 bugs where simplifying of integers, floats and sets was suboptimal.
13762 • Integers in basic data are now required to fit into 64 bits. As a re‐
13764 sult python integer types are now serialized as strings, and some
13765 types have stopped using quite so needlessly large random seeds.
13766 Hypothesis Stateful testing was then turned upon Hypothesis itself,
13768 which lead to an amazing number of minor bugs being found in Hypothesis
13769 itself.
13770 Bugs fixed (most but not all from the result of stateful testing) in‐
13772 clude:
13773 • Serialization of streaming examples was flaky in a way that you would
13775 probably never notice: If you generate a template, simplify it, seri‐
13776 alize it, deserialize it, serialize it again and then deserialize it
13777 you would get the original stream instead of the simplified one.
13778 • If you reduced max_examples below the number of examples already
13780 saved in the database, you would have got a ValueError. Additionally,
13781 if you had more than max_examples in the database all of them would
13782 have been considered.
13783 • @given will no longer count duplicate examples (which it never called
13785 your function with) towards max_examples. This may result in your
13786 tests running slower, but that's probably just because they're trying
13787 more examples.
13788 • General improvements to example search which should result in better
13790 performance and higher quality examples. In particular parameters
13791 which have a history of producing useless results will be more ag‐
13792 gressively culled. This is useful both because it decreases the
13793 chance of useless examples and also because it's much faster to not
13794 check parameters which we were unlikely to ever pick!
13795 • integers_from and lists of types with only one value (e.g. [None])
13797 would previously have had a very high duplication rate so you were
13798 probably only getting a handful of examples. They now have a much
13799 lower duplication rate, as well as the improvements to search making
13800 this less of a problem in the first place.
13801 • You would sometimes see simplification taking significantly longer
13803 than your defined timeout. This would happen because timeout was only
13804 being checked after each successful simplification, so if Hypothesis
13805 was spending a lot of time unsuccessfully simplifying things it
13806 wouldn't stop in time. The timeout is now applied for unsuccessful
13807 simplifications too.
13808 • In Python 2.7, integers_from strategies would have failed during sim‐
13810 plification with an OverflowError if their starting point was at or
13811 near to the maximum size of a 64-bit integer.
13812 • flatmap and map would have failed if called with a function without a
13814 __name__ attribute.
13815 • If max_examples was less than min_satisfying_examples this would al‐
13817 ways error. Now min_satisfying_examples is capped to max_examples.
13818 Note that if you have assumptions to satisfy here this will still
13819 cause an error.
13820 Some minor quality improvements:
13822 • Lists of streams, flatmapped strategies and basic strategies should
13824 now now have slightly better simplification.
13825 1.3.0 - 2015-05-22
13827 New features:
13828 • New verbosity level API for printing intermediate results and excep‐
13830 tions.
13831 • New specifier for strings generated from a specified alphabet.
13833 • Better error messages for tests that are failing because of a lack of
13835 enough examples.
13836 Bug fixes:
13838 • Fix error where use of ForkingTestCase would sometimes result in too
13840 many open files.
13841 • Fix error where saving a failing example that used flatmap could er‐
13843 ror.
13844 • Implement simplification for sampled_from, which apparently never
13846 supported it previously. Oops.
13847 General improvements:
13849 • Better range of examples when using one_of or sampled_from.
13851 • Fix some pathological performance issues when simplifying lists of
13853 complex values.
13854 • Fix some pathological performance issues when simplifying examples
13856 that require unicode strings with high codepoints.
13857 • Random will now simplify to more readable examples.
13859 1.2.1 - 2015-04-16
13861 A small patch release for a bug in the new executors feature. Tests
13862 which require doing something to their result in order to fail would
13863 have instead reported as flaky.
13864 1.2.0 - 2015-04-15
13866 Codename: Finders keepers.
13867 A bunch of new features and improvements.
13869 • Provide a mechanism for customizing how your tests are executed.
13871 • Provide a test runner that forks before running each example. This
13873 allows better support for testing native code which might trigger a
13874 segfault or a C level assertion failure.
13875 • Support for using Hypothesis to find examples directly rather than as
13877 just as a test runner.
13878 • New streaming type which lets you generate infinite lazily loaded
13880 streams of data - perfect for if you need a number of examples but
13881 don't know how many.
13882 • Better support for large integer ranges. You can now use inte‐
13884 gers_in_range with ranges of basically any size. Previously large
13885 ranges would have eaten up all your memory and taken forever.
13886 • Integers produce a wider range of data than before - previously they
13888 would only rarely produce integers which didn't fit into a machine
13889 word. Now it's much more common. This percolates to other numeric
13890 types which build on integers.
13891 • Better validation of arguments to @given. Some situations that would
13893 previously have caused silently wrong behaviour will now raise an er‐
13894 ror.
13895 • Include +/- sys.float_info.max in the set of floating point edge
13897 cases that Hypothesis specifically tries.
13898 • Fix some bugs in floating point ranges which happen when given +/-
13900 sys.float_info.max as one of the endpoints... (really any two floats
13901 that are sufficiently far apart so that x, y are finite but y - x is
13902 infinite). This would have resulted in generating infinite values
13903 instead of ones inside the range.
13904 1.1.1 - 2015-04-07
13906 Codename: Nothing to see here
13907 This is just a patch release put out because it fixed some internal
13909 bugs that would block the Django integration release but did not actu‐
13910 ally affect anything anyone could previously have been using. It also
13911 contained a minor quality fix for floats that I'd happened to have fin‐
13912 ished in time.
13913 • Fix some internal bugs with object lifecycle management that were im‐
13915 possible to hit with the previously released versions but broke hy‐
13916 pothesis-django.
13917 • Bias floating point numbers somewhat less aggressively towards very
13919 small numbers
13920 1.1.0 - 2015-04-06
13922 Codename: No-one mention the M word.
13923 • Unicode strings are more strongly biased towards ascii characters.
13925 Previously they would generate all over the space. This is mostly so
13926 that people who try to shape their unicode strings with assume() have
13927 less of a bad time.
13928 • A number of fixes to data deserialization code that could theoreti‐
13930 cally have caused mysterious bugs when using an old version of a Hy‐
13931 pothesis example database with a newer version. To the best of my
13932 knowledge a change that could have triggered this bug has never actu‐
13933 ally been seen in the wild. Certainly no-one ever reported a bug of
13934 this nature.
13935 • Out of the box support for Decimal and Fraction.
13937 • new dictionary specifier for dictionaries with variable keys.
13939 • Significantly faster and higher quality simplification, especially
13941 for collections of data.
13942 • New filter() and flatmap() methods on Strategy for better ways of
13944 building strategies out of other strategies.
13945 • New BasicStrategy class which allows you to define your own strate‐
13947 gies from scratch without needing an existing matching strategy or
13948 being exposed to the full horror or non-public nature of the Search‐
13949 Strategy interface.
13950 1.0.0 - 2015-03-27
13952 Codename: Blast-off!
13953 There are no code changes in this release. This is precisely the 0.9.2
13955 release with some updated documentation.
13956 0.9.2 - 2015-03-26
13958 Codename: T-1 days.
13959 • floats_in_range would not actually have produced floats_in_range un‐
13961 less that range happened to be (0, 1). Fix this.
13962 0.9.1 - 2015-03-25
13964 Codename: T-2 days.
13965 • Fix a bug where if you defined a strategy using map on a lambda then
13967 the results would not be saved in the database.
13968 • Significant performance improvements when simplifying examples using
13970 lists, strings or bounded integer ranges.
13971 0.9.0 - 2015-03-23
13973 Codename: The final countdown
13974 This release could also be called 1.0-RC1.
13976 It contains a teeny tiny bugfix, but the real point of this release is
13978 to declare feature freeze. There will be zero functionality changes be‐
13979 tween 0.9.0 and 1.0 unless something goes really really wrong. No new
13980 features will be added, no breaking API changes will occur, etc. This
13981 is the final shakedown before I declare Hypothesis stable and ready to
13982 use and throw a party to celebrate.
13983 Bug bounty for any bugs found between now and 1.0: I will buy you a
13985 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
13986 we ever find ourselves in the same city at the same time.
13987 The one tiny bugfix:
13989 • Under pypy, databases would fail to close correctly when garbage col‐
13991 lected, leading to a memory leak and a confusing error message if you
13992 were repeatedly creating databases and not closing them. It is very
13993 unlikely you were doing this and the chances of you ever having no‐
13994 ticed this bug are very low.
13995 0.7.2 - 2015-03-22
13997 Codename: Hygienic macros or bust
13998 • You can now name an argument to @given 'f' and it won't break (issue
14000 #38)
14001 • strategy_test_suite is now named strategy_test_suite as the documen‐
14003 tation claims and not in fact strategy_test_suitee
14004 • Settings objects can now be used as a context manager to temporarily
14006 override the default values inside their context.
14007 0.7.1 - 2015-03-21
14009 Codename: Point releases go faster
14010 • Better string generation by parametrizing by a limited alphabet
14012 • Faster string simplification - previously if simplifying a string
14014 with high range unicode characters it would try every unicode charac‐
14015 ter smaller than that. This was pretty pointless. Now it stops after
14016 it's a short range (it can still reach smaller ones through recursive
14017 calls because of other simplifying operations).
14018 • Faster list simplification by first trying a binary chop down the
14020 middle
14021 • Simultaneous simplification of identical elements in a list. So if a
14023 bug only triggers when you have duplicates but you drew e.g. [-17,
14024 -17], this will now simplify to [0, 0].
14025 0.7.0, - 2015-03-20
14027 Codename: Starting to look suspiciously real
14028 This is probably the last minor release prior to 1.0. It consists of
14030 stability improvements, a few usability things designed to make Hypoth‐
14031 esis easier to try out, and filing off some final rough edges from the
14032 API.
14033 • Significant speed and memory usage improvements
14035 • Add an example() method to strategy objects to give an example of the
14037 sort of data that the strategy generates.
14038 • Remove .descriptor attribute of strategies
14040 • Rename descriptor_test_suite to strategy_test_suite
14042 • Rename the few remaining uses of descriptor to specifier (descriptor
14044 already has a defined meaning in Python)
14045 0.6.0 - 2015-03-13
14047 Codename: I'm sorry, were you using that API?
14048 This is primarily a "simplify all the weird bits of the API" release.
14050 As a result there are a lot of breaking changes. If you just use @given
14051 with core types then you're probably fine.
14052 In particular:
14054 • Stateful testing has been removed from the API
14056 • The way the database is used has been rendered less useful (sorry).
14058 The feature for reassembling values saved from other tests doesn't
14059 currently work. This will probably be brought back in post 1.0.
14060 • SpecificationMapper is no longer a thing. Instead there is an
14062 ExtMethod called strategy which you extend to specify how to convert
14063 other types to strategies.
14064 • Settings are now extensible so you can add your own for configuring a
14066 strategy
14067 • MappedSearchStrategy no longer needs an unpack method
14069 • Basically all the SearchStrategy internals have changed massively. If
14071 you implemented SearchStrategy directly rather than using Mapped‐
14072 SearchStrategy talk to me about fixing it.
14073 • Change to the way extra packages work. You now specify the package.
14075 This must have a load() method. Additionally any modules in the pack‐
14076 age will be loaded in under hypothesis.extra
14077 Bug fixes:
14079 • Fix for a bug where calling falsify on a lambda with a non-ascii
14081 character in its body would error.
14082 Hypothesis Extra:
14084 hypothesis-fakefactory: An extension for using faker data in hypothe‐
14086 sis. Depends
14087 on fake-factory.
14088 0.5.0 - 2015-02-10
14090 Codename: Read all about it.
14091 Core hypothesis:
14093 • Add support back in for pypy and python 3.2
14095 • @given functions can now be invoked with some arguments explicitly
14097 provided. If all arguments that hypothesis would have provided are
14098 passed in then no falsification is run.
14099 • Related to the above, this means that you can now use pytest fixtures
14101 and mark.parametrize with Hypothesis without either interfering with
14102 the other.
14103 • Breaking change: @given no longer works for functions with varargs
14105 (varkwargs are fine). This might be added back in at a later date.
14106 • Windows is now fully supported. A limited version (just the tests
14108 with none of the extras) of the test suite is run on windows with
14109 each commit so it is now a first class citizen of the Hypothesis
14110 world.
14111 • Fix a bug for fuzzy equality of equal complex numbers with different
14113 reprs (this can happen when one coordinate is zero). This shouldn't
14114 affect users - that feature isn't used anywhere public facing.
14115 • Fix generation of floats on windows and 32-bit builds of python. I
14117 was using some struct.pack logic that only worked on certain word
14118 sizes.
14119 • When a test times out and hasn't produced enough examples this now
14121 raises a Timeout subclass of Unfalsifiable.
14122 • Small search spaces are better supported. Previously something like a
14124 @given(bool, bool) would have failed because it couldn't find enough
14125 examples. Hypothesis is now aware of the fact that these are small
14126 search spaces and will not error in this case.
14127 • Improvements to parameter search in the case of hard to satisfy as‐
14129 sume. Hypothesis will now spend less time exploring parameters that
14130 are unlikely to provide anything useful.
14131 • Increase chance of generating "nasty" floats
14133 • Fix a bug that would have caused unicode warnings if you had a sam‐
14135 pled_from that was mixing unicode and byte strings.
14136 • Added a standard test suite that you can use to validate a custom
14138 strategy you've defined is working correctly.
14139 Hypothesis extra:
14141 First off, introducing Hypothesis extra packages!
14143 These are packages that are separated out from core Hypothesis because
14145 they have one or more dependencies. Every hypothesis-extra package is
14146 pinned to a specific point release of Hypothesis and will have some
14147 version requirements on its dependency. They use entry_points so you
14148 will usually not need to explicitly import them, just have them in‐
14149 stalled on the path.
14150 This release introduces two of them:
14152 hypothesis-datetime:
14154 Does what it says on the tin: Generates datetimes for Hypothesis. Just
14156 install the package and datetime support will start working.
14157 Depends on pytz for timezone support
14159 hypothesis-pytest:
14161 A very rudimentary pytest plugin. All it does right now is hook the
14163 display of falsifying examples into pytest reporting.
14164 Depends on pytest.
14166 0.4.3 - 2015-02-05
14168 Codename: TIL narrow Python builds are a thing
14169 This just fixes the one bug.
14171 • Apparently there is such a thing as a "narrow python build" and OS X
14173 ships with these by default for python 2.7. These are builds where
14174 you only have two bytes worth of unicode. As a result, generating
14175 unicode was completely broken on OS X. Fix this by only generating
14176 unicode codepoints in the range supported by the system.
14177 0.4.2 - 2015-02-04
14179 Codename: O(dear)
14180 This is purely a bugfix release:
14182 • Provide sensible external hashing for all core types. This will sig‐
14184 nificantly improve performance of tracking seen examples which hap‐
14185 pens in literally every falsification run. For Hypothesis fixing this
14186 cut 40% off the runtime of the test suite. The behaviour is quadratic
14187 in the number of examples so if you're running the default configura‐
14188 tion this will be less extreme (Hypothesis's test suite runs at a
14189 higher number of examples than default), but you should still see a
14190 significant improvement.
14191 • Fix a bug in formatting of complex numbers where the string could get
14193 incorrectly truncated.
14194 0.4.1 - 2015-02-03
14196 Codename: Cruel and unusual edge cases
14197 This release is mostly about better test case generation.
14199 Enhancements:
14201 • Has a cool release name
14203 • text_type (str in python 3, unicode in python 2) example generation
14205 now actually produces interesting unicode instead of boring ascii
14206 strings.
14207 • floating point numbers are generated over a much wider range, with
14209 particular attention paid to generating nasty numbers - nan, infin‐
14210 ity, large and small values, etc.
14211 • examples can be generated using pieces of examples previously saved
14213 in the database. This allows interesting behaviour that has previ‐
14214 ously been discovered to be propagated to other examples.
14215 • improved parameter exploration algorithm which should allow it to
14217 more reliably hit interesting edge cases.
14218 • Timeout can now be disabled entirely by setting it to any value <= 0.
14220 Bug fixes:
14222 • The descriptor on a OneOfStrategy could be wrong if you had descrip‐
14224 tors which were equal but should not be coalesced. e.g. a strategy
14225 for one_of((frozenset({int}), {int})) would have reported its de‐
14226 scriptor as {int}. This is unlikely to have caused you any problems
14227 • If you had strategies that could produce NaN (which float previously
14229 couldn't but e.g. a Just(float('nan')) could) then this would have
14230 sent hypothesis into an infinite loop that would have only been ter‐
14231 minated when it hit the timeout.
14232 • Given elements that can take a long time to minimize, minimization of
14234 floats or tuples could be quadratic or worse in the that value. You
14235 should now see much better performance for simplification, albeit at
14236 some cost in quality.
14237 Other:
14239 • A lot of internals have been been rewritten. This shouldn't affect
14241 you at all, but it opens the way for certain of hypothesis's oddities
14242 to be a lot more extensible by users. Whether this is a good thing
14243 may be up for debate...
14244 0.4.0 - 2015-01-21
14246 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
14247 stores data in a local SQLite database and will reuse it for all tests
14248 of the same type.
14249 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
14251 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
14252 ther notice. The previous license remains applicable to all code prior
14253 to 0.4.0.
14254 Enhancements:
14256 • Printing of failing examples. I was finding that the pytest runner
14258 was not doing a good job of displaying these, and that Hypothesis it‐
14259 self could do much better.
14260 • Drop dependency on six for cross-version compatibility. It was easy
14262 enough to write the shim for the small set of features that we care
14263 about and this lets us avoid a moderately complex dependency.
14264 • Some improvements to statistical distribution of selecting from small
14266 (<= 3 elements)
14267 • Improvements to parameter selection for finding examples.
14269 Bugs fixed:
14271 • could_have_produced for lists, dicts and other collections would not
14273 have examined the elements and thus when using a union of different
14274 types of list this could result in Hypothesis getting confused and
14275 passing a value to the wrong strategy. This could potentially result
14276 in exceptions being thrown from within simplification.
14277 • sampled_from would not work correctly on a single element list.
14279 • Hypothesis could get very confused by values which are equal despite
14281 having different types being used in descriptors. Hypothesis now has
14282 its own more specific version of equality it uses for descriptors and
14283 tracking. It is always more fine grained than Python equality: Things
14284 considered != are not considered equal by hypothesis, but some things
14285 that are considered == are distinguished. If your test suite uses
14286 both frozenset and set tests this bug is probably affecting you.
14287 0.3.2 - 2015-01-16
14289 • Fix a bug where if you specified floats_in_range with integer argu‐
14290 ments Hypothesis would error in example simplification.
14291 • Improve the statistical distribution of the floats you get for the
14293 floats_in_range strategy. I'm not sure whether this will affect users
14294 in practice but it took my tests for various conditions from flaky to
14295 rock solid so it at the very least improves discovery of the artifi‐
14296 cial cases I'm looking for.
14297 • Improved repr() for strategies and RandomWithSeed instances.
14299 • Add detection for flaky test cases where hypothesis managed to find
14301 an example which breaks it but on the final invocation of the test it
14302 does not raise an error. This will typically happen with too much re‐
14303 cursion errors but could conceivably happen in other circumstances
14304 too.
14305 • Provide a "derandomized" mode. This allows you to run hypothesis with
14307 zero real randomization, making your build nice and deterministic.
14308 The tests run with a seed calculated from the function they're test‐
14309 ing so you should still get a good distribution of test cases.
14310 • Add a mechanism for more conveniently defining tests which just sam‐
14312 ple from some collection.
14313 • Fix for a really subtle bug deep in the internals of the strategy ta‐
14315 ble. In some circumstances if you were to define instance strategies
14316 for both a parent class and one or more of its subclasses you would
14317 under some circumstances get the strategy for the wrong superclass of
14318 an instance. It is very unlikely anyone has ever encountered this in
14319 the wild, but it is conceivably possible given that a mix of namedtu‐
14320 ple and tuple are used fairly extensively inside hypothesis which do
14321 exhibit this pattern of strategy.
14322 0.3.1 - 2015-01-13
14324 • Support for generation of frozenset and Random values
14325 • Correct handling of the case where a called function mutates it argu‐
14327 ment. This involved introducing a notion of a strategies knowing how
14328 to copy their argument. The default method should be entirely accept‐
14329 able and the worst case is that it will continue to have the old be‐
14330 haviour if you don't mark your strategy as mutable, so this shouldn't
14331 break anything.
14332 • Fix for a bug where some strategies did not correctly implement
14334 could_have_produced. It is very unlikely that any of these would have
14335 been seen in the wild, and the consequences if they had been would
14336 have been minor.
14337 • Re-export the @given decorator from the main hypothesis namespace.
14339 It's still available at the old location too.
14340 • Minor performance optimisation for simplifying long lists.
14342 0.3.0 - 2015-01-12
14344 • Complete redesign of the data generation system. Extreme breaking
14345 change for anyone who was previously writing their own SearchStrategy
14346 implementations. These will not work any more and you'll need to mod‐
14347 ify them.
14348 • New settings system allowing more global and modular control of Veri‐
14350 fier behaviour.
14351 • Decouple SearchStrategy from the StrategyTable. This leads to much
14353 more composable code which is a lot easier to understand.
14354 • A significant amount of internal API renaming and moving. This may
14356 also break your code.
14357 • Expanded available descriptors, allowing for generating integers or
14359 floats in a specific range.
14360 • Significantly more robust. A very large number of small bug fixes,
14362 none of which anyone is likely to have ever noticed.
14363 • Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
14365 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
14366 these to remain officially supported for a very long time. I would
14367 not be surprised to add pypy support back in later but I'm not going
14368 to do so until I know someone cares about it. In the meantime it will
14369 probably still work.
14370 0.2.2 - 2015-01-08
14372 • Fix an embarrassing complete failure of the installer caused by my
14373 being bad at version control
14374 0.2.1 - 2015-01-07
14376 • Fix a bug in the new stateful testing feature where you could make
14377 __init__ a @requires method. Simplification would not always work if
14378 the prune method was able to successfully shrink the test.
14379 0.2.0 - 2015-01-07
14381 • It's aliiive.
14382 • Improve python 3 support using six.
14384 • Distinguish between byte and unicode types.
14386 • Fix issues where FloatStrategy could raise.
14388 • Allow stateful testing to request constructor args.
14390 • Fix for issue where test annotations would timeout based on when the
14392 module was loaded instead of when the test started
14393 0.1.4 - 2013-12-14
14395 • Make verification runs time bounded with a configurable timeout
14396 0.1.3 - 2013-05-03
14398 • Bugfix: Stateful testing behaved incorrectly with subclassing.
14399 • Complex number support
14401 • support for recursive strategies
14403 • different error for hypotheses with unsatisfiable assumptions
14405 0.1.2 - 2013-03-24
14407 • Bugfix: Stateful testing was not minimizing correctly and could throw
14408 exceptions.
14409 • Better support for recursive strategies.
14411 • Support for named tuples.
14413 • Much faster integer generation.
14415 0.1.1 - 2013-03-24
14417 • Python 3.x support via 2to3.
14418 • Use new style classes (oops).
14420 0.1.0 - 2013-03-23
14422 • Introduce stateful testing.
14423 • Massive rewrite of internals to add flags and strategies.
14425 0.0.5 - 2013-03-13
14427 • No changes except trying to fix packaging
14428 0.0.4 - 2013-03-13
14430 • No changes except that I checked in a failing test case for 0.0.3 so
14431 had to replace the release. Doh
14432 0.0.3 - 2013-03-13
14434 • Improved a few internals.
14435 • Opened up creating generators from instances as a general API.
14437 • Test integration.
14439 0.0.2 - 2013-03-12
14441 • Starting to tighten up on the internals.
14442 • Change API to allow more flexibility in configuration.
14444 • More testing.
14446 0.0.1 - 2013-03-10
14448 • Initial release.
14449 • Basic working prototype. Demonstrates idea, probably shouldn't be
14451 used.
14452
14454 Hypothesis development is managed by David R. MacIver and Zac Hat‐
14455 field-Dodds, respectively the first author and lead maintainer.
14456 However, these roles don't include unpaid feature development on Hy‐
14458 pothesis. Our roles as leaders of the project are:
14459 1. Helping other people do feature development on Hypothesis
14461 2. Fixing bugs and other code health issues
14463 3. Improving documentation
14465 4. General release management work
14467 5. Planning the general roadmap of the project
14469 6. Doing sponsored development on tasks that are too large or in depth
14471 for other people to take on
14472 So all new features must either be sponsored or implemented by someone
14474 else. That being said, the maintenance team takes an active role in
14475 shepherding pull requests and helping people write a new feature (see
14476 CONTRIBUTING.rst for details and these examples of how the process
14477 goes). This isn't "patches welcome", it's "we will help you write a
14478 patch".
14479 Release policy
14481 Hypothesis releases follow semantic versioning.
14482 We maintain backwards-compatibility wherever possible, and use depreca‐
14484 tion warnings to mark features that have been superseded by a newer al‐
14485 ternative. If you want to detect this, you can upgrade warnings to er‐
14486 rors in the usual ways.
14487 We use continuous deployment to ensure that you can always use our new‐
14489 est and shiniest features - every change to the source tree is automat‐
14490 ically built and published on PyPI as soon as it's merged onto master,
14491 after code review and passing our extensive test suite.
14492 Project roadmap
14494 Hypothesis does not have a long-term release plan. We respond to bug
14495 reports as they are made; new features are released as and when someone
14496 volunteers to write and maintain them.
14497
14499 For questions you are happy to ask in public, the Hypothesis community
14500 is a friendly place where I or others will be more than happy to help
14501 you out. You're also welcome to ask questions on Stack Overflow. If you
14502 do, please tag them with 'python-hypothesis' so someone sees them.
14503 For bugs and enhancements, please file an issue on the GitHub issue
14505 tracker. Note that as per the development policy, enhancements will
14506 probably not get implemented unless you're willing to pay for develop‐
14507 ment or implement them yourself (with assistance from the maintainers).
14508 Bugs will tend to get fixed reasonably promptly, though it is of course
14509 on a best effort basis.
14510 To see the versions of Python, optional dependencies, test runners, and
14512 operating systems Hypothesis supports (meaning incompatibility is
14513 treated as a bug), see Compatibility.
14514 If you need to ask questions privately or want more of a guarantee of
14516 bugs being fixed promptly, please contact me on
14517 hypothesis-support@drmaciver.com to talk about availability of support
14518 contracts.
14519
14521 Downstream packagers often want to package Hypothesis. Here are some
14522 guidelines.
14523 The primary guideline is this: If you are not prepared to keep up with
14525 the Hypothesis release schedule, don't. You will annoy me and are doing
14526 your users a disservice.
14527 Hypothesis has a very frequent release schedule. It's rare that it goes
14529 a week without a release, and there are often multiple releases in a
14530 given week.
14531 If you are prepared to keep up with this schedule, you might find the
14533 rest of this document useful.
14534 Release tarballs
14536 These are available from the GitHub releases page. The tarballs on PyPI
14537 are intended for installation from a Python tool such as pip and should
14538 not be considered complete releases. Requests to include additional
14539 files in them will not be granted. Their absence is not a bug.
14540 Dependencies
14542 Python versions
14543 Hypothesis is designed to work with a range of Python versions - we
14544 support all versions of CPython with upstream support. We also support
14545 the latest versions of PyPy for Python 3.
14546 Other Python libraries
14548 Hypothesis has mandatory dependencies on the following libraries:
14549 • attrs
14551 • sortedcontainers
14553 Hypothesis has optional dependencies on the following libraries:
14555 extras_require = {
14557 "cli": ["click>=7.0", "black>=19.10b0", "rich>=9.0.0"],
14558 "codemods": ["libcst>=0.3.16"],
14559 "ghostwriter": ["black>=19.10b0"],
14560 "pytz": ["pytz>=2014.1"],
14561 "dateutil": ["python-dateutil>=1.4"],
14562 "lark": ["lark-parser>=0.9"], # probably still works with old `lark-parser` too
14563 "numpy": ["numpy>=1.17.3"], # oldest with wheels for non-EOL Python (for now)
14564 "pandas": ["pandas>=1.1"],
14565 "pytest": ["pytest>=4.6"],
14566 "dpcontracts": ["dpcontracts>=0.4"],
14567 "redis": ["redis>=3.0.0"],
14568 # zoneinfo is an odd one: every dependency is conditional, because they're
14569 # only necessary on old versions of Python or Windows systems.
14570 "zoneinfo": [
14571 "tzdata>=2023.3 ; sys_platform == 'win32'",
14572 "backports.zoneinfo>=0.2.1 ; python_version<'3.9'",
14573 ],
14574 # We only support Django versions with upstream support - see
14575 # https://www.djangoproject.com/download/#supported-versions
14576 # We also leave the choice of timezone library to the user, since it
14577 # might be zoneinfo or pytz depending on version and configuration.
14578 "django": ["django>=3.2"],
14579 }
14580 The way this works when installing Hypothesis normally is that these
14583 features become available if the relevant library is installed.
14584 Specifically for pytest, our plugin supports versions of pytest which
14586 have been out of upstream support for some time. Hypothesis tests can
14587 still be executed by even older versions of pytest - you just won't
14588 have the plugin to provide automatic marks, helpful usage warnings, and
14589 per-test statistics.
14590 Testing Hypothesis
14592 If you want to test Hypothesis as part of your packaging you will prob‐
14593 ably not want to use the mechanisms Hypothesis itself uses for running
14594 its tests, because it has a lot of logic for installing and testing
14595 against different versions of Python.
14596 The tests must be run with fairly recent tooling; check the
14598 tree/master/requirements/ directory for details.
14599 The organisation of the tests is described in the
14601 hypothesis-python/tests/README.rst.
14602 Examples
14604 • arch linux
14605 • fedora
14607 • gentoo
14609
14611 One of the things that is often concerning for people using randomized
14612 testing is the question of how to reproduce failing test cases.
14613 NOTE:
14615 It is better to think about the data Hypothesis generates as being
14616 arbitrary, rather than random. We deliberately generate any valid
14617 data that seems likely to cause errors, so you shouldn't rely on any
14618 expected distribution of or relationships between generated data.
14619 You can read about "swarm testing" and "coverage guided fuzzing" if
14620 you're interested, because you don't need to know for Hypothesis!
14621 Fortunately Hypothesis has a number of features to support reproducing
14623 test failures. The one you will use most commonly when developing lo‐
14624 cally is the example database, which means that you shouldn't have to
14625 think about the problem at all for local use - test failures will just
14626 automatically reproduce without you having to do anything.
14627 The example database is perfectly suitable for sharing between ma‐
14629 chines, but there currently aren't very good work flows for that, so
14630 Hypothesis provides a number of ways to make examples reproducible by
14631 adding them to the source code of your tests. This is particularly use‐
14632 ful when e.g. you are trying to run an example that has failed on your
14633 CI, or otherwise share them between machines.
14634 Providing explicit examples
14636 The simplest way to reproduce a failed test is to ask Hypothesis to run
14637 the failing example it printed. For example, if Falsifying example:
14638 test(n=1) was printed you can decorate test with @example(n=1).
14639 @example can also be used to ensure a specific example is always exe‐
14641 cuted as a regression test or to cover some edge case - basically com‐
14642 bining a Hypothesis test and a traditional parametrized test.
14643 class hypothesis.example(*args, **kwargs)
14645 A decorator which ensures a specific example is always tested.
14646 Hypothesis will run all examples you've asked for first. If any of them
14648 fail it will not go on to look for more examples.
14649 It doesn't matter whether you put the example decorator before or after
14651 given. Any permutation of the decorators in the above will do the same
14652 thing.
14653 Note that examples can be positional or keyword based. If they're posi‐
14655 tional then they will be filled in from the right when calling, so ei‐
14656 ther of the following styles will work as expected:
14657 @given(text())
14659 @example("Hello world")
14660 @example(x="Some very long string")
14661 def test_some_code(x):
14662 pass
14663 from unittest import TestCase
14666 class TestThings(TestCase):
14669 @given(text())
14670 @example("Hello world")
14671 @example(x="Some very long string")
14672 def test_some_code(self, x):
14673 pass
14674 As with @given, it is not permitted for a single example to be a mix of
14676 positional and keyword arguments. Either are fine, and you can use one
14677 in one example and the other in another example if for some reason you
14678 really want to, but a single example must be consistent.
14679 example.xfail(condition=True, *, reason='', raises=<class 'BaseExcep‐
14681 tion'>)
14682 Mark this example as an expected failure, like
14683 pytest.mark.xfail().
14684 Expected-failing examples allow you to check that your test does
14686 fail on some examples, and therefore build confidence that pass‐
14687 ing tests are because your code is working, not because the test
14688 is missing something.
14689 @example(...).xfail()
14691 @example(...).xfail(reason="Prices must be non-negative")
14692 @example(...).xfail(raises=(KeyError, ValueError))
14693 @example(...).xfail(sys.version_info[:2] >= (3, 9), reason="needs py39+")
14694 @example(...).xfail(condition=sys.platform != "linux", raises=OSError)
14695 def test(x):
14696 pass
14697 example.via(whence, /)
14699 Attach a machine-readable label noting whence this example came.
14700 The idea is that tools will be able to add @example() cases for
14702 you, e.g. to maintain a high-coverage set of explicit examples,
14703 but also remove them if they become redundant - without ever
14704 deleting manually-added examples:
14705 # You can choose to annotate examples, or not, as you prefer
14707 @example(...)
14708 @example(...).via("regression test for issue #42")
14709 # The `hy-` prefix is reserved for automated tooling
14711 @example(...).via("hy-failing")
14712 @example(...).via("hy-coverage")
14713 @example(...).via("hy-target-$label")
14714 def test(x):
14715 pass
14716 Note that this "method chaining" syntax requires Python 3.9 or
14718 later, for PEP 614 relaxing grammar restrictions on decorators.
14719 If you need to support older versions of Python, you can use an
14720 identity function:
14721 def identity(x):
14723 return x
14724 @identity(example(...).via("label"))
14727 def test(x):
14728 pass
14729 Reproducing a test run with @seed
14731 hypothesis.seed(seed)
14732 seed: Start the test execution from a specific seed.
14733 May be any hashable object. No exact meaning for seed is pro‐
14735 vided other than that for a fixed seed value Hypothesis will try
14736 the same actions (insofar as it can given external sources of
14737 non- determinism. e.g. timing and hash randomization).
14738 Overrides the derandomize setting, which is designed to enable
14740 deterministic builds rather than reproducing observed failures.
14741 When a test fails unexpectedly, usually due to a health check failure,
14743 Hypothesis will print out a seed that led to that failure, if the test
14744 is not already running with a fixed seed. You can then recreate that
14745 failure using either the @seed decorator or (if you are running pytest)
14746 with --hypothesis-seed. For example, the following test function and
14747 RuleBasedStateMachine will each check the same examples each time they
14748 are executed, thanks to @seed():
14749 @seed(1234)
14751 @given(x=...)
14752 def test(x):
14753 ...
14754 @seed(6789)
14757 class MyModel(RuleBasedStateMachine):
14758 ...
14759 The seed will not be printed if you could simply use @example instead.
14761 Reproducing an example with @reproduce_failure
14763 Hypothesis has an opaque binary representation that it uses for all ex‐
14764 amples it generates. This representation is not intended to be stable
14765 across versions or with respect to changes in the test, but can be used
14766 to to reproduce failures with the @reproduce_failure decorator.
14767 hypothesis.reproduce_failure(version, blob)
14769 Run the example that corresponds to this data blob in order to
14770 reproduce a failure.
14771 A test with this decorator always runs only one example and al‐
14773 ways fails. If the provided example does not cause a failure,
14774 or is in some way invalid for this test, then this will fail
14775 with a DidNotReproduce error.
14776 This decorator is not intended to be a permanent addition to
14778 your test suite. It's simply some code you can add to ease re‐
14779 production of a problem in the event that you don't have access
14780 to the test database. Because of this, no compatibility guaran‐
14781 tees are made between different versions of Hypothesis - its API
14782 may change arbitrarily from version to version.
14783 The intent is that you should never write this decorator by hand, but
14785 it is instead provided by Hypothesis. When a test fails with a falsi‐
14786 fying example, Hypothesis may print out a suggestion to use @repro‐
14787 duce_failure on the test to recreate the problem as follows:
14788 >>> from hypothesis import settings, given, PrintSettings
14790 >>> import hypothesis.strategies as st
14791 >>> @given(st.floats())
14792 ... @settings(print_blob=True)
14793 ... def test(f):
14794 ... assert f == f
14795 ...
14796 >>> try:
14797 ... test()
14798 ... except AssertionError:
14799 ... pass
14800 ...
14801 Falsifying example: test(f=nan)
14802 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
14804 Adding the suggested decorator to the test should reproduce the failure
14806 (as long as everything else is the same - changing the versions of
14807 Python or anything else involved, might of course affect the behaviour
14808 of the test! Note that changing the version of Hypothesis will result
14809 in a different error - each @reproduce_failure invocation is specific
14810 to a Hypothesis version).
14811 By default these messages are not printed. If you want to see these
14813 you must set the print_blob setting to True.
14814
14816 David R. MacIver
14817
14819 2013-2023, David R. MacIver
148206.82.0 Jul 31, 2023 HYPOTHESIS(1)
