1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property-based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the fu‐
41 ture.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 • Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 • If you delete an object, it is no longer visible.
53 • If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ""
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ""
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Let's see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127 Anyway, this test immediately finds a bug in the code:
129 Falsifying example: test_decode_inverts_encode(s='')
131 UnboundLocalError: local variable 'character' referenced before assignment
133 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136 If we fix that by just adding the following code to the beginning of
138 the function then Hypothesis tells us the code is correct (by doing
139 nothing as you'd expect a passing test to).
140 if not input_string:
142 return []
143 If we wanted to make sure this example was always checked we could add
145 it in explicitly by using the @example decorator:
146 from hypothesis import example, given, strategies as st
148 @given(st.text())
151 @example("")
152 def test_decode_inverts_encode(s):
153 assert decode(encode(s)) == s
154 This can be useful to show other developers (or your future self) what
156 kinds of data are valid inputs, or to ensure that particular edge cases
157 such as "" are tested every time. It's also great for regression tests
158 because although Hypothesis will remember failing examples, we don't
159 recommend distributing that database.
160 It's also worth noting that both example and given support keyword ar‐
162 guments as well as positional. The following would have worked just as
163 well:
164 @given(s=st.text())
166 @example(s="")
167 def test_decode_inverts_encode(s):
168 assert decode(encode(s)) == s
169 Suppose we had a more interesting bug and forgot to reset the count
171 each time. Say we missed a line in our encode method:
172 def encode(input_string):
174 count = 1
175 prev = ""
176 lst = []
177 for character in input_string:
178 if character != prev:
179 if prev:
180 entry = (prev, count)
181 lst.append(entry)
182 # count = 1 # Missing reset operation
183 prev = character
184 else:
185 count += 1
186 entry = (character, count)
187 lst.append(entry)
188 return lst
189 Hypothesis quickly informs us of the following example:
191 Falsifying example: test_decode_inverts_encode(s='001')
193 Note that the example provided is really quite simple. Hypothesis
195 doesn't just find any counter-example to your tests, it knows how to
196 simplify the examples it finds to produce small easy to understand
197 ones. In this case, two identical values are enough to set the count to
198 a number different from one, followed by another distinct value which
199 should have reset the count but in this case didn't.
200 The examples Hypothesis provides are valid Python code you can run. Any
202 arguments that you explicitly provide when calling the function are not
203 generated by Hypothesis, and if you explicitly provide all the argu‐
204 ments Hypothesis will just call the underlying function once rather
205 than running it multiple times.
206 Installing
208 Hypothesis is available on PyPI as "hypothesis". You can install it
209 with:
210 pip install hypothesis
212 You can install the dependencies for optional extensions with e.g. pip
214 install hypothesis[pandas,django].
215 If you want to install directly from the source code (e.g. because you
217 want to make changes and install the changed version), check out the
218 instructions in CONTRIBUTING.rst.
219 Running tests
221 In our example above we just let pytest discover and run our tests, but
222 we could also have run it explicitly ourselves:
223 if __name__ == "__main__":
225 test_decode_inverts_encode()
226 We could also have done this as a python:unittest.TestCase:
228 import unittest
230 class TestEncoding(unittest.TestCase):
233 @given(text())
234 def test_decode_inverts_encode(self, s):
235 self.assertEqual(decode(encode(s)), s)
236 if __name__ == "__main__":
239 unittest.main()
240 A detail: This works because Hypothesis ignores any arguments it hasn't
242 been told to provide (positional arguments start from the right), so
243 the self argument to the test is simply ignored and works as normal.
244 This also means that Hypothesis will play nicely with other ways of pa‐
245 rameterizing tests. e.g it works fine if you use pytest fixtures for
246 some arguments and Hypothesis for others.
247 Writing tests
249 A test in Hypothesis consists of two parts: A function that looks like
250 a normal test in your test framework of choice but with some additional
251 arguments, and a @given decorator that specifies how to provide those
252 arguments.
253 Here are some other examples of how you could use that:
255 from hypothesis import given, strategies as st
257 @given(st.integers(), st.integers())
260 def test_ints_are_commutative(x, y):
261 assert x + y == y + x
262 @given(x=st.integers(), y=st.integers())
265 def test_ints_cancel(x, y):
266 assert (x + y) - y == x
267 @given(st.lists(st.integers()))
270 def test_reversing_twice_gives_same_list(xs):
271 # This will generate lists of arbitrary length (usually between 0 and
272 # 100 elements) whose elements are integers.
273 ys = list(xs)
274 ys.reverse()
275 ys.reverse()
276 assert xs == ys
277 @given(st.tuples(st.booleans(), st.text()))
280 def test_look_tuples_work_too(t):
281 # A tuple is generated as the one you provided, with the corresponding
282 # types in those positions.
283 assert len(t) == 2
284 assert isinstance(t[0], bool)
285 assert isinstance(t[1], str)
286 Note that as we saw in the above example you can pass arguments to
288 @given either as positional or as keywords.
289 Where to start
291 You should now know enough of the basics to write some tests for your
292 code using Hypothesis. The best way to learn is by doing, so go have a
293 try.
294 If you're stuck for ideas for how to use this sort of test for your
296 code, here are some good starting points:
297 1. Try just calling functions with appropriate arbitrary data and see
299 if they crash. You may be surprised how often this works. e.g. note
300 that the first bug we found in the encoding example didn't even get
301 as far as our assertion: It crashed because it couldn't handle the
302 data we gave it, not because it did the wrong thing.
303 2. Look for duplication in your tests. Are there any cases where you're
305 testing the same thing with multiple different examples? Can you
306 generalise that to a single test using Hypothesis?
307 3. This piece is designed for an F# implementation, but is still very
309 good advice which you may find helps give you good ideas for using
310 Hypothesis.
311 If you have any trouble getting started, don't feel shy about asking
313 for help.
314
316 This is an account of slightly less common Hypothesis features that you
317 don't need to get started but will nevertheless make your life easier.
318 Additional test output
320 Normally the output of a failing test will look something like:
321 Falsifying example: test_a_thing(x=1, y="foo")
323 With the repr of each keyword argument being printed.
325 Sometimes this isn't enough, either because you have a value with a
327 __repr__() method that isn't very descriptive or because you need to
328 see the output of some intermediate steps of your test. That's where
329 the note function comes in:
330 hypothesis.note(value)
332 Report this value in the final execution.
333 >>> from hypothesis import given, note, strategies as st
335 >>> @given(st.lists(st.integers()), st.randoms())
336 ... def test_shuffle_is_noop(ls, r):
337 ... ls2 = list(ls)
338 ... r.shuffle(ls2)
339 ... note(f"Shuffle: {ls2!r}")
340 ... assert ls == ls2
341 ...
342 >>> try:
343 ... test_shuffle_is_noop()
344 ... except AssertionError:
345 ... print("ls != ls2")
346 ...
347 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
348 Shuffle: [1, 0]
349 ls != ls2
350 The note is printed in the final run of the test in order to include
352 any additional information you might need in your test.
353 Test statistics
355 If you are using pytest you can see a number of statistics about the
356 executed tests by passing the command line argument --hypothe‐
357 sis-show-statistics. This will include some general statistics about
358 the test:
359 For example if you ran the following with --hypothesis-show-statistics:
361 from hypothesis import given, strategies as st
363 @given(st.integers())
366 def test_integers(i):
367 pass
368 You would see:
370 - during generate phase (0.06 seconds):
372 - Typical runtimes: < 1ms, ~ 47% in data generation
373 - 100 passing examples, 0 failing examples, 0 invalid examples
374 - Stopped because settings.max_examples=100
375 The final "Stopped because" line is particularly important to note: It
377 tells you the setting value that determined when the test should stop
378 trying new examples. This can be useful for understanding the behaviour
379 of your tests. Ideally you'd always want this to be max_examples.
380 In some cases (such as filtered and recursive strategies) you will see
382 events mentioned which describe some aspect of the data generation:
383 from hypothesis import given, strategies as st
385 @given(st.integers().filter(lambda x: x % 2 == 0))
388 def test_even_integers(i):
389 pass
390 You would see something like:
392 test_even_integers:
394 - during generate phase (0.08 seconds):
396 - Typical runtimes: < 1ms, ~ 57% in data generation
397 - 100 passing examples, 0 failing examples, 12 invalid examples
398 - Events:
399 * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
400 * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
401 - Stopped because settings.max_examples=100
402 You can also mark custom events in a test using the event function:
404 hypothesis.event(value)
406 Record an event that occurred this test. Statistics on number of
407 test runs with each event will be reported at the end if you run
408 Hypothesis in statistics reporting mode.
409 Events should be strings or convertible to them.
411 from hypothesis import event, given, strategies as st
413 @given(st.integers().filter(lambda x: x % 2 == 0))
416 def test_even_integers(i):
417 event(f"i mod 3 = {i%3}")
418 You will then see output like:
420 test_even_integers:
422 - during generate phase (0.09 seconds):
424 - Typical runtimes: < 1ms, ~ 59% in data generation
425 - 100 passing examples, 0 failing examples, 32 invalid examples
426 - Events:
427 * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
428 * 31.06%, i mod 3 = 2
429 * 28.79%, i mod 3 = 0
430 * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
431 * 15.91%, i mod 3 = 1
432 - Stopped because settings.max_examples=100
433 Arguments to event can be any hashable type, but two events will be
435 considered the same if they are the same when converted to a string
436 with python:str.
437 Making assumptions
439 Sometimes Hypothesis doesn't give you exactly the right sort of data
440 you want - it's mostly of the right shape, but some examples won't work
441 and you don't want to care about them. You can just ignore these by
442 aborting the test early, but this runs the risk of accidentally testing
443 a lot less than you think you are. Also it would be nice to spend less
444 time on bad examples - if you're running 100 examples per test (the de‐
445 fault) and it turns out 70 of those examples don't match your needs,
446 that's a lot of wasted time.
447 hypothesis.assume(condition)
449 Calling assume is like an assert that marks the example as bad,
450 rather than failing the test.
451 This allows you to specify properties that you assume will be
453 true, and let Hypothesis try to avoid similar examples in fu‐
454 ture.
455 For example suppose you had the following test:
457 @given(floats())
459 def test_negation_is_self_inverse(x):
460 assert x == -(-x)
461 Running this gives us:
463 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
465 AssertionError
466 This is annoying. We know about NaN and don't really care about it, but
468 as soon as Hypothesis finds a NaN example it will get distracted by
469 that and tell us about it. Also the test will fail and we want it to
470 pass.
471 So let's block off this particular example:
473 from math import isnan
475 @given(floats())
478 def test_negation_is_self_inverse_for_non_nan(x):
479 assume(not isnan(x))
480 assert x == -(-x)
481 And this passes without a problem.
483 In order to avoid the easy trap where you assume a lot more than you
485 intended, Hypothesis will fail a test when it can't find enough exam‐
486 ples passing the assumption.
487 If we'd written:
489 @given(floats())
491 def test_negation_is_self_inverse_for_non_nan(x):
492 assume(False)
493 assert x == -(-x)
494 Then on running we'd have got the exception:
496 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
498 How good is assume?
500 Hypothesis has an adaptive exploration strategy to try to avoid things
501 which falsify assumptions, which should generally result in it still
502 being able to find examples in hard to find situations.
503 Suppose we had the following:
505 @given(lists(integers()))
507 def test_sum_is_positive(xs):
508 assert sum(xs) > 0
509 Unsurprisingly this fails and gives the falsifying example [].
511 Adding assume(xs) to this removes the trivial empty example and gives
513 us [0].
514 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
516 of positive integers is positive.
517 The reason that this should be surprising is not that it doesn't find a
519 counter-example, but that it finds enough examples at all.
520 In order to make sure something interesting is happening, suppose we
522 wanted to try this for long lists. e.g. suppose we added an as‐
523 sume(len(xs) > 10) to it. This should basically never find an example:
524 a naive strategy would find fewer than one in a thousand examples, be‐
525 cause if each element of the list is negative with probability
526 one-half, you'd have to have ten of these go the right way by chance.
527 In the default configuration Hypothesis gives up long before it's tried
528 1000 examples (by default it tries 200).
529 Here's what happens if we try to run this:
531 @given(lists(integers()))
533 def test_sum_is_positive(xs):
534 assume(len(xs) > 1)
535 assume(all(x > 0 for x in xs))
536 print(xs)
537 assert sum(xs) > 0
538 In: test_sum_is_positive()
540 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
542 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
543 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
544 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
545 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
546 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
547 As you can see, Hypothesis doesn't find many examples here, but it
549 finds some - enough to keep it happy.
550 In general if you can shape your strategies better to your tests you
552 should - for example integers(1, 1000) is a lot better than assume(1 <=
553 x <= 1000), but assume will take you a long way if you can't.
554 Defining strategies
556 The type of object that is used to explore the examples given to your
557 test function is called a SearchStrategy. These are created using the
558 functions exposed in the hypothesis.strategies module.
559 Many of these strategies expose a variety of arguments you can use to
561 customize generation. For example for integers you can specify min and
562 max values of integers you want. If you want to see exactly what a
563 strategy produces you can ask for an example:
564 >>> integers(min_value=0, max_value=10).example()
566 1
567 Many strategies are built out of other strategies. For example, if you
569 want to define a tuple you need to say what goes in each element:
570 >>> from hypothesis.strategies import tuples
572 >>> tuples(integers(), integers()).example()
573 (-24597, 12566)
574 Further details are available in a separate document.
576 The gory details of given parameters
578 hypothesis.given(*_given_arguments, **_given_kwargs)
579 A decorator for turning a test function that accepts arguments
580 into a randomized test.
581 This is the main entry point to Hypothesis.
583 The @given decorator may be used to specify which arguments of a func‐
585 tion should be parametrized over. You can use either positional or key‐
586 word arguments, but not a mixture of both.
587 For example all of the following are valid uses:
589 @given(integers(), integers())
591 def a(x, y):
592 pass
593 @given(integers())
596 def b(x, y):
597 pass
598 @given(y=integers())
601 def c(x, y):
602 pass
603 @given(x=integers())
606 def d(x, y):
607 pass
608 @given(x=integers(), y=integers())
611 def e(x, **kwargs):
612 pass
613 @given(x=integers(), y=integers())
616 def f(x, *args, **kwargs):
617 pass
618 class SomeTest(TestCase):
621 @given(integers())
622 def test_a_thing(self, x):
623 pass
624 The following are not:
626 @given(integers(), integers(), integers())
628 def g(x, y):
629 pass
630 @given(integers())
633 def h(x, *args):
634 pass
635 @given(integers(), x=integers())
638 def i(x, y):
639 pass
640 @given()
643 def j(x, y):
644 pass
645 The rules for determining what are valid uses of given are as follows:
647 1. You may pass any keyword argument to given.
649 2. Positional arguments to given are equivalent to the rightmost named
651 arguments for the test function.
652 3. Positional arguments may not be used if the underlying test function
654 has varargs, arbitrary keywords, or keyword-only arguments.
655 4. Functions tested with given may not have any defaults.
657 The reason for the "rightmost named arguments" behaviour is so that us‐
659 ing @given with instance methods works: self will be passed to the
660 function as normal and not be parametrized over.
661 The function returned by given has all the same arguments as the origi‐
663 nal test, minus those that are filled in by @given. Check the notes on
664 framework compatibility to see how this affects other testing libraries
665 you may be using.
666 Targeted example generation
668 Targeted property-based testing combines the advantages of both
669 search-based and property-based testing. Instead of being completely
670 random, T-PBT uses a search-based component to guide the input genera‐
671 tion towards values that have a higher probability of falsifying a
672 property. This explores the input space more effectively and requires
673 fewer tests to find a bug or achieve a high confidence in the system
674 being tested than random PBT. (Löscher and Sagonas)
675 This is not always a good idea - for example calculating the search
677 metric might take time better spent running more uniformly-random test
678 cases - but Hypothesis has experimental support for targeted PBT you
679 may wish to try.
680 hypothesis.target(observation, *, label='')
682 Calling this function with an int or float observation gives it
683 feedback with which to guide our search for inputs that will
684 cause an error, in addition to all the usual heuristics. Obser‐
685 vations must always be finite.
686 Hypothesis will try to maximize the observed value over several
688 examples; almost any metric will work so long as it makes sense
689 to increase it. For example, -abs(error) is a metric that in‐
690 creases as error approaches zero.
691 Example metrics:
693 • Number of elements in a collection, or tasks in a queue
695 • Mean or maximum runtime of a task (or both, if you use label)
697 • Compression ratio for data (perhaps per-algorithm or
699 per-level)
700 • Number of steps taken by a state machine
702 The optional label argument can be used to distinguish between
704 and therefore separately optimise distinct observations, such as
705 the mean and standard deviation of a dataset. It is an error to
706 call target() with any label more than once per test case.
707 NOTE:
709 The more examples you run, the better this technique works.
710 As a rule of thumb, the targeting effect is noticeable above
712 max_examples=1000, and immediately obvious by around ten
713 thousand examples per label used by your test.
714 Test statistics include the best score seen for each label,
716 which can help avoid the threshold problem when the minimal ex‐
717 ample shrinks right down to the threshold of failure (issue
718 #2180).
719 We recommend that users also skim the papers introducing targeted PBT;
721 from ISSTA 2017 and ICST 2018. For the curious, the initial implemen‐
722 tation in Hypothesis uses hill-climbing search via a mutating fuzzer,
723 with some tactics inspired by simulated annealing to avoid getting
724 stuck and endlessly mutating a local maximum.
725 Custom function execution
727 Hypothesis provides you with a hook that lets you control how it runs
728 examples.
729 This lets you do things like set up and tear down around each example,
731 run examples in a subprocess, transform coroutine tests into normal
732 tests, etc. For example, TransactionTestCase in the Django extra runs
733 each example in a separate database transaction.
734 The way this works is by introducing the concept of an executor. An ex‐
736 ecutor is essentially a function that takes a block of code and run it.
737 The default executor is:
738 def default_executor(function):
740 return function()
741 You define executors by defining a method execute_example on a class.
743 Any test methods on that class with @given used on them will use
744 self.execute_example as an executor with which to run tests. For exam‐
745 ple, the following executor runs all its code twice:
746 from unittest import TestCase
748 class TestTryReallyHard(TestCase):
751 @given(integers())
752 def test_something(self, i):
753 perform_some_unreliable_operation(i)
754 def execute_example(self, f):
756 f()
757 return f()
758 Note: The functions you use in map, etc. will run inside the executor.
760 i.e. they will not be called until you invoke the function passed to
761 execute_example.
762 An executor must be able to handle being passed a function which re‐
764 turns None, otherwise it won't be able to run normal test cases. So for
765 example the following executor is invalid:
766 from unittest import TestCase
768 class TestRunTwice(TestCase):
771 def execute_example(self, f):
772 return f()()
773 and should be rewritten as:
775 from unittest import TestCase
777 class TestRunTwice(TestCase):
780 def execute_example(self, f):
781 result = f()
782 if callable(result):
783 result = result()
784 return result
785 An alternative hook is provided for use by test runner extensions such
787 as pytest-trio, which cannot use the execute_example method. This is
788 not recommended for end-users - it is better to write a complete test
789 function directly, perhaps by using a decorator to perform the same
790 transformation before applying @given.
791 @given(x=integers())
793 @pytest.mark.trio
794 async def test(x):
795 ...
796 # Illustrative code, inside the pytest-trio plugin
799 test.hypothesis.inner_test = lambda x: trio.run(test, x)
800 For authors of test runners however, assigning to the inner_test attri‐
802 bute of the hypothesis attribute of the test will replace the interior
803 test.
804 NOTE:
806 The new inner_test must accept and pass through all the *args and
807 **kwargs expected by the original test.
808 If the end user has also specified a custom executor using the exe‐
810 cute_example method, it - and all other execution-time logic - will be
811 applied to the new inner test assigned by the test runner.
812 Making random code deterministic
814 While Hypothesis' example generation can be used for nondeterministic
815 tests, debugging anything nondeterministic is usually a very frustrat‐
816 ing exercise. To make things worse, our example shrinking relies on
817 the same input causing the same failure each time - though we show the
818 un-shrunk failure and a decent error message if it doesn't.
819 By default, Hypothesis will handle the global random and numpy.random
821 random number generators for you, and you can register others:
822 hypothesis.register_random(r)
824 Register the given Random instance for management by Hypothesis.
825 You can pass random.Random instances (or other objects with
827 seed, getstate, and setstate methods) to register_random(r) to
828 have their states seeded and restored in the same way as the
829 global PRNGs from the random and numpy.random modules.
830 All global PRNGs, from e.g. simulation or scheduling frameworks,
832 should be registered to prevent flaky tests. Hypothesis will
833 ensure that the PRNG state is consistent for all test runs, or
834 reproducibly varied if you choose to use the random_module()
835 strategy.
836 Inferred strategies
838 In some cases, Hypothesis can work out what to do when you omit argu‐
839 ments. This is based on introspection, not magic, and therefore has
840 well-defined limits.
841 builds() will check the signature of the target (using get‐
843 fullargspec()). If there are required arguments with type annotations
844 and no strategy was passed to builds(), from_type() is used to fill
845 them in. You can also pass the special value hypothesis.infer as a
846 keyword argument, to force this inference for arguments with a default
847 value.
848 >>> def func(a: int, b: str):
850 ... return [a, b]
851 ...
852 >>> builds(func).example()
853 [-6993, '']
854 hypothesis.infer
856 @given does not perform any implicit inference for required arguments,
858 as this would break compatibility with pytest fixtures. infer can be
859 used as a keyword argument to explicitly fill in an argument from its
860 type annotation.
861 @given(a=infer)
863 def test(a: int):
864 pass
865 # is equivalent to
868 @given(a=integers())
869 def test(a):
870 pass
871 Limitations
873 Hypothesis does not inspect PEP 484 type comments at runtime. While
874 from_type() will work as usual, inference in builds() and @given will
875 only work if you manually create the __annotations__ attribute (e.g. by
876 using @annotations(...) and @returns(...) decorators).
877 The python:typing module is provisional and has a number of internal
879 changes between Python 3.5.0 and 3.6.1, including at minor versions.
880 These are all supported on a best-effort basis, but you may encounter
881 problems with an old version of the module. Please report them to us,
882 and consider updating to a newer version of Python as a workaround.
883 Type annotations in Hypothesis
885 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
886 patible tool, the type checker should automatically pick up our type
887 hints.
888 NOTE:
890 Hypothesis' type hints may make breaking changes between minor re‐
891 leases.
892 Upstream tools and conventions about type hints remain in flux - for
894 example the python:typing module itself is provisional, and Mypy has
895 not yet reached version 1.0 - and we plan to support the latest ver‐
896 sion of this ecosystem, as well as older versions where practical.
897 We may also find more precise ways to describe the type of various
899 interfaces, or change their type and runtime behaviour together in a
900 way which is otherwise backwards-compatible. We often omit type
901 hints for deprecated features or arguments, as an additional form of
902 warning.
903 There are known issues inferring the type of examples generated by de‐
905 ferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
906 ies(). We will fix these, and require correspondingly newer versions
907 of Mypy for type hinting, as the ecosystem improves.
908 Writing downstream type hints
910 Projects that provide Hypothesis strategies and use type hints may wish
911 to annotate their strategies too. This is a supported use-case, again
912 on a best-effort provisional basis. For example:
913 def foo_strategy() -> SearchStrategy[Foo]:
915 ...
916 class hypothesis.strategies.SearchStrategy
918 SearchStrategy is the type of all strategy objects. It is a generic
920 type, and covariant in the type of the examples it creates. For exam‐
921 ple:
922 • integers() is of type SearchStrategy[int].
924 • lists(integers()) is of type SearchStrategy[List[int]].
926 • SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
928 a subtype of Animal (as seems likely).
929 WARNING:
931 SearchStrategy should only be used in type hints. Please do not in‐
932 herit from, compare to, or otherwise use it in any way outside of
933 type hints. The only supported way to construct objects of this
934 type is to use the functions provided by the hypothesis.strategies
935 module!
936 The Hypothesis pytest plugin
938 Hypothesis includes a tiny plugin to improve integration with pytest,
939 which is activated by default (but does not affect other test runners).
940 It aims to improve the integration between Hypothesis and Pytest by
941 providing extra information and convenient access to config options.
942 • pytest --hypothesis-show-statistics can be used to display test and
944 data generation statistics.
945 • pytest --hypothesis-profile=<profile name> can be used to load a set‐
947 tings profile. pytest --hypothesis-verbosity=<level name> can be
948 used to override the current verbosity level.
949 • pytest --hypothesis-seed=<an int> can be used to reproduce a failure
951 with a particular seed.
952 • pytest --hypothesis-explain can be used to temporarily enable the ex‐
954 plain phase.
955 Finally, all tests that are defined with Hypothesis automatically have
957 @pytest.mark.hypothesis applied to them. See here for information on
958 working with markers.
959 NOTE:
961 Pytest will load the plugin automatically if Hypothesis is in‐
962 stalled. You don't need to do anything at all to use it.
963 Use with external fuzzers
965 TIP:
966 Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
967 Use HypoFuzz to fuzz your whole test suite,
968 and find more bugs without more tests!
969 Sometimes, you might want to point a traditional fuzzer such as
972 python-afl, pythonfuzz, or Google's atheris (for Python and native ex‐
973 tensions) at your code. Wouldn't it be nice if you could use any of
974 your @given tests as fuzz targets, instead of converting bytestrings
975 into your objects by hand?
976 @given(st.text())
978 def test_foo(s):
979 ...
980 # This is a traditional fuzz target - call it with a bytestring,
983 # or a binary IO object, and it runs the test once.
984 fuzz_target = test_foo.hypothesis.fuzz_one_input
985 # For example:
987 fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
988 fuzz_target(io.BytesIO(...))
989 Depending on the input to fuzz_one_input, one of three things will hap‐
991 pen:
992 • If the bytestring was invalid, for example because it was too short
994 or failed a filter or assume() too many times, fuzz_one_input returns
995 None.
996 • If the bytestring was valid and the test passed, fuzz_one_input re‐
998 turns a canonicalised and pruned buffer which will replay that test
999 case. This is provided as an option to improve the performance of
1000 mutating fuzzers, but can safely be ignored.
1001 • If the test failed, i.e. raised an exception, fuzz_one_input will add
1003 the pruned buffer to the Hypothesis example database and then
1004 re-raise that exception. All you need to do to reproduce, minimize,
1005 and de-duplicate all the failures found via fuzzing is run your test
1006 suite!
1007 Note that the interpretation of both input and output bytestrings is
1009 specific to the exact version of Hypothesis you are using and the
1010 strategies given to the test, just like the example database and @re‐
1011 produce_failure decorator.
1012 Interaction with settings
1014 fuzz_one_input uses just enough of Hypothesis' internals to drive your
1015 test function with a fuzzer-provided bytestring, and most settings
1016 therefore have no effect in this mode. We recommend running your tests
1017 the usual way before fuzzing to get the benefits of healthchecks, as
1018 well as afterwards to replay, shrink, deduplicate, and report whatever
1019 errors were discovered.
1020 • The database setting is used by fuzzing mode - adding failures to the
1022 database to be replayed when you next run your tests is our preferred
1023 reporting mechanism and response to the 'fuzzer taming' problem.
1024 • The verbosity and stateful_step_count settings work as usual.
1026 The deadline, derandomize, max_examples, phases, print_blob, re‐
1028 port_multiple_bugs, and suppress_health_check settings do not affect
1029 fuzzing mode.
1030 Thread-Safety Policy
1032 As discussed in issue #2719, Hypothesis is not truly thread-safe and
1033 that's unlikely to change in the future. This policy therefore de‐
1034 scribes what you can expect if you use Hypothesis with multiple
1035 threads.
1036 Running tests in multiple processes, e.g. with pytest -n auto, is fully
1038 supported and we test this regularly in CI - thanks to process isola‐
1039 tion, we only need to ensure that DirectoryBasedExampleDatabase can't
1040 tread on its own toes too badly. If you find a bug here we will fix it
1041 ASAP.
1042 Running separate tests in multiple threads is not something we design
1044 or test for, and is not formally supported. That said, anecdotally it
1045 does mostly work and we would like it to keep working - we accept rea‐
1046 sonable patches and low-priority bug reports. The main risks here are
1047 global state, shared caches, and cached strategies.
1048 Using multiple threads within a single test , or running a single test
1050 simultaneously in multiple threads, makes it pretty easy to trigger in‐
1051 ternal errors. We usually accept patches for such issues unless read‐
1052 ability or single-thread performance suffer.
1053 Hypothesis assumes that tests are single-threaded, or do a suffi‐
1055 ciently-good job of pretending to be single-threaded. Tests that use
1056 helper threads internally should be OK, but the user must be careful to
1057 ensure that test outcomes are still deterministic. In particular it
1058 counts as nondeterministic if helper-thread timing changes the sequence
1059 of dynamic draws using e.g. the data().
1060 Interacting with any Hypothesis APIs from helper threads might do
1062 weird/bad things, so avoid that too - we rely on thread-local variables
1063 in a few places, and haven't explicitly tested/audited how they respond
1064 to cross-thread API calls. While data() and equivalents are the most
1065 obvious danger, other APIs might also be subtly affected.
1066
1068 Hypothesis tries to have good defaults for its behaviour, but sometimes
1069 that's not enough and you need to tweak it.
1070 The mechanism for doing this is the settings object. You can set up a
1072 @given based test to use this using a settings decorator:
1073 @given invocation is as follows:
1075 from hypothesis import given, settings
1077 @given(integers())
1080 @settings(max_examples=500)
1081 def test_this_thoroughly(x):
1082 pass
1083 This uses a settings object which causes the test to receive a much
1085 larger set of examples than normal.
1086 This may be applied either before or after the given and the results
1088 are the same. The following is exactly equivalent:
1089 from hypothesis import given, settings
1091 @settings(max_examples=500)
1094 @given(integers())
1095 def test_this_thoroughly(x):
1096 pass
1097 Available settings
1099 class hypothesis.settings(parent=None, *, max_examples=not_set, deran‐
1100 domize=not_set, database=not_set, verbosity=not_set, phases=not_set,
1101 stateful_step_count=not_set, report_multiple_bugs=not_set, sup‐
1102 press_health_check=not_set, deadline=not_set, print_blob=not_set)
1103 A settings object controls a variety of parameters that are used
1104 in falsification. These may control both the falsification
1105 strategy and the details of the data that is generated.
1106 Default values are picked up from the settings.default object
1108 and changes made there will be picked up in newly created set‐
1109 tings.
1110 database
1112 An instance of ExampleDatabase that will be used to save
1113 examples to and load previous examples from. May be None
1114 in which case no storage will be used.
1115 See the example database documentation for a list of
1117 built-in example database implementations, and how to de‐
1118 fine custom implementations.
1119 default value: (dynamically calculated)
1121 deadline
1123 If set, a duration (as timedelta, or integer or float
1124 number of milliseconds) that each individual example
1125 (i.e. each time your test function is called, not the
1126 whole decorated test) within a test is not allowed to ex‐
1127 ceed. Tests which take longer than that may be converted
1128 into errors (but will not necessarily be if close to the
1129 deadline, to allow some variability in test run time).
1130 Set this to None to disable this behaviour entirely.
1132 default value: timedelta(milliseconds=200)
1134 derandomize
1136 If True, seed Hypothesis' random number generator using a
1137 hash of the test function, so that every run will test
1138 the same set of examples until you update Hypothesis,
1139 Python, or the test function.
1140 This allows you to check for regressions and look for
1142 bugs using separate settings profiles - for example run‐
1143 ning quick deterministic tests on every commit, and a
1144 longer non-deterministic nightly testing run.
1145 default value: False
1147 max_examples
1149 Once this many satisfying examples have been considered
1150 without finding any counter-example, falsification will
1151 terminate.
1152 The default value is chosen to suit a workflow where the
1154 test will be part of a suite that is regularly executed
1155 locally or on a CI server, balancing total running time
1156 against the chance of missing a bug.
1157 If you are writing one-off tests, running tens of thou‐
1159 sands of examples is quite reasonable as Hypothesis may
1160 miss uncommon bugs with default settings. For very com‐
1161 plex code, we have observed Hypothesis finding novel bugs
1162 after several million examples while testing SymPy. If
1163 you are running more than 100k examples for a test, con‐
1164 sider using our integration for coverage-guided fuzzing -
1165 it really shines when given minutes or hours to run.
1166 default value: 100
1168 phases Control which phases should be run. See the full documen‐
1170 tation for more details
1171 default value: (Phase.explicit, Phase.reuse, Phase.gener‐
1173 ate, Phase.target, Phase.shrink)
1174 print_blob
1176 If set to True, Hypothesis will print code for failing
1177 examples that can be used with @reproduce_failure to re‐
1178 produce the failing example. The default is True if the
1179 CI or TF_BUILD env vars are set, False otherwise.
1180 default value: (dynamically calculated)
1182 report_multiple_bugs
1184 Because Hypothesis runs the test many times, it can some‐
1185 times find multiple bugs in a single run. Reporting all
1186 of them at once is usually very useful, but replacing the
1187 exceptions can occasionally clash with debuggers. If
1188 disabled, only the exception with the smallest minimal
1189 example is raised.
1190 default value: True
1192 stateful_step_count
1194 Number of steps to run a stateful program for before giv‐
1195 ing up on it breaking.
1196 default value: 50
1198 suppress_health_check
1200 A list of HealthCheck items to disable.
1201 default value: ()
1203 verbosity
1205 Control the verbosity level of Hypothesis messages
1206 default value: Verbosity.normal
1208 Controlling what runs
1210 Hypothesis divides tests into logically distinct phases:
1211 1. Running explicit examples provided with the @example decorator.
1213 2. Rerunning a selection of previously failing examples to reproduce a
1215 previously seen error
1216 3. Generating new examples.
1218 4. Mutating examples for targeted property-based testing.
1220 5. Attempting to shrink an example found in previous phases (other than
1222 phase 1 - explicit examples cannot be shrunk). This turns poten‐
1223 tially large and complicated examples which may be hard to read into
1224 smaller and simpler ones.
1225 6. Attempting to explain the cause of the failure, by identifying sus‐
1227 picious lines of code (e.g. the earliest lines which are never run
1228 on passing inputs, and always run on failures). This relies on
1229 python:sys.settrace(), and is therefore automatically disabled on
1230 PyPy or if you are using coverage or a debugger. If there are no
1231 clearly suspicious lines of code, we refuse the temptation to guess.
1232 The phases setting provides you with fine grained control over which of
1234 these run, with each phase corresponding to a value on the Phase enum:
1235 class hypothesis.Phase
1237 1. Phase.explicit controls whether explicit examples are run.
1239 2. Phase.reuse controls whether previous examples will be reused.
1241 3. Phase.generate controls whether new examples will be generated.
1243 4. Phase.target controls whether examples will be mutated for target‐
1245 ing.
1246 5. Phase.shrink controls whether examples will be shrunk.
1248 6. Phase.explain controls whether Hypothesis attempts to explain test
1250 failures.
1251 The phases argument accepts a collection with any subset of these. e.g.
1253 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1254 ples and shrink them, but will not run explicit examples or reuse pre‐
1255 vious failures, while settings(phases=[Phase.explicit]) will only run
1256 the explicit examples.
1257 Seeing intermediate result
1259 To see what's going on while Hypothesis runs your tests, you can turn
1260 up the verbosity setting.
1261 >>> from hypothesis import find, settings, Verbosity
1263 >>> from hypothesis.strategies import lists, integers
1264 >>> @given(lists(integers()))
1265 ... @settings(verbosity=Verbosity.verbose)
1266 ... def f(x):
1267 ... assert not any(x)
1268 ... f()
1269 Trying example: []
1270 Falsifying example: [-1198601713, -67, 116, -29578]
1271 Shrunk example to [-1198601713]
1272 Shrunk example to [-1198601600]
1273 Shrunk example to [-1191228800]
1274 Shrunk example to [-8421504]
1275 Shrunk example to [-32896]
1276 Shrunk example to [-128]
1277 Shrunk example to [64]
1278 Shrunk example to [32]
1279 Shrunk example to [16]
1280 Shrunk example to [8]
1281 Shrunk example to [4]
1282 Shrunk example to [3]
1283 Shrunk example to [2]
1284 Shrunk example to [1]
1285 [1]
1286 The four levels are quiet, normal, verbose and debug. normal is the de‐
1288 fault, while in quiet mode Hypothesis will not print anything out, not
1289 even the final falsifying example. debug is basically verbose but a bit
1290 more so. You probably don't want it.
1291 If you are using pytest, you may also need to disable output capturing
1293 for passing tests.
1294 Building settings objects
1296 Settings can be created by calling settings with any of the available
1297 settings values. Any absent ones will be set to defaults:
1298 >>> from hypothesis import settings
1300 >>> settings().max_examples
1301 100
1302 >>> settings(max_examples=10).max_examples
1303 10
1304 You can also pass a 'parent' settings object as the first argument, and
1306 any settings you do not specify as keyword arguments will be copied
1307 from the parent settings:
1308 >>> parent = settings(max_examples=10)
1310 >>> child = settings(parent, deadline=None)
1311 >>> parent.max_examples == child.max_examples == 10
1312 True
1313 >>> parent.deadline
1314 200
1315 >>> child.deadline is None
1316 True
1317 Default settings
1319 At any given point in your program there is a current default settings,
1320 available as settings.default. As well as being a settings object in
1321 its own right, all newly created settings objects which are not explic‐
1322 itly based off another settings are based off the default, so will in‐
1323 herit any values that are not explicitly set from it.
1324 You can change the defaults by using profiles.
1326 Settings profiles
1328 Depending on your environment you may want different default settings.
1329 For example: during development you may want to lower the number of ex‐
1330 amples to speed up the tests. However, in a CI environment you may want
1331 more examples so you are more likely to find bugs.
1332 Hypothesis allows you to define different settings profiles. These pro‐
1334 files can be loaded at any time.
1335 static settings.register_profile(name, parent=None, **kwargs)
1337 Registers a collection of values to be used as a settings pro‐
1338 file.
1339 Settings profiles can be loaded by name - for example, you might
1341 create a 'fast' profile which runs fewer examples, keep the 'de‐
1342 fault' profile, and create a 'ci' profile that increases the
1343 number of examples and uses a different database to store fail‐
1344 ures.
1345 The arguments to this method are exactly as for settings: op‐
1347 tional parent settings, and keyword arguments for each setting
1348 that will be set differently to parent (or settings.default, if
1349 parent is None).
1350 static settings.get_profile(name)
1352 Return the profile with the given name.
1353 static settings.load_profile(name)
1355 Loads in the settings defined in the profile provided.
1356 If the profile does not exist, InvalidArgument will be raised.
1358 Any setting not defined in the profile will be the library de‐
1359 fined default for that setting.
1360 Loading a profile changes the default settings but will not change the
1362 behaviour of tests that explicitly change the settings.
1363 >>> from hypothesis import settings
1365 >>> settings.register_profile("ci", max_examples=1000)
1366 >>> settings().max_examples
1367 100
1368 >>> settings.load_profile("ci")
1369 >>> settings().max_examples
1370 1000
1371 Instead of loading the profile and overriding the defaults you can re‐
1373 trieve profiles for specific tests.
1374 >>> settings.get_profile("ci").max_examples
1376 1000
1377 Optionally, you may define the environment variable to load a profile
1379 for you. This is the suggested pattern for running your tests on CI.
1380 The code below should run in a conftest.py or any setup/initialization
1381 section of your test suite. If this variable is not defined the Hy‐
1382 pothesis defined defaults will be loaded.
1383 >>> import os
1385 >>> from hypothesis import settings, Verbosity
1386 >>> settings.register_profile("ci", max_examples=1000)
1387 >>> settings.register_profile("dev", max_examples=10)
1388 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1389 >>> settings.load_profile(os.getenv(u"HYPOTHESIS_PROFILE", "default"))
1390 If you are using the hypothesis pytest plugin and your profiles are
1392 registered by your conftest you can load one with the command line op‐
1393 tion --hypothesis-profile.
1394 $ pytest tests --hypothesis-profile <profile-name>
1396
1398 Most things should be easy to generate and everything should be possi‐
1399 ble.
1400 To support this principle Hypothesis provides strategies for most
1402 built-in types with arguments to constrain or adjust the output, as
1403 well as higher-order strategies that can be composed to generate more
1404 complex types.
1405 This document is a guide to what strategies are available for generat‐
1407 ing data and how to build them. Strategies have a variety of other im‐
1408 portant internal features, such as how they simplify, but the data they
1409 can generate is the only public part of their API.
1410 Core strategies
1412 Functions for building strategies are all available in the hypothe‐
1413 sis.strategies module. The salient functions from it are as follows:
1414 hypothesis.strategies.binary(*, min_size=0, max_size=None)
1416 Generates python:bytes.
1417 The generated python:bytes will have a length of at least
1419 min_size and at most max_size. If max_size is None there is no
1420 upper limit.
1421 Examples from this strategy shrink towards smaller strings and
1423 lower byte values.
1424 hypothesis.strategies.booleans()
1426 Returns a strategy which generates instances of python:bool.
1427 Examples from this strategy will shrink towards False (i.e.
1429 shrinking will replace True with False where possible).
1430 hypothesis.strategies.builds(target, /, *args, **kwargs)
1432 Generates values by drawing from args and kwargs and passing
1433 them to the callable (provided as the first positional argument)
1434 in the appropriate argument position.
1435 e.g. builds(target, integers(), flag=booleans()) would draw an
1437 integer i and a boolean b and call target(i, flag=b).
1438 If the callable has type annotations, they will be used to infer
1440 a strategy for required arguments that were not passed to
1441 builds. You can also tell builds to infer a strategy for an op‐
1442 tional argument by passing the special value hypothesis.infer as
1443 a keyword argument to builds, instead of a strategy for that ar‐
1444 gument to the callable.
1445 If the callable is a class defined with attrs, missing required
1447 arguments will be inferred from the attribute on a best-effort
1448 basis, e.g. by checking attrs standard validators. Dataclasses
1449 are handled natively by the inference from type hints.
1450 Examples from this strategy shrink by shrinking the argument
1452 values to the callable.
1453 hypothesis.strategies.characters(*, whitelist_categories=None, black‐
1455 list_categories=None, blacklist_characters=None, min_codepoint=None,
1456 max_codepoint=None, whitelist_characters=None)
1457 Generates characters, length-one python:strings, following spec‐
1458 ified filtering rules.
1459 • When no filtering rules are specified, any character can be
1461 produced.
1462 • If min_codepoint or max_codepoint is specified, then only
1464 characters having a codepoint in that range will be produced.
1465 • If whitelist_categories is specified, then only characters
1467 from those Unicode categories will be produced. This is a fur‐
1468 ther restriction, characters must also satisfy min_codepoint
1469 and max_codepoint.
1470 • If blacklist_categories is specified, then any character from
1472 those categories will not be produced. Any overlap between
1473 whitelist_categories and blacklist_categories will raise an
1474 exception, as each character can only belong to a single
1475 class.
1476 • If whitelist_characters is specified, then any additional
1478 characters in that list will also be produced.
1479 • If blacklist_characters is specified, then any characters in
1481 that list will be not be produced. Any overlap between
1482 whitelist_characters and blacklist_characters will raise an
1483 exception.
1484 The _codepoint arguments must be integers between zero and
1486 python:sys.maxunicode. The _characters arguments must be col‐
1487 lections of length-one unicode strings, such as a unicode
1488 string.
1489 The _categories arguments must be used to specify either the
1491 one-letter Unicode major category or the two-letter Unicode
1492 general category. For example, ('Nd', 'Lu') signifies "Number,
1493 decimal digit" and "Letter, uppercase". A single letter ('major
1494 category') can be given to match all corresponding categories,
1495 for example 'P' for characters in any punctuation category.
1496 Examples from this strategy shrink towards the codepoint for
1498 '0', or the first allowable codepoint after it if '0' is ex‐
1499 cluded.
1500 hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magni‐
1502 tude=None, allow_infinity=None, allow_nan=None)
1503 Returns a strategy that generates complex numbers.
1504 This strategy draws complex numbers with constrained magnitudes.
1506 The min_magnitude and max_magnitude parameters should be
1507 non-negative Real numbers; a value of None corresponds an infi‐
1508 nite upper bound.
1509 If min_magnitude is nonzero or max_magnitude is finite, it is an
1511 error to enable allow_nan. If max_magnitude is finite, it is an
1512 error to enable allow_infinity.
1513 The magnitude constraints are respected up to a relative error
1515 of (around) floating-point epsilon, due to implementation via
1516 the system sqrt function.
1517 Examples from this strategy shrink by shrinking their real and
1519 imaginary parts, as floats().
1520 If you need to generate complex numbers with particular real and
1522 imaginary parts or relationships between parts, consider using
1523 builds(complex, ...) or @composite respectively.
1524 hypothesis.strategies.composite(f)
1526 Defines a strategy that is built out of potentially arbitrarily
1527 many other strategies.
1528 This is intended to be used as a decorator. See the full docu‐
1530 mentation for more details about how to use this function.
1531 Examples from this strategy shrink by shrinking the output of
1533 each draw call.
1534 hypothesis.strategies.data()
1536 This isn't really a normal strategy, but instead gives you an
1537 object which can be used to draw data interactively from other
1538 strategies.
1539 See the rest of the documentation for more complete information.
1541 Examples from this strategy do not shrink (because there is only
1543 one), but the result of calls to each draw() call shrink as they
1544 normally would.
1545 class hypothesis.strategies.DataObject(data)
1547 This type only exists so that you can write type hints for tests
1548 using the data() strategy. Do not use it directly!
1549 hypothesis.strategies.dates(min_value=datetime.date.min,
1551 max_value=datetime.date.max)
1552 A strategy for dates between min_value and max_value.
1553 Examples from this strategy shrink towards January 1st 2000.
1555 hypothesis.strategies.datetimes(min_value=datetime.datetime.min,
1557 max_value=datetime.datetime.max, *, timezones=none(), allow_imagi‐
1558 nary=True)
1559 A strategy for generating datetimes, which may be time‐
1560 zone-aware.
1561 This strategy works by drawing a naive datetime between
1563 min_value and max_value, which must both be naive (have no time‐
1564 zone).
1565 timezones must be a strategy that generates either None, for
1567 naive datetimes, or tzinfo objects for 'aware' datetimes. You
1568 can construct your own, though we recommend using the dateutil
1569 package and hypothesis.extra.dateutil.timezones() strategy, and
1570 also provide hypothesis.extra.pytz.timezones().
1571 You may pass allow_imaginary=False to filter out "imaginary"
1573 datetimes which did not (or will not) occur due to daylight sav‐
1574 ings, leap seconds, timezone and calendar adjustments, etc.
1575 Imaginary datetimes are allowed by default, because malformed
1576 timestamps are a common source of bugs. Note that because pytz
1577 predates PEP 495, this does not work correctly with timezones
1578 that use a negative DST offset (such as "Europe/Dublin").
1579 Examples from this strategy shrink towards midnight on January
1581 1st 2000, local time.
1582 hypothesis.strategies.decimals(min_value=None, max_value=None, *, al‐
1584 low_nan=None, allow_infinity=None, places=None)
1585 Generates instances of python:decimal.Decimal, which may be:
1586 • A finite rational number, between min_value and max_value.
1588 • Not a Number, if allow_nan is True. None means "allow NaN,
1590 unless min_value and max_value are not None".
1591 • Positive or negative infinity, if max_value and min_value re‐
1593 spectively are None, and allow_infinity is not False. None
1594 means "allow infinity, unless excluded by the min and max val‐
1595 ues".
1596 Note that where floats have one NaN value, Decimals have four:
1598 signed, and either quiet or signalling. See the decimal module
1599 docs for more information on special values.
1600 If places is not None, all finite values drawn from the strategy
1602 will have that number of digits after the decimal place.
1603 Examples from this strategy do not have a well defined shrink
1605 order but try to maximize human readability when shrinking.
1606 hypothesis.strategies.deferred(definition)
1608 A deferred strategy allows you to write a strategy that refer‐
1609 ences other strategies that have not yet been defined. This al‐
1610 lows for the easy definition of recursive and mutually recursive
1611 strategies.
1612 The definition argument should be a zero-argument function that
1614 returns a strategy. It will be evaluated the first time the
1615 strategy is used to produce an example.
1616 Example usage:
1618 >>> import hypothesis.strategies as st
1620 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1621 >>> x.example()
1622 (((False, (True, True)), (False, True)), (True, True))
1623 >>> x.example()
1624 True
1625 Mutual recursion also works fine:
1627 >>> a = st.deferred(lambda: st.booleans() | b)
1629 >>> b = st.deferred(lambda: st.tuples(a, a))
1630 >>> a.example()
1631 True
1632 >>> b.example()
1633 (False, (False, ((False, True), False)))
1634 Examples from this strategy shrink as they normally would from
1636 the strategy returned by the definition.
1637 hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class
1639 'dict'>, min_size=0, max_size=None)
1640 Generates dictionaries of type dict_class with keys drawn from
1641 the keys argument and values drawn from the values argument.
1642 The size parameters have the same interpretation as for lists().
1644 Examples from this strategy shrink by trying to remove keys from
1646 the generated dictionary, and by shrinking each generated key
1647 and value.
1648 hypothesis.strategies.emails()
1650 A strategy for generating email addresses as unicode strings.
1651 The address format is specified in RFC 5322#section-3.4.1. Val‐
1652 ues shrink towards shorter local-parts and host domains.
1653 This strategy is useful for generating "user data" for tests, as
1655 mishandling of email addresses is a common source of bugs.
1656 hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)
1658 Generates a dictionary of the same type as mapping with a fixed
1659 set of keys mapping to strategies. mapping must be a dict sub‐
1660 class.
1661 Generated values have all keys present in mapping, in iteration
1663 order, with the corresponding values drawn from mapping[key].
1664 If optional is passed, the generated value may or may not con‐
1666 tain each key from optional and a value drawn from the corre‐
1667 sponding strategy. Generated values may contain optional keys
1668 in an arbitrary order.
1669 Examples from this strategy shrink by shrinking each individual
1671 value in the generated dictionary, and omitting optional
1672 key-value pairs.
1673 hypothesis.strategies.floats(min_value=None, max_value=None, *, al‐
1675 low_nan=None, allow_infinity=None, width=64, exclude_min=False, ex‐
1676 clude_max=False)
1677 Returns a strategy which generates floats.
1678 • If min_value is not None, all values will be >= min_value (or
1680 > min_value if exclude_min).
1681 • If max_value is not None, all values will be <= max_value (or
1683 < max_value if exclude_max).
1684 • If min_value or max_value is not None, it is an error to en‐
1686 able allow_nan.
1687 • If both min_value and max_value are not None, it is an error
1689 to enable allow_infinity.
1690 Where not explicitly ruled out by the bounds, all of infinity,
1692 -infinity and NaN are possible values generated by this strat‐
1693 egy.
1694 The width argument specifies the maximum number of bits of pre‐
1696 cision required to represent the generated float. Valid values
1697 are 16, 32, or 64. Passing width=32 will still use the builtin
1698 64-bit float class, but always for values which can be exactly
1699 represented as a 32-bit float.
1700 The exclude_min and exclude_max argument can be used to generate
1702 numbers from open or half-open intervals, by excluding the re‐
1703 spective endpoints. Excluding either signed zero will also ex‐
1704 clude the other. Attempting to exclude an endpoint which is
1705 None will raise an error; use allow_infinity=False to generate
1706 finite floats. You can however use e.g. min_value=-math.inf,
1707 exclude_min=True to exclude only one infinite endpoint.
1708 Examples from this strategy have a complicated and hard to ex‐
1710 plain shrinking behaviour, but it tries to improve "human read‐
1711 ability". Finite numbers will be preferred to infinity and in‐
1712 finity will be preferred to NaN.
1713 hypothesis.strategies.fractions(min_value=None, max_value=None, *,
1715 max_denominator=None)
1716 Returns a strategy which generates Fractions.
1717 If min_value is not None then all generated values are no less
1719 than min_value. If max_value is not None then all generated
1720 values are no greater than max_value. min_value and max_value
1721 may be anything accepted by the Fraction constructor.
1722 If max_denominator is not None then the denominator of any gen‐
1724 erated values is no greater than max_denominator. Note that
1725 max_denominator must be None or a positive integer.
1726 Examples from this strategy shrink towards smaller denominators,
1728 then closer to zero.
1729 hypothesis.strategies.from_regex(regex, *, fullmatch=False)
1731 Generates strings that contain a match for the given regex (i.e.
1732 ones for which python:re.search() will return a non-None re‐
1733 sult).
1734 regex may be a pattern or compiled regex. Both byte-strings and
1736 unicode strings are supported, and will generate examples of the
1737 same type.
1738 You can use regex flags such as python:re.IGNORECASE or
1740 python:re.DOTALL to control generation. Flags can be passed ei‐
1741 ther in compiled regex or inside the pattern with a (?iLmsux)
1742 group.
1743 Some regular expressions are only partly supported - the under‐
1745 lying strategy checks local matching and relies on filtering to
1746 resolve context-dependent expressions. Using too many of these
1747 constructs may cause health-check errors as too many examples
1748 are filtered out. This mainly includes (positive or negative)
1749 lookahead and lookbehind groups.
1750 If you want the generated string to match the whole regex you
1752 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1753 gle character string, while "." will return any string, and
1754 r"\A.$" will return a single character optionally followed by a
1755 "\n". Alternatively, passing fullmatch=True will ensure that
1756 the whole string is a match, as if you had used the \A and \Z
1757 markers.
1758 Examples from this strategy shrink towards shorter strings and
1760 lower character values, with exact behaviour that may depend on
1761 the pattern.
1762 hypothesis.strategies.from_type(thing)
1764 Looks up the appropriate search strategy for the given type.
1765 from_type is used internally to fill in missing arguments to
1767 builds() and can be used interactively to explore what strate‐
1768 gies are available or to debug type resolution.
1769 You can use register_type_strategy() to handle your custom
1771 types, or to globally redefine certain strategies - for example
1772 excluding NaN from floats, or use timezone-aware instead of
1773 naive time and datetime strategies.
1774 The resolution logic may be changed in a future version, but
1776 currently tries these five options:
1777 1. If thing is in the default lookup mapping or user-registered
1779 lookup, return the corresponding strategy. The default
1780 lookup covers all types with Hypothesis strategies, including
1781 extras where possible.
1782 2. If thing is from the python:typing module, return the corre‐
1784 sponding strategy (special logic).
1785 3. If thing has one or more subtypes in the merged lookup, re‐
1787 turn the union of the strategies for those types that are not
1788 subtypes of other elements in the lookup.
1789 4. Finally, if thing has type annotations for all required argu‐
1791 ments, and is not an abstract class, it is resolved via
1792 builds().
1793 5. Because abstract types cannot be instantiated, we treat ab‐
1795 stract types as the union of their concrete subclasses. Note
1796 that this lookup works via inheritance but not via register,
1797 so you may still need to use register_type_strategy().
1798 There is a valuable recipe for leveraging from_type() to gener‐
1800 ate "everything except" values from a specified type. I.e.
1801 def everything_except(excluded_types):
1803 return (
1804 from_type(type)
1805 .flatmap(from_type)
1806 .filter(lambda x: not isinstance(x, excluded_types))
1807 )
1808 For example, everything_except(int) returns a strategy that can
1810 generate anything that from_type() can ever generate, except for
1811 instances of python:int, and excluding instances of types added
1812 via register_type_strategy().
1813 This is useful when writing tests which check that invalid input
1815 is rejected in a certain way.
1816 hypothesis.strategies.frozensets(elements, *, min_size=0,
1818 max_size=None)
1819 This is identical to the sets function but instead returns
1820 frozensets.
1821 hypothesis.strategies.functions(*, like=lambda : ..., returns=none(),
1823 pure=False)
1824 A strategy for functions, which can be used in callbacks.
1825 The generated functions will mimic the interface of like, which
1827 must be a callable (including a class, method, or function).
1828 The return value for the function is drawn from the returns ar‐
1829 gument, which must be a strategy.
1830 If pure=True, all arguments passed to the generated function
1832 must be hashable, and if passed identical arguments the original
1833 return value will be returned again - not regenerated, so beware
1834 mutable values.
1835 If pure=False, generated functions do not validate their argu‐
1837 ments, and may return a different value if called again with the
1838 same arguments.
1839 Generated functions can only be called within the scope of the
1841 @given which created them. This strategy does not support .ex‐
1842 ample().
1843 hypothesis.strategies.integers(min_value=None, max_value=None)
1845 Returns a strategy which generates integers.
1846 If min_value is not None then all values will be >= min_value.
1848 If max_value is not None then all values will be <= max_value
1849 Examples from this strategy will shrink towards zero, and nega‐
1851 tive values will also shrink towards positive (i.e. -n may be
1852 replaced by +n).
1853 hypothesis.strategies.ip_addresses(*, v=None, network=None)
1855 Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Ad‐
1856 dresses, or leave unspecified to allow both versions.
1857 network may be an IPv4Network or IPv6Network, or a string repre‐
1859 senting a network such as "127.0.0.0/24" or "2001:db8::/32". As
1860 well as generating addresses within a particular routable net‐
1861 work, this can be used to generate addresses from a reserved
1862 range listed in the IANA registries.
1863 If you pass both v and network, they must be for the same ver‐
1865 sion.
1866 hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None,
1868 unique_by=None, unique=False)
1869 This has the same behaviour as lists, but returns iterables in‐
1870 stead.
1871 Some iterables cannot be indexed (e.g. sets) and some do not
1873 have a fixed length (e.g. generators). This strategy produces
1874 iterators, which cannot be indexed and do not have a fixed
1875 length. This ensures that you do not accidentally depend on se‐
1876 quence behaviour.
1877 hypothesis.strategies.just(value)
1879 Return a strategy which only generates value.
1880 Note: value is not copied. Be wary of using mutable values.
1882 If value is the result of a callable, you can use
1884 builds(callable) instead of just(callable()) to get a fresh
1885 value each time.
1886 Examples from this strategy do not shrink (because there is only
1888 one).
1889 hypothesis.strategies.lists(elements, *, min_size=0, max_size=None,
1891 unique_by=None, unique=False)
1892 Returns a list containing values drawn from elements with length
1893 in the interval [min_size, max_size] (no bounds in that direc‐
1894 tion if these are None). If max_size is 0, only the empty list
1895 will be drawn.
1896 If unique is True (or something that evaluates to True), we com‐
1898 pare direct object equality, as if unique_by was lambda x: x.
1899 This comparison only works for hashable types.
1900 If unique_by is not None it must be a callable or tuple of
1902 callables returning a hashable type when given a value drawn
1903 from elements. The resulting list will satisfy the condition
1904 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1905 If unique_by is a tuple of callables the uniqueness will be re‐
1907 spective to each callable.
1908 For example, the following will produce two columns of integers
1910 with both columns being unique respectively.
1911 >>> twoints = st.tuples(st.integers(), st.integers())
1913 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1914 Examples from this strategy shrink by trying to remove elements
1916 from the list, and by shrinking each individual element of the
1917 list.
1918 hypothesis.strategies.none()
1920 Return a strategy which only generates None.
1921 Examples from this strategy do not shrink (because there is only
1923 one).
1924 hypothesis.strategies.nothing()
1926 This strategy never successfully draws a value and will always
1927 reject on an attempt to draw.
1928 Examples from this strategy do not shrink (because there are
1930 none).
1931 hypothesis.strategies.one_of(*args)
1933 Return a strategy which generates values from any of the argu‐
1934 ment strategies.
1935 This may be called with one iterable argument instead of multi‐
1937 ple strategy arguments, in which case one_of(x) and one_of(*x)
1938 are equivalent.
1939 Examples from this strategy will generally shrink to ones that
1941 come from strategies earlier in the list, then shrink according
1942 to behaviour of the strategy that produced them. In order to get
1943 good shrinking behaviour, try to put simpler strategies first.
1944 e.g. one_of(none(), text()) is better than one_of(text(),
1945 none()).
1946 This is especially important when using recursive strategies.
1948 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1949 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1950 st.none()) will shrink very badly indeed.
1951 hypothesis.strategies.permutations(values)
1953 Return a strategy which returns permutations of the ordered col‐
1954 lection values.
1955 Examples from this strategy shrink by trying to become closer to
1957 the original order of values.
1958 hypothesis.strategies.random_module()
1960 The Hypothesis engine handles PRNG state for the stdlib and
1961 Numpy random modules internally, always seeding them to zero and
1962 restoring the previous state after the test.
1963 If having a fixed seed would unacceptably weaken your tests, and
1965 you cannot use a random.Random instance provided by randoms(),
1966 this strategy calls python:random.seed() with an arbitrary inte‐
1967 ger and passes you an opaque object whose repr displays the seed
1968 value for debugging. If numpy.random is available, that state
1969 is also managed.
1970 Examples from these strategy shrink to seeds closer to zero.
1972 hypothesis.strategies.randoms(*, note_method_calls=False, use_true_ran‐
1974 dom=False)
1975 Generates instances of random.Random. The generated Random in‐
1976 stances are of a special HypothesisRandom subclass.
1977 • If note_method_calls is set to True, Hypothesis will print the
1979 randomly drawn values in any falsifying test case. This can be
1980 helpful for debugging the behaviour of randomized algorithms.
1981 • If use_true_random is set to True then values will be drawn
1983 from their usual distribution, otherwise they will actually be
1984 Hypothesis generated values (and will be shrunk accordingly
1985 for any failing test case). Setting use_true_random=False will
1986 tend to expose bugs that would occur with very low probability
1987 when it is set to True, and this flag should only be set to
1988 True when your code relies on the distribution of values for
1989 correctness.
1990 hypothesis.strategies.recursive(base, extend, *, max_leaves=100)
1992 base: A strategy to start from.
1993 extend: A function which takes a strategy and returns a new
1995 strategy.
1996 max_leaves: The maximum number of elements to be drawn from base
1998 on a given run.
1999 This returns a strategy S such that S = extend(base | S). That
2001 is, values may be drawn from base, or from any strategy reach‐
2002 able by mixing applications of | and extend.
2003 An example may clarify: recursive(booleans(), lists) would re‐
2005 turn a strategy that may return arbitrarily nested and mixed
2006 lists of booleans. So e.g. False, [True], [False, []], and
2007 [[[[True]]]] are all valid values to be drawn from that strat‐
2008 egy.
2009 Examples from this strategy shrink by trying to reduce the
2011 amount of recursion and by shrinking according to the shrinking
2012 behaviour of base and the result of extend.
2013 hypothesis.strategies.register_type_strategy(custom_type, strategy)
2015 Add an entry to the global type-to-strategy lookup.
2016 This lookup is used in builds() and @given.
2018 builds() will be used automatically for classes with type anno‐
2020 tations on __init__ , so you only need to register a strategy if
2021 one or more arguments need to be more tightly defined than their
2022 type-based default, or if you want to supply a strategy for an
2023 argument with a default value.
2024 strategy may be a search strategy, or a function that takes a
2026 type and returns a strategy (useful for generic types).
2027 Note that you may not register a parametrised generic type (such
2029 as MyCollection[int]) directly, because the resolution logic
2030 does not handle this case correctly. Instead, you may register
2031 a function for MyCollection and inspect the type parameters
2032 within that function.
2033 hypothesis.strategies.runner(*, default=not_set)
2035 A strategy for getting "the current test runner", whatever that
2036 may be. The exact meaning depends on the entry point, but it
2037 will usually be the associated 'self' value for it.
2038 If there is no current test runner and a default is provided,
2040 return that default. If no default is provided, raises Invali‐
2041 dArgument.
2042 Examples from this strategy do not shrink (because there is only
2044 one).
2045 hypothesis.strategies.sampled_from(elements)
2047 Returns a strategy which generates any value present in ele‐
2048 ments.
2049 Note that as with just(), values will not be copied and thus you
2051 should be careful of using mutable data.
2052 sampled_from supports ordered collections, as well as Enum ob‐
2054 jects. Flag objects may also generate any combination of their
2055 members.
2056 Examples from this strategy shrink by replacing them with values
2058 earlier in the list. So e.g. sampled_from([10, 1]) will shrink
2059 by trying to replace 1 values with 10, and sampled_from([1, 10])
2060 will shrink by trying to replace 10 values with 1.
2061 It is an error to sample from an empty sequence, because return‐
2063 ing nothing() makes it too easy to silently drop parts of com‐
2064 pound strategies. If you need that behaviour, use sam‐
2065 pled_from(seq) if seq else nothing().
2066 hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)
2068 This has the same behaviour as lists, but returns sets instead.
2069 Note that Hypothesis cannot tell if values are drawn from ele‐
2071 ments are hashable until running the test, so you can define a
2072 strategy for sets of an unhashable type but it will fail at test
2073 time.
2074 Examples from this strategy shrink by trying to remove elements
2076 from the set, and by shrinking each individual element of the
2077 set.
2078 hypothesis.strategies.shared(base, *, key=None)
2080 Returns a strategy that draws a single shared value per run,
2081 drawn from base. Any two shared instances with the same key will
2082 share the same value, otherwise the identity of this strategy
2083 will be used. That is:
2084 >>> s = integers() # or any other strategy
2086 >>> x = shared(s)
2087 >>> y = shared(s)
2088 In the above x and y may draw different (or potentially the
2090 same) values. In the following they will always draw the same:
2091 >>> x = shared(s, key="hi")
2093 >>> y = shared(s, key="hi")
2094 Examples from this strategy shrink as per their base strategy.
2096 hypothesis.strategies.slices(size)
2098 Generates slices that will select indices up to the supplied
2099 size
2100 Generated slices will have start and stop indices that range
2102 from -size to size - 1 and will step in the appropriate direc‐
2103 tion. Slices should only produce an empty selection if the start
2104 and end are the same.
2105 Examples from this strategy shrink toward 0 and smaller values
2107 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
2109 gories=('Cs')), *, min_size=0, max_size=None)
2110 Generates strings with characters drawn from alphabet, which
2111 should be a collection of length one strings or a strategy gen‐
2112 erating such strings.
2113 The default alphabet strategy can generate the full unicode
2115 range but excludes surrogate characters because they are invalid
2116 in the UTF-8 encoding. You can use characters() without argu‐
2117 ments to find surrogate-related bugs such as bpo-34454.
2118 min_size and max_size have the usual interpretations. Note that
2120 Python measures string length by counting codepoints: U+00C5 Å
2121 is a single character, while U+0041 U+030A Å is two - the A,
2122 and a combining ring above.
2123 Examples from this strategy shrink towards shorter strings, and
2125 with the characters in the text shrinking as per the alphabet
2126 strategy. This strategy does not normalize() examples, so gen‐
2127 erated strings may be in any or none of the 'normal forms'.
2128 hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min,
2130 max_value=datetime.timedelta.max)
2131 A strategy for timedeltas between min_value and max_value.
2132 Examples from this strategy shrink towards zero.
2134 hypothesis.strategies.times(min_value=datetime.time.min,
2136 max_value=datetime.time.max, *, timezones=none())
2137 A strategy for times between min_value and max_value.
2138 The timezones argument is handled as for datetimes().
2140 Examples from this strategy shrink towards midnight, with the
2142 timezone component shrinking as for the strategy that provided
2143 it.
2144 hypothesis.strategies.timezone_keys(*, allow_prefix=True)
2146 A strategy for IANA timezone names.
2147 As well as timezone names like "UTC", "Australia/Sydney", or
2149 "America/New_York", this strategy can generate:
2150 • Aliases such as "Antarctica/McMurdo", which links to "Pa‐
2152 cific/Auckland".
2153 • Deprecated names such as "Antarctica/South_Pole", which also
2155 links to "Pacific/Auckland". Note that most but not all dep‐
2156 recated timezone names are also aliases.
2157 • Timezone names with the "posix/" or "right/" prefixes, unless
2159 allow_prefix=False.
2160 These strings are provided separately from Tzinfo objects - such
2162 as ZoneInfo instances from the timezones() strategy - to facili‐
2163 tate testing of timezone logic without needing workarounds to
2164 access non-canonical names.
2165 NOTE:
2167 The python:zoneinfo module is new in Python 3.9, so you will
2168 need to install the backports.zoneinfo module on earlier ver‐
2169 sions, and the importlib_resources backport on Python 3.6.
2170 pip install hypothesis[zoneinfo] will install these condi‐
2172 tional dependencies if and only if they are needed.
2173 On Windows, you may need to access IANA timezone data via the
2175 tzdata package. For non-IANA timezones, such as Windows-native
2176 names or GNU TZ strings, we recommend using sampled_from() with
2177 the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().
2178 hypothesis.strategies.timezones(*, no_cache=False)
2180 A strategy for python:zoneinfo.ZoneInfo objects.
2181 If no_cache=True, the generated instances are constructed using
2183 ZoneInfo.no_cache instead of the usual constructor. This may
2184 change the semantics of your datetimes in surprising ways, so
2185 only use it if you know that you need to!
2186 NOTE:
2188 The python:zoneinfo module is new in Python 3.9, so you will
2189 need to install the backports.zoneinfo module on earlier ver‐
2190 sions, and the importlib_resources backport on Python 3.6.
2191 pip install hypothesis[zoneinfo] will install these condi‐
2193 tional dependencies if and only if they are needed.
2194 hypothesis.strategies.tuples(*args)
2196 Return a strategy which generates a tuple of the same length as
2197 args by generating the value at index i from args[i].
2198 e.g. tuples(integers(), integers()) would generate a tuple of
2200 length two with both values an integer.
2201 Examples from this strategy shrink by shrinking their component
2203 parts.
2204 hypothesis.strategies.uuids(*, version=None)
2206 Returns a strategy that generates UUIDs.
2207 If the optional version argument is given, value is passed
2209 through to UUID and only UUIDs of that version will be gener‐
2210 ated.
2211 All returned values from this will be unique, so e.g. if you do
2213 lists(uuids()) the resulting list will never contain duplicates.
2214 Examples from this strategy don't have any meaningful shrink or‐
2216 der.
2217 Provisional strategies
2219 This module contains various provisional APIs and strategies.
2220 It is intended for internal use, to ease code reuse, and is not stable.
2222 Point releases may move or break the contents at any time!
2223 Internet strategies should conform to RFC 3986 or the authoritative
2225 definitions it links to. If not, report the bug!
2226 hypothesis.provisional.domains(*, max_length=255, max_ele‐
2228 ment_length=63)
2229 Generate RFC 1035 compliant fully qualified domain names.
2230 hypothesis.provisional.urls()
2232 A strategy for RFC 3986, generating http/https URLs.
2233 Shrinking
2235 When using strategies it is worth thinking about how the data shrinks.
2236 Shrinking is the process by which Hypothesis tries to produce human
2237 readable examples when it finds a failure - it takes a complex example
2238 and turns it into a simpler one.
2239 Each strategy defines an order in which it shrinks - you won't usually
2241 need to care about this much, but it can be worth being aware of as it
2242 can affect what the best way to write your own strategies is.
2243 The exact shrinking behaviour is not a guaranteed part of the API, but
2245 it doesn't change that often and when it does it's usually because we
2246 think the new way produces nicer examples.
2247 Possibly the most important one to be aware of is one_of(), which has a
2249 preference for values produced by strategies earlier in its argument
2250 list. Most of the others should largely "do the right thing" without
2251 you having to think about it.
2252 Adapting strategies
2254 Often it is the case that a strategy doesn't produce exactly what you
2255 want it to and you need to adapt it. Sometimes you can do this in the
2256 test, but this hurts reuse because you then have to repeat the adaption
2257 in every test.
2258 Hypothesis gives you ways to build strategies from other strategies
2260 given functions for transforming the data.
2261 Mapping
2263 map is probably the easiest and most useful of these to use. If you
2264 have a strategy s and a function f, then an example s.map(f).example()
2265 is f(s.example()), i.e. we draw an example from s and then apply f to
2266 it.
2267 e.g.:
2269 >>> lists(integers()).map(sorted).example()
2271 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2272 Note that many things that you might use mapping for can also be done
2274 with builds(), and if you find yourself indexing into a tuple within
2275 .map() it's probably time to use that instead.
2276 Filtering
2278 filter lets you reject some examples. s.filter(f).example() is some ex‐
2279 ample of s such that f(example) is truthy.
2280 >>> integers().filter(lambda x: x > 11).example()
2282 26126
2283 >>> integers().filter(lambda x: x > 11).example()
2284 23324
2285 It's important to note that filter isn't magic and if your condition is
2287 too hard to satisfy then this can fail:
2288 >>> integers().filter(lambda x: False).example()
2290 Traceback (most recent call last):
2291 ...
2292 hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries
2293 In general you should try to use filter only to avoid corner cases that
2295 you don't want rather than attempting to cut out a large chunk of the
2296 search space.
2297 A technique that often works well here is to use map to first transform
2299 the data and then use filter to remove things that didn't work out. So
2300 for example if you wanted pairs of integers (x,y) such that x < y you
2301 could do the following:
2302 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2304 [-8543729478746591815, 3760495307320535691]
2305 Chaining strategies together
2307 Finally there is flatmap. flatmap draws an example, then turns that ex‐
2308 ample into a strategy, then draws an example from that strategy.
2309 It may not be obvious why you want this at first, but it turns out to
2311 be quite useful because it lets you generate different types of data
2312 with relationships to each other.
2313 For example suppose we wanted to generate a list of lists of the same
2315 length:
2316 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2318 ... lambda n: lists(lists(integers(), min_size=n, max_size=n))
2319 ... )
2320 >>> rectangle_lists.example()
2321 []
2322 >>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
2323 [[], [], [], [], [], [], [], [], [], []]
2324 >>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
2325 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2326 >>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
2327 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2328 In this example we first choose a length for our tuples, then we build
2330 a strategy which generates lists containing lists precisely of that
2331 length. The finds show what simple examples for this look like.
2332 Most of the time you probably don't want flatmap, but unlike filter and
2334 map which are just conveniences for things you could just do in your
2335 tests, flatmap allows genuinely new data generation that you wouldn't
2336 otherwise be able to easily do.
2337 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2339 don't know Haskell, ignore everything in these parentheses. You do not
2340 need to understand anything about monads to use this, or anything else
2341 in Hypothesis).
2342 Recursive data
2344 Sometimes the data you want to generate has a recursive definition.
2345 e.g. if you wanted to generate JSON data, valid JSON is:
2346 1. Any float, any boolean, any unicode string.
2348 2. Any list of valid JSON data
2350 3. Any dictionary mapping unicode strings to valid JSON data.
2352 The problem is that you cannot call a strategy recursively and expect
2354 it to not just blow up and eat all your memory. The other problem here
2355 is that not all unicode strings display consistently on different ma‐
2356 chines, so we'll restrict them in our doctest.
2357 The way Hypothesis handles this is with the recursive() strategy which
2359 you pass in a base case and a function that, given a strategy for your
2360 data type, returns a new strategy for it. So for example:
2361 >>> from string import printable
2363 ... from pprint import pprint
2364 >>> json = recursive(
2365 ... none() | booleans() | floats() | text(printable),
2366 ... lambda children: lists(children, 1)
2367 ... | dictionaries(text(printable), children, min_size=1),
2368 ... )
2369 >>> pprint(json.example())
2370 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2371 >>> pprint(json.example())
2372 {'de(l': None,
2373 'nK': {'(Rt)': None,
2374 '+hoZh1YU]gy8': True,
2375 '8z]EIFA06^li^': 'LFE{Q',
2376 '9,': 'l{cA=/'}}
2377 That is, we start with our leaf data and then we augment it by allowing
2379 lists and dictionaries of anything we can generate as JSON data.
2380 The size control of this works by limiting the maximum number of values
2382 that can be drawn from the base strategy. So for example if we wanted
2383 to only generate really small JSON we could do this as:
2384 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2386 >>> small_lists.example()
2387 True
2388 >>> small_lists.example()
2389 [False]
2390 Composite strategies
2392 The @composite decorator lets you combine other strategies in more or
2393 less arbitrary ways. It's probably the main thing you'll want to use
2394 for complicated custom strategies.
2395 The composite decorator works by converting a function that returns one
2397 example into a function that returns a strategy that produces such ex‐
2398 amples - which you can pass to @given, modify with .map or .filter, and
2399 generally use like any other strategy.
2400 It does this by giving you a special function draw as the first argu‐
2402 ment, which can be used just like the corresponding method of the
2403 data() strategy within a test. In fact, the implementation is almost
2404 the same - but defining a strategy with @composite makes code reuse
2405 easier, and usually improves the display of failing examples.
2406 For example, the following gives you a list and an index into it:
2408 >>> @composite
2410 ... def list_and_index(draw, elements=integers()):
2411 ... xs = draw(lists(elements, min_size=1))
2412 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2413 ... return (xs, i)
2414 ...
2415 draw(s) is a function that should be thought of as returning s.exam‐
2417 ple(), except that the result is reproducible and will minimize cor‐
2418 rectly. The decorated function has the initial argument removed from
2419 the list, but will accept all the others in the expected order. De‐
2420 faults are preserved.
2421 >>> list_and_index()
2423 list_and_index()
2424 >>> list_and_index().example()
2425 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2426 >>> list_and_index(booleans())
2428 list_and_index(elements=booleans())
2429 >>> list_and_index(booleans()).example()
2430 ([True, False], 0)
2431 Note that the repr will work exactly like it does for all the built-in
2433 strategies: it will be a function that you can call to get the strategy
2434 in question, with values provided only if they do not match the de‐
2435 faults.
2436 You can use assume inside composite functions:
2438 @composite
2440 def distinct_strings_with_common_characters(draw):
2441 x = draw(text(min_size=1))
2442 y = draw(text(alphabet=x))
2443 assume(x != y)
2444 return (x, y)
2445 This works as assume normally would, filtering out any examples for
2447 which the passed in argument is falsey.
2448 Take care that your function can cope with adversarial draws, or ex‐
2450 plicitly rejects them using the .filter() method or assume() - our mu‐
2451 tation and shrinking logic can do some strange things, and a naive im‐
2452 plementation might lead to serious performance problems. For example:
2453 @composite
2455 def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
2456 # The bad way: if Hypothesis keeps generating e.g. zero,
2457 # we'll keep looping for a very long time.
2458 result = set()
2459 while len(result) < size:
2460 result.add(draw(elements))
2461 # The good way: use a filter, so Hypothesis can tell what's valid!
2462 for _ in range(size):
2463 result.add(draw(elements.filter(lambda x: x not in result)))
2464 return result
2465 If @composite is used to decorate a method or classmethod, the draw ar‐
2467 gument must come before self or cls. While we therefore recommend
2468 writing strategies as standalone functions and using the
2469 register_type_strategy() function to associate them with a class, meth‐
2470 ods are supported and the @composite decorator may be applied either
2471 before or after @classmethod or @staticmethod. See issue #2578 and
2472 pull request #2634 for more details.
2473 Drawing interactively in tests
2475 There is also the data() strategy, which gives you a means of using
2476 strategies interactively. Rather than having to specify everything up
2477 front in @given you can draw from strategies in the body of your test.
2478 This is similar to @composite, but even more powerful as it allows you
2480 to mix test code with example generation. The downside of this power
2481 is that data() is incompatible with explicit @example(...)s - and the
2482 mixed code is often harder to debug when something goes wrong.
2483 If you need values that are affected by previous draws but which don't
2485 depend on the execution of your test, stick to the simpler @composite.
2486 @given(data())
2488 def test_draw_sequentially(data):
2489 x = data.draw(integers())
2490 y = data.draw(integers(min_value=x))
2491 assert x < y
2492 If the test fails, each draw will be printed with the falsifying exam‐
2494 ple. e.g. the above is wrong (it has a boundary condition error), so
2495 will print:
2496 Falsifying example: test_draw_sequentially(data=data(...))
2498 Draw 1: 0
2499 Draw 2: 0
2500 As you can see, data drawn this way is simplified as usual.
2502 Optionally, you can provide a label to identify values generated by
2504 each call to data.draw(). These labels can be used to identify values
2505 in the output of a falsifying example.
2506 For instance:
2508 @given(data())
2510 def test_draw_sequentially(data):
2511 x = data.draw(integers(), label="First number")
2512 y = data.draw(integers(min_value=x), label="Second number")
2513 assert x < y
2514 will produce the output:
2516 Falsifying example: test_draw_sequentially(data=data(...))
2518 Draw 1 (First number): 0
2519 Draw 2 (Second number): 0
2520
2522 Hypothesis has minimal dependencies, to maximise compatibility and make
2523 installing Hypothesis as easy as possible.
2524 Our integrations with specific packages are therefore provided by extra
2526 modules that need their individual dependencies installed in order to
2527 work. You can install these dependencies using the setuptools extra
2528 feature as e.g. pip install hypothesis[django]. This will check in‐
2529 stallation of compatible versions.
2530 You can also just install hypothesis into a project using them, ignore
2532 the version constraints, and hope for the best.
2533 In general "Which version is Hypothesis compatible with?" is a hard
2535 question to answer and even harder to regularly test. Hypothesis is al‐
2536 ways tested against the latest compatible version and each package will
2537 note the expected compatibility range. If you run into a bug with any
2538 of these please specify the dependency version.
2539 There are separate pages for django and numpy.
2541 hypothesis[cli]
2543 $ hypothesis --help
2544 Usage: hypothesis [OPTIONS] COMMAND [ARGS]...
2545 Options:
2547 --version Show the version and exit.
2548 -h, --help Show this message and exit.
2549 Commands:
2551 codemod `hypothesis codemod` refactors deprecated or inefficient code.
2552 fuzz [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
2553 write `hypothesis write` writes property-based tests for you!
2554 This module requires the click package, and provides Hypothesis' com‐
2556 mand-line interface, for e.g. 'ghostwriting' tests via the terminal.
2557 It's also where HypoFuzz adds the hypothesis fuzz command (learn more
2558 about that here).
2559 hypothesis[lark]
2561 This extra can be used to generate strings matching any context-free
2562 grammar, using the Lark parser library.
2563 It currently only supports Lark's native EBNF syntax, but we plan to
2565 extend this to support other common syntaxes such as ANTLR and RFC 5234
2566 ABNF. Lark already supports loading grammars from nearley.js, so you
2567 may not have to write your own at all.
2568 Note that as Lark is at version 0.x, this module may break API compati‐
2570 bility in minor releases if supporting the latest version of Lark would
2571 otherwise be infeasible. We may also be quite aggressive in bumping
2572 the minimum version of Lark, unless someone volunteers to either fund
2573 or do the maintenance.
2574 hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None)
2576 A strategy for strings accepted by the given context-free gram‐
2577 mar.
2578 grammar must be a Lark object, which wraps an EBNF specifica‐
2580 tion. The Lark EBNF grammar reference can be found here.
2581 from_lark will automatically generate strings matching the non‐
2583 terminal start symbol in the grammar, which was supplied as an
2584 argument to the Lark class. To generate strings matching a dif‐
2585 ferent symbol, including terminals, you can override this by
2586 passing the start argument to from_lark. Note that Lark may re‐
2587 move unreachable productions when the grammar is compiled, so
2588 you should probably pass the same value for start to both.
2589 Currently from_lark does not support grammars that need custom
2591 lexing. Any lexers will be ignored, and any undefined terminals
2592 from the use of %declare will result in generation errors. To
2593 define strategies for such terminals, pass a dictionary mapping
2594 their name to a corresponding strategy as the explicit argument.
2595 The hypothesmith project includes a strategy for Python source,
2597 based on a grammar and careful post-processing.
2598 Example grammars, which may provide a useful starting point for your
2600 tests, can be found in the Lark repository and in this third-party col‐
2601 lection.
2602 hypothesis[pytz]
2604 This module provides pytz timezones.
2605 You can use this strategy to make hypothesis.strategies.datetimes() and
2607 hypothesis.strategies.times() produce timezone-aware values.
2608 hypothesis.extra.pytz.timezones()
2610 Any timezone in the Olsen database, as a pytz tzinfo object.
2611 This strategy minimises to UTC, or the smallest possible fixed
2613 offset, and is designed for use with hypothesis.strategies.date‐
2614 times().
2615 hypothesis[dateutil]
2617 This module provides dateutil timezones.
2618 You can use this strategy to make datetimes() and times() produce time‐
2620 zone-aware values.
2621 hypothesis.extra.dateutil.timezones()
2623 Any timezone from dateutil.
2624 This strategy minimises to UTC, or the timezone with the small‐
2626 est offset from UTC as of 2000-01-01, and is designed for use
2627 with datetimes().
2628 Note that the timezones generated by the strategy may vary de‐
2630 pending on the configuration of your machine. See the dateutil
2631 documentation for more information.
2632
2634 Writing tests with Hypothesis frees you from the tedium of deciding on
2635 and writing out specific inputs to test. Now, the hypothesis.ex‐
2636 tra.ghostwriter module can write your test functions for you too!
2637 The idea is to provide an easy way to start property-based testing, and
2639 a seamless transition to more complex test code - because ghostwritten
2640 tests are source code that you could have written for yourself.
2641 So just pick a function you'd like tested, and feed it to one of the
2643 functions below. They follow imports, use but do not require type an‐
2644 notations, and generally do their best to write you a useful test. You
2645 can also use our command-line interface:
2646 $ hypothesis write --help
2648 Usage: hypothesis write [OPTIONS] FUNC...
2649 `hypothesis write` writes property-based tests for you!
2651 Type annotations are helpful but not required for our advanced
2653 introspection and templating logic. Try running the examples below to see
2654 how it works:
2655 hypothesis write gzip
2657 hypothesis write numpy.matmul
2658 hypothesis write re.compile --except re.error
2659 hypothesis write --equivalent ast.literal_eval eval
2660 hypothesis write --roundtrip json.dumps json.loads
2661 hypothesis write --style=unittest --idempotent sorted
2662 hypothesis write --binary-op operator.add
2663 Options:
2665 --roundtrip start by testing write/read or encode/decode!
2666 --equivalent very useful when optimising or refactoring code
2667 --idempotent
2668 --binary-op
2669 --style [pytest|unittest] pytest-style function, or unittest-style method?
2670 -e, --except OBJ_NAME dotted name of exception(s) to ignore
2671 -h, --help Show this message and exit.
2672 NOTE:
2674 The ghostwriter requires black, but the generated code only requires
2675 Hypothesis itself.
2676 NOTE:
2678 Legal questions? While the ghostwriter fragments and logic is under
2679 the MPL-2.0 license like the rest of Hypothesis, the output from the
2680 ghostwriter is made available under the Creative Commons Zero (CC0)
2681 public domain dedication, so you can use it without any restric‐
2682 tions.
2683 hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(),
2685 style='pytest')
2686 Guess which ghostwriters to use, for a module or collection of
2687 functions.
2688 As for all ghostwriters, the except_ argument should be an
2690 python:Exception or tuple of exceptions, and style may be either
2691 "pytest" to write test functions or "unittest" to write test
2692 methods and TestCase.
2693 After finding the public functions attached to any modules, the
2695 magic ghostwriter looks for pairs of functions to pass to
2696 roundtrip(), then checks for binary_operation() and ufunc()
2697 functions, and any others are passed to fuzz().
2698 For example, try hypothesis write gzip on the command line!
2700 hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest')
2702 Write source code for a property-based test of func.
2703 The resulting test checks that valid input only leads to ex‐
2705 pected exceptions. For example:
2706 from re import compile, error
2708 from hypothesis.extra import ghostwriter
2710 ghostwriter.fuzz(compile, except_=error)
2712 Gives:
2714 # This test code was written by the `hypothesis.extra.ghostwriter` module
2716 # and is provided under the Creative Commons Zero public domain dedication.
2717 import re
2718 from hypothesis import given, reject, strategies as st
2720 # TODO: replace st.nothing() with an appropriate strategy
2722 @given(pattern=st.nothing(), flags=st.just(0))
2725 def test_fuzz_compile(pattern, flags):
2726 try:
2727 re.compile(pattern=pattern, flags=flags)
2728 except re.error:
2729 reject()
2730 Note that it includes all the required imports. Because the
2732 pattern parameter doesn't have annotations or a default argu‐
2733 ment, you'll need to specify a strategy - for example text() or
2734 binary(). After that, you have a test!
2735 hypothesis.extra.ghostwriter.idempotent(func, *, except_=(),
2737 style='pytest')
2738 Write source code for a property-based test of func.
2739 The resulting test checks that if you call func on it's own out‐
2741 put, the result does not change. For example:
2742 from typing import Sequence
2744 from hypothesis.extra import ghostwriter
2746 def timsort(seq: Sequence[int]) -> Sequence[int]:
2749 return sorted(seq)
2750 ghostwriter.idempotent(timsort)
2753 Gives:
2755 # This test code was written by the `hypothesis.extra.ghostwriter` module
2757 # and is provided under the Creative Commons Zero public domain dedication.
2758 from hypothesis import given, strategies as st
2760 @given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
2763 def test_idempotent_timsort(seq):
2764 result = timsort(seq=seq)
2765 repeat = timsort(seq=result)
2766 assert result == repeat, (result, repeat)
2767 hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(),
2769 style='pytest')
2770 Write source code for a property-based test of funcs.
2771 The resulting test checks that if you call the first function,
2773 pass the result to the second (and so on), the final result is
2774 equal to the first input argument.
2775 This is a very powerful property to test, especially when the
2777 config options are varied along with the object to round-trip.
2778 For example, try ghostwriting a test for python:json.dumps() -
2779 would you have thought of all that?
2780 hypothesis write --roundtrip json.dumps json.loads
2782 hypothesis.extra.ghostwriter.equivalent(*funcs, except_=(),
2784 style='pytest')
2785 Write source code for a property-based test of funcs.
2786 The resulting test checks that calling each of the functions
2788 gives the same result. This can be used as a classic 'oracle',
2789 such as testing a fast sorting algorithm against the
2790 python:sorted() builtin, or for differential testing where none
2791 of the compared functions are fully trusted but any difference
2792 indicates a bug (e.g. running a function on different numbers of
2793 threads, or simply multiple times).
2794 The functions should have reasonably similar signatures, as only
2796 the common parameters will be passed the same arguments - any
2797 other parameters will be allowed to vary.
2798 hypothesis.extra.ghostwriter.binary_operation(func, *, associa‐
2800 tive=True, commutative=True, identity=infer, distributes_over=None, ex‐
2801 cept_=(), style='pytest')
2802 Write property tests for the binary operation func.
2803 While binary operations are not particularly common, they have
2805 such nice properties to test that it seems a shame not to demon‐
2806 strate them with a ghostwriter. For an operator f, test that:
2807 • if associative, f(a, f(b, c)) == f(f(a, b), c)
2809 • if commutative, f(a, b) == f(b, a)
2811 • if identity is not None, f(a, identity) == a
2813 • if distributes_over is +, f(a, b) + f(a, c) == f(a, b+c)
2815 For example:
2817 ghostwriter.binary_operation(
2819 operator.mul,
2820 identity=1,
2821 distributes_over=operator.add,
2822 style="unittest",
2823 )
2824 hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest')
2826 Write a property-based test for the array unfunc func.
2827 The resulting test checks that your ufunc or gufunc has the ex‐
2829 pected broadcasting and dtype casting behaviour. You will prob‐
2830 ably want to add extra assertions, but as with the other ghost‐
2831 writers this gives you a great place to start.
2832 hypothesis write numpy.matmul
2834
2836 Hypothesis offers a number of features specific for Django testing,
2837 available in the hypothesis[django] extra. This is tested against each
2838 supported series with mainstream or extended support - if you're still
2839 getting security patches, you can test with Hypothesis.
2840 class hypothesis.extra.django.TestCase
2842 Using it is quite straightforward: All you need to do is subclass
2844 hypothesis.extra.django.TestCase or
2845 hypothesis.extra.django.TransactionTestCase and you can use @given as
2846 normal, and the transactions will be per example rather than per test
2847 function as they would be if you used @given with a normal django test
2848 suite (this is important because your test function will be called mul‐
2849 tiple times and you don't want them to interfere with each other). Test
2850 cases on these classes that do not use @given will be run as normal.
2851 class hypothesis.extra.django.TransactionTestCase
2853 We recommend avoiding TransactionTestCase unless you really have to run
2855 each test case in a database transaction. Because Hypothesis runs this
2856 in a loop, the performance problems it normally has are significantly
2857 exacerbated and your tests will be really slow. If you are using
2858 TransactionTestCase, you may need to use @settings(sup‐
2859 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2860 example generation.
2861 Having set up a test class, you can now pass @given a strategy for
2863 Django models:
2864 hypothesis.extra.django.from_model(model, /, **field_strategies)
2866 Return a strategy for examples of model.
2867 WARNING:
2869 Hypothesis creates saved models. This will run inside your
2870 testing transaction when using the test runner, but if you
2871 use the dev console this will leave debris in your database.
2872 model must be an subclass of Model. Strategies for fields may
2874 be passed as keyword arguments, for example
2875 is_staff=st.just(False). In order to support models with fields
2876 named "model", this is a positional-only parameter.
2877 Hypothesis can often infer a strategy based the field type and
2879 validators, and will attempt to do so for any required fields.
2880 No strategy will be inferred for an AutoField, nullable field,
2881 foreign key, or field for which a keyword argument is passed to
2882 from_model(). For example, a Shop type with a foreign key to
2883 Company could be generated with:
2884 shop_strategy = from_model(Shop, company=from_model(Company))
2886 Like for builds(), you can pass infer as a keyword argument to
2888 infer a strategy for a field which has a default value instead
2889 of using the default.
2890 For example, using the trivial django project we have for testing:
2892 >>> from hypothesis.extra.django import from_model
2894 >>> from toystore.models import Customer
2895 >>> c = from_model(Customer).example()
2896 >>> c
2897 <Customer: Customer object>
2898 >>> c.email
2899 'jaime.urbina@gmail.com'
2900 >>> c.name
2901 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2902 >>> c.age
2903 -873375803
2904 Hypothesis has just created this with whatever the relevant type of
2906 data is.
2907 Obviously the customer's age is implausible, which is only possible be‐
2909 cause we have not used (eg) MinValueValidator to set the valid range
2910 for this field (or used a PositiveSmallIntegerField, which would only
2911 need a maximum value validator).
2912 If you do have validators attached, Hypothesis will only generate exam‐
2914 ples that pass validation. Sometimes that will mean that we fail a
2915 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2916 egy to skip validation at the strategy level:
2917 NOTE:
2919 Inference from validators will be much more powerful when issue
2920 #1116 is implemented, but there will always be some edge cases that
2921 require you to pass an explicit strategy.
2922 >>> from hypothesis.strategies import integers
2924 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2925 >>> c
2926 <Customer: Customer object>
2927 >>> c.age
2928 5
2929 hypothesis.extra.django.from_form(form, form_kwargs=None,
2931 **field_strategies)
2932 Return a strategy for examples of form.
2933 form must be an subclass of Form. Strategies for fields may be
2935 passed as keyword arguments, for example
2936 is_staff=st.just(False).
2937 Hypothesis can often infer a strategy based the field type and
2939 validators, and will attempt to do so for any required fields.
2940 No strategy will be inferred for a disabled field or field for
2941 which a keyword argument is passed to from_form().
2942 This function uses the fields of an unbound form instance to de‐
2944 termine field strategies, any keyword arguments needed to in‐
2945 stantiate the unbound form instance can be passed into
2946 from_form() as a dict with the keyword form_kwargs. E.g.:
2947 shop_strategy = from_form(Shop, form_kwargs={"company_id": 5})
2949 Like for builds(), you can pass infer as a keyword argument to
2951 infer a strategy for a field which has a default value instead
2952 of using the default.
2953 Tips and tricks
2955 Custom field types
2956 If you have a custom Django field type you can register it with Hypoth‐
2957 esis's model deriving functionality by registering a default strategy
2958 for it:
2959 >>> from toystore.models import CustomishField, Customish
2961 >>> from_model(Customish).example()
2962 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2963 customish for model Customish
2964 >>> from hypothesis.extra.django import register_field_strategy
2965 >>> from hypothesis.strategies import just
2966 >>> register_field_strategy(CustomishField, just("hi"))
2967 >>> x = from_model(Customish).example()
2968 >>> x.customish
2969 'hi'
2970 Note that this mapping is on exact type. Subtypes will not inherit it.
2972 hypothesis.extra.django.register_field_strategy(field_type, strategy)
2974 Add an entry to the global field-to-strategy lookup used by
2975 from_field().
2976 field_type must be a subtype of django.db.models.Field or
2978 django.forms.Field, which must not already be registered.
2979 strategy must be a SearchStrategy.
2980 hypothesis.extra.django.from_field(field)
2982 Return a strategy for values that fit the given field.
2983 This function is used by from_form() and from_model() for any
2985 fields that require a value, or for which you passed hypothe‐
2986 sis.infer.
2987 It's pretty similar to the core from_type() function, with a
2989 subtle but important difference: from_field takes a Field in‐
2990 stance, rather than a Field subtype, so that it has access to
2991 instance attributes such as string length and validators.
2992 Generating child models
2994 For the moment there's no explicit support in hypothesis-django for
2995 generating dependent models. i.e. a Company model will generate no
2996 Shops. However if you want to generate some dependent models as well,
2997 you can emulate this by using the flatmap function as follows:
2998 from hypothesis.strategies import just, lists
3000 def generate_with_shops(company):
3003 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
3004 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
3007 Let's unpack what this is doing:
3009 The way flatmap works is that we draw a value from the original strat‐
3011 egy, then apply a function to it which gives us a new strategy. We then
3012 draw a value from that strategy. So in this case we're first drawing a
3013 company, and then we're drawing a list of shops belonging to that com‐
3014 pany: The just strategy is a strategy such that drawing it always pro‐
3015 duces the individual value, so from_model(Shop, company=just(company))
3016 is a strategy that generates a Shop belonging to the original company.
3017 So the following code would give us a list of shops all belonging to
3019 the same company:
3020 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
3022 The only difference from this and the above is that we want the com‐
3024 pany, not the shops. This is where the inner map comes in. We build the
3025 list of shops and then throw it away, instead returning the company we
3026 started for. This works because the models that Hypothesis generates
3027 are saved in the database, so we're essentially running the inner
3028 strategy purely for the side effect of creating those children in the
3029 database.
3030 Generating primary key values
3032 If your model includes a custom primary key that you want to generate
3033 using a strategy (rather than a default auto-increment primary key)
3034 then Hypothesis has to deal with the possibility of a duplicate primary
3035 key.
3036 If a model strategy generates a value for the primary key field, Hy‐
3038 pothesis will create the model instance with update_or_create(), over‐
3039 writing any existing instance in the database for this test case with
3040 the same primary key.
3041 On the subject of MultiValueField
3043 Django forms feature the MultiValueField which allows for several
3044 fields to be combined under a single named field, the default example
3045 of this is the SplitDateTimeField.
3046 class CustomerForm(forms.Form):
3048 name = forms.CharField()
3049 birth_date_time = forms.SplitDateTimeField()
3050 from_form supports MultiValueField subclasses directly, however if you
3052 want to define your own strategy be forewarned that Django binds data
3053 for a MultiValueField in a peculiar way. Specifically each sub-field is
3054 expected to have its own entry in data addressed by the field name
3055 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
3056 ValueField, so form data for the example above might look like this:
3057 {
3059 "name": "Samuel John",
3060 "birth_date_time_0": "2018-05-19", # the date, as the first sub-field
3061 "birth_date_time_1": "15:18:00", # the time, as the second sub-field
3062 }
3063 Thus, if you want to define your own strategies for such a field you
3065 must address your sub-fields appropriately:
3066 from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))
3068
3070 numpy
3071 Hypothesis offers a number of strategies for NumPy testing, available
3072 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
3073 package.
3074 The centerpiece is the arrays() strategy, which generates arrays with
3076 any dtype, shape, and contents you can specify or give a strategy for.
3077 To make this as useful as possible, strategies are provided to generate
3078 array shapes and generate all kinds of fixed-size or compound dtypes.
3079 hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0,
3081 max_size=None, min_value=None, max_value=None, allow_nan=None, al‐
3082 low_infinity=None, exclude_min=None, exclude_max=None)
3083 Creates a strategy which can generate any value of the given
3084 dtype.
3085 Compatible **kwargs are passed to the inferred strategy function
3087 for integers, floats, and strings. This allows you to customise
3088 the min and max values, control the length or contents of
3089 strings, or exclude non-finite numbers. This is particularly
3090 useful when kwargs are passed through from arrays() which allow
3091 a variety of numeric dtypes, as it seamlessly handles the width
3092 or representable bounds for you. See issue #2552 for more de‐
3093 tail.
3094 hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None,
3096 fill=None, unique=False)
3097 Returns a strategy for generating numpy:numpy.ndarrays.
3098 • dtype may be any valid input to dtype (this includes dtype ob‐
3100 jects), or a strategy that generates such values.
3101 • shape may be an integer >= 0, a tuple of such integers, or a
3103 strategy that generates such values.
3104 • elements is a strategy for generating values to put in the ar‐
3106 ray. If it is None a suitable value will be inferred based on
3107 the dtype, which may give any legal value (including eg NaN
3108 for floats). If a mapping, it will be passed as **kwargs to
3109 from_dtype()
3110 • fill is a strategy that may be used to generate a single back‐
3112 ground value for the array. If None, a suitable default will
3113 be inferred based on the other arguments. If set to nothing()
3114 then filling behaviour will be disabled entirely and every el‐
3115 ement will be generated independently.
3116 • unique specifies if the elements of the array should all be
3118 distinct from one another. Note that in this case multiple NaN
3119 values may still be allowed. If fill is also set, the only
3120 valid values for it to return are NaN values (anything for
3121 which numpy:numpy.isnan returns True. So e.g. for complex num‐
3122 bers (nan+1j) is also a valid fill). Note that if unique is
3123 set to True the generated values must be hashable.
3124 Arrays of specified dtype and shape are generated for example
3126 like this:
3127 >>> import numpy as np
3129 >>> arrays(np.int8, (2, 3)).example()
3130 array([[-8, 6, 3],
3131 [-6, 4, 6]], dtype=int8)
3132 • See What you can generate and how.
3134 >>> import numpy as np
3136 >>> from hypothesis.strategies import floats
3137 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
3138 array([ 0.88974794, 0.77387938, 0.1977879 ])
3139 Array values are generated in two parts:
3141 1. Some subset of the coordinates of the array are populated
3143 with a value drawn from the elements strategy (or its in‐
3144 ferred form).
3145 2. If any coordinates were not assigned in the previous step, a
3147 single value is drawn from the fill strategy and is assigned
3148 to all remaining places.
3149 You can set fill to nothing() if you want to disable this behav‐
3151 iour and draw a value for every element.
3152 If fill is set to None then it will attempt to infer the correct
3154 behaviour automatically: If unique is True, no filling will oc‐
3155 cur by default. Otherwise, if it looks safe to reuse the values
3156 of elements across multiple coordinates (this will be the case
3157 for any inferred strategy, and for most of the builtins, but is
3158 not the case for mutable values or strategies built with
3159 flatmap, map, composite, etc) then it will use the elements
3160 strategy as the fill, else it will default to having no fill.
3161 Having a fill helps Hypothesis craft high quality examples, but
3163 its main importance is when the array generated is large: Hy‐
3164 pothesis is primarily designed around testing small examples. If
3165 you have arrays with hundreds or more elements, having a fill
3166 value is essential if you want your tests to run in reasonable
3167 time.
3168 hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None,
3170 min_side=1, max_side=None)
3171 Return a strategy for array shapes (tuples of int >= 1).
3172 hypothesis.extra.numpy.scalar_dtypes()
3174 Return a strategy that can return any non-flexible scalar dtype.
3175 hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?',
3177 sizes=(8, 16, 32, 64))
3178 Return a strategy for unsigned integer dtypes.
3179 endianness may be < for little-endian, > for big-endian, = for
3181 native byte order, or ? to allow either byte order. This argu‐
3182 ment only applies to dtypes of more than one byte.
3183 sizes must be a collection of integer sizes in bits. The de‐
3185 fault (8, 16, 32, 64) covers the full range of sizes.
3186 hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16,
3188 32, 64))
3189 Return a strategy for signed integer dtypes.
3190 endianness and sizes are treated as for
3192 unsigned_integer_dtypes().
3193 hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16,
3195 32, 64))
3196 Return a strategy for floating-point dtypes.
3197 sizes is the size in bits of floating-point number. Some ma‐
3199 chines support 96- or 128-bit floats, but these are not gener‐
3200 ated by default.
3201 Larger floats (96 and 128 bit real parts) are not supported on
3203 all platforms and therefore disabled by default. To generate
3204 these dtypes, include these values in the sizes argument.
3205 hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?',
3207 sizes=(64, 128))
3208 Return a strategy for complex-number dtypes.
3209 sizes is the total size in bits of a complex number, which con‐
3211 sists of two floats. Complex halves (a 16-bit real part) are
3212 not supported by numpy and will not be generated by this strat‐
3213 egy.
3214 hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_pe‐
3216 riod='ns', endianness='?')
3217 Return a strategy for datetime64 dtypes, with various precisions
3218 from year to attosecond.
3219 hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_pe‐
3221 riod='ns', endianness='?')
3222 Return a strategy for timedelta64 dtypes, with various preci‐
3223 sions from year to attosecond.
3224 hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1,
3226 max_len=16)
3227 Return a strategy for generating bytestring dtypes, of various
3228 lengths and byteorder.
3229 While Hypothesis' string strategies can generate empty strings,
3231 string dtypes with length 0 indicate that size is still to be
3232 determined, so the minimum length for string dtypes is 1.
3233 hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?',
3235 min_len=1, max_len=16)
3236 Return a strategy for generating unicode string dtypes, of vari‐
3237 ous lengths and byteorder.
3238 While Hypothesis' string strategies can generate empty strings,
3240 string dtypes with length 0 indicate that size is still to be
3241 determined, so the minimum length for string dtypes is 1.
3242 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
3244 *, min_size=1, max_size=5, allow_subarrays=False)
3245 Return a strategy for generating array (compound) dtypes, with
3246 members drawn from the given subtype strategy.
3247 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
3249 *, max_leaves=10, max_itemsize=None)
3250 Return the most-general dtype strategy.
3251 Elements drawn from this strategy may be simple (from the sub‐
3253 type_strategy), or several such values drawn from array_dtypes()
3254 with allow_subarrays=True. Subdtypes in an array dtype may be
3255 nested to any depth, subject to the max_leaves argument.
3256 hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0,
3258 max_size=None)
3259 Return a strategy for generating permissible tuple-values for
3260 the axis argument for a numpy sequential function (e.g.
3261 numpy:numpy.sum()), given an array of the specified dimensional‐
3262 ity.
3263 All tuples will have an length >= min_size and <= max_size. The
3265 default value for max_size is ndim.
3266 Examples from this strategy shrink towards an empty tuple, which
3268 render most sequential functions as no-ops.
3269 The following are some examples drawn from this strategy.
3271 >>> [valid_tuple_axes(3).example() for i in range(4)]
3273 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3274 valid_tuple_axes can be joined with other strategies to generate
3276 any type of valid axis object, i.e. integers, tuples, and None:
3277 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3279 hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0,
3281 max_dims=None, min_side=1, max_side=None)
3282 Return a strategy for generating shapes that are broadcast-com‐
3283 patible with the provided shape.
3284 Examples from this strategy shrink towards a shape with length
3286 min_dims. The size of an aligned dimension shrinks towards size
3287 1. The size of an unaligned dimension shrink towards min_side.
3288 • shape a tuple of integers
3290 • min_dims The smallest length that the generated shape can pos‐
3292 sess.
3293 • max_dims The largest length that the generated shape can pos‐
3295 sess. The default-value for max_dims is min(32,
3296 max(len(shape), min_dims) + 2).
3297 • min_side The smallest size that an unaligned dimension can
3299 possess.
3300 • max_side The largest size that an unaligned dimension can pos‐
3302 sess. The default value is 2 + 'size-of-largest-aligned-di‐
3303 mension'.
3304 The following are some examples drawn from this strategy.
3306 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3308 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3309 hypothesis.extra.numpy.mutually_broadcastable_shapes(*,
3311 num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0,
3312 max_dims=None, min_side=1, max_side=None)
3313 Return a strategy for generating a specified number of shapes,
3314 N, that are mutually-broadcastable with one another and with the
3315 provided "base-shape".
3316 The strategy will generate a named-tuple of:
3318 • input_shapes: the N generated shapes
3320 • result_shape: the resulting shape, produced by broadcasting
3322 the N shapes with the base-shape
3323 Each shape produced from this strategy shrinks towards a shape
3325 with length min_dims. The size of an aligned dimension shrinks
3326 towards being having a size of 1. The size of an unaligned di‐
3327 mension shrink towards min_side.
3328 • num_shapes The number of mutually broadcast-compatible shapes
3330 to generate.
3331 • base-shape The shape against which all generated shapes can
3333 broadcast. The default shape is empty, which corresponds to a
3334 scalar and thus does not constrain broadcasting at all.
3335 • min_dims The smallest length that any generated shape can pos‐
3337 sess.
3338 • max_dims The largest length that any generated shape can pos‐
3340 sess. It cannot exceed 32, which is the greatest supported
3341 dimensionality for a numpy array. The default-value for
3342 max_dims is 2 + max(len(shape), min_dims), capped at 32.
3343 • min_side The smallest size that an unaligned dimension can
3345 possess.
3346 • max_side The largest size that an unaligned dimension can pos‐
3348 sess. The default value is 2 + 'size-of-largest-aligned-di‐
3349 mension'.
3350 The following are some examples drawn from this strategy.
3352 >>> # Draw three shapes, and each shape is broadcast-compatible with `(2, 3)`
3354 >>> for _ in range(5):
3355 ... mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3)).example()
3356 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3357 BroadcastableShapes(input_shapes=((3,), (1,), (2, 1)), result_shape=(2, 3))
3358 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3359 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
3360 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
3361 Use with Generalised Universal Function signatures
3363 A universal function (or ufunc for short) is a function that op‐
3365 erates on ndarrays in an element-by-element fashion, supporting
3366 array broadcasting, type casting, and several other standard
3367 features. A generalised ufunc operates on sub-arrays rather
3368 than elements, based on the "signature" of the function. Com‐
3369 pare e.g. numpy:numpy.add (ufunc) to numpy:numpy.matmul (gu‐
3370 func).
3371 To generate shapes for a gufunc, you can pass the signature ar‐
3373 gument instead of num_shapes. This must be a gufunc signature
3374 string; which you can write by hand or access as e.g. np.mat‐
3375 mul.signature on generalised ufuncs.
3376 In this case, the side arguments are applied to the 'core dimen‐
3378 sions' as well, ignoring any frozen dimensions. base_shape and
3379 the dims arguments are applied to the 'loop dimensions', and if
3380 necessary, the dimensionality of each shape is silently capped
3381 to respect the 32-dimension limit.
3382 The generated result_shape is the real result shape of applying
3384 the gufunc to arrays of the generated input_shapes, even where
3385 this is different to broadcasting the loop dimensions.
3386 gufunc-compatible shapes shrink their loop dimensions as above,
3388 towards omitting optional core dimensions, and smaller-size core
3389 dimensions.
3390 >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
3392 >>> for _ in range(3):
3393 ... mutually_broadcastable_shapes(signature=np.matmul.signature).example()
3394 BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
3395 BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
3396 BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))
3397 hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0,
3399 max_dims=None, allow_newaxis=False, allow_ellipsis=True)
3400 The basic_indices strategy generates basic indexes for arrays of
3401 the specified shape, which may include dimensions of size zero.
3402 It generates tuples containing some mix of integers,
3404 python:slice objects, ... (Ellipsis), and numpy:numpy.newaxis;
3405 which when used to index a shape-shaped array will produce ei‐
3406 ther a scalar or a shared-memory view. When a length-one tuple
3407 would be generated, this strategy may instead return the element
3408 which will index the first axis, e.g. 5 instead of (5,).
3409 • shape: the array shape that will be indexed, as a tuple of in‐
3411 tegers >= 0. This must be at least two-dimensional for a tu‐
3412 ple to be a valid basic index; for one-dimensional arrays use
3413 slices() instead.
3414 • min_dims: the minimum dimensionality of the resulting view
3416 from use of the generated index. When min_dims == 0, scalars
3417 and zero-dimensional arrays are both allowed.
3418 • max_dims: the maximum dimensionality of the resulting view.
3420 If not specified, it defaults to max(len(shape), min_dims) +
3421 2.
3422 • allow_ellipsis: whether ...` is allowed in the index.
3424 • allow_newaxis: whether numpy:numpy.newaxis is allowed in the
3426 index.
3427 Note that the length of the generated tuple may be anywhere be‐
3429 tween zero and min_dims. It may not match the length of shape,
3430 or even the dimensionality of the array view resulting from its
3431 use!
3432 hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=ar‐
3434 ray_shapes(), dtype='int')
3435 Return a search strategy for tuples of integer-arrays that, when
3436 used to index into an array of shape shape, given an array whose
3437 shape was drawn from result_shape.
3438 Examples from this strategy shrink towards the tuple of in‐
3440 dex-arrays:
3441 len(shape) * (np.zeros(drawn_result_shape, dtype), )
3443 • shape a tuple of integers that indicates the shape of the ar‐
3445 ray, whose indices are being generated.
3446 • result_shape a strategy for generating tuples of integers,
3448 which describe the shape of the resulting index arrays. The
3449 default is array_shapes(). The shape drawn from this strategy
3450 determines the shape of the array that will be produced when
3451 the corresponding example from integer_array_indices is used
3452 as an index.
3453 • dtype the integer data type of the generated index-arrays.
3455 Negative integer indices can be generated if a signed integer
3456 type is specified.
3457 Recall that an array can be indexed using a tuple of integer-ar‐
3459 rays to access its members in an arbitrary order, producing an
3460 array with an arbitrary shape. For example:
3461 >>> from numpy import array
3463 >>> x = array([-0, -1, -2, -3, -4])
3464 >>> ind = (array([[4, 0], [0, 1]]),) # a tuple containing a 2D integer-array
3465 >>> x[ind] # the resulting array is commensurate with the indexing array(s)
3466 array([[-4, 0],
3467 [0, -1]])
3468 Note that this strategy does not accommodate all variations of
3470 so-called 'advanced indexing', as prescribed by NumPy's nomen‐
3471 clature. Combinations of basic and advanced indexes are too
3472 complex to usefully define in a standard strategy; we leave ap‐
3473 plication-specific strategies to the user. Advanced-boolean in‐
3474 dexing can be defined as arrays(shape=..., dtype=bool), and is
3475 similarly left to the user.
3476 pandas
3478 Hypothesis provides strategies for several of the core pandas data
3479 types: pandas.Index, pandas.Series and pandas.DataFrame.
3480 The general approach taken by the pandas module is that there are mul‐
3482 tiple strategies for generating indexes, and all of the other strate‐
3483 gies take the number of entries they contain from their index strategy
3484 (with sensible defaults). So e.g. a Series is specified by specifying
3485 its numpy.dtype (and/or a strategy for generating elements for it).
3486 hypothesis.extra.pandas.indexes(*, elements=None, dtype=None,
3488 min_size=0, max_size=None, unique=True)
3489 Provides a strategy for producing a pandas.Index.
3490 Arguments:
3492 • elements is a strategy which will be used to generate the in‐
3494 dividual values of the index. If None, it will be inferred
3495 from the dtype. Note: even if the elements strategy produces
3496 tuples, the generated value will not be a MultiIndex, but in‐
3497 stead be a normal index whose elements are tuples.
3498 • dtype is the dtype of the resulting index. If None, it will be
3500 inferred from the elements strategy. At least one of dtype or
3501 elements must be provided.
3502 • min_size is the minimum number of elements in the index.
3504 • max_size is the maximum number of elements in the index. If
3506 None then it will default to a suitable small size. If you
3507 want larger indexes you should pass a max_size explicitly.
3508 • unique specifies whether all of the elements in the resulting
3510 index should be distinct.
3511 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
3513 Provides a strategy which generates an Index whose values are 0,
3514 1, ..., n for some n.
3515 Arguments:
3517 • min_size is the smallest number of elements the index can
3519 have.
3520 • max_size is the largest number of elements the index can have.
3522 If None it will default to some suitable value based on
3523 min_size.
3524 hypothesis.extra.pandas.series(*, elements=None, dtype=None, in‐
3526 dex=None, fill=None, unique=False)
3527 Provides a strategy for producing a pandas.Series.
3528 Arguments:
3530 • elements: a strategy that will be used to generate the indi‐
3532 vidual values in the series. If None, we will attempt to infer
3533 a suitable default from the dtype.
3534 • dtype: the dtype of the resulting series and may be any value
3536 that can be passed to numpy.dtype. If None, will use pandas's
3537 standard behaviour to infer it from the type of the elements
3538 values. Note that if the type of values that comes out of your
3539 elements strategy varies, then so will the resulting dtype of
3540 the series.
3541 • index: If not None, a strategy for generating indexes for the
3543 resulting Series. This can generate either pandas.Index ob‐
3544 jects or any sequence of values (which will be passed to the
3545 Index constructor).
3546 You will probably find it most convenient to use the indexes()
3548 or range_indexes() function to produce values for this argu‐
3549 ment.
3550 Usage:
3552 >>> series(dtype=int).example()
3554 0 -2001747478
3555 1 1153062837
3556 class hypothesis.extra.pandas.column(name=None, elements=None,
3558 dtype=None, fill=None, unique=False)
3559 Data object for describing a column in a DataFrame.
3560 Arguments:
3562 • name: the column name, or None to default to the column posi‐
3564 tion. Must be hashable, but can otherwise be any value sup‐
3565 ported as a pandas column name.
3566 • elements: the strategy for generating values in this column,
3568 or None to infer it from the dtype.
3569 • dtype: the dtype of the column, or None to infer it from the
3571 element strategy. At least one of dtype or elements must be
3572 provided.
3573 • fill: A default value for elements of the column. See arrays()
3575 for a full explanation.
3576 • unique: If all values in this column should be distinct.
3578 hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, ele‐
3580 ments=None, fill=None, unique=False)
3581 A convenience function for producing a list of column objects of
3582 the same general shape.
3583 The names_or_number argument is either a sequence of values, the
3585 elements of which will be used as the name for individual column
3586 objects, or a number, in which case that many unnamed columns
3587 will be created. All other arguments are passed through verbatim
3588 to create the columns.
3589 hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, in‐
3591 dex=None)
3592 Provides a strategy for producing a pandas.DataFrame.
3593 Arguments:
3595 • columns: An iterable of column objects describing the shape of
3597 the generated DataFrame.
3598 • rows: A strategy for generating a row object. Should generate
3600 either dicts mapping column names to values or a sequence map‐
3601 ping column position to the value in that position (note that
3602 unlike the pandas.DataFrame constructor, single values are not
3603 allowed here. Passing e.g. an integer is an error, even if
3604 there is only one column).
3605 At least one of rows and columns must be provided. If both are
3607 provided then the generated rows will be validated against the
3608 columns and an error will be raised if they don't match.
3609 Caveats on using rows:
3611 • In general you should prefer using columns to rows, and only
3613 use rows if the columns interface is insufficiently flexible
3614 to describe what you need - you will get better performance
3615 and example quality that way.
3616 • If you provide rows and not columns, then the shape and
3618 dtype of the resulting DataFrame may vary. e.g. if you have
3619 a mix of int and float in the values for one column in your
3620 row entries, the column will sometimes have an integral
3621 dtype and sometimes a float.
3622 • index: If not None, a strategy for generating indexes for the
3624 resulting DataFrame. This can generate either pandas.Index ob‐
3625 jects or any sequence of values (which will be passed to the
3626 Index constructor).
3627 You will probably find it most convenient to use the indexes()
3629 or range_indexes() function to produce values for this argu‐
3630 ment.
3631 Usage:
3633 The expected usage pattern is that you use column and columns()
3635 to specify a fixed shape of the DataFrame you want as follows.
3636 For example the following gives a two column data frame:
3637 >>> from hypothesis.extra.pandas import column, data_frames
3639 >>> data_frames([
3640 ... column('A', dtype=int), column('B', dtype=float)]).example()
3641 A B
3642 0 2021915903 1.793898e+232
3643 1 1146643993 inf
3644 2 -2096165693 1.000000e+07
3645 If you want the values in different columns to interact in some
3647 way you can use the rows argument. For example the following
3648 gives a two column DataFrame where the value in the first column
3649 is always at most the value in the second:
3650 >>> from hypothesis.extra.pandas import column, data_frames
3652 >>> import hypothesis.strategies as st
3653 >>> data_frames(
3654 ... rows=st.tuples(st.floats(allow_nan=False),
3655 ... st.floats(allow_nan=False)).map(sorted)
3656 ... ).example()
3657 0 1
3658 0 -3.402823e+38 9.007199e+15
3659 1 -1.562796e-298 5.000000e-01
3660 You can also combine the two:
3662 >>> from hypothesis.extra.pandas import columns, data_frames
3664 >>> import hypothesis.strategies as st
3665 >>> data_frames(
3666 ... columns=columns(["lo", "hi"], dtype=float),
3667 ... rows=st.tuples(st.floats(allow_nan=False),
3668 ... st.floats(allow_nan=False)).map(sorted)
3669 ... ).example()
3670 lo hi
3671 0 9.314723e-49 4.353037e+45
3672 1 -9.999900e-01 1.000000e+07
3673 2 -2.152861e+134 -1.069317e-73
3674 (Note that the column dtype must still be specified and will not
3676 be inferred from the rows. This restriction may be lifted in fu‐
3677 ture).
3678 Combining rows and columns has the following behaviour:
3680 • The column names and dtypes will be used.
3682 • If the column is required to be unique, this will be enforced.
3684 • Any values missing from the generated rows will be provided
3686 using the column's fill.
3687 • Any values in the row not present in the column specification
3689 (if dicts are passed, if there are keys with no corresponding
3690 column name, if sequences are passed if there are too many
3691 items) will result in InvalidArgument being raised.
3692 Supported versions
3694 There is quite a lot of variation between pandas versions. We only com‐
3695 mit to supporting the latest version of pandas, but older minor ver‐
3696 sions are supported on a "best effort" basis. Hypothesis is currently
3697 tested against and confirmed working with every Pandas minor version
3698 from 0.25 through to 1.1.
3699 Releases that are not the latest patch release of their minor version
3701 are not tested or officially supported, but will probably also work un‐
3702 less you hit a pandas bug.
3703
3705 Hypothesis tries to detect common mistakes and things that will cause
3706 difficulty at run time in the form of a number of 'health checks'.
3707 These include detecting and warning about:
3709 • Strategies with very slow data generation
3711 • Strategies which filter out too much
3713 • Recursive strategies which branch too much
3715 • Tests that are unlikely to complete in a reasonable amount of time.
3717 If any of these scenarios are detected, Hypothesis will emit a warning
3719 about them.
3720 The general goal of these health checks is to warn you about things
3722 that you are doing that might appear to work but will either cause Hy‐
3723 pothesis to not work correctly or to perform badly.
3724 To selectively disable health checks, use the suppress_health_check
3726 setting. The argument for this parameter is a list with elements drawn
3727 from any of the class-level attributes of the HealthCheck class. Using
3728 a value of HealthCheck.all() will disable all health checks.
3729 class hypothesis.HealthCheck(value)
3731 Arguments for suppress_health_check.
3732 Each member of this enum is a type of health check to suppress.
3734 data_too_large = 1
3736 Check for when the typical size of the examples you are
3737 generating exceeds the maximum allowed size too often.
3738 filter_too_much = 2
3740 Check for when the test is filtering out too many exam‐
3741 ples, either through use of assume() or filter(), or oc‐
3742 casionally for Hypothesis internal reasons.
3743 too_slow = 3
3745 Check for when your data generation is extremely slow and
3746 likely to hurt testing.
3747 return_value = 5
3749 Checks if your tests return a non-None value (which will
3750 be ignored and is unlikely to do what you want).
3751 large_base_example = 7
3753 Checks if the natural example to shrink towards is very
3754 large.
3755 not_a_test_method = 8
3757 Checks if @given has been applied to a method defined by
3758 python:unittest.TestCase (i.e. not a test).
3759 function_scoped_fixture = 9
3761 Check if @given has been applied to a test with a pytest
3762 function-scoped fixture. Function-scoped fixtures run
3763 once for the whole function, not once per example, and
3764 this is usually not what you want.
3765 Because of this limitation, tests that need to need to
3767 set up or reset state for every example need to do so
3768 manually within the test itself, typically using an ap‐
3769 propriate context manager.
3770 Suppress this health check only in the rare case that you
3772 are using a function-scoped fixture that does not need to
3773 be reset between individual examples, but for some reason
3774 you cannot use a wider fixture scope (e.g. session scope,
3775 module scope, class scope).
3776 This check requires the Hypothesis pytest plugin, which
3778 is enabled by default when running Hypothesis inside
3779 pytest.
3780 Deprecations
3782 We also use a range of custom exception and warning types, so you can
3783 see exactly where an error came from - or turn only our warnings into
3784 errors.
3785 class hypothesis.errors.HypothesisDeprecationWarning
3787 A deprecation warning issued by Hypothesis.
3788 Actually inherits from FutureWarning, because DeprecationWarning
3790 is hidden by the default warnings filter.
3791 You can configure the Python python:warnings to handle these
3793 warnings differently to others, either turning them into errors
3794 or suppressing them entirely. Obviously we would prefer the
3795 former!
3796 Deprecated features will be continue to emit warnings for at least six
3798 months, and then be removed in the following major release. Note how‐
3799 ever that not all warnings are subject to this grace period; sometimes
3800 we strengthen validation by adding a warning and these may become er‐
3801 rors immediately at a major release.
3802
3804 When Hypothesis finds a bug it stores enough information in its data‐
3805 base to reproduce it. This enables you to have a classic testing work‐
3806 flow of find a bug, fix a bug, and be confident that this is actually
3807 doing the right thing because Hypothesis will start by retrying the ex‐
3808 amples that broke things last time.
3809 Limitations
3811 The database is best thought of as a cache that you never need to in‐
3812 validate: Information may be lost when you upgrade a Hypothesis version
3813 or change your test, so you shouldn't rely on it for correctness - if
3814 there's an example you want to ensure occurs each time then there's a
3815 feature for including them in your source code - but it helps the de‐
3816 velopment workflow considerably by making sure that the examples you've
3817 just found are reproduced.
3818 The database also records examples that exercise less-used parts of
3820 your code, so the database may update even when no failing examples
3821 were found.
3822 Upgrading Hypothesis and changing your tests
3824 The design of the Hypothesis database is such that you can put arbi‐
3825 trary data in the database and not get wrong behaviour. When you up‐
3826 grade Hypothesis, old data might be invalidated, but this should happen
3827 transparently. It can never be the case that e.g. changing the strategy
3828 that generates an argument gives you data from the old strategy.
3829 ExampleDatabase implementations
3831 Hypothesis' default database setting creates a
3832 DirectoryBasedExampleDatabase in your current working directory, under
3833 .hypothesis/examples. If this location is unusable, e.g. because you
3834 do not have read or write permissions, Hypothesis will emit a warning
3835 and fall back to an InMemoryExampleDatabase.
3836 Hypothesis provides the following ExampleDatabase implementations:
3838 class hypothesis.database.InMemoryExampleDatabase
3840 A non-persistent example database, implemented in terms of a
3841 dict of sets.
3842 This can be useful if you call a test function several times in
3844 a single session, or for testing other database implementations,
3845 but because it does not persist between runs we do not recommend
3846 it for general use.
3847 class hypothesis.database.DirectoryBasedExampleDatabase(path)
3849 Use a directory to store Hypothesis examples as files.
3850 Each test corresponds to a directory, and each example to a file
3852 within that directory. While the contents are fairly opaque, a
3853 DirectoryBasedExampleDatabase can be shared by checking the di‐
3854 rectory into version control, for example with the following
3855 .gitignore:
3856 # Ignore files cached by Hypothesis...
3858 .hypothesis/*
3859 # except for the examples directory
3860 !.hypothesis/examples/
3861 Note however that this only makes sense if you also pin to an
3863 exact version of Hypothesis, and we would usually recommend im‐
3864 plementing a shared database with a network datastore - see
3865 ExampleDatabase, and the MultiplexedDatabase helper.
3866 class hypothesis.database.ReadOnlyDatabase(db)
3868 A wrapper to make the given database read-only.
3869 The implementation passes through fetch, and turns save, delete,
3871 and move into silent no-ops.
3872 Note that this disables Hypothesis' automatic discarding of
3874 stale examples. It is designed to allow local machines to ac‐
3875 cess a shared database (e.g. from CI servers), without propagat‐
3876 ing changes back from a local or in-development branch.
3877 class hypothesis.database.MultiplexedDatabase(*dbs)
3879 A wrapper around multiple databases.
3880 Each save, fetch, move, or delete operation will be run against
3882 all of the wrapped databases. fetch does not yield duplicate
3883 values, even if the same value is present in two or more of the
3884 wrapped databases.
3885 This combines well with a ReadOnlyDatabase, as follows:
3887 local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
3889 shared = CustomNetworkDatabase()
3890 settings.register_profile("ci", database=shared)
3892 settings.register_profile(
3893 "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
3894 )
3895 settings.load_profile("ci" if os.environ.get("CI") else "dev")
3896 So your CI system or fuzzing runs can populate a central shared
3898 database; while local runs on development machines can reproduce
3899 any failures from CI but will only cache their own failures lo‐
3900 cally and cannot remove examples from the shared database.
3901 class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_af‐
3903 ter=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:')
3904 Store Hypothesis examples as sets in the given redis.Redis data‐
3905 store.
3906 This is particularly useful for shared databases, as per the
3908 recipe for a MultiplexedDatabase.
3909 NOTE:
3911 If a test has not been run for expire_after, those examples
3912 will be allowed to expire. The default time-to-live persists
3913 examples between weekly runs.
3914 Defining your own ExampleDatabase
3916 You can define your ExampleDatabase, for example to use a shared datas‐
3917 tore, with just a few methods:
3918 class hypothesis.database.ExampleDatabase(*args, **kwargs)
3920 An abstract base class for storing examples in Hypothesis' in‐
3921 ternal format.
3922 An ExampleDatabase maps each bytes key to many distinct bytes
3924 values, like a Mapping[bytes, AbstractSet[bytes]].
3925 abstract save(key, value)
3927 Save value under key.
3928 If this value is already present for this key, silently
3930 do nothing.
3931 abstract fetch(key)
3933 Return an iterable over all values matching this key.
3934 abstract delete(key, value)
3936 Remove this value from this key.
3937 If this value is not present, silently do nothing.
3939 move(src, dest, value)
3941 Move value from key src to key dest. Equivalent to
3942 delete(src, value) followed by save(src, value), but may
3943 have a more efficient implementation.
3944 Note that value will be inserted at dest regardless of
3946 whether it is currently present at src.
3947
3949 With @given, your tests are still something that you mostly write your‐
3950 self, with Hypothesis providing some data. With Hypothesis's stateful
3951 testing, Hypothesis instead tries to generate not just data but entire
3952 tests. You specify a number of primitive actions that can be combined
3953 together, and then Hypothesis will try to find sequences of those ac‐
3954 tions that result in a failure.
3955 TIP:
3957 Before reading this reference documentation, we recommend reading
3958 How not to Die Hard with Hypothesis and An Introduction to
3959 Rule-Based Stateful Testing, in that order. The implementation de‐
3960 tails will make more sense once you've seen them used in practice,
3961 and know why each method or decorator is available.
3962 NOTE:
3964 This style of testing is often called model-based testing, but in
3965 Hypothesis is called stateful testing (mostly for historical reasons
3966 - the original implementation of this idea in Hypothesis was more
3967 closely based on ScalaCheck's stateful testing where the name is
3968 more apt). Both of these names are somewhat misleading: You don't
3969 really need any sort of formal model of your code to use this, and
3970 it can be just as useful for pure APIs that don't involve any state
3971 as it is for stateful ones.
3972 It's perhaps best to not take the name of this sort of testing too
3974 seriously. Regardless of what you call it, it is a powerful form of
3975 testing which is useful for most non-trivial APIs.
3976 You may not need state machines
3978 The basic idea of stateful testing is to make Hypothesis choose actions
3979 as well as values for your test, and state machines are a great declar‐
3980 ative way to do just that.
3981 For simpler cases though, you might not need them at all - a standard
3983 test with @given might be enough, since you can use data() in branches
3984 or loops. In fact, that's how the state machine explorer works inter‐
3985 nally. For more complex workloads though, where a higher level API
3986 comes into it's own, keep reading!
3987 Rule-based state machines
3989 class hypothesis.stateful.RuleBasedStateMachine
3990 A RuleBasedStateMachine gives you a structured way to define
3991 state machines.
3992 The idea is that a state machine carries a bunch of types of
3994 data divided into Bundles, and has a set of rules which may read
3995 data from bundles (or just from normal strategies) and push data
3996 onto bundles. At any given point a random applicable rule will
3997 be executed.
3998 A rule is very similar to a normal @given based test in that it takes
4000 values drawn from strategies and passes them to a user defined test
4001 function. The key difference is that where @given based tests must be
4002 independent, rules can be chained together - a single test run may in‐
4003 volve multiple rule invocations, which may interact in various ways.
4004 Rules can take normal strategies as arguments, or a specific kind of
4006 strategy called a Bundle. A Bundle is a named collection of generated
4007 values that can be reused by other operations in the test. They are
4008 populated with the results of rules, and may be used as arguments to
4009 rules, allowing data to flow from one rule to another, and rules to
4010 work on the results of previous computations or actions.
4011 You can think of each value that gets added to any Bundle as being as‐
4013 signed to a new variable. Drawing a value from the bundle strategy
4014 means choosing one of the corresponding variables and using that value,
4015 and consumes() as a del statement for that variable. If you can re‐
4016 place use of Bundles with instance attributes of the class that is of‐
4017 ten simpler, but often Bundles are strictly more powerful.
4018 The following rule based state machine example is a simplified version
4020 of a test for Hypothesis's example database implementation. An example
4021 database maps keys to sets of values, and in this test we compare one
4022 implementation of it to a simplified in memory model of its behaviour,
4023 which just stores the same values in a Python dict. The test then runs
4024 operations against both the real database and the in-memory representa‐
4025 tion of it and looks for discrepancies in their behaviour.
4026 import shutil
4028 import tempfile
4029 from collections import defaultdict
4030 import hypothesis.strategies as st
4032 from hypothesis.database import DirectoryBasedExampleDatabase
4033 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
4034 class DatabaseComparison(RuleBasedStateMachine):
4037 def __init__(self):
4038 super().__init__()
4039 self.tempd = tempfile.mkdtemp()
4040 self.database = DirectoryBasedExampleDatabase(self.tempd)
4041 self.model = defaultdict(set)
4042 keys = Bundle("keys")
4044 values = Bundle("values")
4045 @rule(target=keys, k=st.binary())
4047 def add_key(self, k):
4048 return k
4049 @rule(target=values, v=st.binary())
4051 def add_value(self, v):
4052 return v
4053 @rule(k=keys, v=values)
4055 def save(self, k, v):
4056 self.model[k].add(v)
4057 self.database.save(k, v)
4058 @rule(k=keys, v=values)
4060 def delete(self, k, v):
4061 self.model[k].discard(v)
4062 self.database.delete(k, v)
4063 @rule(k=keys)
4065 def values_agree(self, k):
4066 assert set(self.database.fetch(k)) == self.model[k]
4067 def teardown(self):
4069 shutil.rmtree(self.tempd)
4070 TestDBComparison = DatabaseComparison.TestCase
4073 In this we declare two bundles - one for keys, and one for values. We
4075 have two trivial rules which just populate them with data (k and v),
4076 and three non-trivial rules: save saves a value under a key and delete
4077 removes a value from a key, in both cases also updating the model of
4078 what should be in the database. values_agree then checks that the con‐
4079 tents of the database agrees with the model for a particular key.
4080 We can then integrate this into our test suite by getting a unittest
4082 TestCase from it:
4083 TestTrees = DatabaseComparison.TestCase
4085 # Or just run with pytest's unittest support
4087 if __name__ == "__main__":
4088 unittest.main()
4089 This test currently passes, but if we comment out the line where we
4091 call self.model[k].discard(v), we would see the following output when
4092 run under pytest:
4093 AssertionError: assert set() == {b''}
4095 ------------ Hypothesis ------------
4097 state = DatabaseComparison()
4099 var1 = state.add_key(k=b'')
4100 var2 = state.add_value(v=var1)
4101 state.save(k=var1, v=var2)
4102 state.delete(k=var1, v=var2)
4103 state.values_agree(k=var1)
4104 state.teardown()
4105 Note how it's printed out a very short program that will demonstrate
4107 the problem. The output from a rule based state machine should gener‐
4108 ally be pretty close to Python code - if you have custom repr implemen‐
4109 tations that don't return valid Python then it might not be, but most
4110 of the time you should just be able to copy and paste the code into a
4111 test to reproduce it.
4112 You can control the detailed behaviour with a settings object on the
4114 TestCase (this is a normal hypothesis settings object using the de‐
4115 faults at the time the TestCase class was first referenced). For exam‐
4116 ple if you wanted to run fewer examples with larger programs you could
4117 change the settings to:
4118 DatabaseComparison.TestCase.settings = settings(
4120 max_examples=50, stateful_step_count=100
4121 )
4122 Which doubles the number of steps each program runs and halves the num‐
4124 ber of test cases that will be run.
4125 Rules
4127 As said earlier, rules are the most common feature used in RuleBased‐
4128 StateMachine. They are defined by applying the rule() decorator on a
4129 function. Note that RuleBasedStateMachine must have at least one rule
4130 defined and that a single function cannot be used to define multiple
4131 rules (this to avoid having multiple rules doing the same things). Due
4132 to the stateful execution method, rules generally cannot take arguments
4133 from other sources such as fixtures or pytest.mark.parametrize - con‐
4134 sider providing them via a strategy such as sampled_from() instead.
4135 hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)
4137 Decorator for RuleBasedStateMachine. Any name present in target
4138 or targets will define where the end result of this function
4139 should go. If both are empty then the end result will be dis‐
4140 carded.
4141 target must be a Bundle, or if the result should go to multiple
4143 bundles you can pass a tuple of them as the targets argument.
4144 It is invalid to use both arguments for a single rule. If the
4145 result should go to exactly one of several bundles, define a
4146 separate rule for each case.
4147 kwargs then define the arguments that will be passed to the
4149 function invocation. If their value is a Bundle, or if it is
4150 consumes(b) where b is a Bundle, then values that have previ‐
4151 ously been produced for that bundle will be provided. If con‐
4152 sumes is used, the value will also be removed from the bundle.
4153 Any other kwargs should be strategies and values from them will
4155 be provided.
4156 hypothesis.stateful.consumes(bundle)
4158 When introducing a rule in a RuleBasedStateMachine, this func‐
4159 tion can be used to mark bundles from which each value used in a
4160 step with the given rule should be removed. This function re‐
4161 turns a strategy object that can be manipulated and combined
4162 like any other.
4163 For example, a rule declared with
4165 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
4167 sumes(b3)))
4168 will consume a value from Bundle b2 and several values from Bun‐
4170 dle b3 to populate value2 and value3 each time it is executed.
4171 hypothesis.stateful.multiple(*args)
4173 This function can be used to pass multiple results to the tar‐
4174 get(s) of a rule. Just use return multiple(result1, result2,
4175 ...) in your rule.
4176 It is also possible to use return multiple() with no arguments
4178 in order to end a rule without passing any result.
4179 Initializes
4181 Initializes are a special case of rules that are guaranteed to be run
4182 at most once at the beginning of a run (i.e. before any normal rule is
4183 called). Note if multiple initialize rules are defined, they may be
4184 called in any order, and that order will vary from run to run.
4185 Initializes are typically useful to populate bundles:
4187 hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)
4189 Decorator for RuleBasedStateMachine.
4190 An initialize decorator behaves like a rule, but all @initial‐
4192 ize() decorated methods will be called before any @rule() deco‐
4193 rated methods, in an arbitrary order. Each @initialize() method
4194 will be called exactly once per run, unless one raises an excep‐
4195 tion - after which only the .teardown() method will be run.
4196 @initialize() methods may not have preconditions.
4197 import hypothesis.strategies as st
4199 from hypothesis.stateful import Bundle, RuleBasedStateMachine, initialize, rule
4200 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
4202 class NumberModifier(RuleBasedStateMachine):
4205 folders = Bundle("folders")
4207 files = Bundle("files")
4208 @initialize(target=folders)
4210 def init_folders(self):
4211 return "/"
4212 @rule(target=folders, name=name_strategy)
4214 def create_folder(self, parent, name):
4215 return f"{parent}/{name}"
4216 @rule(target=files, name=name_strategy)
4218 def create_file(self, parent, name):
4219 return f"{parent}/{name}"
4220 Preconditions
4222 While it's possible to use assume() in RuleBasedStateMachine rules, if
4223 you use it in only a few rules you can quickly run into a situation
4224 where few or none of your rules pass their assumptions. Thus, Hypothe‐
4225 sis provides a precondition() decorator to avoid this problem. The
4226 precondition() decorator is used on rule-decorated functions, and must
4227 be given a function that returns True or False based on the RuleBased‐
4228 StateMachine instance.
4229 hypothesis.stateful.precondition(precond)
4231 Decorator to apply a precondition for rules in a RuleBased‐
4232 StateMachine. Specifies a precondition for a rule to be consid‐
4233 ered as a valid step in the state machine, which is more effi‐
4234 cient than using assume() within the rule. The precond function
4235 will be called with the instance of RuleBasedStateMachine and
4236 should return True or False. Usually it will need to look at at‐
4237 tributes on that instance.
4238 For example:
4240 class MyTestMachine(RuleBasedStateMachine):
4242 state = 1
4243 @precondition(lambda self: self.state != 0)
4245 @rule(numerator=integers())
4246 def divide_with(self, numerator):
4247 self.state = numerator / self.state
4248 If multiple preconditions are applied to a single rule, it is
4250 only considered a valid step when all of them return True. Pre‐
4251 conditions may be applied to invariants as well as rules.
4252 from hypothesis.stateful import RuleBasedStateMachine, precondition, rule
4254 class NumberModifier(RuleBasedStateMachine):
4257 num = 0
4259 @rule()
4261 def add_one(self):
4262 self.num += 1
4263 @precondition(lambda self: self.num != 0)
4265 @rule()
4266 def divide_with_one(self):
4267 self.num = 1 / self.num
4268 By using precondition() here instead of assume(), Hypothesis can filter
4270 the inapplicable rules before running them. This makes it much more
4271 likely that a useful sequence of steps will be generated.
4272 Note that currently preconditions can't access bundles; if you need to
4274 use preconditions, you should store relevant data on the instance in‐
4275 stead.
4276 Invariants
4278 Often there are invariants that you want to ensure are met after every
4279 step in a process. It would be possible to add these as rules that are
4280 run, but they would be run zero or multiple times between other rules.
4281 Hypothesis provides a decorator that marks a function to be run after
4282 every step.
4283 hypothesis.stateful.invariant(*, check_during_init=False)
4285 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
4286 chine. The decorated function will be run after every rule and
4287 can raise an exception to indicate failed invariants.
4288 For example:
4290 class MyTestMachine(RuleBasedStateMachine):
4292 state = 1
4293 @invariant()
4295 def is_nonzero(self):
4296 assert self.state != 0
4297 By default, invariants are only checked after all @initialize()
4299 rules have been run. Pass check_during_init=True for invariants
4300 which can also be checked during initialization.
4301 from hypothesis.stateful import RuleBasedStateMachine, invariant, rule
4303 class NumberModifier(RuleBasedStateMachine):
4306 num = 0
4308 @rule()
4310 def add_two(self):
4311 self.num += 2
4312 if self.num > 50:
4313 self.num += 1
4314 @invariant()
4316 def divide_with_one(self):
4317 assert self.num % 2 == 0
4318 NumberTest = NumberModifier.TestCase
4321 Invariants can also have precondition()s applied to them, in which case
4323 they will only be run if the precondition function returns true.
4324 Note that currently invariants can't access bundles; if you need to use
4326 invariants, you should store relevant data on the instance instead.
4327 More fine grained control
4329 If you want to bypass the TestCase infrastructure you can invoke these
4330 manually. The stateful module exposes the function run_state_ma‐
4331 chine_as_test, which takes an arbitrary function returning a RuleBased‐
4332 StateMachine and an optional settings parameter and does the same as
4333 the class based runTest provided.
4334 This is not recommended as it bypasses some important internal func‐
4336 tions, including reporting of statistics such as runtimes and event()
4337 calls. It was originally added to support custom __init__ methods, but
4338 you can now use initialize() rules instead.
4339
4341 Hypothesis does its level best to be compatible with everything you
4342 could possibly need it to be compatible with. Generally you should just
4343 try it and expect it to work. If it doesn't, you can be surprised and
4344 check this document for the details.
4345 Hypothesis versions
4347 Backwards compatibility is better than backporting fixes, so we use se‐
4348 mantic versioning and only support the most recent version of Hypothe‐
4349 sis. See support for more information.
4350 Documented APIs will not break except between major version bumps. All
4352 APIs mentioned in this documentation are public unless explicitly noted
4353 as provisional, in which case they may be changed in minor releases.
4354 Undocumented attributes, modules, and behaviour may include breaking
4355 changes in patch releases.
4356 Python versions
4358 Hypothesis is supported and tested on CPython 3.6+, i.e. all versions
4359 of CPython with upstream support,
4360 Hypothesis also supports the latest PyPy for Python 3.6. 32-bit builds
4362 of CPython also work, though they are currently only tested on Windows.
4363 In general Hypothesis does not officially support anything except the
4365 latest patch release of any version of Python it supports. Earlier re‐
4366 leases should work and bugs in them will get fixed if reported, but
4367 they're not tested in CI and no guarantees are made.
4368 Operating systems
4370 In theory Hypothesis should work anywhere that Python does. In practice
4371 it is only known to work and regularly tested on OS X, Windows and
4372 Linux, and you may experience issues running it elsewhere.
4373 If you're using something else and it doesn't work, do get in touch and
4375 I'll try to help, but unless you can come up with a way for me to run a
4376 CI server on that operating system it probably won't stay fixed due to
4377 the inevitable march of time.
4378 Testing frameworks
4380 In general Hypothesis goes to quite a lot of effort to generate things
4381 that look like normal Python test functions that behave as closely to
4382 the originals as possible, so it should work sensibly out of the box
4383 with every test framework.
4384 If your testing relies on doing something other than calling a function
4386 and seeing if it raises an exception then it probably won't work out of
4387 the box. In particular things like tests which return generators and
4388 expect you to do something with them (e.g. nose's yield based tests)
4389 will not work. Use a decorator or similar to wrap the test to take this
4390 form, or ask the framework maintainer to support our hooks for insert‐
4391 ing such a wrapper later.
4392 In terms of what's actually known to work:
4394 • Hypothesis integrates as smoothly with pytest and unittest as we
4396 can make it, and this is verified as part of the CI.
4397 • pytest fixtures work in the usual way for tests that have been
4399 decorated with @given - just avoid passing a strategy for each ar‐
4400 gument that will be supplied by a fixture. However, each fixture
4401 will run once for the whole function, not once per example. Deco‐
4402 rating a fixture function with @given is meaningless.
4403 • The python:unittest.mock.patch() decorator works with @given, but
4405 we recommend using it as a context manager within the decorated
4406 test to ensure that the mock is per-test-case and avoid poor in‐
4407 teractions with Pytest fixtures.
4408 • Nose works fine with Hypothesis, and this is tested as part of the
4410 CI. yield based tests simply won't work.
4411 • Integration with Django's testing requires use of the hypothe‐
4413 sis-django package. The issue is that in Django's tests' normal
4414 mode of execution it will reset the database once per test rather
4415 than once per example, which is not what you want.
4416 • Coverage works out of the box with Hypothesis; our own test suite
4418 has 100% branch coverage.
4419 Optional packages
4421 The supported versions of optional packages, for strategies in hypothe‐
4422 sis.extra, are listed in the documentation for that extra. Our general
4423 goal is to support all versions that are supported upstream.
4424 Regularly verifying this
4426 Everything mentioned above as explicitly supported is checked on every
4427 commit with GitHub Actions. Our continuous delivery pipeline runs all
4428 of these checks before publishing each release, so when we say they're
4429 supported we really mean it.
4430
4432 This is a collection of examples of how to use Hypothesis in interest‐
4433 ing ways. It's small for now but will grow over time.
4434 All of these examples are designed to be run under pytest, and nose
4436 should work too.
4437 How not to sort by a partial order
4439 The following is an example that's been extracted and simplified from a
4440 real bug that occurred in an earlier version of Hypothesis. The real
4441 bug was a lot harder to find.
4442 Suppose we've got the following type:
4444 class Node:
4446 def __init__(self, label, value):
4447 self.label = label
4448 self.value = tuple(value)
4449 def __repr__(self):
4451 return f"Node({self.label!r}, {self.value!r})"
4452 def sorts_before(self, other):
4454 if len(self.value) >= len(other.value):
4455 return False
4456 return other.value[: len(self.value)] == self.value
4457 Each node is a label and a sequence of some data, and we have the rela‐
4459 tionship sorts_before meaning the data of the left is an initial seg‐
4460 ment of the right. So e.g. a node with value [1, 2] will sort before a
4461 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
4462 before the other.
4463 We have a list of nodes, and we want to topologically sort them with
4465 respect to this ordering. That is, we want to arrange the list so that
4466 if x.sorts_before(y) then x appears earlier in the list than y. We
4467 naively think that the easiest way to do this is to extend the partial
4468 order defined here to a total order by breaking ties arbitrarily and
4469 then using a normal sorting algorithm. So we define the following code:
4470 from functools import total_ordering
4472 @total_ordering
4475 class TopoKey:
4476 def __init__(self, node):
4477 self.value = node
4478 def __lt__(self, other):
4480 if self.value.sorts_before(other.value):
4481 return True
4482 if other.value.sorts_before(self.value):
4483 return False
4484 return self.value.label < other.value.label
4486 def sort_nodes(xs):
4489 xs.sort(key=TopoKey)
4490 This takes the order defined by sorts_before and extends it by breaking
4492 ties by comparing the node labels.
4493 But now we want to test that it works.
4495 First we write a function to verify that our desired outcome holds:
4497 def is_prefix_sorted(xs):
4499 for i in range(len(xs)):
4500 for j in range(i + 1, len(xs)):
4501 if xs[j].sorts_before(xs[i]):
4502 return False
4503 return True
4504 This will return false if it ever finds a pair in the wrong order and
4506 return true otherwise.
4507 Given this function, what we want to do with Hypothesis is assert that
4509 for all sequences of nodes, the result of calling sort_nodes on it is
4510 sorted.
4511 First we need to define a strategy for Node:
4513 import hypothesis.strategies as st
4515 NodeStrategy = st.builds(Node, st.integers(), st.lists(st.booleans(), max_size=10))
4517 We want to generate short lists of values so that there's a decent
4519 chance of one being a prefix of the other (this is also why the choice
4520 of bool as the elements). We then define a strategy which builds a node
4521 out of an integer and one of those short lists of booleans.
4522 We can now write a test:
4524 from hypothesis import given
4526 @given(st.lists(NodeStrategy))
4529 def test_sorting_nodes_is_prefix_sorted(xs):
4530 sort_nodes(xs)
4531 assert is_prefix_sorted(xs)
4532 this immediately fails with the following example:
4534 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
4536 The reason for this is that because False is not a prefix of (True,
4538 True) nor vice versa, sorting things the first two nodes are equal be‐
4539 cause they have equal labels. This makes the whole order non-transi‐
4540 tive and produces basically nonsense results.
4541 But this is pretty unsatisfying. It only works because they have the
4543 same label. Perhaps we actually wanted our labels to be unique. Let's
4544 change the test to do that.
4545 def deduplicate_nodes_by_label(nodes):
4547 table = {node.label: node for node in nodes}
4548 return list(table.values())
4549 We define a function to deduplicate nodes by labels, and can now map
4551 that over a strategy for lists of nodes to give us a strategy for lists
4552 of nodes with unique labels:
4553 @given(st.lists(NodeStrategy).map(deduplicate_nodes_by_label))
4555 def test_sorting_nodes_is_prefix_sorted(xs):
4556 sort_nodes(xs)
4557 assert is_prefix_sorted(xs)
4558 Hypothesis quickly gives us an example of this still being wrong:
4560 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]
4562 Now this is a more interesting example. None of the nodes will sort
4564 equal. What is happening here is that the first node is strictly less
4565 than the last node because (False,) is a prefix of (False, False). This
4566 is in turn strictly less than the middle node because neither is a pre‐
4567 fix of the other and -2 < -1. The middle node is then less than the
4568 first node because -1 < 0.
4569 So, convinced that our implementation is broken, we write a better one:
4571 def sort_nodes(xs):
4573 for i in range(1, len(xs)):
4574 j = i - 1
4575 while j >= 0:
4576 if xs[j].sorts_before(xs[j + 1]):
4577 break
4578 xs[j], xs[j + 1] = xs[j + 1], xs[j]
4579 j -= 1
4580 This is just insertion sort slightly modified - we swap a node back‐
4582 wards until swapping it further would violate the order constraints.
4583 The reason this works is because our order is a partial order already
4584 (this wouldn't produce a valid result for a general topological sorting
4585 - you need the transitivity).
4586 We now run our test again and it passes, telling us that this time
4588 we've successfully managed to sort some nodes without getting it com‐
4589 pletely wrong. Go us.
4590 Time zone arithmetic
4592 This is an example of some tests for pytz which check that various
4593 timezone conversions behave as you would expect them to. These tests
4594 should all pass, and are mostly a demonstration of some useful sorts of
4595 thing to test with Hypothesis, and how the datetimes() strategy works.
4596 from datetime import timedelta
4598 # The datetimes strategy is naive by default, so tell it to use timezones
4600 aware_datetimes = st.datetimes(timezones=st.timezones())
4601 @given(aware_datetimes, st.timezones(), st.timezones())
4604 def test_convert_via_intermediary(dt, tz1, tz2):
4605 """Test that converting between timezones is not affected
4606 by a detour via another timezone.
4607 """
4608 assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
4609 @given(aware_datetimes, st.timezones())
4612 def test_convert_to_and_fro(dt, tz2):
4613 """If we convert to a new timezone and back to the old one
4614 this should leave the result unchanged.
4615 """
4616 tz1 = dt.tzinfo
4617 assert dt == dt.astimezone(tz2).astimezone(tz1)
4618 @given(aware_datetimes, st.timezones())
4621 def test_adding_an_hour_commutes(dt, tz):
4622 """When converting between timezones it shouldn't matter
4623 if we add an hour here or add an hour there.
4624 """
4625 an_hour = timedelta(hours=1)
4626 assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
4627 @given(aware_datetimes, st.timezones())
4630 def test_adding_a_day_commutes(dt, tz):
4631 """When converting between timezones it shouldn't matter
4632 if we add a day here or add a day there.
4633 """
4634 a_day = timedelta(days=1)
4635 assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
4636 Condorcet's paradox
4638 A classic paradox in voting theory, called Condorcet's paradox, is that
4639 majority preferences are not transitive. That is, there is a population
4640 and a set of three candidates A, B and C such that the majority of the
4641 population prefer A to B, B to C and C to A.
4642 Wouldn't it be neat if we could use Hypothesis to provide an example of
4644 this?
4645 Well as you can probably guess from the presence of this section, we
4647 can! The main trick is to decide how we want to represent the result
4648 of an election - for this example, we'll use a list of "votes", where
4649 each vote is a list of candidates in the voters preferred order. With‐
4650 out further ado, here is the code:
4651 from collections import Counter
4653 from hypothesis import given
4655 from hypothesis.strategies import lists, permutations
4656 # We need at least three candidates and at least three voters to have a
4659 # paradox; anything less can only lead to victories or at worst ties.
4660 @given(lists(permutations(["A", "B", "C"]), min_size=3))
4661 def test_elections_are_transitive(election):
4662 all_candidates = {"A", "B", "C"}
4663 # First calculate the pairwise counts of how many prefer each candidate
4665 # to the other
4666 counts = Counter()
4667 for vote in election:
4668 for i in range(len(vote)):
4669 for j in range(i + 1, len(vote)):
4670 counts[(vote[i], vote[j])] += 1
4671 # Now look at which pairs of candidates one has a majority over the
4673 # other and store that.
4674 graph = {}
4675 for i in all_candidates:
4676 for j in all_candidates:
4677 if counts[(i, j)] > counts[(j, i)]:
4678 graph.setdefault(i, set()).add(j)
4679 # Now for each triple assert that it is transitive.
4681 for x in all_candidates:
4682 for y in graph.get(x, ()):
4683 for z in graph.get(y, ()):
4684 assert x not in graph.get(z, ())
4685 The example Hypothesis gives me on my first run (your mileage may of
4687 course vary) is:
4688 [["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]
4690 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
4692 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
4693 and 2) prefer C to A. This is in fact basically the canonical example
4694 of the voting paradox.
4695 Fuzzing an HTTP API
4697 Hypothesis's support for testing HTTP services is somewhat nascent.
4698 There are plans for some fully featured things around this, but right
4699 now they're probably quite far down the line.
4700 But you can do a lot yourself without any explicit support! Here's a
4702 script I wrote to throw arbitrary data against the API for an entirely
4703 fictitious service called Waspfinder (this is only lightly obfuscated
4704 and you can easily figure out who I'm actually talking about, but I
4705 don't want you to run this code and hammer their API without their per‐
4706 mission).
4707 All this does is use Hypothesis to generate arbitrary JSON data match‐
4709 ing the format their API asks for and check for 500 errors. More ad‐
4710 vanced tests which then use the result and go on to do other things are
4711 definitely also possible. The schemathesis package provides an excel‐
4712 lent example of this!
4713 import math
4715 import os
4716 import random
4717 import time
4718 import unittest
4719 from collections import namedtuple
4720 import requests
4722 from hypothesis import assume, given, strategies as st
4724 Goal = namedtuple("Goal", ("slug",))
4726 # We just pass in our API credentials via environment variables.
4729 waspfinder_token = os.getenv("WASPFINDER_TOKEN")
4730 waspfinder_user = os.getenv("WASPFINDER_USER")
4731 assert waspfinder_token is not None
4732 assert waspfinder_user is not None
4733 GoalData = st.fixed_dictionaries(
4735 {
4736 "title": st.text(),
4737 "goal_type": st.sampled_from(
4738 ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
4739 ),
4740 "goaldate": st.one_of(st.none(), st.floats()),
4741 "goalval": st.one_of(st.none(), st.floats()),
4742 "rate": st.one_of(st.none(), st.floats()),
4743 "initval": st.floats(),
4744 "panic": st.floats(),
4745 "secret": st.booleans(),
4746 "datapublic": st.booleans(),
4747 }
4748 )
4749 needs2 = ["goaldate", "goalval", "rate"]
4752 class WaspfinderTest(unittest.TestCase):
4755 @given(GoalData)
4756 def test_create_goal_dry_run(self, data):
4757 # We want slug to be unique for each run so that multiple test runs
4758 # don't interfere with each other. If for some reason some slugs trigger
4759 # an error and others don't we'll get a Flaky error, but that's OK.
4760 slug = hex(random.getrandbits(32))[2:]
4761 # Use assume to guide us through validation we know about, otherwise
4763 # we'll spend a lot of time generating boring examples.
4764 # Title must not be empty
4766 assume(data["title"])
4767 # Exactly two of these values should be not None. The other will be
4769 # inferred by the API.
4770 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
4772 for v in data.values():
4773 if isinstance(v, float):
4774 assume(not math.isnan(v))
4775 data["slug"] = slug
4776 # The API nicely supports a dry run option, which means we don't have
4778 # to worry about the user account being spammed with lots of fake goals
4779 # Otherwise we would have to make sure we cleaned up after ourselves
4780 # in this test.
4781 data["dryrun"] = True
4782 data["auth_token"] = waspfinder_token
4783 for d, v in data.items():
4784 if v is None:
4785 data[d] = "null"
4786 else:
4787 data[d] = str(v)
4788 result = requests.post(
4789 "https://waspfinder.example.com/api/v1/users/"
4790 "%s/goals.json" % (waspfinder_user,),
4791 data=data,
4792 )
4793 # Let's not hammer the API too badly. This will of course make the
4795 # tests even slower than they otherwise would have been, but that's
4796 # life.
4797 time.sleep(1.0)
4798 # For the moment all we're testing is that this doesn't generate an
4800 # internal error. If we didn't use the dry run option we could have
4801 # then tried doing more with the result, but this is a good start.
4802 self.assertNotEqual(result.status_code, 500)
4803 if __name__ == "__main__":
4806 unittest.main()
4807
4809 The Hypothesis community is small for the moment but is full of excel‐
4810 lent people who can answer your questions and help you out. Please do
4811 join us. The two major places for community discussion are:
4812 • The mailing list.
4814 • An IRC channel, #hypothesis on freenode, which is more active than
4816 the mailing list.
4817 Feel free to use these to ask for help, provide feedback, or discuss
4819 anything remotely Hypothesis related at all. If you post a question on
4820 Stack Overflow, please use the python-hypothesis tag!
4821 Please note that the Hypothesis code of conduct applies in all Hypothe‐
4823 sis community spaces.
4824 If you would like to cite Hypothesis, please consider our suggested ci‐
4826 tation.
4827 If you like repo badges, we suggest the following badge, which you can
4829 add with reStructuredText or Markdown, respectively: [image]
4830 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
4832 :alt: Tested with Hypothesis
4833 :target: https://hypothesis.readthedocs.io
4834 [](https://hypothesis.readthedocs.io/)
4836 Finally, we have a beautiful logo which appears online, and often on
4838 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
4839 [image]
4840 As well as being beautiful, dragonflies actively hunt down bugs for a
4842 living! You can find the images and a usage guide in the brand direc‐
4843 tory on GitHub, or find us at conferences where we often have stickers
4844 and sometimes other swag.
4845
4847 What is Hypothesis for?
4848 From the perspective of a user, the purpose of Hypothesis is to make it
4850 easier for you to write better tests.
4851 From my perspective as the author, that is of course also a purpose of
4853 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
4854 mania for a moment), the larger purpose of Hypothesis is to drag the
4855 world kicking and screaming into a new and terrifying age of high qual‐
4856 ity software.
4857 Software is, as they say, eating the world. Software is also terrible.
4859 It's buggy, insecure and generally poorly thought out. This combination
4860 is clearly a recipe for disaster.
4861 And the state of software testing is even worse. Although it's fairly
4863 uncontroversial at this point that you should be testing your code, can
4864 you really say with a straight face that most projects you've worked on
4865 are adequately tested?
4866 A lot of the problem here is that it's too hard to write good tests.
4868 Your tests encode exactly the same assumptions and fallacies that you
4869 had when you wrote the code, so they miss exactly the same bugs that
4870 you missed when you wrote the code.
4871 Meanwhile, there are all sorts of tools for making testing better that
4873 are basically unused. The original Quickcheck is from 1999 and the ma‐
4874 jority of developers have not even heard of it, let alone used it.
4875 There are a bunch of half-baked implementations for most languages, but
4876 very few of them are worth using.
4877 The goal of Hypothesis is to bring advanced testing techniques to the
4879 masses, and to provide an implementation that is so high quality that
4880 it is easier to use them than it is not to use them. Where I can, I
4881 will beg, borrow and steal every good idea I can find that someone has
4882 had to make software testing better. Where I can't, I will invent new
4883 ones.
4884 Quickcheck is the start, but I also plan to integrate ideas from fuzz
4886 testing (a planned future feature is to use coverage information to
4887 drive example selection, and the example saving database is already in‐
4888 spired by the workflows people use for fuzz testing), and am open to
4889 and actively seeking out other suggestions and ideas.
4890 The plan is to treat the social problem of people not using these ideas
4892 as a bug to which there is a technical solution: Does property-based
4893 testing not match your workflow? That's a bug, let's fix it by figur‐
4894 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
4895 tom data for your application? That's a bug. Let's fix it by figuring
4896 out how to make it easier, or how to take something you're already us‐
4897 ing to specify your data and derive a generator from that automati‐
4898 cally. Find the explanations of these advanced ideas hopelessly obtuse
4899 and hard to follow? That's a bug. Let's provide you with an easy API
4900 that lets you test your code better without a PhD in software verifica‐
4901 tion.
4902 Grand ambitions, I know, and I expect ultimately the reality will be
4904 somewhat less grand, but so far in about three months of development,
4905 Hypothesis has become the most solid implementation of Quickcheck ever
4906 seen in a mainstream language (as long as we don't count Scala as main‐
4907 stream yet), and at the same time managed to significantly push forward
4908 the state of the art, so I think there's reason to be optimistic.
4909
4911 This is a page for listing people who are using Hypothesis and how ex‐
4912 cited they are about that. If that's you and your name is not on the
4913 list, this file is in Git and I'd love it if you sent me a pull request
4914 to fix that.
4915 Stripe
4917 At Stripe we use Hypothesis to test every piece of our machine learning
4918 model training pipeline (powered by scikit). Before we migrated, our
4919 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
4920 resentative at all of our actual very complex data. Because we needed
4921 to craft examples for each test, we took the easy way out and lived
4922 with extremely low test coverage.
4923 Hypothesis changed all that. Once we had our strategies for generating
4925 Dataframes of features it became trivial to slightly customize each
4926 strategy for new tests. Our coverage is now close to 90%.
4927 Full-stop, property-based testing is profoundly more powerful - and has
4929 caught or prevented far more bugs - than our old style of example-based
4930 testing.
4931 Kristian Glass - Director of Technology at LaterPay GmbH
4933 Hypothesis has been brilliant for expanding the coverage of our test
4934 cases, and also for making them much easier to read and understand, so
4935 we're sure we're testing the things we want in the way we want.
4936 Seth Morton
4938 When I first heard about Hypothesis, I knew I had to include it in my
4939 two open-source Python libraries, natsort and fastnumbers . Quite
4940 frankly, I was a little appalled at the number of bugs and "holes" I
4941 found in the code. I can now say with confidence that my libraries are
4942 more robust to "the wild." In addition, Hypothesis gave me the confi‐
4943 dence to expand these libraries to fully support Unicode input, which I
4944 never would have had the stomach for without such thorough testing ca‐
4945 pabilities. Thanks!
4946 Sixty North
4948 At Sixty North we use Hypothesis for testing Segpy an open source
4949 Python library for shifting data between Python data structures and SEG
4950 Y files which contain geophysical data from the seismic reflection sur‐
4951 veys used in oil and gas exploration.
4952 This is our first experience of property-based testing – as opposed to
4954 example-based testing. Not only are our tests more powerful, they are
4955 also much better explanations of what we expect of the production code.
4956 In fact, the tests are much closer to being specifications. Hypothesis
4957 has located real defects in our code which went undetected by tradi‐
4958 tional test cases, simply because Hypothesis is more relentlessly devi‐
4959 ous about test case generation than us mere humans! We found Hypothe‐
4960 sis particularly beneficial for Segpy because SEG Y is an antiquated
4961 format that uses legacy text encodings (EBCDIC) and even a legacy
4962 floating point format we implemented from scratch in Python.
4963 Hypothesis is sure to find a place in most of our future Python code‐
4965 bases and many existing ones too.
4966 mulkieran
4968 Just found out about this excellent QuickCheck for Python implementa‐
4969 tion and ran up a few tests for my bytesize package last night. Refuted
4970 a few hypotheses in the process.
4971 Looking forward to using it with a bunch of other projects as well.
4973 Adam Johnson
4975 I have written a small library to serialize dicts to MariaDB's dynamic
4976 columns binary format, mariadb-dyncol. When I first developed it, I
4977 thought I had tested it really well - there were hundreds of test
4978 cases, some of them even taken from MariaDB's test suite itself. I was
4979 ready to release.
4980 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
4982 Wow! It found bug after bug after bug. Even after a first release, I
4983 thought of a way to make the tests do more validation, which revealed a
4984 further round of bugs! Most impressively, Hypothesis found a compli‐
4985 cated off-by-one error in a condition with 4095 versus 4096 bytes of
4986 data - something that I would never have found.
4987 Long live Hypothesis! (Or at least, property-based testing).
4989 Josh Bronson
4991 Adopting Hypothesis improved bidict's test coverage and significantly
4992 increased our ability to make changes to the code with confidence that
4993 correct behavior would be preserved. Thank you, David, for the great
4994 testing tool.
4995 Cory Benfield
4997 Hypothesis is the single most powerful tool in my toolbox for working
4998 with algorithmic code, or any software that produces predictable output
4999 from a wide range of sources. When using it with Priority, Hypothesis
5000 consistently found errors in my assumptions and extremely subtle bugs
5001 that would have taken months of real-world use to locate. In some
5002 cases, Hypothesis found subtle deviations from the correct output of
5003 the algorithm that may never have been noticed at all.
5004 When it comes to validating the correctness of your tools, nothing
5006 comes close to the thoroughness and power of Hypothesis.
5007 Jon Moore
5009 One extremely satisfied user here. Hypothesis is a really solid imple‐
5010 mentation of property-based testing, adapted well to Python, and with
5011 good features such as failure-case shrinkers. I first used it on a
5012 project where we needed to verify that a vendor's Python and non-Python
5013 implementations of an algorithm matched, and it found about a dozen
5014 cases that previous example-based testing and code inspections had not.
5015 Since then I've been evangelizing for it at our firm.
5016 Russel Winder
5018 I am using Hypothesis as an integral part of my Python workshops. Test‐
5019 ing is an integral part of Python programming and whilst unittest and,
5020 better, pytest can handle example-based testing, property-based testing
5021 is increasingly far more important than example-base testing, and Hy‐
5022 pothesis fits the bill.
5023 Wellfire Interactive
5025 We've been using Hypothesis in a variety of client projects, from test‐
5026 ing Django-related functionality to domain-specific calculations. It
5027 both speeds up and simplifies the testing process since there's so much
5028 less tedious and error-prone work to do in identifying edge cases. Test
5029 coverage is nice but test depth is even nicer, and it's much easier to
5030 get meaningful test depth using Hypothesis.
5031 Cody Kochmann
5033 Hypothesis is being used as the engine for random object generation
5034 with my open source function fuzzer battle_tested which maps all behav‐
5035 iors of a function allowing you to minimize the chance of unexpected
5036 crashes when running code in production.
5037 With how efficient Hypothesis is at generating the edge cases that
5039 cause unexpected behavior occur, battle_tested is able to map out the
5040 entire behavior of most functions in less than a few seconds.
5041 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
5043 ing it.
5044 Merchise Autrement
5046 Just minutes after our first use of hypothesis we uncovered a subtle
5047 bug in one of our most used library. Since then, we have increasingly
5048 used hypothesis to improve the quality of our testing in libraries and
5049 applications as well.
5050 Florian Kromer
5052 At Roboception GmbH I use Hypothesis to implement fully automated
5053 stateless and stateful reliability tests for the 3D sensor rc_visard
5054 and robotic software components .
5055 Thank you very much for creating the (probably) most powerful prop‐
5057 erty-based testing framework.
5058 Reposit Power
5060 With a micro-service architecture, testing between services is made
5061 easy using Hypothesis in integration testing. Ensuring everything is
5062 running smoothly is vital to help maintain a secure network of Virtual
5063 Power Plants.
5064 It allows us to find potential bugs and edge cases with relative ease
5066 and minimal overhead. As our architecture relies on services communi‐
5067 cating effectively, Hypothesis allows us to strictly test for the kind
5068 of data which moves around our services, particularly our backend
5069 Python applications.
5070 Your name goes here
5072 I know there are many more, because I keep finding out about new people
5073 I'd never even heard of using Hypothesis. If you're looking to way to
5074 give back to a tool you love, adding your name here only takes a moment
5075 and would really help a lot. As per instructions at the top, just send
5076 me a pull request and I'll add you to the list.
5077
5079 The following is a non-exhaustive list of open source projects I know
5080 are using Hypothesis. If you're aware of any others please add them to
5081 the list! The only inclusion criterion right now is that if it's a
5082 Python library then it should be available on PyPI.
5083 You can find hundreds more from the Hypothesis page at libraries.io,
5085 and thousands on GitHub. Hypothesis has over 100,000 downloads per
5086 week, and was used by more than 4% of Python users surveyed by the PSF
5087 in 2020.
5088 • aur
5090 • argon2_cffi
5092 • attrs
5094 • axelrod
5096 • bidict
5098 • binaryornot
5100 • brotlicffi
5102 • chardet
5104 • cmph-cffi
5106 • cryptography
5108 • dbus-signature-pyparsing
5110 • dry-python/returns
5112 • fastnumbers
5114 • flocker
5116 • flownetpy
5118 • funsize
5120 • fusion-index
5122 • hyper-h2
5124 • into-dbus-python
5126 • justbases
5128 • justbytes
5130 • loris
5132 • mariadb-dyncol
5134 • mercurial
5136 • natsort
5138 • poliastro
5140 • pretext
5142 • priority
5144 • PyCEbox
5146 • PyPy
5148 • pyrsistent
5150 • python-humble-utils
5152 • pyudev
5154 • qutebrowser
5156 • RubyMarshal
5158 • Segpy
5160 • simoa
5162 • srt
5164 • tchannel
5166 • vdirsyncer
5168 • wcag-contrast-ratio
5170 • yacluster
5172 • yturl
5174
5176 Hypothesis has been eagerly used and extended by the open source commu‐
5177 nity. This page lists extensions and applications; you can find more
5178 or newer packages by searching PyPI by keyword or filter by classifier,
5179 or search libraries.io.
5180 If there's something missing which you think should be here, let us
5182 know!
5183 NOTE:
5185 Being listed on this page does not imply that the Hypothesis main‐
5186 tainers endorse a package.
5187 External strategies
5189 Some packages provide strategies directly:
5190 • hypothesis-fspaths - strategy to generate filesystem paths.
5192 • hypothesis-geojson - strategy to generate GeoJson.
5194 • hypothesis-geometry - strategies to generate geometric objects.
5196 • hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
5198 • hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
5200 • hypothesis-ros - strategies to generate messages and parameters for
5202 the Robot Operating System.
5203 • hypothesis-csv - strategy to generate CSV files.
5205 • hypothesis-networkx - strategy to generate networkx graphs.
5207 • hypothesis-bio - strategies for bioinformatics data, such as DNA,
5209 codons, FASTA, and FASTQ formats.
5210 • hypothesmith - strategy to generate syntatically-valid Python code.
5212 Others provide a function to infer a strategy from some other schema:
5214 • hypothesis-jsonschema - infer strategies from JSON schemas.
5216 • lollipop-hypothesis - infer strategies from lollipop schemas.
5218 • hypothesis-drf - infer strategies from a djangorestframework seri‐
5220 aliser.
5221 • hypothesis-graphql - infer strategies from GraphQL schemas.
5223 • hypothesis-mongoengine - infer strategies from a mongoengine model.
5225 • hypothesis-pb - infer strategies from Protocol Buffer schemas.
5227 Or some other custom integration, such as a "hypothesis" entry point:
5229 • deal is a design-by-contract library with built-in Hypothesis sup‐
5231 port.
5232 • icontract-hypothesis infers strategies from icontract code contracts.
5234 • Pandera schemas all have a .strategy() method, which returns a strat‐
5236 egy for matching DataFrames.
5237 • Pydantic automatically registers constrained types - so builds() and
5239 from_type() "just work" regardless of the underlying implementation.
5240 Other cool things
5242 schemathesis is a tool for testing web applications built with Open API
5243 / Swagger specifications. It reads the schema and generates test cases
5244 which will ensure that the application is compliant with its schema.
5245 The application under test could be written in any language, the only
5246 thing you need is a valid API schema in a supported format. Includes
5247 CLI and convenient pytest integration. Powered by Hypothesis and
5248 hypothesis-jsonschema, inspired by the earlier swagger-conformance li‐
5249 brary.
5250 Trio is an async framework with "an obsessive focus on usability and
5252 correctness", so naturally it works with Hypothesis! pytest-trio in‐
5253 cludes a custom hook that allows @given(...) to work with Trio-style
5254 async test functions, and hypothesis-trio includes stateful testing ex‐
5255 tensions to support concurrent programs.
5256 pymtl3 is "an open-source Python-based hardware generation, simulation,
5258 and verification framework with multi-level hardware modeling support",
5259 which ships with Hypothesis integrations to check that all of those
5260 levels are equivalent, from function-level to register-transfer level
5261 and even to hardware.
5262 libarchimedes makes it easy to use Hypothesis in the Hy language, a
5264 Lisp embedded in Python.
5265 battle_tested is a fuzzing tool that will show you how your code can
5267 fail - by trying all kinds of inputs and reporting whatever happens.
5268 pytest-subtesthack functions as a workaround for issue #377.
5270 returns uses Hypothesis to verify that Higher Kinded Types correctly
5272 implement functor, applicative, monad, and other laws; allowing a
5273 declarative approach to be combined with traditional pythonic code.
5274 icontract-hypothesis includes a ghostwriter for test files and IDE in‐
5276 tegrations such as icontract-hypothesis-vim,
5277 icontract-hypothesis-pycharm, and icontract-hypothesis-vscode - you can
5278 run a quick 'smoke test' with only a few keystrokes for any type-anno‐
5279 tated function, even if it doesn't have any contracts!
5280 Writing an extension
5282 See CONTRIBUTING.rst for more information.
5283 New strategies can be added to Hypothesis, or published as an external
5285 package on PyPI - either is fine for most strategies. If in doubt, ask!
5286 It's generally much easier to get things working outside, because
5288 there's more freedom to experiment and fewer requirements in stability
5289 and API style. We're happy to review and help with external packages as
5290 well as pull requests!
5291 If you're thinking about writing an extension, please name it hypothe‐
5293 sis-{something} - a standard prefix makes the community more visible
5294 and searching for extensions easier. And make sure you use the Frame‐
5295 work :: Hypothesis trove classifier!
5296 On the other hand, being inside gets you access to some deeper imple‐
5298 mentation features (if you need them) and better long-term guarantees
5299 about maintenance. We particularly encourage pull requests for new
5300 composable primitives that make implementing other strategies easier,
5301 or for widely used types in the standard library. Strategies for other
5302 things are also welcome; anything with external dependencies just goes
5303 in hypothesis.extra.
5304 Tools such as assertion helpers may also need to check whether the cur‐
5306 rent test is using Hypothesis:
5307 hypothesis.currently_in_test_context()
5309 Return True if the calling code is currently running inside an
5310 @given or stateful test, False otherwise.
5311 This is useful for third-party integrations and assertion
5313 helpers which may be called from traditional or property-based
5314 tests, but can only use assume() or target() in the latter case.
5315 Registering strategies via setuptools entry points
5317 If you would like to ship Hypothesis strategies for a custom type - ei‐
5318 ther as part of the upstream library, or as a third-party extension,
5319 there's a catch: from_type() only works after the corresponding call to
5320 register_type_strategy(). This means that either
5321 • you have to try importing Hypothesis to register the strategy when
5323 your library is imported, though that's only useful at test time, or
5324 • the user has to call a 'register the strategies' helper that you pro‐
5326 vide before running their tests
5327 Entry points are Python's standard way of automating the latter: when
5329 you register a "hypothesis" entry point in your setup.py, we'll import
5330 and run it automatically when hypothesis is imported. Nothing happens
5331 unless Hypothesis is already in use, and it's totally seamless for
5332 downstream users!
5333 Let's look at an example. You start by adding a function somewhere in
5335 your package that does all the Hypothesis-related setup work:
5336 # mymodule.py
5338 class MyCustomType:
5341 def __init__(self, x: int):
5342 assert x >= 0, f"got {x}, but only positive numbers are allowed"
5343 self.x = x
5344 def _hypothesis_setup_hook():
5347 import hypothesis.strategies as st
5348 st.register_type_strategy(MyCustomType, st.integers(min_value=0))
5350 and then tell setuptools that this is your "hypothesis" entry point:
5352 # setup.py
5354 # You can list a module to import by dotted name
5356 entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}
5357 # Or name a specific function too, and Hypothesis will call it for you
5359 entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}
5360 And that's all it takes!
5362 NOTE:
5364 On Python 3.6 and 3.7, where the importlib.metadata module is not in
5365 the standard library, loading entry points requires either the
5366 importlib_metadata (preferred) or setuptools (fallback) package to
5367 be installed.
5368 Interaction with pytest-cov
5370 Because pytest does not load plugins from entrypoints in any particular
5371 order, using the Hypothesis entrypoint may import your module before
5372 pytest-cov starts. This is a known issue, but there are workarounds.
5373 You can use coverage run pytest ... instead of pytest --cov ..., opting
5375 out of the pytest plugin entirely. Alternatively, you can ensure that
5376 Hypothesis is loaded after coverage measurement is started by disabling
5377 the entrypoint, and loading our pytest plugin from your conftest.py in‐
5378 stead:
5379 echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
5381 pytest -p "no:hypothesispytest" ...
5382
5384 This is a record of all past Hypothesis releases and what went into
5385 them, in reverse chronological order. All previous releases should
5386 still be available on PyPI.
5387 Hypothesis 6.x
5389 6.14.3 - 2021-07-18
5390 This patch updates our autoformatting tools, improving our code style
5391 without any API changes.
5392 6.14.2 - 2021-07-12
5394 This patch ensures that we shorten tracebacks for tests which fail due
5395 to inconsistent data generation between runs (i.e. raise Flaky).
5396 6.14.1 - 2021-07-02
5398 This patch updates some internal type annotations. There is no
5399 user-visible change.
5400 6.14.0 - 2021-06-09
5402 The explain phase now requires shrinking to be enabled, and will be au‐
5403 tomatically skipped for deadline-exceeded errors.
5404 6.13.14 - 2021-06-04
5406 This patch improves the tuples() strategy type annotations, to preserve
5407 the element types for up to length-five tuples (issue #3005).
5408 As for one_of(), this is the best we can do before a planned extension
5410 to PEP 646 is released, hopefully in Python 3.11.
5411 6.13.13 - 2021-06-04
5413 This patch teaches the Ghostwriter how to find custom ufuncs from any
5414 module that defines them, and that yaml.unsafe_load() does not undo
5415 yaml.safe_load().
5416 6.13.12 - 2021-06-03
5418 This patch reduces the amount of internal code excluded from our test
5419 suite's code coverage checks.
5420 There is no user-visible change.
5422 6.13.11 - 2021-06-02
5424 This patch removes some old internal helper code that previously ex‐
5425 isted to make Python 2 compatibility easier.
5426 There is no user-visible change.
5428 6.13.10 - 2021-05-30
5430 This release adjusts some internal code to help make our test suite
5431 more reliable.
5432 There is no user-visible change.
5434 6.13.9 - 2021-05-30
5436 This patch cleans up some internal code related to filtering strate‐
5437 gies.
5438 There is no user-visible change.
5440 6.13.8 - 2021-05-28
5442 This patch slightly improves the performance of some internal code for
5443 generating integers.
5444 6.13.7 - 2021-05-27
5446 This patch fixes a bug in from_regex() that caused from_regex("", full‐
5447 match=True) to unintentionally generate non-empty strings (issue
5448 #4982).
5449 The only strings that completely match an empty regex pattern are empty
5451 strings.
5452 6.13.6 - 2021-05-26
5454 This patch fixes a bug that caused integers() to shrink towards nega‐
5455 tive values instead of positive values in some cases.
5456 6.13.5 - 2021-05-24
5458 This patch fixes rare cases where hypothesis write --binary-op could
5459 print reproducing instructions from the internal search for an identity
5460 element.
5461 6.13.4 - 2021-05-24
5463 This patch removes some unnecessary intermediate list-comprehensions,
5464 using the latest versions of pyupgrade and shed.
5465 6.13.3 - 2021-05-23
5467 This patch adds a .hypothesis property to invalid test functions,
5468 bringing them inline with valid tests and fixing a bug where
5469 pytest-asyncio would swallow the real error message and mistakenly
5470 raise a version incompatibility error.
5471 6.13.2 - 2021-05-23
5473 Some of Hypothesis's numpy/pandas strategies use a fill argument to
5474 speed up generating large arrays, by generating a single fill value and
5475 sharing that value among many array slots instead of filling every sin‐
5476 gle slot individually.
5477 When no fill argument is provided, Hypothesis tries to detect whether
5479 it is OK to automatically use the elements argument as a fill strategy,
5480 so that it can still use the faster approach.
5481 This patch fixes a bug that would cause that optimization to trigger in
5483 some cases where it isn't 100% guaranteed to be OK.
5484 If this makes some of your numpy/pandas tests run more slowly, try
5486 adding an explicit fill argument to the relevant strategies to ensure
5487 that Hypothesis always uses the faster approach.
5488 6.13.1 - 2021-05-20
5490 This patch strengthens some internal import-time consistency checks for
5491 the built-in strategies.
5492 There is no user-visible change.
5494 6.13.0 - 2021-05-18
5496 This release adds URL fragment generation to the urls() strategy (issue
5497 #2908). Thanks to Pax (R. Margret) for contributing this patch at the
5498 PyCon US Mentored Sprints!
5499 6.12.1 - 2021-05-17
5501 This patch fixes issue #2964, where .map() and .filter() methods were
5502 omitted from the repr() of just() and sampled_from() strategies, since
5503 version 5.43.7.
5504 6.12.0 - 2021-05-06
5506 This release automatically rewrites some simple filters, such as inte‐
5507 gers().filter(lambda x: x > 9) to the more efficient inte‐
5508 gers(min_value=10), based on the AST of the predicate.
5509 We continue to recommend using the efficient form directly wherever
5511 possible, but this should be useful for e.g. pandera "Checks" where you
5512 already have a simple predicate and translating manually is really an‐
5513 noying. See issue #2701 for ideas about floats and simple text strate‐
5514 gies.
5515 6.11.0 - 2021-05-06
5517 hypothesis.target() now returns the observation value, allowing it to
5518 be conveniently used inline in expressions such as assert target(abs(a
5519 - b)) < 0.1.
5520 6.10.1 - 2021-04-26
5522 This patch fixes a deprecation warning if you're using recent versions
5523 of importlib-metadata (issue #2934), which we use to load third-party
5524 plugins such as Pydantic's integration. On older versions of
5525 importlib-metadata, there is no change and you don't need to upgrade.
5526 6.10.0 - 2021-04-17
5528 This release teaches the Ghostwriter to read parameter types from
5529 Sphinx, Google, or Numpy-style structured docstrings, and improves some
5530 related heuristics about how to test scientific and numerical programs.
5531 6.9.2 - 2021-04-15
5533 This release improves the Ghostwriter's handling of exceptions, by
5534 reading :raises ...: entries in function docstrings and ensuring that
5535 we don't suppresss the error raised by test assertions.
5536 6.9.1 - 2021-04-12
5538 This patch updates our autoformatting tools, improving our code style
5539 without any API changes.
5540 6.9.0 - 2021-04-11
5542 This release teaches from_type() how to see through python:typing.Anno‐
5543 tated. Thanks to Vytautas Strimaitis for reporting and fixing issue
5544 #2919!
5545 6.8.12 - 2021-04-11
5547 If rich is installed, the hypothesis write command will use it to syn‐
5548 tax-highlight the Ghostwritten code.
5549 6.8.11 - 2021-04-11
5551 This patch improves an error message from from_type() when builds()
5552 would be more suitable (issue #2930).
5553 6.8.10 - 2021-04-11
5555 This patch updates the type annotations for arrays() to reflect that
5556 shape: SearchStrategy[int] is supported.
5557 6.8.9 - 2021-04-07
5559 This patch fixes from_type() with abstract types which have either re‐
5560 quired but non-type-annotated arguments to __init__, or where
5561 from_type() can handle some concrete subclasses but not others.
5562 6.8.8 - 2021-04-07
5564 This patch teaches hypothesis write to check for possible roundtrips in
5565 several more cases, such as by looking for an inverse in the module
5566 which defines the function to test.
5567 6.8.7 - 2021-04-07
5569 This patch adds a more helpful error message if you try to call sam‐
5570 pled_from() on an Enum which has no members, but does have data‐
5571 class()-style annotations (issue #2923).
5572 6.8.6 - 2021-04-06
5574 The fixed_dictionaries() strategy now preserves dict iteration order
5575 instead of sorting the keys. This also affects the pretty-printing of
5576 keyword arguments to @given() (issue #2913).
5577 6.8.5 - 2021-04-05
5579 This patch teaches hypothesis write to default to ghostwriting tests
5580 with --style=pytest only if pytest is installed, or --style=unittest
5581 otherwise.
5582 6.8.4 - 2021-04-01
5584 This patch adds type annotations for the settings decorator, to avoid
5585 an error when running mypy in strict mode.
5586 6.8.3 - 2021-03-28
5588 This patch improves the Ghostwriter's handling of strategies to gener‐
5589 ate various fiddly types including frozensets, keysviews, valuesviews,
5590 regex matches and patterns, and so on.
5591 6.8.2 - 2021-03-27
5593 This patch fixes some internal typos. There is no user-visible change.
5594 6.8.1 - 2021-03-14
5596 This patch lays more groundwork for filter rewriting (issue #2701).
5597 There is no user-visible change... yet.
5598 6.8.0 - 2021-03-11
5600 This release registers the remaining builtin types, and teaches
5601 from_type() to try resolving ForwardRef and Type references to built-in
5602 types.
5603 6.7.0 - 2021-03-10
5605 This release teaches RuleBasedStateMachine to avoid checking invari‐
5606 ant()s until all initialize() rules have been run. You can enable
5607 checking of specific invariants for incompletely initialized machines
5608 by using @invariant(check_during_init=True) (issue #2868).
5609 In previous versions, it was possible if awkward to implement this be‐
5611 haviour using precondition() and an auxiliary variable.
5612 6.6.1 - 2021-03-09
5614 This patch improves the error message when from_type() fails to resolve
5615 a forward-reference inside a python:typing.Type such as Type["int"] (‐
5616 issue #2565).
5617 6.6.0 - 2021-03-07
5619 This release makes it an explicit error to apply invariant() to a
5620 rule() or initialize() rule in stateful testing. Such a combination
5621 had unclear semantics, especially in combination with precondition(),
5622 and was never meant to be allowed (issue #2681).
5623 6.5.0 - 2021-03-07
5625 This release adds the explain phase, in which Hypothesis attempts to
5626 explain why your test failed by pointing to suspicious lines of code
5627 (i.e. those which were always, and only, run on failing inputs). We
5628 plan to include "generalising" failing examples in this phase in a fu‐
5629 ture release (issue #2192).
5630 6.4.3 - 2021-03-04
5632 This patch fixes issue #2794, where nesting deferred() strategies
5633 within recursive() strategies could trigger an internal assertion.
5634 While it was always possible to get the same results from a more sensi‐
5635 ble strategy, the convoluted form now works too.
5636 6.4.2 - 2021-03-04
5638 This patch fixes several problems with mypy when --no-implicit-reexport
5639 was activated in user projects.
5640 Thanks to Nikita Sobolev for fixing issue #2884!
5642 6.4.1 - 2021-03-04
5644 This patch fixes an exception that occurs when using type unions of the
5645 typing_extensions Literal backport on Python 3.6.
5646 Thanks to Ben Anhalt for identifying and fixing this bug.
5648 6.4.0 - 2021-03-02
5650 This release fixes stateful testing methods with multiple precondi‐
5651 tion() decorators. Previously, only the outer-most precondition was
5652 checked (issue #2681).
5653 6.3.4 - 2021-02-28
5655 This patch refactors some internals of RuleBasedStateMachine. There is
5656 no change to the public API or behaviour.
5657 6.3.3 - 2021-02-26
5659 This patch moves some internal code, so that future work can avoid cre‐
5660 ating import cycles. There is no user-visible change.
5661 6.3.2 - 2021-02-25
5663 This patch enables register_type_strategy() for subclasses of
5664 python:typing.TypedDict. Previously, from_type() would ignore the reg‐
5665 istered strategy (issue #2872).
5666 Thanks to Ilya Lebedev for identifying and fixing this bug!
5668 6.3.1 - 2021-02-24
5670 This release lays the groundwork for automatic rewriting of simple fil‐
5671 ters, for example converting integers().filter(lambda x: x > 9) to in‐
5672 tegers(min_value=10).
5673 Note that this is not supported yet, and we will continue to recommend
5675 writing the efficient form directly wherever possible - predicate
5676 rewriting is provided mainly for the benefit of downstream libraries
5677 which would otherwise have to implement it for themselves (e.g. pandera
5678 and icontract-hypothesis). See issue #2701 for details.
5679 6.3.0 - 2021-02-20
5681 The Hypothesis pytest plugin now requires pytest version 4.6 or later.
5682 If the plugin detects an earlier version of pytest, it will automati‐
5683 cally deactivate itself.
5684 (4.6.x is the earliest pytest branch that still accepts community bug‐
5686 fixes.)
5687 Hypothesis-based tests should continue to work in earlier versions of
5689 pytest, but enhanced integrations provided by the plugin (such as --hy‐
5690 pothesis-show-statistics and other command-line flags) will no longer
5691 be available in obsolete pytest versions.
5692 6.2.0 - 2021-02-12
5694 If you use pytest-html, Hypothesis now includes the summary statistics
5695 for each test in the HTML report, whether or not the --hypothe‐
5696 sis-show-statistics argument was passed to show them in the com‐
5697 mand-line output.
5698 6.1.1 - 2021-01-31
5700 This patch updates our automatic code formatting to use shed, which in‐
5701 cludes autoflake, black, isort, and pyupgrade (issue #2780).
5702 6.1.0 - 2021-01-29
5704 This release teaches Hypothesis to distinguish between errors based on
5705 the __cause__ or __context__ of otherwise identical exceptions, which
5706 is particularly useful when internal errors can be wrapped by a li‐
5707 brary-specific or semantically appropriate exception such as:
5708 try:
5710 do_the_thing(foo, timeout=10)
5711 except Exception as err:
5712 raise FooError("Failed to do the thing") from err
5713 Earlier versions of Hypothesis only see the FooError, while we can now
5715 distinguish a FooError raised because of e.g. an internal assertion
5716 from one raised because of a TimeoutExceeded exception.
5717 6.0.4 - 2021-01-27
5719 This release prevents a race condition inside recursive() strategies.
5720 The race condition occurs when the same recursive() strategy is shared
5721 among tests that are running in multiple threads (issue #2717).
5722 6.0.3 - 2021-01-23
5724 This patch improves the type annotations for one_of(), by adding over‐
5725 loads to handle up to five distinct arguments as Union before falling
5726 back to Any, as well as annotating the | (__or__) operator for strate‐
5727 gies (issue #2765).
5728 6.0.2 - 2021-01-14
5730 This release makes some small improvements to how filtered strategies
5731 work. It should improve the performance of shrinking filtered strate‐
5732 gies, and may under some (probably rare) circumstances improve the di‐
5733 versity of generated examples.
5734 6.0.1 - 2021-01-13
5736 This patch fixes an interaction where our test statistics handling made
5737 Pytest's --junit-xml output fail to validate against the strict xunit2
5738 schema (issue #1975).
5739 6.0.0 - 2021-01-08
5741 Welcome to the next major version of Hypothesis!
5742 There are no new features here, as we release those in minor versions.
5744 Instead, 6.0 is a chance for us to remove deprecated features (many al‐
5745 ready converted into no-ops), and turn a variety of warnings into er‐
5746 rors.
5747 If you were running on the last version of Hypothesis 5.x without any
5749 Hypothesis deprecation warnings, this will be a very boring upgrade.
5750 In fact, nothing will change for you at all.
5751 Changes
5753 • Many functions now use PEP 3102 keyword-only arguments where passing
5754 positional arguments was deprecated since 5.5.
5755 • hypothesis.extra.django.from_model() no longer accepts model as a
5757 keyword argument, where it could conflict with fields named "model".
5758 • randoms() now defaults to use_true_random=False.
5760 • complex_numbers() no longer accepts min_magnitude=None; either use
5762 min_magnitude=0 or just omit the argument.
5763 • hypothesis.provisional.ip4_addr_strings and ip6_addr_strings are re‐
5765 moved in favor of ip_addresses(v=...).map(str).
5766 • register_type_strategy() no longer accepts generic types with type
5768 arguments, which were always pretty badly broken.
5769 • Using function-scoped pytest fixtures is now a health-check error,
5771 instead of a warning.
5772 TIP:
5774 The hypothesis codemod command can automatically refactor your code,
5775 particularly to convert positional to keyword arguments where those
5776 are now required.
5777 Hypothesis 5.x
5779 5.49.0 - 2021-01-07
5780 This release adds the function_scoped_fixture health check value, which
5781 can be used to suppress the existing warning that appears when @given
5782 is applied to a test that uses pytest function-scoped fixtures.
5783 (This warning exists because function-scoped fixtures only run once per
5785 function, not once per example, which is usually unexpected and can
5786 cause subtle problems.)
5787 When this warning becomes a health check error in a future release,
5789 suppressing it via Python warning settings will no longer be possible.
5790 In the rare case that once-per-function behaviour is intended, it will
5791 still be possible to use function_scoped_fixture to opt out of the
5792 health check error for specific tests.
5793 5.48.0 - 2021-01-06
5795 This release adds hypothesis.currently_in_test_context(), which can be
5796 used to check whether the calling code is currently running inside an
5797 @given or stateful test.
5798 This is most useful for third-party integrations and assertion helpers
5800 which may wish to use assume() or target(), without also requiring that
5801 the helper only be used from property-based tests (issue #2581).
5802 5.47.0 - 2021-01-05
5804 This release upgrades the import logic for ghostwritten tests, handling
5805 many cases where imports would previously be missing or from unexpected
5806 locations.
5807 5.46.0 - 2021-01-04
5809 This release upgrades from_type(), to infer strategies for type-anno‐
5810 tated arguments even if they have defaults when it otherwise falls back
5811 to builds() (issue #2708).
5812 5.45.0 - 2021-01-04
5814 This release adds the codemods extra, which you can use to check for
5815 and automatically fix issues such as use of deprecated Hypothesis APIs
5816 (issue #2705).
5817 5.44.0 - 2021-01-03
5819 This patch fixes from_type() with the typing_extensions Literal back‐
5820 port on Python 3.6.
5821 5.43.9 - 2021-01-02
5823 This patch fixes issue #2722, where certain orderings of regis‐
5824 ter_type_strategy(), ForwardRef, and from_type() could trigger an in‐
5825 ternal error.
5826 5.43.8 - 2021-01-02
5828 This patch makes some strategies for collections with a uniqueness con‐
5829 straint much more efficient, including dictionaries(keys=sam‐
5830 pled_from(...), values=..) and lists(tuples(sampled_from(...), ...),
5831 unique_by=lambda x: x[0]). (related to issue #2036)
5832 5.43.7 - 2021-01-02
5834 This patch extends our faster special case for sampled_from() elements
5835 in unique lists() to account for chains of .map(...) and .filter(...)
5836 calls (issue #2036).
5837 5.43.6 - 2021-01-02
5839 This patch improves the type annotations on assume() and @repro‐
5840 duce_failure().
5841 5.43.5 - 2021-01-01
5843 This patch updates our copyright headers to include 2021. Happy new
5844 year!
5845 5.43.4 - 2020-12-24
5847 This change fixes a documentation error in the database setting.
5848 The previous documentation suggested that callers could specify a data‐
5850 base path string, or the special string ":memory:", but this setting
5851 has never actually allowed string arguments.
5852 Permitted values are None, and instances of ExampleDatabase.
5854 5.43.3 - 2020-12-11
5856 This patch fixes issue #2696, an internal error triggered when the @ex‐
5857 ample decorator was used and the verbosity setting was quiet.
5858 5.43.2 - 2020-12-10
5860 This patch improves the error message from the data_frames() strategy
5861 when both the rows and columns arguments are given, but there is a
5862 missing entry in rows and the corresponding column has no fill value (‐
5863 issue #2678).
5864 5.43.1 - 2020-12-10
5866 This patch improves the error message if builds() is passed an Enum
5867 which cannot be called without arguments, to suggest using sam‐
5868 pled_from() (issue #2693).
5869 5.43.0 - 2020-12-09
5871 This release adds new timezones() and timezone_keys() strategies (issue
5872 #2630) based on the new python:zoneinfo module in Python 3.9.
5873 pip install hypothesis[zoneinfo] will ensure that you have the appro‐
5875 priate backports installed if you need them.
5876 5.42.3 - 2020-12-09
5878 This patch fixes an internal error in datetimes() with allow_imagi‐
5879 nary=False where the timezones argument can generate tzinfo=None (issue
5880 #2662).
5881 5.42.2 - 2020-12-09
5883 This patch teaches hypothesis.extra.django.from_field() to infer more
5884 efficient strategies by inspecting (not just filtering by) field val‐
5885 idators for numeric and string fields (issue #1116).
5886 5.42.1 - 2020-12-09
5888 This patch refactors hypothesis.settings to use type-annotated keyword
5889 arguments instead of **kwargs, which makes tab-completion much more
5890 useful - as well as type-checkers like mypy.
5891 5.42.0 - 2020-12-09
5893 This patch teaches the magic() ghostwriter to recognise "en/de" func‐
5894 tion roundtrips other than the common encode/decode pattern, such as
5895 encrypt/decrypt or, encipher/decipher.
5896 5.41.5 - 2020-12-05
5898 This patch adds a performance optimisation to avoid saving redundant
5899 seeds when using the .fuzz_one_input hook.
5900 5.41.4 - 2020-11-28
5902 This patch fixes issue #2657, where passing unicode patterns compiled
5903 with python:re.IGNORECASE to from_regex() could trigger an internal er‐
5904 ror when casefolding a character creates a longer string (e.g.
5905 "\u0130".lower() -> "i\u0370").
5906 5.41.3 - 2020-11-18
5908 This patch adds a final fallback clause to our plugin logic to fail
5909 with a warning rather than error on Python < 3.8 when neither the
5910 importlib_metadata (preferred) or setuptools (fallback) packages are
5911 available.
5912 5.41.2 - 2020-11-08
5914 This patch fixes urls() strategy ensuring that ~ (tilde) is treated as
5915 one of the url-safe characters (issue #2658).
5916 5.41.1 - 2020-11-03
5918 This patch improves our CLI help and documentation.
5919 5.41.0 - 2020-10-30
5921 Hypothesis now shrinks examples where the error is raised while drawing
5922 from a strategy. This makes complicated custom strategies much easier
5923 to debug, at the cost of a slowdown for use-cases where you catch and
5924 ignore such errors.
5925 5.40.0 - 2020-10-30
5927 This release teaches from_type() how to handle ChainMap, Counter,
5928 Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).
5929 5.39.0 - 2020-10-30
5931 from_type() now knows how to resolve PEP 585 parameterized standard
5932 collection types, which are new in Python 3.9 (issue #2629).
5933 5.38.1 - 2020-10-26
5935 This patch fixes builds(), so that when passed infer for an argument
5936 with a non-Optional type annotation and a default value of None to
5937 build a class which defines an explicit __signature__ attribute, either
5938 None or that type may be generated.
5939 This is unlikely to happen unless you are using pydantic (issue #2648).
5941 5.38.0 - 2020-10-24
5943 This release improves our support for @st.composite on a python:class‐
5944 method or python:staticmethod (issue #2578).
5945 5.37.5 - 2020-10-24
5947 This patch fixes from_type() with Iterable[T] (issue #2645).
5948 5.37.4 - 2020-10-20
5950 This patch teaches the magic() ghostwriter to recognise that pairs of
5951 functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().
5952 5.37.3 - 2020-10-15
5954 This patch improves builds() and from_type() support for explicitly de‐
5955 fined __signature__ attributes, from version 5.8.3, to support generic
5956 types from the python:typing module.
5957 Thanks to Rónán Carrigan for identifying and fixing this problem!
5959 5.37.2 - 2020-10-14
5961 This patch fixes from_lark() with version 0.10.1+ of the lark-parser
5962 package.
5963 5.37.1 - 2020-10-07
5965 This patch fixes some broken links in the lark extra documentation.
5966 5.37.0 - 2020-10-03
5968 This release adds a new RedisExampleDatabase, along with the ReadOnly‐
5969 Database and MultiplexedDatabase helpers, to support team workflows
5970 where failing examples can be seamlessly shared between everyone on the
5971 team - and your CI servers or buildbots.
5972 5.36.2 - 2020-10-02
5974 This patch ensures that if the "hypothesis" entry point is callable, we
5975 call it after importing it. You can still use non-callable entry
5976 points (like modules), which are only imported.
5977 We also prefer importlib.metadata or the backport over pkg_resources,
5979 which makes import hypothesis around 200 milliseconds faster (issue
5980 #2571).
5981 5.36.1 - 2020-09-25
5983 This patch adds some helpful suggestions to error messages you might
5984 see while learning to use the @example() decorator (issue #2611) or the
5985 one_of() strategy.
5986 5.36.0 - 2020-09-24
5988 This release upgrades the from_dtype() strategy to pass optional
5989 **kwargs to the inferred strategy, and upgrades the arrays() strategy
5990 to accept an elements=kwargs dict to pass through to from_dtype().
5991 arrays(floating_dtypes(), shape, elements={"min_value": -10,
5993 "max_value": 10}) is a particularly useful pattern, as it allows for
5994 any floating dtype without triggering the roundoff warning for smaller
5995 types or sacrificing variety for larger types (issue #2552).
5996 5.35.4 - 2020-09-21
5998 This patch reformats our code with the latest black to take advantage
5999 of the support for magic trailing commas.
6000 5.35.3 - 2020-09-15
6002 This release significantly improves the performance of Hypothesis's in‐
6003 ternal implementation of automaton learning. However this code does not
6004 run as part of the user-accessible API so this has no user-visible im‐
6005 pact.
6006 5.35.2 - 2020-09-14
6008 This patch ensures that, when the generate phases is disabled, we can
6009 replay up to max_examples examples from the database - which is very
6010 useful when using Hypothesis with a fuzzer.
6011 Thanks to Afrida Tabassum for fixing issue #2585!
6013 5.35.1 - 2020-09-14
6015 This patch changes some internal python:struct.Struct.format strings
6016 from bytes to str, to avoid python:BytesWarning when running python
6017 -bb.
6018 Thanks to everyone involved in pytest-xdist issue 596, bpo-16349,
6020 bpo-21071, and bpo-41777 for their work on this - it was a remarkably
6021 subtle issue!
6022 5.35.0 - 2020-09-11
6024 The target() function now accepts integers as well as floats.
6025 5.34.1 - 2020-09-11
6027 This patch adds explicit Optional annotations to our public API, to
6028 better support users who run mypy with --strict or no_implicit_op‐
6029 tional=True.
6030 Thanks to Krzysztof Przybyła for bringing this to our attention and
6032 writing the patch!
6033 5.34.0 - 2020-09-11
6035 This release drops support for Python 3.5, which reached end of life
6036 upstream on 2020-09-13.
6037 5.33.2 - 2020-09-09
6039 This patch fixes a problem with builds() that was not able to generate
6040 valid data for annotated classes with constructors.
6041 Thanks to Nikita Sobolev for fixing issue #2603!
6043 5.33.1 - 2020-09-07
6045 This patch improves the error message from the hypothesis write command
6046 if black (required for the ghostwriter) is not installed.
6047 Thanks to Nikita Sobolev for fixing issue #2604!
6049 5.33.0 - 2020-09-06
6051 When reporting failing examples, or tried examples in verbose mode, Hy‐
6052 pothesis now identifies which were from @example(...) explicit exam‐
6053 ples.
6054 5.32.1 - 2020-09-06
6056 This patch contains some internal refactoring. Thanks to Felix Sheldon
6057 for fixing issue #2516!
6058 5.32.0 - 2020-09-04
6060 An array drawn from arrays() will own its own memory; previously most
6061 arrays returned by this strategy were views.
6062 5.31.0 - 2020-09-04
6064 builds() will use the __signature__ attribute of the target, if it ex‐
6065 ists, to retrieve type hints. Previously python:typ‐
6066 ing.get_type_hints(), was used by default. If argument names varied
6067 between the __annotations__ and __signature__, they would not be sup‐
6068 plied to the target.
6069 This was particularly an issue for pydantic models which use an alias
6071 generator.
6072 5.30.1 - 2020-09-04
6074 This patch makes the ghostwriter much more robust when passed unusual
6075 modules.
6076 • improved support for non-resolvable type annotations
6078 • magic() can now write equivalent() tests
6080 • running magic() on modules where some names in __all__ are undefined
6082 skips such names, instead of raising an error
6083 • magic() now knows to skip mocks
6085 • improved handling of import-time errors found by the ghostwriter CLI
6087 5.30.0 - 2020-08-30
6089 register_type_strategy() now supports python:typing.TypeVar, which was
6090 previously hard-coded, and allows a variety of types to be generated
6091 for an unconstrained TypeVar instead of just text().
6092 Thanks again to Nikita Sobolev for all your work on advanced types!
6094 5.29.4 - 2020-08-28
6096 This release fixes some hard to trigger bugs in Hypothesis's automata
6097 learning code. This code is only run as part of the Hypothesis build
6098 process, and not for user code, so this release has no user visible im‐
6099 pact.
6100 5.29.3 - 2020-08-27
6102 This patch adds type annotations to the hypothesis.database module.
6103 There is no runtime change, but your typechecker might notice.
6104 5.29.2 - 2020-08-27
6106 This patch tracks some additional information in Hypothesis internals,
6107 and has no user-visible impact.
6108 5.29.1 - 2020-08-27
6110 This release fixes a bug in some Hypothesis internal support code for
6111 learning automata. This mostly doesn't have any user visible impact,
6112 although it slightly affects the learned shrink passes so shrinking may
6113 be subtly different.
6114 5.29.0 - 2020-08-24
6116 This release adds support for entry-points, which allows for smoother
6117 integration of third-party Hypothesis extensions and external li‐
6118 braries. Unless you're publishing a library with Hypothesis integra‐
6119 tion, you'll probably only ever use this indirectly!
6120 5.28.0 - 2020-08-24
6122 from_type() can now resolve TypeVar instances when the bound is a For‐
6123 wardRef, so long as that name is in fact defined in the same module as
6124 the typevar (no TYPE_CHECKING tricks, sorry). This feature requires
6125 Python 3.7 or later.
6126 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
6128 5.27.0 - 2020-08-20
6130 This patch adds two new ghostwriters to test binary operations, like
6131 python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().
6132 5.26.1 - 2020-08-19
6134 This release improves the performance of some methods in Hypothesis's
6135 internal automaton library. These are currently only lightly used by
6136 user code, but this may result in slightly faster shrinking.
6137 5.26.0 - 2020-08-17
6139 register_type_strategy() no longer accepts parametrised user-defined
6140 generic types, because the resolution logic was quite badly broken (‐
6141 issue #2537).
6142 Instead of registering a strategy for e.g. MyCollection[int], you
6144 should register a function for MyCollection and inspect the type param‐
6145 eters within that function.
6146 Thanks to Nikita Sobolev for the bug report, design assistance, and
6148 pull request to implement this feature!
6149 5.25.0 - 2020-08-16
6151 Tired of writing tests? Or new to Hypothesis and not sure where to
6152 start?
6153 This release is for you! With our new Ghostwriter functions and hy‐
6155 pothesis write ... command-line interface, you can stop writing tests
6156 entirely... or take the source code Hypothesis writes for you as a
6157 starting point.
6158 This has been in the works for months, from issue #2118 to versions
6160 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who
6161 reviewed pull requests or commented on demos, and to Timothy Crosley's
6162 hypothesis-auto project for inspiration.
6163 5.24.4 - 2020-08-14
6165 This patch adds yet more internal functions to support a new feature
6166 we're working on, like version 5.18.3 and version 5.23.6. We promise
6167 it's worth the wait!
6168 5.24.3 - 2020-08-13
6170 This release fixes a small internal bug in Hypothesis's internal autom‐
6171 aton library. Fortunately this bug was currently impossible to hit in
6172 user facing code, so this has no user visible impact.
6173 5.24.2 - 2020-08-12
6175 This release improves shrink quality by allowing Hypothesis to automat‐
6176 ically learn new shrink passes for difficult to shrink tests.
6177 The automatic learning is not currently accessible in user code (it
6179 still needs significant work on robustness and performance before it is
6180 ready for that), but this release includes learned passes that should
6181 improve shrinking quality for tests which use any of the text(),
6182 floats(), datetimes(), emails(), and complex_numbers() strategies.
6183 5.24.1 - 2020-08-12
6185 This patch updates some docstrings, without changing runtime behaviour.
6186 5.24.0 - 2020-08-10
6188 The functions() strategy has a new argument pure=True, which ensures
6189 that the same return value is generated for identical calls to the gen‐
6190 erated function (issue #2538).
6191 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
6193 5.23.12 - 2020-08-10
6195 This release removes a number of Hypothesis's internal "shrink passes"
6196 - transformations it makes to a generated test case during shrinking -
6197 which appeared to be redundant with other transformations.
6198 It is unlikely that you will see much impact from this. If you do, it
6200 will likely show up as a change in shrinking performance (probably
6201 slower, maybe faster), or possibly in worse shrunk results. If you en‐
6202 counter the latter, please let us know.
6203 5.23.11 - 2020-08-04
6205 This release fixes a bug in some internal Hypothesis support code. It
6206 has no user visible impact.
6207 5.23.10 - 2020-08-04
6209 This release improves the quality of shrunk test cases in some special
6210 cases. Specifically, it should get shrinking unstuck in some scenarios
6211 which require simultaneously changing two parts of the generated test
6212 case.
6213 5.23.9 - 2020-08-03
6215 This release improves the performance of some internal support code. It
6216 has no user visible impact, as that code is not currently run during
6217 normal Hypothesis operation.
6218 5.23.8 - 2020-07-31
6220 This release adds a heuristic to detect when shrinking has finished de‐
6221 spite the fact that there are many more possible transformations to
6222 try. This will be particularly useful for tests where the minimum fail‐
6223 ing test case is very large despite there being many smaller test cases
6224 possible, where it is likely to speed up shrinking dramatically.
6225 In some cases it is likely that this will result in worse shrunk test
6227 cases. In those cases rerunning the test will result in further shrink‐
6228 ing.
6229 5.23.7 - 2020-07-29
6231 This release makes some performance improvements to shrinking. They
6232 should only be noticeable for tests that are currently particularly
6233 slow to shrink.
6234 5.23.6 - 2020-07-29
6236 This patch adds some more internal functions to support a new feature
6237 we're working on, like version 5.18.3. There is still no user-visible
6238 change... yet.
6239 5.23.5 - 2020-07-29
6241 This release makes some changes to internal support code that is not
6242 currently used in production Hypothesis. It has no user visible effect
6243 at present.
6244 5.23.4 - 2020-07-29
6246 This release improves shrinking quality in some special cases.
6247 5.23.3 - 2020-07-27
6249 This release fixes issue #2507, where lazy evaluation meant that the
6250 values drawn from a sampled_from() strategy could depend on mutations
6251 of the sampled sequence that happened after the strategy was construc‐
6252 ted.
6253 5.23.2 - 2020-07-27
6255 This patch fixes issue #2462, a bug in our handling of unittest.Test‐
6256 Case.subTest(). Thanks to Israel Fruchter for fixing this at the Eu‐
6257 roPython sprints!
6258 5.23.1 - 2020-07-26
6260 This release improves the behaviour of the characters() strategy when
6261 shrinking, by changing which characters are considered smallest to pre‐
6262 fer more "normal" ascii characters where available.
6263 5.23.0 - 2020-07-26
6265 The default print_blob setting is now smarter. It defaults to True in
6266 CI and False for local development.
6267 Thanks to Hugo van Kemenade for implementing this feature at the Eu‐
6269 roPython sprints!
6270 5.22.0 - 2020-07-25
6272 The slices() strategy can now generate slices for empty sequences,
6273 slices with negative start and stop indices (from the end of the se‐
6274 quence), and step=None in place of step=1.
6275 Thanks to Sangarshanan for implementing this feature at the EuroPython
6277 sprints!
6278 5.21.0 - 2020-07-23
6280 This release ensures that tests which raise RecursionError are not re‐
6281 ported as flaky simply because we run them from different initial stack
6282 depths (issue #2494).
6283 5.20.4 - 2020-07-23
6285 This release improves the performance of the sample method on objects
6286 obtained from randoms() when use_true_random=False. This should mostly
6287 only be noticeable when the sample size is a large fraction of the pop‐
6288 ulation size, but may also help avoid health check failures in some
6289 other cases.
6290 5.20.3 - 2020-07-21
6292 This release makes some internal changes for testing purposes and
6293 should have no user visible effect.
6294 5.20.2 - 2020-07-18
6296 This release fixes a small caching bug in Hypothesis internals that may
6297 under some circumstances have resulted in a less diverse set of test
6298 cases being generated than was intended.
6299 Fixing this problem revealed some performance problems that could occur
6301 during targeted property based testing, so this release also fixes
6302 those. Targeted property-based testing should now be significantly
6303 faster in some cases, but this may be at the cost of reduced effective‐
6304 ness.
6305 5.20.1 - 2020-07-17
6307 This patch updates our formatting to use isort 5. There is no
6308 user-visible change.
6309 5.20.0 - 2020-07-17
6311 The basic_indices() strategy can now generate bare indexers in place of
6312 length-one tuples. Thanks to Andrea for this patch!
6313 5.19.3 - 2020-07-15
6315 This patch removes an internal use of distutils in order to avoid this
6316 setuptools warning for some users.
6317 5.19.2 - 2020-07-13
6319 This patch contains a small internal refactoring with no user-visible
6320 impact.
6321 Thanks to Andrea for writing this at the SciPy 2020 Sprints!
6323 5.19.1 - 2020-07-12
6325 This release slightly improves shrinking behaviour. This should mainly
6326 only impact stateful tests, but may have some minor positive impact on
6327 shrinking collections (lists, sets, etc).
6328 5.19.0 - 2020-06-30
6330 This release improves the randoms() strategy by adding support for Ran‐
6331 dom instances where Hypothesis generates the random values rather than
6332 having them be "truly" random.
6333 5.18.3 - 2020-06-27
6335 This patch adds some internal functions to support a new feature we're
6336 working on. There is no user-visible change... yet.
6337 5.18.2 - 2020-06-26
6339 This patch improves our docs for the derandomize setting.
6340 5.18.1 - 2020-06-25
6342 This release consists of some internal refactoring to the shrinker in
6343 preparation for future work. It has no user visible impact.
6344 5.18.0 - 2020-06-22
6346 This release teaches Hypothesis to shorten tracebacks for explicit ex‐
6347 amples, as we already do for generated examples, so that you can focus
6348 on your code rather than ours.
6349 If you have multiple failing explicit examples, they will now all be
6351 reported. To report only the first failure, you can use the re‐
6352 port_multiple_bugs=False setting as for generated examples.
6353 5.17.0 - 2020-06-22
6355 This patch adds strategy inference for the Literal, NewType, Type, De‐
6356 faultDict, and TypedDict types from the typing_extensions backport on
6357 PyPI.
6358 5.16.3 - 2020-06-21
6360 This patch precomputes some of the setup logic for our experimental ex‐
6361 ternal fuzzer integration and sets deadline=None in fuzzing mode, sav‐
6362 ing around 150us on each iteration.
6363 This is around two-thirds the runtime to fuzz an empty test with
6365 @given(st.none()), and nice to have even as a much smaller fraction of
6366 the runtime for non-trivial tests.
6367 5.16.2 - 2020-06-19
6369 This patch fixes an internal error when warning about the use of func‐
6370 tion-scoped fixtures for parametrised tests where the parametrised
6371 value contained a % character. Thanks to Bryant for reporting and fix‐
6372 ing this bug!
6373 5.16.1 - 2020-06-10
6375 If you pass a python:list or python:tuple where a strategy was ex‐
6376 pected, the error message now mentions sampled_from() as an example
6377 strategy.
6378 Thanks to the enthusiastic participants in the PyCon Mentored Sprints
6380 who suggested adding this hint.
6381 5.16.0 - 2020-05-27
6383 functions() can now infer the appropriate returns strategy if you pass
6384 a like function with a return-type annotation. Before, omitting the
6385 returns argument would generate functions that always returned None.
6386 5.15.1 - 2020-05-21
6388 Fix from_type() with generic types under Python 3.9.
6389 5.15.0 - 2020-05-19
6391 This patch fixes an error that happens when multiple threads create new
6392 strategies.
6393 5.14.0 - 2020-05-13
6395 Passing min_magnitude=None to complex_numbers() is now deprecated - you
6396 can explicitly pass min_magnitude=0, or omit the argument entirely.
6397 5.13.1 - 2020-05-13
6399 This patch fixes an internal error in from_type() for python:typ‐
6400 ing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and
6401 fixing issue #2427!
6402 5.13.0 - 2020-05-12
6404 This release upgrades the test statistics available via the --hypothe‐
6405 sis-show-statistics option to include separate information on each of
6406 the phases (issue #1555).
6407 5.12.2 - 2020-05-12
6409 This patch teaches the from_type() internals to return slightly more
6410 efficient strategies for some generic sets and mappings.
6411 5.12.1 - 2020-05-12
6413 This patch adds a # noqa comment for flake8 3.8.0, which disagrees with
6414 mypy about how to write the type of ....
6415 5.12.0 - 2020-05-10
6417 This release limits the maximum duration of the shrinking phase to five
6418 minutes, so that Hypothesis does not appear to hang when making very
6419 slow progress shrinking a failing example (issue #2340).
6420 If one of your tests triggers this logic, we would really appreciate a
6422 bug report to help us improve the shrinker for difficult but realistic
6423 workloads.
6424 5.11.0 - 2020-05-07
6426 This release improves the interaction between assume() and the @exam‐
6427 ple() decorator, so that the following test no longer fails with Unsat‐
6428 isfiedAssumption (issue #2125):
6429 @given(value=floats(0, 1))
6431 @example(value=0.56789) # used to make the test fail!
6432 @pytest.mark.parametrize("threshold", [0.5, 1])
6433 def test_foo(threshold, value):
6434 assume(value < threshold)
6435 ...
6436 5.10.5 - 2020-05-04
6438 If you have django installed but don't use it, this patch will make im‐
6439 port hypothesis a few hundred milliseconds faster (e.g. 0.704s ->
6440 0.271s).
6441 Thanks to importtime-waterfall for highlighting this problem and Jake
6443 Vanderplas for the solution - it's impossible to misuse code from a
6444 module you haven't imported!
6445 5.10.4 - 2020-04-24
6447 This patch improves the internals of builds() type inference, to handle
6448 recursive forward references in certain dataclasses. This is useful
6449 for e.g. hypothesmith's forthcoming LibCST mode.
6450 5.10.3 - 2020-04-22
6452 This release reverses the order in which some operations are tried dur‐
6453 ing shrinking. This should generally be a slight performance improve‐
6454 ment, but most tests are unlikely to notice much difference.
6455 5.10.2 - 2020-04-22
6457 This patch fixes issue #2406, where use of pandas:pandas.Timestamp ob‐
6458 jects as bounds for the datetimes() strategy caused an internal error.
6459 This bug was introduced in version 5.8.1.
6460 5.10.1 - 2020-04-19
6462 This release is a small internal refactoring to how shrinking interacts
6463 with targeted property-based testing that should have no user user vis‐
6464 ible impact.
6465 5.10.0 - 2020-04-18
6467 This release improves our support for datetimes and times around DST
6468 transitions.
6469 times() and datetimes() are now sometimes generated with fold=1, indi‐
6471 cating that they represent the second occurrence of a given wall-time
6472 when clocks are set backwards. This may be set even when there is no
6473 transition, in which case the fold value should be ignored.
6474 For consistency, timezones provided by the pytz package can now gener‐
6476 ate imaginary times (such as the hour skipped over when clocks 'spring
6477 forward' to daylight saving time, or during some historical timezone
6478 transitions). All other timezones have always supported generation of
6479 imaginary times.
6480 If you prefer the previous behaviour, datetimes() now takes an argument
6482 allow_imaginary which defaults to True but can be set to False for any
6483 timezones strategy.
6484 5.9.1 - 2020-04-16
6486 This patch fixes the rendering of binary() docstring by using the
6487 proper backticks syntax.
6488 5.9.0 - 2020-04-15
6490 Failing tests which use target() now report the highest score observed
6491 for each target alongside the failing example(s), even without explic‐
6492 itly showing test statistics.
6493 This improves the debugging workflow for tests of accuracy, which as‐
6495 sert that the total imprecision is within some error budget - for exam‐
6496 ple, abs(a - b) < 0.5. Previously, shrinking to a minimal failing ex‐
6497 ample could often make errors seem smaller or more subtle than they re‐
6498 ally are (see the threshold problem, and issue #2180).
6499 5.8.6 - 2020-04-15
6501 This patch improves the docstring of binary(), the python:repr() of
6502 sampled_from() on an python:enum.Enum subclass, and a warning in our
6503 pytest plugin. There is no change in runtime behaviour.
6504 5.8.5 - 2020-04-15
6506 This release (potentially very significantly) improves the performance
6507 of failing tests in some rare cases, mostly only relevant when using
6508 targeted property-based testing, by stopping further optimisation of
6509 unrelated test cases once a failing example is found.
6510 5.8.4 - 2020-04-14
6512 This release fixes issue #2395, where under some circumstances targeted
6513 property-based testing could cause Hypothesis to get caught in an infi‐
6514 nite loop.
6515 5.8.3 - 2020-04-12
6517 This patch teaches builds() and from_type() to use the __signature__
6518 attribute of classes where it has been set, improving our support for
6519 Pydantic models (in pydantic >= 1.5).
6520 5.8.2 - 2020-04-12
6522 This release improves the performance of the part of the core engine
6523 that deliberately generates duplicate values.
6524 5.8.1 - 2020-04-12
6526 This patch improves dates() shrinking, to simplify year, month, and day
6527 like datetimes() rather than minimizing the number of days since
6528 2000-01-01.
6529 5.8.0 - 2020-03-24
6531 This release adds a .hypothesis.fuzz_one_input attribute to @given
6532 tests, for easy integration with external fuzzers such as python-afl
6533 (supporting issue #171).
6534 5.7.2 - 2020-03-24
6536 This patch fixes issue #2341, ensuring that the printed output from a
6537 stateful test cannot use variable names before they are defined.
6538 5.7.1 - 2020-03-23
6540 This patch fixes issue #2375, preventing incorrect failure when a func‐
6541 tion scoped fixture is overridden with a higher scoped fixture.
6542 5.7.0 - 2020-03-19
6544 This release allows the array_dtypes() strategy to generate Numpy
6545 dtypes which have field titles in addition to field names. We expect
6546 this to expose latent bugs where code expects that set(dtype.names) ==
6547 set(dtype.fields), though the latter may include titles.
6548 5.6.1 - 2020-03-18
6550 This makes model a positional-only argument to from_model(), to support
6551 models with a field literally named "model" (issue #2369).
6552 5.6.0 - 2020-02-29
6554 This release adds an explicit warning for tests that are both decorated
6555 with @given(...) and request a function-scoped pytest fixture, because
6556 such fixtures are only executed once for all Hypothesis test cases and
6557 that often causes trouble (issue #377).
6558 It's very difficult to fix this on the pytest side, so since 2015 our
6560 advice has been "just don't use function-scoped fixtures with Hypothe‐
6561 sis". Now we detect and warn about the issue at runtime!
6562 5.5.5 - 2020-02-29
6564 This release cleans up the internal machinery for stateful, after we
6565 dropped the legacy APIs in Hypothesis 5.0 (issue #2218). There is no
6566 user-visible change.
6567 5.5.4 - 2020-02-16
6569 This patch fixes issue #2351, arrays() would raise a confusing error if
6570 we inferred a strategy for datetime64 or timedelta64 values with vary‐
6571 ing time units.
6572 We now infer an internally-consistent strategy for such arrays, and
6574 have a more helpful error message if an inconsistent strategy is ex‐
6575 plicitly specified.
6576 5.5.3 - 2020-02-14
6578 This patch improves the signature of builds() by specifying target as a
6579 positional-only argument on Python 3.8 (see PEP 570). The semantics of
6580 builds() have not changed at all - this just clarifies the documenta‐
6581 tion.
6582 5.5.2 - 2020-02-13
6584 This release makes Hypothesis faster at generating test cases that con‐
6585 tain duplicated values in their inputs.
6586 5.5.1 - 2020-02-07
6588 This patch has some tiny internal code clean-ups, with no user-visible
6589 change.
6590 5.5.0 - 2020-02-07
6592 Our style guide suggests that optional parameters should usually be
6593 keyword-only arguments (see PEP 3102) to prevent confusion based on po‐
6594 sitional arguments - for example, hypothesis.strategies.floats() takes
6595 up to four boolean flags and many of the Numpy strategies have both
6596 dims and side bounds.
6597 This release converts most optional parameters in our API to use key‐
6599 word-only arguments - and adds a compatibility shim so you get warnings
6600 rather than errors everywhere (issue #2130).
6601 5.4.2 - 2020-02-06
6603 This patch fixes compatibility with Python 3.5.2 (issue #2334). Note
6604 that we only test the latest patch of each minor version, though as in
6605 this case we usually accept pull requests for older patch versions.
6606 5.4.1 - 2020-02-01
6608 This patch improves the repr of from_type(), so that in most cases it
6609 will display the strategy it resolves to rather than from_type(...).
6610 The latter form will continue to be used where resolution is not imme‐
6611 diately successful, e.g. invalid arguments or recursive type defini‐
6612 tions involving forward references.
6613 5.4.0 - 2020-01-30
6615 This release removes support for Python 3.5.0 and 3.5.1, where the
6616 python:typing module was quite immature (e.g. missing overload() and
6617 Type).
6618 Note that Python 3.5 will reach its end-of-life in September 2020, and
6620 new releases of Hypothesis may drop support somewhat earlier.
6621 NOTE:
6623 pip install hypothesis should continue to give you the latest com‐
6624 patible version. If you have somehow ended up with an incompatible
6625 version, you need to update your packaging stack to pip >= 9.0 and
6626 setuptools >= 24.2 - see here for details. Then pip uninstall hy‐
6627 pothesis && pip install hypothesis will get you back to a compatible
6628 version.
6629 5.3.1 - 2020-01-26
6631 This patch does some minor internal cleanup; there is no user-visible
6632 change.
6633 5.3.0 - 2020-01-21
6635 The standard library ipaddress module is new in Python 3, and this re‐
6636 lease adds the new ip_addresses() strategy to generate IPv4Addresses
6637 and/or IPv6Addresses (depending on the v and network arguments).
6638 If you use them in type annotations, from_type() now has strategies
6640 registered for ipaddress address, network, and interface types.
6641 The provisional strategies for IP address strings are therefore depre‐
6643 cated.
6644 5.2.1 - 2020-01-21
6646 This patch reverts version 5.2, due to a strange issue where indexing
6647 an array of strings can raise an error instead of returning an item
6648 which contains certain surrogate characters.
6649 5.2.0 - 2020-01-19
6651 This release allows from_dtype() to generate Unicode strings which can‐
6652 not be encoded in UTF-8, but are valid in Numpy arrays (which use
6653 UTF-32).
6654 5.1.6 - 2020-01-19
6656 This patch fixes issue #2320, where from_type(Set[Hashable]) could
6657 raise an internal error because Decimal("snan") is of a hashable type,
6658 but raises an error when hashed. We now ensure that set elements and
6659 dict keys in generic types can actually be hashed.
6660 5.1.5 - 2020-01-12
6662 This patch fixes an internal error when running in an IPython repl or
6663 Jupyter notebook on Windows (issue #2319), and an internal error on
6664 Python 3.5.1 (issue #2318).
6665 5.1.4 - 2020-01-11
6667 This patch fixes a bug where errors in third-party extensions such as
6668 hypothesis-trio or hypothesis-jsonschema were incorrectly considered to
6669 be Hypothesis internal errors, which could result in confusing error
6670 messages.
6671 Thanks to Vincent Michel for reporting and fixing the bug!
6673 5.1.3 - 2020-01-11
6675 This release converts the type hint comments on our public API to PEP
6676 484 type annotations.
6677 Thanks to Ivan Levkivskyi for com2ann - with the refactoring tools from
6679 5.0.1 it made this process remarkably easy!
6680 5.1.2 - 2020-01-09
6682 This patch makes multiple() iterable, so that output like a, b =
6683 state.some_rule() is actually executable and can be used to reproduce
6684 failing examples.
6685 Thanks to Vincent Michel for reporting and fixing issue #2311!
6687 5.1.1 - 2020-01-06
6689 This patch contains many small refactorings to replace our Python 2
6690 compatibility functions with their native Python 3 equivalents. Since
6691 Hypothesis is now Python 3 only, there is no user-visible change.
6692 5.1.0 - 2020-01-03
6694 This release teaches from_type() how to generate python:datetime.time‐
6695 zone. As a result, you can now generate python:datetime.tzinfo objects
6696 without having pytz installed.
6697 If your tests specifically require pytz timezones, you should be using
6699 hypothesis.extra.pytz.timezones() instead of st.from_type(tzinfo).
6700 5.0.1 - 2020-01-01
6702 This patch contains mostly-automated refactorings to remove code that
6703 we only needed to support Python 2. Since Hypothesis is now Python 3
6704 only (hurray!), there is no user-visible change.
6705 Our sincere thanks to the authors of autoflake, black, isort, and
6707 pyupgrade, who have each and collectively made this kind of update
6708 enormously easier.
6709 5.0.0 - 2020-01-01
6711 Welcome to the next major version of Hypothesis!
6712 There are no new features here, as we release those in minor versions.
6714 Instead, 5.0 is a chance for us to remove deprecated features (many al‐
6715 ready converted into no-ops), and turn a variety of warnings into er‐
6716 rors.
6717 If you were running on the last version of Hypothesis 4.x without any
6719 Hypothesis deprecation warnings, this will be a very boring upgrade.
6720 In fact, nothing will change for you at all.
6721 NOTE:
6723 This release drops support for Python 2, which has passed its end of
6724 life date. The Python 3 Statement outlines our reasons, and lists
6725 many other packages that have made the same decision.
6726 pip install hypothesis should continue to give you the latest com‐
6728 patible version. If you have somehow ended up with Hypothesis 5.0
6729 on Python 2, you need to update your packaging stack to pip >= 9.0
6730 and setuptools >= 24.2 - see here for details. Then pip uninstall
6731 hypothesis && pip install hypothesis will get you back to a compati‐
6732 ble version.
6733 Strategies
6735 • integers() bounds must be equal to an integer, though they can still
6736 be other types.
6737 • If fractions() is passed a max_denominator, the bounds must have at
6739 most that denominator.
6740 • floats() bounds must be exactly representable as a floating-point
6742 number with the given width. If not, the error message includes the
6743 nearest such number.
6744 • sampled_from([]) is now an error.
6746 • The values from the elements and fill strategies for hypothesis.ex‐
6748 tra.numpy.arrays() must be losslessly representable in an array of
6749 the given dtype.
6750 • The min_size and max_size arguments to all collection strategies must
6752 be of type python:int (or max_size may be None).
6753 Miscellaneous
6755 • The .example() method of strategies (intended for interactive explo‐
6756 ration) no longer takes a random argument.
6757 • It is now an error to apply @example, @seed, or @reproduce_failure
6759 without also applying @given.
6760 • You may pass either the target or targets argument to stateful rules,
6762 but not both.
6763 • deadline must be None (to disable), a timedelta, or an integer or
6765 float number of milliseconds.
6766 • Both of derandomize and print_blob must be either True or False,
6768 where they previously accepted other values.
6769 • stateful_step_count must be at least one.
6771 • max_examples must be at least one. To disable example generation,
6773 use the phases setting.
6774 Removals
6776 • hypothesis.stateful.GenericStateMachine in favor of hypothesis.state‐
6777 ful.RuleBasedStateMachine
6778 • hypothesis.extra.django.models.models in favor of hypothesis.ex‐
6780 tra.django.from_model() and hypothesis.extra.django.models.add_de‐
6781 fault_field_mapping in favor of hypothesis.extra.django.regis‐
6782 ter_field_strategy()
6783 • hypothesis.HealthCheck.hung_test, without replacement
6785 • hypothesis.settings.buffer, without replacement
6787 • hypothesis.PrintSettings, because hypothesis.settings.print_blob
6789 takes True or False
6790 • hypothesis.settings.timeout, in favor of hypothesis.settings.deadline
6792 • hypothesis.unlimited without replacement (only only useful as argu‐
6794 ment to timeout)
6795 Hypothesis 4.x
6797 4.57.1 - 2019-12-29
6798 This patch improves the type hints and documentation for the django ex‐
6799 tra. There is no runtime change.
6800 4.57.0 - 2019-12-28
6802 This release improves support for the SupportsOp protocols from the
6803 python:typing module when using on from_type() as outlined in issue
6804 #2292. The following types now generate much more varied strategies
6805 when called with from_type():
6806 • python:typing.SupportsAbs
6808 • python:typing.SupportsBytes
6810 • python:typing.SupportsComplex
6812 • python:typing.SupportsInt
6814 • python:typing.SupportsFloat
6816 • python:typing.SupportsRound
6818 Note that using from_type() with one of the above strategies will not
6820 ensure that the the specified function will execute successfully (ie :
6821 the strategy returned for from_type(typing.SupportsAbs) may include
6822 NaNs or things which cause the python:abs() function to error. )
6823 Thanks to Lea Provenzano for this patch.
6825 4.56.3 - 2019-12-22
6827 This release fixes a small internal bug in shrinking which could have
6828 caused it to perform slightly more tests than were necessary. Fixing
6829 this shouldn't have much effect but it will make shrinking slightly
6830 faster.
6831 4.56.2 - 2019-12-21
6833 This release removes an internal heuristic that was no longer providing
6834 much benefit. It is unlikely that there will be any user visible ef‐
6835 fect.
6836 4.56.1 - 2019-12-19
6838 This release further improves the optimisation algorithm for targeted
6839 property-based testing.
6840 4.56.0 - 2019-12-18
6842 This release enables deprecation warnings even when the verbosity set‐
6843 ting is quiet, in preparation for Hypothesis 5.0 (issue #2218).
6844 Warnings can still be filtered by the standard mechanisms provided in
6846 the standard-library python:warnings module.
6847 4.55.4 - 2019-12-18
6849 This release improves Hypothesis's management of the set of test cases
6850 it tracks between runs. It will only do anything if you have Phase.tar‐
6851 get enabled and an example database set. In those circumstances it
6852 should result in a more thorough and faster set of examples that are
6853 tried on each run.
6854 4.55.3 - 2019-12-18
6856 This release makes Hypothesis better at generating test cases where
6857 generated values are duplicated in different parts of the test case.
6858 This will be especially noticeable with reasonably complex values, as
6859 it was already able to do this for simpler ones such as integers or
6860 floats.
6861 4.55.2 - 2019-12-17
6863 This release expands the set of test cases that Hypothesis saves in its
6864 database for future runs to include a representative set of "struc‐
6865 turally different" test cases - e.g. it might try to save test cases
6866 where a given list is empty or not.
6867 Currently this is unlikely to have much user visible impact except to
6869 produce slightly more consistent behaviour between consecutive runs of
6870 a test suite. It is mostly groundwork for future improvements which
6871 will exploit this functionality more effectively.
6872 4.55.1 - 2019-12-16
6874 This patch fixes issue #2257, where from_type() could incorrectly gen‐
6875 erate bytestrings when passed a generic python:typing.Sequence such as
6876 Sequence[set].
6877 4.55.0 - 2019-12-16
6879 This release adds database support for targeted property-based testing,
6880 so the best examples based on the targeting will be saved and reused
6881 between runs. This is mostly laying groundwork for future features in
6882 this area, but will also make targeted property-based tests more useful
6883 during development, where the same tests tend to get run over and over
6884 again.
6885 If max_examples is large, this may increase memory usage significantly
6887 under some circumstances, but these should be relatively rare.
6888 This release also adds a dependency on the sortedcontainers package.
6890 4.54.2 - 2019-12-16
6892 This release improves the optimisation algorithm for targeted prop‐
6893 erty-based testing, so that it will find higher quality results more
6894 reliably. Specifically, in cases where it would previously have got
6895 near a local optimum, it will now tend to achieve the locally optimal
6896 value.
6897 4.54.1 - 2019-12-16
6899 This release is mostly internal changes in support of better testing of
6900 the core engine. You are unlikely to see much effect, although some in‐
6901 ternal heuristics have changed slightly.
6902 4.54.0 - 2019-12-15
6904 This release adds a dedicated phase for targeted property-based test‐
6905 ing, and (somewhat) improves the targeting algorithm so that it will
6906 find higher quality results more reliably. This comes at a cost of
6907 making it more likely to get stuck in a local optimum.
6908 4.53.3 - 2019-12-15
6910 This patch fixes from_type() with python:typing.Hashable and
6911 python:typing.Sized, which previously failed with an internal error on
6912 Python 3.7 or later.
6913 Thanks to Lea Provenzano for both reporting issue #2272 and writing the
6915 patch!
6916 4.53.2 - 2019-12-11
6918 This release reorganises a number of the Hypothesis internal modules
6919 into a package structure. If you are only depending on the public API
6920 it should have no effect. If you are depending on the internal API
6921 (which you shouldn't be, and which we don't guarantee compatibility on)
6922 you may have to rename some imports.
6923 4.53.1 - 2019-12-09
6925 This release changes the size distribution of the number of steps run
6926 in stateful testing: It will now almost always run the maximum number
6927 of steps permitted.
6928 4.53.0 - 2019-12-09
6930 statistics now include the best score seen for each label, which can
6931 help avoid the threshold problem when the minimal example shrinks
6932 right down to the threshold of failure (issue #2180).
6933 4.52.0 - 2019-12-09
6935 This release changes the stateful_step_count setting to raise an error
6936 if set to 0. This is a backwards compatible change because a value of 0
6937 would never have worked and attempting to run it would have resulted in
6938 an internal assertion error.
6939 4.51.1 - 2019-12-09
6941 This release makes a small internal change to the distribution of test
6942 cases. It is unlikely to have much user visible impact.
6943 4.51.0 - 2019-12-07
6945 This release deprecates use of @example, @seed, or @reproduce_failure
6946 without @given.
6947 Thanks to Nick Anyos for the patch!
6949 4.50.8 - 2019-12-05
6951 This patch makes certain uses of Bundles more efficient in stateful
6952 testing (issue #2078).
6953 4.50.7 - 2019-12-05
6955 This release refactors some of Hypothesis's internal interfaces for
6956 representing data generation. It should have no user visible effect.
6957 4.50.6 - 2019-12-02
6959 This patch removes some old debugging helpers in our Numpy extra which
6960 have not been needed since issue #1963 and issue #2245.
6961 4.50.5 - 2019-12-01
6963 This patch fixes issue #2229, where Numpy arrays of unsized strings
6964 would only ever have strings of size one due to an interaction between
6965 our generation logic and Numpy's allocation strategy.
6966 4.50.4 - 2019-12-01
6968 This patch fixes a rare internal error in strategies for a list of
6969 unique items sampled from a short non-unique sequence (issue #2247).
6970 The bug was discovered via hypothesis-jsonschema.
6971 4.50.3 - 2019-12-01
6973 This release improves the error message when @settings tries to inherit
6974 settings from a parent argument that isn't a settings instance.
6975 4.50.2 - 2019-11-29
6977 This release improves Hypothesis's "Falsifying example" output, by
6978 breaking output across multiple lines where necessary, and by removing
6979 irrelevant information from the stateful testing output.
6980 4.50.1 - 2019-11-29
6982 This patch adds flake8-comprehensions to our linter suite. There is no
6983 user-visible change - expect perhaps via some strange microbenchmarks -
6984 but certain parts of the code now have a clear and more consistent
6985 style.
6986 4.50.0 - 2019-11-28
6988 This release fixes some cases where we might previously have failed to
6989 run the validation logic for some strategies. As a result tests which
6990 would previously have been silently testing significantly less than
6991 they should may now start to raise InvalidArgument now that these er‐
6992 rors are caught.
6993 4.49.0 - 2019-11-28
6995 This release significantly improves the data distribution in rule based
6996 stateful testing, by using a technique called Swarm Testing (Groce,
6997 Alex, et al. "Swarm testing." Proceedings of the 2012 International
6998 Symposium on Software Testing and Analysis. ACM, 2012.) to select
6999 which rules are run in any given test case. This should allow it to
7000 find many issues that it would previously have missed.
7001 This change is likely to be especially beneficial for stateful tests
7003 with large numbers of rules.
7004 4.48.1 - 2019-11-28
7006 This release adds some heuristics to test case generation that try to
7007 ensure that test cases generated early on will be relatively small.
7008 This fixes a bug introduced in Hypothesis 4.42.0 which would cause oc‐
7010 casional too_slow failures on some tests.
7011 4.48.0 - 2019-11-28
7013 This release revokes the deprecation of find, as we've now rebuilt it
7014 on top of @given, which means it has minimal maintenance burden and
7015 we're happy to support it.
7016 4.47.5 - 2019-11-28
7018 This release rebuilds find() on top of @given in order to have more
7019 code in common. It should have minimal user visible effect.
7020 4.47.4 - 2019-11-27
7022 This patch removes an internal compatibility shim that we no longer
7023 need.
7024 4.47.3 - 2019-11-26
7026 This patch fixes several typos in our docstrings and comments, with no
7027 change in behaviour. Thanks to Dmitry Dygalo for identifying and fix‐
7028 ing them!
7029 4.47.2 - 2019-11-25
7031 This release fixes an internal issue where Hypothesis would sometimes
7032 generate test cases that were above its intended maximum size. This
7033 would only have happened rarely and probably would not have caused ma‐
7034 jor problems when it did.
7035 Users of the new targeted property-based testing might see minor im‐
7037 pact (possibly slightly faster tests and slightly worse target scores),
7038 but only in the unlikely event that they were hitting this problem.
7039 Other users should not see any effect at all.
7040 4.47.1 - 2019-11-24
7042 This release removes some unused code from the core engine. There is
7043 no user-visible change.
7044 4.47.0 - 2019-11-24
7046 This release commonizes some code between running explicit examples and
7047 normal test execution. The main user visible impact of this is that
7048 deadlines are now enforced when running explicit examples.
7049 4.46.1 - 2019-11-23
7051 This patch ensures that a KeyboardInterrupt received during example
7052 generation is not treated as a mystery test failure but instead propa‐
7053 gates to the top level, not recording the interrupted generation in the
7054 conjecture data tree. Thanks to Anne Archibald for identifying and
7055 fixing the problem.
7056 4.46.0 - 2019-11-22
7058 This release changes the behaviour of floats() when excluding signed
7059 zeros - floats(max_value=0.0, exclude_max=True) can no longer generate
7060 -0.0 nor the much rarer floats(min_value=-0.0, exclude_min=True) gener‐
7061 ate +0.0.
7062 The correct interaction between signed zeros and exclusive endpoints
7064 was unclear; we now enforce the invariant that floats() will never gen‐
7065 erate a value equal to an excluded endpoint (issue #2201).
7066 If you prefer the old behaviour, you can pass floats(max_value=-0.0) or
7068 floats(min_value=0.0) which is exactly equivalent and has not changed.
7069 If you had two endpoints equal to zero, we recommend clarifying your
7070 tests by using just() or sampled_from() instead of floats().
7071 4.45.1 - 2019-11-20
7073 This patch improves the error message when invalid arguments are passed
7074 to rule() or invariant() (issue #2149).
7075 Thanks to Benjamin Palmer for this bugfix!
7077 4.45.0 - 2019-11-20
7079 This release supports python:typing.Final and python:typing.TypedDict
7080 in from_type().
7081 4.44.5 - 2019-11-20
7083 This patch disables our pytest plugin when running on versions of
7084 pytest before 4.3, the oldest our plugin supports. Note that at time
7085 of writing the Pytest developers only support 4.6 and later!
7086 Hypothesis tests using @given() work on any test runner, but our inte‐
7088 grations to e.g. avoid example database collisions when combined with
7089 @pytest.mark.parametrize eventually drop support for obsolete versions.
7090 4.44.4 - 2019-11-20
7092 This patch adds some internal comments and clarifications to the Hy‐
7093 pothesis implementation. There is no user-visible change.
7094 4.44.3 - 2019-11-20
7096 This patch avoids importing test runners such as pytest, unittest2, or
7097 nose solely to access their special "skip test" exception types - if
7098 the module is not in sys.modules, the exception can't be raised anyway.
7099 This fixes a problem where importing an otherwise unused module could
7101 cause spurious errors due to import-time side effects (and possibly
7102 -Werror).
7103 4.44.2 - 2019-11-12
7105 This release fixes @given to only complain about missing keyword-only
7106 arguments if the associated test function is actually called.
7107 This matches the behaviour of other InvalidArgument errors produced by
7109 @given.
7110 4.44.1 - 2019-11-11
7112 This patch allows Hypothesis to run in environments that do not specify
7113 a __file__, such as a python:zipapp (issue #2196).
7114 4.44.0 - 2019-11-11
7116 This release adds a signature argument to mutually_broad‐
7117 castable_shapes() (issue #2174), which allows us to generate shapes
7118 which are valid for functions like numpy:numpy.matmul that require
7119 shapes which are not simply broadcastable.
7120 Thanks to everyone who has contributed to this feature over the last
7122 year, and a particular shout-out to Zac Hatfield-Dodds and Ryan Sok‐
7123 laski for mutually_broadcastable_shapes() and to Ryan Turner for the
7124 downstream hypothesis-gufunc project.
7125 4.43.9 - 2019-11-11
7127 This patch fixes issue #2108, where the first test using data() to draw
7128 from characters() or text() would be flaky due to unreliable test tim‐
7129 ings.
7130 Time taken by lazy instantiation of strategies is now counted towards
7132 drawing from the strategy, rather than towards the deadline for the
7133 test function.
7134 4.43.8 - 2019-11-08
7136 This release ensures that the strategies passed to @given are properly
7137 validated when applied to a test method inside a test class.
7138 This should result in clearer error messages when some of those strate‐
7140 gies are invalid.
7141 4.43.7 - 2019-11-08
7143 This release changes how Hypothesis manages its search space in cases
7144 where it generates redundant data. This should cause it to generate
7145 significantly fewer duplicated examples (especially with short integer
7146 ranges), and may cause it to produce more useful examples in some cases
7147 (especially ones where there is a significant amount of filtering).
7148 4.43.6 - 2019-11-07
7150 This patch refactors width handling in floats(); you may notice small
7151 performance improvements but the main purpose is to enable work on
7152 issue #1704 (improving shrinking of bounded floats).
7153 4.43.5 - 2019-11-06
7155 This patch removes an unused internal flag. There is no user-visible
7156 change.
7157 4.43.4 - 2019-11-05
7159 This patch corrects the exception type and error message you get if you
7160 attempt to use data() to draw from something which is not a strategy.
7161 This never worked, but the error is more helpful now.
7162 4.43.3 - 2019-11-05
7164 We've adopted flake8-bugbear to check for a few more style issues, and
7165 this patch implements the minor internal cleanups it suggested. There
7166 is no user-visible change.
7167 4.43.2 - 2019-11-05
7169 This patch fixes the formatting of some documentation, but there is no
7170 change to any executed code.
7171 4.43.1 - 2019-11-04
7173 Python 3.8's new python:typing.Literal type - see PEP 586 for details -
7174 is now supported in from_type().
7175 4.43.0 - 2019-11-04
7177 This release adds the strategy mutually_broadcastable_shapes(), which
7178 generates multiple array shapes that are mutually broadcast-compatible
7179 with an optional user-specified base-shape.
7180 This is a generalisation of broadcastable_shapes(). It relies heavily
7182 on non-public internals for performance when generating and shrinking
7183 examples. We intend to support generating shapes matching a ufunc sig‐
7184 nature in a future version (issue #2174).
7185 Thanks to Ryan Soklaski, Zac Hatfield-Dodds, and @rdturnermtl who con‐
7187 tributed to this new feature.
7188 4.42.10 - 2019-11-03
7190 This release fixes from_type() when used with bounded or constrained
7191 python:typing.TypeVar objects (issue #2094).
7192 Previously, distinct typevars with the same constraints would be
7194 treated as all single typevar, and in cases where a typevar bound was
7195 resolved to a union of subclasses this could result in mixed types be‐
7196 ing generated for that typevar.
7197 4.42.9 - 2019-11-03
7199 This patch ensures that the default value broadcastable_shapes()
7200 chooses for max_dims is always valid (at most 32), even if you pass
7201 min_dims=32.
7202 4.42.8 - 2019-11-02
7204 This patch ensures that we only add profile information to the pytest
7205 header if running either pytest or Hypothesis in verbose mode, matching
7206 the builtin cache plugin (issue #2155).
7207 4.42.7 - 2019-11-02
7209 This patch makes stateful step printing expand the result of a step
7210 into multiple variables when you return multiple() (issue #2139).
7211 Thanks to Joseph Weston for reporting and fixing this bug!
7212 4.42.6 - 2019-11-02
7214 This release fixes a bug (issue #2166) where a Unicode character info
7215 cache file was generated but never used on subsequent test runs, caus‐
7216 ing tests to run more slowly than they should have.
7217 Thanks to Robert Knight for this bugfix!
7219 4.42.5 - 2019-11-01
7221 This patch corrects some internal documentation. There is no user-vis‐
7222 ible change.
7223 4.42.4 - 2019-11-01
7225 This release fixes a bug (issue #2160) where decorators applied after
7226 @settings and before @given were ignored.
7227 Thanks to Tom Milligan for this bugfix!
7229 4.42.3 - 2019-10-30
7231 This release updates Hypothesis's formatting to the new version of
7232 black, and has absolutely no user visible effect.
7233 4.42.2 - 2019-10-30
7235 This release fixes a bug in recursive() which would have meant that in
7236 practice max_leaves was treated as if it was lower than it actually is
7237 - specifically it would be capped at the largest power of two smaller
7238 than it. It is now handled correctly.
7239 4.42.1 - 2019-10-30
7241 Python 3.8's new python:typing.SupportsIndex type - see PEP 357 for de‐
7242 tails - is now supported in from_type().
7243 Thanks to Grigorios Giannakopoulos for the patch!
7245 4.42.0 - 2019-10-27
7247 This release significantly simplifies Hypothesis's internal logic for
7248 data generation, by removing a number of heuristics of questionable or
7249 unproven value.
7250 The results of this change will vary significantly from test to test.
7252 Most test suites will see significantly faster data generation and
7253 lower memory usage. The "quality" of the generated data may go up or
7254 down depending on your particular test suites.
7255 If you see any significant regressions in Hypothesis's ability to find
7257 bugs in your code as a result of this release, please file an issue to
7258 let us know.
7259 Users of the new targeted property-based testing functionality are
7261 reasonably likely to see improvements in data generation, as this re‐
7262 lease changes the search algorithm for targeted property based testing
7263 to one that is more likely to be productive than the existing approach.
7264 4.41.3 - 2019-10-21
7266 This patch is to ensure that our internals remain comprehensible to
7267 mypy 0.740 - there is no user-visible change.
7268 4.41.2 - 2019-10-17
7270 This patch changes some internal hashes to SHA384, to better support
7271 users subject to FIPS-140. There is no user-visible API change.
7272 Thanks to Paul Kehrer for this contribution!
7274 4.41.1 - 2019-10-16
7276 This release makes --hypothesis-show-statistics much more useful for
7277 tests using a RuleBasedStateMachine, by simplifying the reprs so that
7278 events are aggregated correctly.
7279 4.41.0 - 2019-10-16
7281 This release upgrades the fixed_dictionaries() strategy to support op‐
7282 tional keys (issue #1913).
7283 4.40.2 - 2019-10-16
7285 This release makes some minor internal changes in support of improving
7286 the Hypothesis test suite. It should not have any user visible impact.
7287 4.40.1 - 2019-10-14
7289 This release changes how Hypothesis checks if a parameter to a test
7290 function is a mock object. It is unlikely to have any noticeable ef‐
7291 fect, but may result in a small performance improvement, especially for
7292 test functions where a mock object is being passed as the first argu‐
7293 ment.
7294 4.40.0 - 2019-10-09
7296 This release fixes a bug where our example database logic did not dis‐
7297 tinguish between failing examples based on arguments from a
7298 @pytest.mark.parametrize(...). This could in theory cause data loss if
7299 a common failure overwrote a rare one, and in practice caused occa‐
7300 sional file-access collisions in highly concurrent workloads (e.g. dur‐
7301 ing a 300-way parametrize on 16 cores).
7302 For internal reasons this also involves bumping the minimum supported
7304 version of pytest to 4.3
7305 Thanks to Peter C Kroon for the Hacktoberfest patch!
7307 4.39.3 - 2019-10-09
7309 This patch improves our type hints on the emails(), functions(), inte‐
7310 gers(), iterables(), and slices() strategies, as well as the .filter()
7311 method.
7312 There is no runtime change, but if you use mypy or a similar
7314 type-checker on your tests the results will be a bit more precise.
7315 4.39.2 - 2019-10-09
7317 This patch improves the performance of unique collections such as
7318 sets() of just() or booleans() strategies. They were already pretty
7319 good though, so you're unlikely to notice much!
7320 4.39.1 - 2019-10-09
7322 If a value in a dict passed to fixed_dictionaries() is not a strategy,
7323 Hypothesis now tells you which one.
7324 4.39.0 - 2019-10-07
7326 This release adds the basic_indices() strategy, to generate basic in‐
7327 dexes for arrays of the specified shape (issue #1930).
7328 It generates tuples containing some mix of integers, python:slice ob‐
7330 jects, ... (Ellipsis), and numpy:numpy.newaxis; which when used to in‐
7331 dex an array of the specified shape produce either a scalar or a
7332 shared-memory view of the array. Note that the index tuple may be
7333 longer or shorter than the array shape, and may produce a view with an‐
7334 other dimensionality again!
7335 Thanks to Lampros Mountrakis, Ryan Soklaski, and Zac Hatfield-Dodds for
7337 their collaboration on this surprisingly subtle strategy!
7338 4.38.3 - 2019-10-04
7340 This patch defers creation of the .hypothesis directory until we have
7341 something to store in it, meaning that it will appear when Hypothesis
7342 is used rather than simply installed.
7343 Thanks to Peter C Kroon for the Hacktoberfest patch!
7345 4.38.2 - 2019-10-02
7347 This patch bumps our dependency on attrs to >=19.2.0; but there are no
7348 user-visible changes to Hypothesis.
7349 4.38.1 - 2019-10-01
7351 This is a comment-only patch which tells mypy 0.730 to ignore some in‐
7352 ternal compatibility shims we use to support older Pythons.
7353 4.38.0 - 2019-10-01
7355 This release adds the hypothesis.target() function, which implements
7356 experimental support for targeted property-based testing (issue #1779).
7357 By calling target() in your test function, Hypothesis can do a
7359 hill-climbing search for bugs. If you can calculate a suitable metric
7360 such as the load factor or length of a queue, this can help you find
7361 bugs with inputs that are highly improbably from unguided generation -
7362 however good our heuristics, example diversity, and deduplication logic
7363 might be. After all, those features are at work in targeted PBT too!
7364 4.37.0 - 2019-09-28
7366 This release emits a warning if you use the .example() method of a
7367 strategy in a non-interactive context.
7368 given() is a much better choice for writing tests, whether you care
7370 about performance, minimal examples, reproducing failures, or even just
7371 the variety of inputs that will be tested!
7372 4.36.2 - 2019-09-20
7374 This patch disables part of the typing-based inference for the attrs
7375 package under Python 3.5.0, which has some incompatible internal de‐
7376 tails (issue #2095).
7377 4.36.1 - 2019-09-17
7379 This patch fixes a bug in strategy inference for attrs classes where
7380 Hypothesis would fail to infer a strategy for attributes of a generic
7381 type such as Union[int, str] or List[bool] (issue #2091).
7382 Thanks to Jonathan Gayvallet for the bug report and this patch!
7384 4.36.0 - 2019-09-09
7386 This patch deprecates min_len or max_len of 0 in byte_string_dtypes()
7387 and unicode_string_dtypes(). The lower limit is now 1.
7388 Numpy uses a length of 0 in these dtypes to indicate an undetermined
7390 size, chosen from the data at array creation. However, as the arrays()
7391 strategy creates arrays before filling them, strings were truncated to
7392 1 byte.
7393 4.35.1 - 2019-09-09
7395 This patch improves the messaging that comes from invalid size argu‐
7396 ments to collection strategies such as lists().
7397 4.35.0 - 2019-09-04
7399 This release improves the from_lark() strategy, tightening argument
7400 validation and adding the explicit argument to allow use with terminals
7401 that use @declare instead of a string or regular expression.
7402 This feature is required to handle features such as indent and dedent
7404 tokens in Python code, which can be generated with the hypothesmith
7405 package.
7406 4.34.0 - 2019-08-23
7408 The from_type() strategy now knows to look up the subclasses of ab‐
7409 stract types, which cannot be instantiated directly.
7410 This is very useful for hypothesmith to support libCST.
7412 4.33.1 - 2019-08-21
7414 This patch works around a crash when an incompatible version of Numpy
7415 is installed under PyPy 5.10 (Python 2.7).
7416 If you are still using Python 2, please upgrade to Python 3 as soon as
7418 possible - it will be unsupported at the end of this year.
7419 4.33.0 - 2019-08-20
7421 This release improves the domains() strategy, as well as the urls() and
7422 the emails() strategies which use it. These strategies now use the
7423 full IANA list of Top Level Domains and are correct as per RFC 1035.
7424 Passing tests using these strategies may now fail.
7426 Thanks to TechDragon for this improvement.
7428 4.32.3 - 2019-08-05
7430 This patch tidies up the repr of several settings-related objects, at
7431 runtime and in the documentation, and deprecates the undocumented edge
7432 case that phases=None was treated like phases=tuple(Phase).
7433 It also fixes from_lark() with lark 0.7.2 and later.
7435 4.32.2 - 2019-07-30
7437 This patch updates some internal comments for mypy 0.720. There is no
7438 user-visible impact.
7439 4.32.1 - 2019-07-29
7441 This release changes how the shrinker represents its progress inter‐
7442 nally. For large generated test cases this should result in signifi‐
7443 cantly less memory usage and possibly faster shrinking. Small generated
7444 test cases may be slightly slower to shrink but this shouldn't be very
7445 noticeable.
7446 4.32.0 - 2019-07-28
7448 This release makes arrays() more pedantic about elements strategies
7449 that cannot be exactly represented as array elements.
7450 In practice, you will see new warnings if you were using a float16 or
7452 float32 dtype without passing floats() the width=16 or width=32 argu‐
7453 ments respectively.
7454 The previous behaviour could lead to silent truncation, and thus some
7456 elements being equal to an explicitly excluded bound (issue #1899).
7457 4.31.1 - 2019-07-28
7459 This patch changes an internal use of MD5 to SHA hashes, to better sup‐
7460 port users subject to FIPS-140. There is no user-visible or API
7461 change.
7462 Thanks to Alex Gaynor for this patch.
7464 4.31.0 - 2019-07-24
7466 This release simplifies the logic of the print_blob setting by removing
7467 the option to set it to PrintSettings.INFER. As a result the
7468 print_blob setting now takes a single boolean value, and the use of
7469 PrintSettings is deprecated.
7470 4.28.2 - 2019-07-14
7472 This patch improves the docstrings of several Hypothesis strategies, by
7473 clarifying markup and adding cross-references. There is no runtime
7474 change.
7475 Thanks to Elizabeth Williams and Serah Njambi Rono for their contribu‐
7477 tions at the SciPy 2019 sprints!
7478 4.28.1 - 2019-07-12
7480 This patch improves the behaviour of the text() strategy when passed an
7481 alphabet which is not a strategy. The value is now interpreted as
7482 whitelist_characters to characters() instead of a sequence for sam‐
7483 pled_from(), which standardises the distribution of examples and the
7484 shrinking behaviour.
7485 You can get the previous behaviour by using lists(sampled_from(alpha‐
7487 bet)).map("".map) instead.
7488 4.28.0 - 2019-07-11
7490 This release deprecates find(). The .example() method is a better re‐
7491 placement if you want an example, and for the rare occasions where you
7492 want the minimal example you can get it from @given.
7493 @given has steadily outstripped find() in both features and performance
7495 over recent years, and as we do not have the resources to maintain and
7496 test both we think it is better to focus on just one.
7497 4.27.0 - 2019-07-08
7499 This release refactors the implementation of the .example() method, to
7500 more accurately represent the data which will be generated by @given.
7501 As a result, calling s.example() on an empty strategy s (such as noth‐
7503 ing()) now raises Unsatisfiable instead of the old NoExamples excep‐
7504 tion.
7505 4.26.4 - 2019-07-07
7507 This patch ensures that the Pandas extra will keep working when Python
7508 3.8 removes abstract base classes from the top-level python:collections
7509 namespace. This also fixes the relevant warning in Python 3.7, but
7510 there is no other difference in behaviour and you do not need to do
7511 anything.
7512 4.26.3 - 2019-07-05
7514 This release fixes issue #2027, by changing the way Hypothesis tries
7515 to generate distinct examples to be more efficient.
7516 This may result in slightly different data distribution, and should im‐
7518 prove generation performance in general, but should otherwise have min‐
7519 imal user impact.
7520 4.26.2 - 2019-07-04
7522 This release fixes issue #1864, where some simple tests would perform
7523 very slowly, because they would run many times with each subsequent run
7524 being progressively slower. They will now stop after a more reasonable
7525 number of runs without hitting this problem.
7526 Unless you are hitting exactly this issue, it is unlikely that this re‐
7528 lease will have any effect, but certain classes of custom generators
7529 that are currently very slow may become a bit faster, or start to trig‐
7530 ger health check failures.
7531 4.26.1 - 2019-07-04
7533 This release adds the strategy integer_array_indices(), which generates
7534 tuples of Numpy arrays that can be used for advanced indexing to select
7535 an array of a specified shape.
7536 4.26.0 - 2019-07-04
7538 This release significantly improves the performance of drawing unique
7539 collections whose elements are drawn from sampled_from() strategies.
7540 As a side effect, this detects an error condition that would previously
7542 have passed silently: When the min_size argument on a collection with
7543 distinct elements is greater than the number of elements being sampled,
7544 this will now raise an error.
7545 4.25.1 - 2019-07-03
7547 This release removes some defunct internal functionality that was only
7548 being used for testing. It should have no user visible impact.
7549 4.25.0 - 2019-07-03
7551 This release deprecates and disables the buffer_size setting, which
7552 should have been treated as a private implementation detail all along.
7553 We recommend simply deleting this settings argument.
7554 4.24.6 - 2019-06-26
7556 This patch makes datetimes() more efficient, as it now handles short
7557 months correctly by construction instead of filtering.
7558 4.24.5 - 2019-06-23
7560 This patch improves the development experience by simplifying the
7561 tracebacks you will see when e.g. you have used the .map(...) method of
7562 a strategy and the mapped function raises an exception.
7563 No new exceptions can be raised, nor existing exceptions change any‐
7565 thing but their traceback. We're simply using if-statements rather
7566 than exceptions for control flow in a certain part of the internals!
7567 4.24.4 - 2019-06-21
7569 This patch fixes issue #2014, where our compatibility layer broke with
7570 version 3.7.4 of the typing module backport on PyPI.
7571 This issue only affects Python 2. We remind users that Hypothesis,
7573 like many other packages, will drop Python 2 support on 2020-01-01 and
7574 already has several features that are only available on Python 3.
7575 4.24.3 - 2019-06-07
7577 This patch improves the implementation of an internal wrapper on Python
7578 3.8 beta1 (and will break on the alphas; but they're not meant to be
7579 stable). On other versions, there is no change at all.
7580 Thanks to Daniel Hahler for the patch, and Victor Stinner for his work
7582 on bpo-37032 that made it possible.
7583 4.24.2 - 2019-06-06
7585 Deprecation messages for functions in hypothesis.extra.django.models
7586 now explicitly name the deprecated function to make it easier to track
7587 down usages. Thanks to Kristian Glass for this contribution!
7588 4.24.1 - 2019-06-04
7590 This patch fixes issue #1999, a spurious bug raised when a @st.compos‐
7591 ite function was passed a keyword-only argument.
7592 Thanks to Jim Nicholls for his fantastic bug report.
7594 4.24.0 - 2019-05-29
7596 This release deprecates GenericStateMachine, in favor of RuleBased‐
7597 StateMachine. Rule-based stateful testing is significantly faster, es‐
7598 pecially during shrinking.
7599 If your use-case truly does not fit rule-based stateful testing, we
7601 recommend writing a custom test function which drives your specific
7602 control-flow using data().
7603 4.23.9 - 2019-05-28
7605 This patch fixes a very rare example database issue with file permis‐
7606 sions.
7607 When running a test that uses both @given and pytest.mark.parametrize,
7609 using pytest-xdist on Windows, with failing examples in the database,
7610 two attempts to read a file could overlap and we caught FileNotFound
7611 but not other OSErrors.
7612 4.23.8 - 2019-05-26
7614 This patch has a minor cleanup of the internal engine. There is no
7615 user-visible impact.
7616 4.23.7 - 2019-05-26
7618 This patch clarifies some error messages when the test function signa‐
7619 ture is incompatible with the arguments to @given, especially when the
7620 @settings() decorator is also used (issue #1978).
7621 4.23.6 - 2019-05-19
7623 This release adds the pyupgrade fixer to our code style, for consistent
7624 use of dict and set literals and comprehensions.
7625 4.23.5 - 2019-05-16
7627 This release slightly simplifies a small part of the core engine.
7628 There is no user-visible change.
7629 4.23.4 - 2019-05-09
7631 Fixes a minor formatting issue the docstring of from_type()
7632 4.23.3 - 2019-05-09
7634 Adds a recipe to the docstring of from_type() that describes a means
7635 for drawing values for "everything except" a specified type. This
7636 recipe is especially useful for writing tests that perform input-type
7637 validation.
7638 4.23.2 - 2019-05-08
7640 This patch uses autoflake to remove some pointless pass statements,
7641 which improves our workflow but has no user-visible impact.
7642 4.23.1 - 2019-05-08
7644 This patch fixes an OverflowError in from_type(xrange) on Python 2.
7645 It turns out that not only do the start and stop values have to fit in
7647 a C long, but so does stop - start. We now handle this even on 32bit
7648 platforms, but remind users that Python2 will not be supported after
7649 2019 without specific funding.
7650 4.23.0 - 2019-05-08
7652 This release implements the slices() strategy, to generate slices of a
7653 length-size sequence.
7654 Thanks to Daniel J. West for writing this patch at the PyCon 2019
7656 sprints!
7657 4.22.3 - 2019-05-07
7659 This patch exposes DataObject, solely to support more precise type
7660 hints. Objects of this type are provided by data(), and can be used to
7661 draw examples from strategies intermixed with your test code.
7662 4.22.2 - 2019-05-07
7664 This patch fixes the very rare issue #1798 in array_dtypes(), which
7665 caused an internal error in our tests.
7666 4.22.1 - 2019-05-07
7668 This patch fixes a rare bug in from_type(range).
7669 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
7671 Sprints.
7672 4.22.0 - 2019-05-07
7674 The unique_by argument to lists now accepts a tuple of callables such
7675 that every element of the generated list will be unique with respect to
7676 each callable in the tuple (issue #1916).
7677 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
7679 4.21.1 - 2019-05-06
7681 This patch cleans up the internals of one_of(). You may see a slight
7682 change to the distribution of examples from this strategy but there is
7683 no change to the public API.
7684 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
7686 sprints!
7687 4.21.0 - 2019-05-05
7689 The from_type() strategy now supports python:slice objects.
7690 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
7692 Mentored Sprints.
7693 4.20.0 - 2019-05-05
7695 This release improves the array_shapes() strategy, to choose an appro‐
7696 priate default for max_side based on the min_side, and max_dims based
7697 on the min_dims. An explicit error is raised for dimensions greater
7698 than 32, which are not supported by Numpy, as for other invalid combi‐
7699 nations of arguments.
7700 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
7702 tored Sprints.
7703 4.19.0 - 2019-05-05
7705 The from_type() strategy now supports python:range objects (or xrange
7706 on Python 2).
7707 Thanks to Katrina Durance for writing this feature at the PyCon 2019
7709 Mentored Sprints.
7710 4.18.3 - 2019-04-30
7712 This release fixes a very rare edge case in the test-case mutator,
7713 which could cause an internal error with certain unusual tests.
7714 4.18.2 - 2019-04-30
7716 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
7717 changed the representation of code objects to support positional-only
7718 arguments. Note however that Hypothesis does not (yet) support such
7719 functions as e.g. arguments to builds() or inputs to @given.
7720 Thanks to Paul Ganssle for identifying and fixing this bug.
7722 4.18.1 - 2019-04-29
7724 This patch improves the performance of unique collections such as
7725 sets() when the elements are drawn from a sampled_from() strategy (‐
7726 issue #1115).
7727 4.18.0 - 2019-04-24
7729 This release adds the functions() strategy, which can be used to imi‐
7730 tate your 'real' function for callbacks.
7731 4.17.2 - 2019-04-19
7733 This release refactors stateful rule selection to share the new machin‐
7734 ery with sampled_from() instead of using the original independent im‐
7735 plementation.
7736 4.17.1 - 2019-04-16
7738 This patch allows Hypothesis to try a few more examples after finding
7739 the first bug, in hopes of reporting multiple distinct bugs. The
7740 heuristics described in issue #847 ensure that we avoid wasting time on
7741 fruitless searches, while still surfacing each bug as soon as possible.
7742 4.17.0 - 2019-04-16
7744 This release adds the strategy broadcastable_shapes(), which generates
7745 array shapes that are broadcast-compatible with a provided shape.
7746 4.16.0 - 2019-04-12
7748 This release allows register_type_strategy() to be used with
7749 python:typing.NewType instances. This may be useful to e.g. provide
7750 only positive integers for from_type(UserId) with a UserId = New‐
7751 Type('UserId', int) type.
7752 Thanks to PJCampi for suggesting and writing the patch!
7754 4.15.0 - 2019-04-09
7756 This release supports passing a timedelta as the deadline setting, so
7757 you no longer have to remember that the number is in milliseconds (‐
7758 issue #1900).
7759 Thanks to Damon Francisco for this change!
7761 4.14.7 - 2019-04-09
7763 This patch makes the type annotations on hypothesis.extra.dateutil com‐
7764 patible with mypy 0.700.
7765 4.14.6 - 2019-04-07
7767 This release fixes a bug introduced in Hypothesis 4.14.3 that would
7768 sometimes cause sampled_from(...).filter(...) to encounter an internal
7769 assertion failure when there are three or fewer elements, and every el‐
7770 ement is rejected by the filter.
7771 4.14.5 - 2019-04-05
7773 This patch takes the previous efficiency improvements to sam‐
7774 pled_from(...).filter(...) strategies that reject most elements, and
7775 generalises them to also apply to sampled_from(...).filter(...).fil‐
7776 ter(...) and longer chains of filters.
7777 4.14.4 - 2019-04-05
7779 This release fixes a bug that prevented random_module() from correctly
7780 restoring the previous state of the random module.
7781 The random state was instead being restored to a temporary determinis‐
7783 tic state, which accidentally caused subsequent tests to see the same
7784 random values across multiple test runs.
7785 4.14.3 - 2019-04-03
7787 This patch adds an internal special case to make sampled_from(...).fil‐
7788 ter(...) much more efficient when the filter rejects most elements (‐
7789 issue #1885).
7790 4.14.2 - 2019-03-31
7792 This patch improves the error message if the function f in s.flatmap(f)
7793 does not return a strategy.
7794 Thanks to Kai Chen for this change!
7796 4.14.1 - 2019-03-30
7798 This release modifies how Hypothesis selects operations to run during
7799 shrinking, by causing it to deprioritise previously useless classes of
7800 shrink until others have reached a fixed point.
7801 This avoids certain pathological cases where the shrinker gets very
7803 close to finishing and then takes a very long time to finish the last
7804 small changes because it tries many useless shrinks for each useful one
7805 towards the end. It also should cause a more modest improvement (prob‐
7806 ably no more than about 30%) in shrinking performance for most tests.
7807 4.14.0 - 2019-03-19
7809 This release blocks installation of Hypothesis on Python 3.4, which
7810 reached its end of life date on 2019-03-18.
7811 This should not be of interest to anyone but downstream maintainers -
7813 if you are affected, migrate to a secure version of Python as soon as
7814 possible or at least seek commercial support.
7815 4.13.0 - 2019-03-19
7817 This release makes it an explicit error to call floats(min_value=inf,
7818 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
7819 are no possible values that can be generated (issue #1859).
7820 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
7822 -0. and we could thus generate either if comparing by value, violating
7823 the sequence ordering of floats is a special case we don't want or
7824 need.
7825 4.12.1 - 2019-03-18
7827 This release should significantly reduce the amount of memory that Hy‐
7828 pothesis uses for representing large test cases, by storing information
7829 in a more compact representation and only unpacking it lazily when it
7830 is first needed.
7831 4.12.0 - 2019-03-18
7833 This update adds the report_multiple_bugs setting, which you can use to
7834 disable multi-bug reporting and only raise whichever bug had the small‐
7835 est minimal example. This is occasionally useful when using a debugger
7836 or tools that annotate tracebacks via introspection.
7837 4.11.7 - 2019-03-18
7839 This change makes a tiny improvement to the core engine's bookkeeping.
7840 There is no user-visible change.
7841 4.11.6 - 2019-03-15
7843 This release changes some of Hypothesis's internal shrinking behaviour
7844 in order to reduce memory usage and hopefully improve performance.
7845 4.11.5 - 2019-03-13
7847 This release adds a micro-optimisation to how Hypothesis handles debug
7848 reporting internally. Hard to shrink test may see a slight performance
7849 improvement, but in most common scenarios it is unlikely to be notice‐
7850 able.
7851 4.11.4 - 2019-03-13
7853 This release removes some redundant code that was no longer needed but
7854 was still running a significant amount of computation and allocation on
7855 the hot path. This should result in a modest speed improvement for
7856 most tests, especially those with large test cases.
7857 4.11.3 - 2019-03-13
7859 This release adds a micro-optimisation to how Hypothesis caches test
7860 cases. This will cause a small improvement in speed and memory usage
7861 for large test cases, but in most common scenarios it is unlikely to be
7862 noticeable.
7863 4.11.2 - 2019-03-13
7865 This release removes some internal code that populates a field that is
7866 no longer used anywhere. This should result in some modest performance
7867 and speed improvements and no other user visible effects.
7868 4.11.1 - 2019-03-13
7870 This is a formatting-only patch, enabled by a new version of isort.
7871 4.11.0 - 2019-03-12
7873 This release deprecates sampled_from() with empty sequences. This re‐
7874 turns nothing(), which gives a clear error if used directly... but sim‐
7875 ply vanishes if combined with another strategy.
7876 Tests that silently generate less than expected are a serious problem
7878 for anyone relying on them to find bugs, and we think reliability more
7879 important than convenience in this case.
7880 4.10.0 - 2019-03-11
7882 This release improves Hypothesis's to detect flaky tests, by noticing
7883 when the behaviour of the test changes between runs. In particular
7884 this will notice many new cases where data generation depends on exter‐
7885 nal state (e.g. external sources of randomness) and flag those as flaky
7886 sooner and more reliably.
7887 The basis of this feature is a considerable reengineering of how Hy‐
7889 pothesis stores its history of test cases, so on top of this its memory
7890 usage should be considerably reduced.
7891 4.9.0 - 2019-03-09
7893 This release adds the strategy valid_tuple_axes(), which generates tu‐
7894 ples of axis-indices that can be passed to the axis argument in NumPy's
7895 sequential functions (e.g. numpy:numpy.sum()).
7896 Thanks to Ryan Soklaski for this strategy.
7898 4.8.0 - 2019-03-06
7900 This release significantly tightens validation in hypothesis.settings.
7901 max_examples, buffer_size, and stateful_step_count must be positive in‐
7902 tegers; deadline must be a positive number or None; and derandomize
7903 must be either True or False.
7904 As usual, this replaces existing errors with a more helpful error and
7906 starts new validation checks as deprecation warnings.
7907 4.7.19 - 2019-03-04
7909 This release makes some micro-optimisations to certain calculations
7910 performed in the shrinker. These should particularly speed up large
7911 test cases where the shrinker makes many small changes. It will also
7912 reduce the amount allocated, but most of this is garbage that would
7913 have been immediately thrown away, so you probably won't see much ef‐
7914 fect specifically from that.
7915 4.7.18 - 2019-03-03
7917 This patch removes some overhead from arrays() with a constant shape
7918 and dtype. The resulting performance improvement is modest, but worth‐
7919 while for small arrays.
7920 4.7.17 - 2019-03-01
7922 This release makes some micro-optimisations within Hypothesis's inter‐
7923 nal representation of test cases. This should cause heavily nested
7924 test cases to allocate less during generation and shrinking, which
7925 should speed things up slightly.
7926 4.7.16 - 2019-02-28
7928 This changes the order in which Hypothesis runs certain operations dur‐
7929 ing shrinking. This should significantly decrease memory usage and
7930 speed up shrinking of large examples.
7931 4.7.15 - 2019-02-28
7933 This release allows Hypothesis to calculate a number of attributes of
7934 generated test cases lazily. This should significantly reduce memory
7935 usage and modestly improve performance, especially for large test
7936 cases.
7937 4.7.14 - 2019-02-28
7939 This release reduces the number of operations the shrinker will try
7940 when reordering parts of a test case. This should in some circum‐
7941 stances significantly speed up shrinking. It may result in different
7942 final test cases, and if so usually slightly worse ones, but it should
7943 not generally have much impact on the end result as the operations re‐
7944 moved were typically useless.
7945 4.7.13 - 2019-02-27
7947 This release changes how Hypothesis reorders examples within a test
7948 case during shrinking. This should make shrinking considerably faster.
7949 4.7.12 - 2019-02-27
7951 This release slightly improves the shrinker's ability to replace parts
7952 of a test case with their minimal version, by allowing it to do so in
7953 bulk rather than one at a time. Where this is effective, shrinker per‐
7954 formance should be modestly improved.
7955 4.7.11 - 2019-02-25
7957 This release makes some micro-optimisations to common operations per‐
7958 formed during shrinking. Shrinking should now be slightly faster, es‐
7959 pecially for large examples with relatively fast test functions.
7960 4.7.10 - 2019-02-25
7962 This release is a purely internal refactoring of Hypothesis's API for
7963 representing test cases. There should be no user visible effect.
7964 4.7.9 - 2019-02-24
7966 This release changes certain shrink passes to make them more efficient
7967 when they aren't making progress.
7968 4.7.8 - 2019-02-23
7970 This patch removes some unused code, which makes the internals a bit
7971 easier to understand. There is no user-visible impact.
7972 4.7.7 - 2019-02-23
7974 This release reduces the number of operations the shrinker will try
7975 when reordering parts of a test case. This should in some circum‐
7976 stances significantly speed up shrinking. It may result in different
7977 final test cases, and if so usually slightly worse ones, but it should
7978 not generally have much impact on the end result as the operations re‐
7979 moved were typically useless.
7980 4.7.6 - 2019-02-23
7982 This patch removes some unused code from the shrinker. There is no
7983 user-visible change.
7984 4.7.5 - 2019-02-23
7986 This release changes certain shrink passes to make them adaptive - that
7987 is, in cases where they are successfully making progress they may now
7988 do so significantly faster.
7989 4.7.4 - 2019-02-22
7991 This is a docs-only patch, noting that because the lark-parser is under
7992 active development at version 0.x, hypothesis[lark] APIs may break in
7993 minor releases if necessary to keep up with the upstream package.
7994 4.7.3 - 2019-02-22
7996 This changes Hypothesis to no longer import various test frameworks by
7997 default (if they are installed). which will speed up the initial im‐
7998 port hypothesis call.
7999 4.7.2 - 2019-02-22
8001 This release changes Hypothesis's internal representation of a test
8002 case to calculate some expensive structural information on demand
8003 rather than eagerly. This should reduce memory usage a fair bit, and
8004 may make generation somewhat faster.
8005 4.7.1 - 2019-02-21
8007 This release refactors the internal representation of previously run
8008 test cases. The main thing you should see as a result is that Hypothe‐
8009 sis becomes somewhat less memory hungry.
8010 4.7.0 - 2019-02-21
8012 This patch allows array_shapes() to generate shapes with side-length or
8013 even dimension zero, though the minimum still defaults to one. These
8014 shapes are rare and have some odd behavior, but are particularly impor‐
8015 tant to test for just that reason!
8016 In a related bigfix, arrays() now supports generating zero-dimensional
8018 arrays with dtype=object and a strategy for iterable elements. Previ‐
8019 ously, the array element would incorrectly be set to the first item in
8020 the generated iterable.
8021 Thanks to Ryan Turner for continuing to improve our Numpy support.
8023 4.6.1 - 2019-02-19
8025 This release is a trivial micro-optimisation inside Hypothesis which
8026 should result in it using significantly less memory.
8027 4.6.0 - 2019-02-18
8029 This release changes some inconsistent behavior of arrays() from the
8030 Numpy extra when asked for an array of shape=(). arrays() will now al‐
8031 ways return a Numpy ndarray, and the array will always be of the re‐
8032 quested dtype.
8033 Thanks to Ryan Turner for this change.
8035 4.5.12 - 2019-02-18
8037 This release fixes a minor typo in an internal comment. There is no
8038 user-visible change.
8039 4.5.11 - 2019-02-15
8041 This release fixes issue #1813, a bug introduced in 3.59.1, which
8042 caused random_module() to no longer affect the body of the test: Al‐
8043 though Hypothesis would claim to be seeding the random module in fact
8044 tests would always run with a seed of zero.
8045 4.5.10 - 2019-02-14
8047 This patch fixes an off-by-one error in the maximum length of emails().
8048 Thanks to Krzysztof Jurewicz for pull request #1812.
8049 4.5.9 - 2019-02-14
8051 This patch removes some unused code from the shrinker. There is no
8052 user-visible change.
8053 4.5.8 - 2019-02-12
8055 This release fixes an internal IndexError in Hypothesis that could
8056 sometimes be triggered during shrinking.
8057 4.5.7 - 2019-02-11
8059 This release modifies the shrinker to interleave different types of re‐
8060 duction operations, e.g. switching between deleting data and lowering
8061 scalar values rather than trying entirely deletions then entirely low‐
8062 ering.
8063 This may slow things down somewhat in the typical case, but has the ma‐
8065 jor advantage that many previously difficult to shrink examples should
8066 become much faster, because the shrinker will no longer tend to stall
8067 when trying some ineffective changes to the shrink target but will in‐
8068 stead interleave it with other more effective operations.
8069 4.5.6 - 2019-02-11
8071 This release makes a number of internal changes to the implementation
8072 of hypothesis.extra.lark.from_lark(). These are primarily intended as
8073 a refactoring, but you may see some minor improvements to performance
8074 when generating large strings, and possibly to shrink quality.
8075 4.5.5 - 2019-02-10
8077 This patch prints an explanatory note when issue #1798 is triggered,
8078 because the error message from Numpy is too terse to locate the prob‐
8079 lem.
8080 4.5.4 - 2019-02-08
8082 In Python 2, long integers are not allowed in the shape argument to ar‐
8083 rays(). Thanks to Ryan Turner for fixing this.
8084 4.5.3 - 2019-02-08
8086 This release makes a small internal refactoring to clarify how Hypothe‐
8087 sis instructs tests to stop running when appropriate. There is no
8088 user-visible change.
8089 4.5.2 - 2019-02-06
8091 This release standardises all of the shrinker's internal operations on
8092 running in a random order.
8093 The main effect you will see from this that it should now be much less
8095 common for the shrinker to stall for a long time before making further
8096 progress. In some cases this will correspond to shrinking more slowly,
8097 but on average it should result in faster shrinking.
8098 4.5.1 - 2019-02-05
8100 This patch updates some docstrings, but has no runtime changes.
8101 4.5.0 - 2019-02-03
8103 This release adds exclude_min and exclude_max arguments to floats(), so
8104 that you can easily generate values from open or half-open intervals (‐
8105 issue #1622).
8106 4.4.6 - 2019-02-03
8108 This patch fixes a bug where from_regex() could throw an internal error
8109 if the python:re.IGNORECASE flag was used (issue #1786).
8110 4.4.5 - 2019-02-02
8112 This release removes two shrink passes that Hypothesis runs late in the
8113 process. These were very expensive when the test function was slow and
8114 often didn't do anything useful.
8115 Shrinking should get faster for most failing tests. If you see any re‐
8117 gression in example quality as a result of this release, please let us
8118 know.
8119 4.4.4 - 2019-02-02
8121 This release modifies the way that Hypothesis deletes data during
8122 shrinking. It will primarily be noticeable for very large examples,
8123 which should now shrink faster.
8124 The shrinker is now also able to perform some deletions that it could
8126 not previously, but this is unlikely to be very noticeable.
8127 4.4.3 - 2019-01-25
8129 This release fixes an open file leak that used to cause ResourceWarn‐
8130 ings.
8131 4.4.2 - 2019-01-24
8133 This release changes Hypothesis's internal approach to caching the re‐
8134 sults of executing test cases. The result should be that it is now
8135 significantly less memory hungry, especially when shrinking large test
8136 cases.
8137 Some tests may get slower or faster depending on whether the new or old
8139 caching strategy was well suited to them, but any change in speed in
8140 either direction should be minor.
8141 4.4.1 - 2019-01-24
8143 This patch tightens up some of our internal heuristics to deal with
8144 shrinking floating point numbers, which will now run in fewer circum‐
8145 stances.
8146 You are fairly unlikely to see much difference from this, but if you do
8148 you are likely to see shrinking become slightly faster and/or producing
8149 slightly worse results.
8150 4.4.0 - 2019-01-24
8152 This release adds the from_form() function, which allows automatic
8153 testing against Django forms. (issue #35)
8154 Thanks to Paul Stiverson for this feature, which resolves our oldest
8156 open issue!
8157 4.3.0 - 2019-01-24
8159 This release deprecates HealthCheck.hung_test and disables the associ‐
8160 ated runtime check for tests that ran for more than five minutes. Such
8161 a check is redundant now that we enforce the deadline and max_examples
8162 setting, which can be adjusted independently.
8163 4.2.0 - 2019-01-23
8165 This release adds a new module, hypothesis.extra.lark, which you can
8166 use to generate strings matching a context-free grammar.
8167 In this initial version, only lark-parser EBNF grammars are supported,
8169 by the new hypothesis.extra.lark.from_lark() function.
8170 4.1.2 - 2019-01-23
8172 This patch fixes a very rare overflow bug (issue #1748) which could
8173 raise an InvalidArgument error in complex_numbers() even though the ar‐
8174 guments were valid.
8175 4.1.1 - 2019-01-23
8177 This release makes some improvements to internal code organisation and
8178 documentation and has no impact on behaviour.
8179 4.1.0 - 2019-01-22
8181 This release adds register_random(), which registers random.Random in‐
8182 stances or compatible objects to be seeded and reset by Hypothesis to
8183 ensure that test cases are deterministic.
8184 We still recommend explicitly passing a random.Random instance from
8186 randoms() if possible, but registering a framework-global state for Hy‐
8187 pothesis to manage is better than flaky tests!
8188 4.0.2 - 2019-01-22
8190 This patch fixes issue #1387, where bounded integers() with a very
8191 large range would almost always generate very large numbers. Now, we
8192 usually use the same tuned distribution as unbounded integers().
8193 4.0.1 - 2019-01-16
8195 This release randomizes the order in which the shrinker tries some of
8196 its initial normalization operations. You are unlikely to see much
8197 difference as a result unless your generated examples are very large.
8198 In this case you may see some performance improvements in shrinking.
8199 4.0.0 - 2019-01-14
8201 Welcome to the next major version of Hypothesis!
8202 There are no new features here, as we release those in minor versions.
8204 Instead, 4.0 is a chance for us to remove deprecated features (many al‐
8205 ready converted into no-ops), and turn a variety of warnings into er‐
8206 rors.
8207 If you were running on the last version of Hypothesis 3.x without any
8209 Hypothesis deprecation warnings (or using private APIs), this will be a
8210 very boring upgrade. In fact, nothing will change for you at all. Per
8211 our deprecation policy, warnings added in the last six months (after
8212 2018-07-05) have not been converted to errors.
8213 Removals
8215 • hypothesis.extra.datetime has been removed, replaced by the core date
8216 and time strategies.
8217 • hypothesis.extra.fakefactory has been removed, replaced by general
8219 expansion of Hypothesis' strategies and the third-party ecosystem.
8220 • The SQLite example database backend has been removed.
8222 Settings
8224 • The deadline is now enforced by default, rather than just emitting a
8225 warning when the default (200 milliseconds per test case) deadline is
8226 exceeded.
8227 • The database_file setting has been removed; use database.
8229 • The perform_health_check setting has been removed; use sup‐
8231 press_health_check.
8232 • The max_shrinks setting has been removed; use phases to disable
8234 shrinking.
8235 • The min_satisfying_examples, max_iterations, strict, timeout, and
8237 use_coverage settings have been removed without user-configurable re‐
8238 placements.
8239 Strategies
8241 • The elements argument is now required for collection strategies.
8242 • The average_size argument was a no-op and has been removed.
8244 • Date and time strategies now only accept min_value and max_value for
8246 bounds.
8247 • builds() now requires that the thing to build is passed as the first
8249 positional argument.
8250 • Alphabet validation for text() raises errors, not warnings, as does
8252 category validation for characters().
8253 • The choices() strategy has been removed. Instead, you can use data()
8255 with sampled_from(), so choice(elements) becomes data.draw(sam‐
8256 pled_from(elements)).
8257 • The streaming() strategy has been removed. Instead, you can use
8259 data() and replace iterating over the stream with data.draw() calls.
8260 • sampled_from() and permutations() raise errors instead of warnings if
8262 passed a collection that is not a sequence.
8263 Miscellaneous
8265 • Applying @given to a test function multiple times was really ineffi‐
8266 cient, and now it's also an error.
8267 • Using the .example() method of a strategy (intended for interactive
8269 exploration) within another strategy or a test function always weak‐
8270 ened data generation and broke shrinking, and now it's an error too.
8271 • The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
8273 ported, as the database_file setting has been removed.
8274 • The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
8276 ported. You can use the --hypothesis-verbosity pytest argument in‐
8277 stead, or write your own setup code using the settings profile system
8278 to replace it.
8279 • Using @seed or derandomize=True now forces database=None to ensure
8281 results are in fact reproducible. If database is not None, doing so
8282 also emits a HypothesisWarning.
8283 • Unused exception types have been removed from hypothesis.errors;
8285 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
8286 ple, Timeout, and WrongFormat.
8287 Hypothesis 3.x
8289 3.88.3 - 2019-01-11
8290 This changes the order that the shrinker tries certain operations in
8291 its "emergency" phase which runs late in the process. The new order
8292 should be better at avoiding long stalls where the shrinker is failing
8293 to make progress, which may be helpful if you have difficult to shrink
8294 test cases. However this will not be noticeable in the vast majority
8295 of use cases.
8296 3.88.2 - 2019-01-11
8298 This is a pure refactoring release that extracts some logic from the
8299 core Hypothesis engine into its own class and file. It should have no
8300 user visible impact.
8301 3.88.1 - 2019-01-11
8303 This patch fixes some markup in our documentation.
8304 3.88.0 - 2019-01-10
8306 Introduces hypothesis.stateful.multiple(), which allows rules in rule
8307 based state machines to send multiple results at once to their target
8308 Bundle, or none at all.
8309 3.87.0 - 2019-01-10
8311 This release contains a massive cleanup of the Hypothesis for Django
8312 extra:
8313 • hypothesis.extra.django.models.models() is deprecated in favor of hy‐
8315 pothesis.extra.django.from_model().
8316 • hypothesis.extra.django.models.add_default_field_mapping() is depre‐
8318 cated in favor of hypothesis.extra.django.register_field_strategy().
8319 • from_model() does not infer a strategy for nullable fields or fields
8321 with a default unless passed infer, like builds(). models.models()
8322 would usually but not always infer, and a special default_value
8323 marker object was required to disable inference.
8324 3.86.9 - 2019-01-09
8326 This release improves some internal logic about when a test case in Hy‐
8327 pothesis's internal representation could lead to a valid test case. In
8328 some circumstances this can lead to a significant speed up during
8329 shrinking. It may have some minor negative impact on the quality of
8330 the final result due to certain shrink passes now having access to less
8331 information about test cases in some circumstances, but this should
8332 rarely matter.
8333 3.86.8 - 2019-01-09
8335 This release has no user visible changes but updates our URLs to use
8336 HTTPS.
8337 3.86.7 - 2019-01-08
8339 Hypothesis can now automatically generate values for Django models with
8340 a URLfield, thanks to a new provisional strategy for URLs (issue
8341 #1388).
8342 3.86.6 - 2019-01-07
8344 This release is a pure refactoring that extracts some internal code
8345 into its own file. It should have no user visible effect.
8346 3.86.5 - 2019-01-06
8348 This is a docs-only patch, which fixes some typos and removes a few hy‐
8349 perlinks for deprecated features.
8350 3.86.4 - 2019-01-04
8352 This release changes the order in which the shrinker tries to delete
8353 data. For large and slow tests this may significantly improve the per‐
8354 formance of shrinking.
8355 3.86.3 - 2019-01-04
8357 This release fixes a bug where certain places Hypothesis internal er‐
8358 rors could be raised during shrinking when a user exception occurred
8359 that suppressed an exception Hypothesis uses internally in its genera‐
8360 tion.
8361 The two known ways to trigger this problem were:
8363 • Errors raised in stateful tests' teardown function.
8365 • Errors raised in finally blocks that wrapped a call to data.draw.
8367 These cases will now be handled correctly.
8369 3.86.2 - 2019-01-04
8371 This patch is a docs-only change to fix a broken hyperlink.
8372 3.86.1 - 2019-01-04
8374 This patch fixes issue #1732, where integers() would always return long
8375 values on Python 2.
8376 3.86.0 - 2019-01-03
8378 This release ensures that infinite numbers are never generated by
8379 floats() with allow_infinity=False, which could previously happen in
8380 some cases where one bound was also provided.
8381 The trivially inconsistent min_value=inf, allow_infinity=False now
8383 raises an InvalidArgumentError, as does the inverse with max_value.
8384 You can still use just(inf) to generate inf without violating other
8385 constraints.
8386 3.85.3 - 2019-01-02
8388 Happy new year everyone! This release has no user visible changes but
8389 updates our copyright headers to include 2019.
8390 3.85.2 - 2018-12-31
8392 This release makes a small change to the way the shrinker works. You
8393 may see some improvements to speed of shrinking on especially large and
8394 hard to shrink examples, but most users are unlikely to see much dif‐
8395 ference.
8396 3.85.1 - 2018-12-30
8398 This patch fixes issue #1700, where a line that contained a Unicode
8399 character before a lambda definition would cause an internal exception.
8400 3.85.0 - 2018-12-29
8402 Introduces the hypothesis.stateful.consumes() function. When defining a
8403 rule in stateful testing, it can be used to mark bundles from which
8404 values should be consumed, i. e. removed after use in the rule. This
8405 has been proposed in issue #136.
8406 Thanks to Jochen Müller for this long-awaited feature.
8408 3.84.6 - 2018-12-28
8410 This patch makes a small internal change to fix an issue in Hypothe‐
8411 sis's own coverage tests (issue #1718).
8412 There is no user-visible change.
8414 3.84.5 - 2018-12-21
8416 This patch refactors the hypothesis.strategies module, so that private
8417 names should no longer appear in tab-completion lists. We previously
8418 relied on __all__ for this, but not all editors respect it.
8419 3.84.4 - 2018-12-21
8421 This is a follow-up patch to ensure that the deprecation date is auto‐
8422 matically recorded for any new deprecations. There is no user-visible
8423 effect.
8424 3.84.3 - 2018-12-20
8426 This patch updates the Hypothesis pytest plugin to avoid a recently
8427 deprecated hook interface. There is no user-visible change.
8428 3.84.2 - 2018-12-19
8430 This patch fixes the internals for integers() with one bound. Values
8431 from this strategy now always shrink towards zero instead of towards
8432 the bound, and should shrink much more efficiently too. On Python 2,
8433 providing a bound incorrectly excluded long integers, which can now be
8434 generated.
8435 3.84.1 - 2018-12-18
8437 This patch adds information about when features were deprecated, but
8438 this is only recorded internally and has no user-visible effect.
8439 3.84.0 - 2018-12-18
8441 This release changes the stateful testing backend from find() to use
8442 @given (issue #1300). This doesn't change how you create stateful
8443 tests, but does make them run more like other Hypothesis tests.
8444 @reproduce_failure and @seed now work for stateful tests.
8446 Stateful tests now respect the deadline and suppress_health_check set‐
8448 tings, though they are disabled by default. You can enable them by us‐
8449 ing @settings(...) as a class decorator with whatever arguments you
8450 prefer.
8451 3.83.2 - 2018-12-17
8453 Hypothesis has adopted Black as our code formatter (issue #1686).
8454 There are no functional changes to the source, but it's prettier!
8455 3.83.1 - 2018-12-13
8457 This patch increases the variety of examples generated by from_type().
8458 3.83.0 - 2018-12-12
8460 Our pytest plugin now warns you when strategy functions have been col‐
8461 lected as tests, which may happen when e.g. using the @composite deco‐
8462 rator when you should be using @given(st.data()) for inline draws.
8463 Such functions always pass when treated as tests, because the lazy cre‐
8464 ation of strategies mean that the function body is never actually exe‐
8465 cuted!
8466 3.82.6 - 2018-12-11
8468 Hypothesis can now show statistics when running under pytest-xdist.
8469 Previously, statistics were only reported when all tests were run in a
8470 single process (issue #700).
8471 3.82.5 - 2018-12-08
8473 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
8474 to integers() could cause errors on Python 3 due to internal rounding.
8475 3.82.4 - 2018-12-08
8477 Hypothesis now seeds and resets the global state of np.random for each
8478 test case, to ensure that tests are reproducible.
8479 This matches and complements the existing handling of the python:random
8481 module - Numpy simply maintains an independent PRNG for performance
8482 reasons.
8483 3.82.3 - 2018-12-08
8485 This is a no-op release to add the new Framework :: Hypothesis trove
8486 classifier to hypothesis on PyPI.
8487 You can use it as a filter to find Hypothesis-related packages such as
8489 extensions as they add the tag over the coming weeks, or simply visit
8490 our curated list.
8491 3.82.2 - 2018-12-08
8493 The Hypothesis for Pandas extension is now listed in setup.py, so you
8494 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
8495 tion.
8496 3.82.1 - 2018-10-29
8498 This patch fixes from_type() on Python 2 for classes where cls.__init__
8499 is object.__init__. Thanks to ccxcz for reporting issue #1656.
8500 3.82.0 - 2018-10-29
8502 The alphabet argument for text() now uses its default value of charac‐
8503 ters(blacklist_categories=('Cs',)) directly, instead of hiding that be‐
8504 hind alphabet=None and replacing it within the function. Passing None
8505 is therefore deprecated.
8506 3.81.0 - 2018-10-27
8508 GenericStateMachine and RuleBasedStateMachine now raise an explicit er‐
8509 ror when instances of settings are assigned to the classes' settings
8510 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
8511 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
8512 tor to handle this automatically.
8513 3.80.0 - 2018-10-25
8515 Since version 3.68.0, arrays() checks that values drawn from the ele‐
8516 ments and fill strategies can be safely cast to the dtype of the array,
8517 and emits a warning otherwise.
8518 This release expands the checks to cover overflow for finite complex64
8520 elements and string truncation caused by too-long elements or trailing
8521 null characters (issue #1591).
8522 3.79.4 - 2018-10-25
8524 Tests using @given now shrink errors raised from pytest helper func‐
8525 tions, instead of reporting the first example found.
8526 This was previously fixed in version 3.56.0, but only for stateful
8528 testing.
8529 3.79.3 - 2018-10-23
8531 Traceback elision is now disabled on Python 2, to avoid an import-time
8532 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
8533 3.79.2: issue #1648).
8534 We encourage all users to upgrade to Python 3 before the end of 2019.
8536 3.79.2 - 2018-10-23
8538 This patch shortens tracebacks from Hypothesis, so you can see exactly
8539 happened in your code without having to skip over irrelevant details
8540 about our internals (issue #848).
8541 In the example test (see pull request #1582), this reduces tracebacks
8543 from nine frames to just three - and for a test with multiple errors,
8544 from seven frames per error to just one!
8545 If you do want to see the internal details, you can disable frame eli‐
8547 sion by setting verbosity to debug.
8548 3.79.1 - 2018-10-22
8550 The abstract number classes Number, Complex, Real, Rational, and Inte‐
8551 gral are now supported by the from_type() strategy. Previously, you
8552 would have to use register_type_strategy() before they could be re‐
8553 solved (issue #1636)
8554 3.79.0 - 2018-10-18
8556 This release adds a CLI flag for verbosity --hypothesis-verbosity to
8557 the Hypothesis pytest plugin, applied after loading the profile speci‐
8558 fied by --hypothesis-profile. Valid options are the names of verbosity
8559 settings, quiet, normal, verbose or debug.
8560 Thanks to Bex Dunn for writing this patch at the PyCon Australia
8562 sprints!
8563 The pytest header now correctly reports the current profile if --hy‐
8565 pothesis-profile has been used.
8566 Thanks to Mathieu Paturel for the contribution at the Canberra Python
8568 Hacktoberfest.
8569 3.78.0 - 2018-10-16
8571 This release has deprecated the generation of integers, floats and
8572 fractions when the conversion of the upper and/ or lower bound is not
8573 100% exact, e.g. when an integer gets passed a bound that is not a
8574 whole number. (issue #1625)
8575 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
8577 3.77.0 - 2018-10-16
8579 This minor release adds functionality to settings allowing it to be
8580 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
8581 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
8583 3.76.1 - 2018-10-16
8585 This patch fixes some warnings added by recent releases of pydocstyle
8586 and mypy.
8587 3.76.0 - 2018-10-11
8589 This release deprecates using floats for min_size and max_size.
8590 The type hint for average_size arguments has been changed from Op‐
8592 tional[int] to None, because non-None values are always ignored and
8593 deprecated.
8594 3.75.4 - 2018-10-10
8596 This patch adds more internal comments to the core engine's se‐
8597 quence-length shrinker. There should be no user-visible change.
8598 3.75.3 - 2018-10-09
8600 This patch adds additional comments to some of the core engine's inter‐
8601 nal data structures. There is no user-visible change.
8602 3.75.2 - 2018-10-09
8604 This patch avoids caching a trivial case, fixing issue #493.
8605 3.75.1 - 2018-10-09
8607 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
8608 for this contribution!
8609 3.75.0 - 2018-10-08
8611 This release deprecates the use of min_size=None, setting the default
8612 min_size to 0 (issue #1618).
8613 3.74.3 - 2018-10-08
8615 This patch makes some small internal changes to comply with a new lint
8616 setting in the build. There should be no user-visible change.
8617 3.74.2 - 2018-10-03
8619 This patch fixes issue #1153, where time spent reifying a strategy was
8620 also counted in the time spent generating the first example. Strate‐
8621 gies are now fully constructed and validated before the timer is
8622 started.
8623 3.74.1 - 2018-10-03
8625 This patch fixes some broken formatting and links in the documentation.
8626 3.74.0 - 2018-10-01
8628 This release checks that the value of the print_blob setting is a
8629 PrintSettings instance.
8630 Being able to specify a boolean value was not intended, and is now dep‐
8632 recated. In addition, specifying True will now cause the blob to al‐
8633 ways be printed, instead of causing it to be suppressed.
8634 Specifying any value that is not a PrintSettings or a boolean is now an
8636 error.
8637 3.73.5 - 2018-10-01
8639 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
8640 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
8641 eter names min_value and max_value instead of the deprecated names
8642 3.73.4 - 2018-09-30
8644 This patch ensures that Hypothesis deprecation warnings display the
8645 code that emitted them when you're not running in -Werror mode (issue
8646 #652).
8647 3.73.3 - 2018-09-27
8649 Tracebacks involving @composite are now slightly shorter due to some
8650 internal refactoring.
8651 3.73.2 - 2018-09-26
8653 This patch fixes errors in the internal comments for one of the
8654 shrinker passes. There is no user-visible change.
8655 3.73.1 - 2018-09-25
8657 This patch substantially improves the distribution of data generated
8658 with recursive(), and fixes a rare internal error (issue #1502).
8659 3.73.0 - 2018-09-24
8661 This release adds the fulfill() function, which is designed for testing
8662 code that uses dpcontracts 0.4 or later for input validation. This
8663 provides some syntactic sugar around use of assume(), to automatically
8664 filter out and retry calls that cause a precondition check to fail (‐
8665 issue #1474).
8666 3.72.0 - 2018-09-24
8668 This release makes setting attributes of the hypothesis.settings class
8669 an explicit error. This has never had any effect, but could mislead
8670 users who confused it with the current settings instance hypothe‐
8671 sis.settings.default (which is also immutable). You can change the
8672 global settings with settings profiles.
8673 3.71.11 - 2018-09-24
8675 This patch factors out some common code in the shrinker for iterating
8676 over pairs of data blocks. There should be no user-visible change.
8677 3.71.10 - 2018-09-18
8679 This patch allows from_type() to handle the empty tuple type, typ‐
8680 ing.Tuple[()].
8681 3.71.9 - 2018-09-17
8683 This patch updates some internal comments for mypy. There is no
8684 user-visible effect, even for Mypy users.
8685 3.71.8 - 2018-09-17
8687 This patch fixes a rare bug that would cause a particular shrinker pass
8688 to raise an IndexError, if a shrink improvement changed the underlying
8689 data in an unexpected way.
8690 3.71.7 - 2018-09-17
8692 This release fixes the broken cross-references in our docs, and adds a
8693 CI check so we don't add new ones.
8694 3.71.6 - 2018-09-16
8696 This patch fixes two bugs (issue #944 and issue #1521), where messages
8697 about @seed did not check the current verbosity setting, and the wrong
8698 settings were active while executing explicit examples.
8699 3.71.5 - 2018-09-15
8701 This patch fixes a DeprecationWarning added in Python 3.8 (issue
8702 #1576).
8703 Thanks to tirkarthi for this contribution!
8705 3.71.4 - 2018-09-14
8707 This is a no-op release, which implements automatic DOI minting and
8708 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
8709 zon 2020 programme for providing this service!
8710 Check our CITATION file for details, or head right on over to
8712 doi.org/10.5281/zenodo.1412597
8713 3.71.3 - 2018-09-10
8715 This release adds the test name to some deprecation warnings, for eas‐
8716 ier debugging.
8717 Thanks to Sanyam Khurana for the patch!
8719 3.71.2 - 2018-09-10
8721 This release makes Hypothesis's memory usage substantially smaller for
8722 tests with many examples, by bounding the number of past examples it
8723 keeps around.
8724 You will not see much difference unless you are running tests with
8726 max_examples set to well over 1000, but if you do have such tests then
8727 you should see memory usage mostly plateau where previously it would
8728 have grown linearly with time.
8729 3.71.1 - 2018-09-09
8731 This patch adds internal comments to some tree traversals in the core
8732 engine. There is no user-visible change.
8733 3.71.0 - 2018-09-08
8735 This release deprecates the coverage-guided testing functionality, as
8736 it has proven brittle and does not really pull its weight.
8737 We intend to replace it with something more useful in the future, but
8739 the feature in its current form does not seem to be worth the cost of
8740 using, and whatever replaces it will likely look very different.
8741 3.70.4 - 2018-09-08
8743 This patch changes the behaviour of reproduce_failure() so that blobs
8744 are only printed in quiet mode when the print_blob setting is set to
8745 ALWAYS.
8746 Thanks to Cameron McGill for writing this patch at the PyCon Australia
8748 sprints!
8749 3.70.3 - 2018-09-03
8751 This patch removes some unnecessary code from the internals. There is
8752 no user-visible change.
8753 3.70.2 - 2018-09-03
8755 This patch fixes an internal bug where a corrupted argument to @repro‐
8756 duce_failure could raise the wrong type of error. Thanks again to
8757 Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
8758 tently provides excellent bug reports including issue #1558.
8759 3.70.1 - 2018-09-03
8761 This patch updates hypothesis to report its version and settings when
8762 run with pytest. (issue #1223).
8763 Thanks to Jack Massey for this feature.
8765 3.70.0 - 2018-09-01
8767 This release adds a fullmatch argument to from_regex(). When full‐
8768 match=True, the whole example will match the regex pattern as for
8769 python:re.fullmatch().
8770 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
8772 sprints!
8773 3.69.12 - 2018-08-30
8775 This release reverts the changes to logging handling in 3.69.11, which
8776 broke test that use the pytest caplog fixture internally because all
8777 logging was disabled (issue #1546).
8778 3.69.11 - 2018-08-29
8780 This patch will hide all logging messages produced by test cases before
8781 the final, minimal, failing test case (issue #356).
8782 Thanks to Gary Donovan for writing this patch at the PyCon Australia
8784 sprints!
8785 3.69.10 - 2018-08-29
8787 This patch fixes a bug that prevents coverage from reporting unexecuted
8788 Python files (issue #1085).
8789 Thanks to Gary Donovan for writing this patch at the PyCon Australia
8791 sprints!
8792 3.69.9 - 2018-08-28
8794 This patch improves the packaging of the Python package by adding LI‐
8795 CENSE.txt to the sdist (issue #1311), clarifying the minimum supported
8796 versions of pytz and dateutil (issue #1383), and adds keywords to the
8797 metadata (issue #1520).
8798 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
8800 tralia sprints!
8801 3.69.8 - 2018-08-28
8803 This is an internal change which replaces pickle with json to prevent
8804 possible security issues.
8805 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
8807 sprints!
8808 3.69.7 - 2018-08-28
8810 This patch ensures that note() prints the note for every test case when
8811 the verbosity setting is Verbosity.verbose. At normal verbosity it
8812 only prints from the final test case.
8813 Thanks to Tom McDermott for writing this patch at the PyCon Australia
8815 sprints!
8816 3.69.6 - 2018-08-27
8818 This patch improves the testing of some internal caching. It should
8819 have no user-visible effect.
8820 3.69.5 - 2018-08-27
8822 This change performs a small rename and refactoring in the core engine.
8823 There is no user-visible change.
8824 3.69.4 - 2018-08-27
8826 This change improves the core engine's ability to avoid unnecessary
8827 work, by consulting its cache of previously-tried inputs in more cases.
8828 3.69.3 - 2018-08-27
8830 This patch handles passing an empty python:enum.Enum to from_type() by
8831 returning nothing(), instead of raising an internal python:Assertion‐
8832 Error.
8833 Thanks to Paul Amazona for writing this patch at the PyCon Australia
8835 sprints!
8836 3.69.2 - 2018-08-23
8838 This patch fixes a small mistake in an internal comment. There is no
8839 user-visible change.
8840 3.69.1 - 2018-08-21
8842 This change fixes a small bug in how the core engine consults its cache
8843 of previously-tried inputs. There is unlikely to be any user-visible
8844 change.
8845 3.69.0 - 2018-08-20
8847 This release improves argument validation for stateful testing.
8848 • If the target or targets of a rule() are invalid, we now raise a use‐
8850 ful validation error rather than an internal exception.
8851 • Passing both the target and targets arguments is deprecated - append
8853 the target bundle to the targets tuple of bundles instead.
8854 • Passing the name of a Bundle rather than the Bundle itself is also
8856 deprecated.
8857 3.68.3 - 2018-08-20
8859 This is a docs-only patch, fixing some typos and formatting issues.
8860 3.68.2 - 2018-08-19
8862 This change fixes a small bug in how the core engine caches the results
8863 of previously-tried inputs. The effect is unlikely to be noticeable,
8864 but it might avoid unnecessary work in some cases.
8865 3.68.1 - 2018-08-18
8867 This patch documents the from_dtype() function, which infers a strategy
8868 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
8869 directly when creating e.g. Pandas objects.
8870 3.68.0 - 2018-08-15
8872 arrays() now checks that integer and float values drawn from elements
8873 and fill strategies can be safely cast to the dtype of the array, and
8874 emits a warning otherwise (issue #1385).
8875 Elements in the resulting array could previously violate constraints on
8877 the elements strategy due to floating-point overflow or truncation of
8878 integers to fit smaller types.
8879 3.67.1 - 2018-08-14
8881 This release contains a tiny refactoring of the internals. There is no
8882 user-visible change.
8883 3.67.0 - 2018-08-10
8885 This release adds a width argument to floats(), to generate lower-pre‐
8886 cision floating point numbers for e.g. Numpy arrays.
8887 The generated examples are always instances of Python's native float
8889 type, which is 64bit, but passing width=32 will ensure that all values
8890 can be exactly represented as 32bit floats. This can be useful to
8891 avoid overflow (to +/- infinity), and for efficiency of generation and
8892 shrinking.
8893 Half-precision floats (width=16) are also supported, but require Numpy
8895 if you are running Python 3.5 or earlier.
8896 3.66.33 - 2018-08-10
8898 This release fixes a bug in floats(), where setting allow_infin‐
8899 ity=False and exactly one of min_value and max_value would allow infi‐
8900 nite values to be generated.
8901 3.66.32 - 2018-08-09
8903 This release adds type hints to the example() and seed() decorators,
8904 and fixes the type hint on register_type_strategy(). The second argu‐
8905 ment to register_type_strategy() must either be a SearchStrategy, or a
8906 callable which takes a type and returns a SearchStrategy.
8907 3.66.31 - 2018-08-08
8909 Another set of changes designed to improve the performance of shrinking
8910 on large examples. In particular the shrinker should now spend consid‐
8911 erably less time running useless shrinks.
8912 3.66.30 - 2018-08-06
8914 "Bug fixes and performance improvements".
8915 This release is a fairly major overhaul of the shrinker designed to im‐
8917 prove its behaviour on large examples, especially around stateful test‐
8918 ing. You should hopefully see shrinking become much faster, with little
8919 to no quality degradation (in some cases quality may even improve).
8920 3.66.29 - 2018-08-05
8922 This release fixes two very minor bugs in the core engine:
8923 • it fixes a corner case that was missing in 3.66.28, which should
8925 cause shrinking to work slightly better.
8926 • it fixes some logic for how shrinking interacts with the database
8928 that was causing Hypothesis to be insufficiently aggressive about
8929 clearing out old keys.
8930 3.66.28 - 2018-08-05
8932 This release improves how Hypothesis handles reducing the size of inte‐
8933 gers' representation. This change should mostly be invisible as it's
8934 purely about the underlying representation and not the generated value,
8935 but it may result in some improvements to shrink performance.
8936 3.66.27 - 2018-08-05
8938 This release changes the order in which Hypothesis chooses parts of the
8939 test case to shrink. For typical usage this should be a significant
8940 performance improvement on large examples. It is unlikely to have a ma‐
8941 jor impact on example quality, but where it does change the result it
8942 should usually be an improvement.
8943 3.66.26 - 2018-08-05
8945 This release improves the debugging information that the shrinker emits
8946 about the operations it performs, giving better summary statistics
8947 about which passes resulted in test executions and whether they were
8948 successful.
8949 3.66.25 - 2018-08-05
8951 This release fixes several bugs that were introduced to the shrinker in
8952 3.66.24 which would have caused it to behave significantly less well
8953 than advertised. With any luck you should actually see the promised
8954 benefits now.
8955 3.66.24 - 2018-08-03
8957 This release changes how Hypothesis deletes data when shrinking in or‐
8958 der to better handle deletion of large numbers of contiguous sequences.
8959 Most tests should see little change, but this will hopefully provide a
8960 significant speed up for stateful testing.
8961 3.66.23 - 2018-08-02
8963 This release makes some internal changes to enable further improvements
8964 to the shrinker. You may see some changes in the final shrunk examples,
8965 but they are unlikely to be significant.
8966 3.66.22 - 2018-08-01
8968 This release adds some more internal caching to the shrinker. This
8969 should cause a significant speed up for shrinking, especially for
8970 stateful testing and large example sizes.
8971 3.66.21 - 2018-08-01
8973 This patch is for downstream packagers - our tests now pass under
8974 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
8975 of Hypothesis itself.
8976 3.66.20 - 2018-08-01
8978 This release removes some functionality from the shrinker that was tak‐
8979 ing a considerable amount of time and does not appear to be useful any
8980 more due to a number of quality improvements in the shrinker.
8981 You may see some degradation in shrink quality as a result of this, but
8983 mostly shrinking should just get much faster.
8984 3.66.19 - 2018-08-01
8986 This release slightly changes the format of some debugging information
8987 emitted during shrinking, and refactors some of the internal interfaces
8988 around that.
8989 3.66.18 - 2018-07-31
8991 This release is a very small internal refactoring which should have no
8992 user visible impact.
8993 3.66.17 - 2018-07-31
8995 This release fixes a bug that could cause an IndexError to be raised
8996 from inside Hypothesis during shrinking. It is likely that it was im‐
8997 possible to trigger this bug in practice - it was only made visible by
8998 some currently unreleased work.
8999 3.66.16 - 2018-07-31
9001 This release is a very small internal refactoring which should have no
9002 user visible impact.
9003 3.66.15 - 2018-07-31
9005 This release makes Hypothesis's shrinking faster by removing some re‐
9006 dundant work that it does when minimizing values in its internal repre‐
9007 sentation.
9008 3.66.14 - 2018-07-30
9010 This release expands the deprecation of timeout from 3.16.0 to also
9011 emit the deprecation warning in find or stateful testing.
9012 3.66.13 - 2018-07-30
9014 This release adds an additional shrink pass that is able to reduce the
9015 size of examples in some cases where the transformation is non-obvious.
9016 In particular this will improve the quality of some examples which
9017 would have regressed in 3.66.12.
9018 3.66.12 - 2018-07-28
9020 This release changes how we group data together for shrinking. It
9021 should result in improved shrinker performance, especially in stateful
9022 testing.
9023 3.66.11 - 2018-07-28
9025 This patch modifies how which rule to run is selected during rule based
9026 stateful testing. This should result in a slight performance increase
9027 during generation and a significant performance and quality improvement
9028 when shrinking.
9029 As a result of this change, some state machines which would previously
9031 have thrown an InvalidDefinition are no longer detected as invalid.
9032 3.66.10 - 2018-07-28
9034 This release weakens some minor functionality in the shrinker that had
9035 only modest benefit and made its behaviour much harder to reason about.
9036 This is unlikely to have much user visible effect, but it is possible
9038 that in some cases shrinking may get slightly slower. It is primarily
9039 to make it easier to work on the shrinker and pave the way for future
9040 work.
9041 3.66.9 - 2018-07-26
9043 This release improves the information that Hypothesis emits about its
9044 shrinking when verbosity is set to debug.
9045 3.66.8 - 2018-07-24
9047 This patch includes some minor fixes in the documentation, and updates
9048 the minimum version of pytest to 3.0 (released August 2016).
9049 3.66.7 - 2018-07-24
9051 This release fixes a bug where difficult to shrink tests could some‐
9052 times trigger an internal assertion error inside the shrinker.
9053 3.66.6 - 2018-07-23
9055 This patch ensures that Hypothesis fully supports Python 3.7, by up‐
9056 grading from_type() (issue #1264) and fixing some minor issues in our
9057 test suite (issue #1148).
9058 3.66.5 - 2018-07-22
9060 This patch fixes the online docs for various extras, by ensuring that
9061 their dependencies are installed on readthedocs.io (issue #1326).
9062 3.66.4 - 2018-07-20
9064 This release improves the shrinker's ability to reorder examples.
9065 For example, consider the following test:
9067 import hypothesis.strategies as st
9069 from hypothesis import given
9070 @given(st.text(), st.text())
9073 def test_non_equal(x, y):
9074 assert x != y
9075 Previously this could have failed with either of x="", y="0" or x="0",
9077 y="". Now it should always fail with x="", y="0".
9078 This will allow the shrinker to produce more consistent results, espe‐
9080 cially in cases where test cases contain some ordered collection whose
9081 actual order does not matter.
9082 3.66.3 - 2018-07-20
9084 This patch fixes inference in the builds() strategy with subtypes of
9085 python:typing.NamedTuple, where the __init__ method is not useful for
9086 introspection. We now use the field types instead - thanks to James
9087 Uther for identifying this bug.
9088 3.66.2 - 2018-07-19
9090 This release improves the shrinker's ability to handle situations where
9091 there is an additive constraint between two values.
9092 For example, consider the following test:
9094 import hypothesis.strategies as st
9096 from hypothesis import given
9097 @given(st.integers(), st.integers())
9100 def test_does_not_exceed_100(m, n):
9101 assert m + n < 100
9102 Previously this could have failed with almost any pair (m, n) with 0 <=
9104 m <= n and m + n == 100. Now it should almost always fail with m=0,
9105 n=100.
9106 This is a relatively niche specialisation, but can be useful in situa‐
9108 tions where e.g. a bug is triggered by an integer overflow.
9109 3.66.1 - 2018-07-09
9111 This patch fixes a rare bug where an incorrect percentage drawtime
9112 could be displayed for a test, when the system clock was changed during
9113 a test running under Python 2 (we use python:time.monotonic() where it
9114 is available to avoid such problems). It also fixes a possible
9115 zero-division error that can occur when the underlying C library dou‐
9116 ble-rounds an intermediate value in python:math.fsum() and gets the
9117 least significant bit wrong.
9118 3.66.0 - 2018-07-05
9120 This release improves validation of the alphabet argument to the text()
9121 strategy. The following misuses are now deprecated, and will be an er‐
9122 ror in a future version:
9123 • passing an unordered collection (such as set('abc')), which violates
9125 invariants about shrinking and reproducibility
9126 • passing an alphabet sequence with elements that are not strings
9128 • passing an alphabet sequence with elements that are not of length
9130 one, which violates any size constraints that may apply
9131 Thanks to Sushobhit for adding these warnings (issue #1329).
9133 3.65.3 - 2018-07-04
9135 This release fixes a mostly theoretical bug where certain usage of the
9136 internal API could trigger an assertion error inside Hypothesis. It is
9137 unlikely that this problem is even possible to trigger through the pub‐
9138 lic API.
9139 3.65.2 - 2018-07-04
9141 This release fixes dependency information for coverage. Previously Hy‐
9142 pothesis would allow installing coverage with any version, but it only
9143 works with coverage 4.0 or later.
9144 We now specify the correct metadata in our setup.py, so Hypothesis will
9146 only allow installation with compatible versions of coverage.
9147 3.65.1 - 2018-07-03
9149 This patch ensures that stateful tests which raise an error from a
9150 pytest helper still print the sequence of steps taken to reach that
9151 point (issue #1372). This reporting was previously broken because the
9152 helpers inherit directly from python:BaseException, and therefore re‐
9153 quire special handling to catch without breaking e.g. the use of ctrl-C
9154 to quit the test.
9155 3.65.0 - 2018-06-30
9157 This release deprecates the max_shrinks setting in favor of an internal
9158 heuristic. If you need to avoid shrinking examples, use the phases
9159 setting instead. (issue #1235)
9160 3.64.2 - 2018-06-27
9162 This release fixes a bug where an internal assertion error could some‐
9163 times be triggered while shrinking a failing test.
9164 3.64.1 - 2018-06-27
9166 This patch fixes type-checking errors in our vendored pretty-printer,
9167 which were ignored by our mypy config but visible for anyone else
9168 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
9169 3.64.0 - 2018-06-26
9171 This release adds an interface which can be used to insert a wrapper
9172 between the original test function and @given (issue #1257). This will
9173 be particularly useful for test runner extensions such as pytest-trio,
9174 but is not recommended for direct use by other users of Hypothesis.
9175 3.63.0 - 2018-06-26
9177 This release adds a new mechanism to infer strategies for classes de‐
9178 fined using attrs, based on the the type, converter, or validator of
9179 each attribute. This inference is now built in to builds() and
9180 from_type().
9181 On Python 2, from_type() no longer generates instances of int when
9183 passed long, or vice-versa.
9184 3.62.0 - 2018-06-26
9186 This release adds PEP 484 type hints to Hypothesis on a provisional ba‐
9187 sis, using the comment-based syntax for Python 2 compatibility. You
9188 can read more about our type hints here.
9189 It also adds the py.typed marker specified in PEP 561. After you pip
9191 install hypothesis, mypy 0.590 or later will therefore type-check your
9192 use of our public interface!
9193 3.61.0 - 2018-06-24
9195 This release deprecates the use of settings as a context manager, the
9196 use of which is somewhat ambiguous.
9197 Users should define settings with global state or with the @set‐
9199 tings(...) decorator.
9200 3.60.1 - 2018-06-20
9202 Fixed a bug in generating an instance of a Django model from a strategy
9203 where the primary key is generated as part of the strategy. See details
9204 here.
9205 Thanks to Tim Martin for this contribution.
9207 3.60.0 - 2018-06-20
9209 This release adds the @initialize decorator for stateful testing (orig‐
9210 inally discussed in issue #1216). All @initialize rules will be called
9211 once each in an arbitrary order before any normal rule is called.
9212 3.59.3 - 2018-06-19
9214 This is a no-op release to take into account some changes to the re‐
9215 lease process. It should have no user visible effect.
9216 3.59.2 - 2018-06-18
9218 This adds support for partially sorting examples which cannot be fully
9219 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
9220 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
9221 0].
9222 Thanks to Luke for contributing.
9224 3.59.1 - 2018-06-16
9226 This patch uses python:random.getstate() and python:random.setstate()
9227 to restore the PRNG state after @given runs deterministic tests. With‐
9228 out restoring state, you might have noticed problems such as issue
9229 #1266. The fix also applies to stateful testing (issue #702).
9230 3.59.0 - 2018-06-14
9232 This release adds the emails() strategy, which generates unicode
9233 strings representing an email address.
9234 Thanks to Sushobhit for moving this to the public API (issue #162).
9236 3.58.1 - 2018-06-13
9238 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
9239 1 2 3.
9240 Thanks to Luke for contributing.
9242 3.58.0 - 2018-06-13
9244 This adds a new extra timezones() strategy that generates dateutil
9245 timezones.
9246 Thanks to Conrad for contributing.
9248 3.57.0 - 2018-05-20
9250 Using an unordered collection with the permutations() strategy has been
9251 deprecated because the order in which e.g. a set shrinks is arbitrary.
9252 This may cause different results between runs.
9253 3.56.10 - 2018-05-16
9255 This release makes hypothesis.settings.define_setting a private method,
9256 which has the effect of hiding it from the documentation.
9257 3.56.9 - 2018-05-11
9259 This is another release with no functionality changes as part of
9260 changes to Hypothesis's new release tagging scheme.
9261 3.56.8 - 2018-05-10
9263 This is a release with no functionality changes that moves Hypothesis
9264 over to a new release tagging scheme.
9265 3.56.7 - 2018-05-10
9267 This release provides a performance improvement for most tests, but in
9268 particular users of sampled_from() who don't have numpy installed
9269 should see a significant performance improvement.
9270 3.56.6 - 2018-05-09
9272 This patch contains further internal work to support Mypy. There are
9273 no user-visible changes... yet.
9274 3.56.5 - 2018-04-22
9276 This patch contains some internal refactoring to run mypy in CI. There
9277 are no user-visible changes.
9278 3.56.4 - 2018-04-21
9280 This release involves some very minor internal clean up and should have
9281 no user visible effect at all.
9282 3.56.3 - 2018-04-20
9284 This release fixes a problem introduced in 3.56.0 where setting the hy‐
9285 pothesis home directory (through currently undocumented means) would no
9286 longer result in the default database location living in the new home
9287 directory.
9288 3.56.2 - 2018-04-20
9290 This release fixes a problem introduced in 3.56.0 where setting max_ex‐
9291 amples to 1 would result in tests failing with Unsatisfiable. This
9292 problem could also occur in other harder to trigger circumstances (e.g.
9293 by setting it to a low value, having a hard to satisfy assumption, and
9294 disabling health checks).
9295 3.56.1 - 2018-04-20
9297 This release fixes a problem that was introduced in 3.56.0: Use of the
9298 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
9299 cated, actually broken due to being read before various setup the dep‐
9300 recation path needed was done. It now works correctly (and emits a dep‐
9301 recation warning).
9302 3.56.0 - 2018-04-17
9304 This release deprecates several redundant or internally oriented set‐
9305 tings, working towards an orthogonal set of configuration options that
9306 are widely useful without requiring any knowledge of our internals (‐
9307 issue #535).
9308 • Deprecated settings that no longer have any effect are no longer
9310 shown in the __repr__ unless set to a non-default value.
9311 • hypothesis.settings.perform_health_check is deprecated, as it dupli‐
9313 cates suppress_health_check.
9314 • hypothesis.settings.max_iterations is deprecated and disabled, be‐
9316 cause we can usually get better behaviour from an internal heuristic
9317 than a user-controlled setting.
9318 • hypothesis.settings.min_satisfying_examples is deprecated and dis‐
9320 abled, due to overlap with the filter_too_much healthcheck and poor
9321 interaction with max_examples.
9322 • HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
9324 the profile system instead.
9325 • Examples tried by find() are now reported at debug verbosity level
9327 (as well as verbose level).
9328 3.55.6 - 2018-04-14
9330 This release fixes a somewhat obscure condition (issue #1230) under
9331 which you could occasionally see a failing test trigger an assertion
9332 error inside Hypothesis instead of failing normally.
9333 3.55.5 - 2018-04-14
9335 This patch fixes one possible cause of issue #966. When running Python
9336 2 with hash randomisation, passing a python:bytes object to python:ran‐
9337 dom.seed() would use version=1, which broke derandomize (because the
9338 seed depended on a randomised hash). If derandomize is still nondeter‐
9339 ministic for you, please open an issue.
9340 3.55.4 - 2018-04-13
9342 This patch makes a variety of minor improvements to the documentation,
9343 and improves a few validation messages for invalid inputs.
9344 3.55.3 - 2018-04-12
9346 This release updates the URL metadata associated with the PyPI package
9347 (again). It has no other user visible effects.
9348 3.55.2 - 2018-04-11
9350 This release updates the URL metadata associated with the PyPI package.
9351 It has no other user visible effects.
9352 3.55.1 - 2018-04-06
9354 This patch relaxes constraints in our tests on the expected values re‐
9355 turned by the standard library function hypot() and the internal helper
9356 function cathetus, to fix near-exact test failures on some 32-bit sys‐
9357 tems used by downstream packagers.
9358 3.55.0 - 2018-04-05
9360 This release includes several improvements to the handling of the data‐
9361 base setting.
9362 • The database_file setting was a historical artefact, and you should
9364 just use database directly.
9365 • The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
9367 favor of load_profile() and the database setting.
9368 • If you have not configured the example database at all and the de‐
9370 fault location is not usable (due to e.g. permissions issues), Hy‐
9371 pothesis will fall back to an in-memory database. This is not per‐
9372 sisted between sessions, but means that the defaults work on
9373 read-only filesystems.
9374 3.54.0 - 2018-04-04
9376 This release improves the complex_numbers() strategy, which now sup‐
9377 ports min_magnitude and max_magnitude arguments, along with allow_nan
9378 and allow_infinity like for floats().
9379 Thanks to J.J. Green for this feature.
9381 3.53.0 - 2018-04-01
9383 This release removes support for Django 1.8, which reached end of life
9384 on 2018-04-01. You can see Django's release and support schedule on
9385 the Django Project website.
9386 3.52.3 - 2018-04-01
9388 This patch fixes the min_satisfying_examples settings documentation, by
9389 explaining that example shrinking is tracked at the level of the under‐
9390 lying bytestream rather than the output value.
9391 The output from find() in verbose mode has also been adjusted - see the
9393 example session - to avoid duplicating lines when the example repr is
9394 constant, even if the underlying representation has been shrunken.
9395 3.52.2 - 2018-03-30
9397 This release improves the output of failures with rule based stateful
9398 testing in two ways:
9399 • The output from it is now usually valid Python code.
9401 • When the same value has two different names because it belongs to two
9403 different bundles, it will now display with the name associated with
9404 the correct bundle for a rule argument where it is used.
9405 3.52.1 - 2018-03-29
9407 This release improves the behaviour of stateful testing in two ways:
9408 • Previously some runs would run no steps (issue #376). This should no
9410 longer happen.
9411 • RuleBasedStateMachine tests which used bundles extensively would of‐
9413 ten shrink terribly. This should now be significantly improved,
9414 though there is likely a lot more room for improvement.
9415 This release also involves a low level change to how ranges of integers
9417 are handles which may result in other improvements to shrink quality in
9418 some cases.
9419 3.52.0 - 2018-03-24
9421 This release deprecates use of @settings(...) as a decorator, on func‐
9422 tions or methods that are not also decorated with @given. You can
9423 still apply these decorators in any order, though you should only do so
9424 once each.
9425 Applying @given twice was already deprecated, and applying @set‐
9427 tings(...) twice is deprecated in this release and will become an error
9428 in a future version. Neither could ever be used twice to good effect.
9429 Using @settings(...) as the sole decorator on a test is completely
9431 pointless, so this common usage error will become an error in a future
9432 version of Hypothesis.
9433 3.51.0 - 2018-03-24
9435 This release deprecates the average_size argument to lists() and other
9436 collection strategies. You should simply delete it wherever it was
9437 used in your tests, as it no longer has any effect.
9438 In early versions of Hypothesis, the average_size argument was treated
9440 as a hint about the distribution of examples from a strategy. Subse‐
9441 quent improvements to the conceptual model and the engine for generat‐
9442 ing and shrinking examples mean it is more effective to simply describe
9443 what constitutes a valid example, and let our internals handle the dis‐
9444 tribution.
9445 3.50.3 - 2018-03-24
9447 This patch contains some internal refactoring so that we can run with
9448 warnings as errors in CI.
9449 3.50.2 - 2018-03-20
9451 This has no user-visible changes except one slight formatting change to
9452 one docstring, to avoid a deprecation warning.
9453 3.50.1 - 2018-03-20
9455 This patch fixes an internal error introduced in 3.48.0, where a check
9456 for the Django test runner would expose import-time errors in Django
9457 configuration (issue #1167).
9458 3.50.0 - 2018-03-19
9460 This release improves validation of numeric bounds for some strategies.
9461 • integers() and floats() now raise InvalidArgument if passed a
9463 min_value or max_value which is not an instance of Real, instead of
9464 various internal errors.
9465 • floats() now converts its bounding values to the nearest float above
9467 or below the min or max bound respectively, instead of just casting
9468 to float. The old behaviour was incorrect in that you could generate
9469 float(min_value), even when this was less than min_value itself (pos‐
9470 sible with eg. fractions).
9471 • When both bounds are provided to floats() but there are no floats in
9473 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
9474 raised.
9475 • decimals() gives a more useful error message if passed a string that
9477 cannot be converted to Decimal in a context where this error is not
9478 trapped.
9479 Code that previously seemed to work may be explicitly broken if there
9481 were no floats between min_value and max_value (only possible with
9482 non-float bounds), or if a bound was not a Real number but still al‐
9483 lowed in python:math.isnan (some custom classes with a __float__
9484 method).
9485 3.49.1 - 2018-03-15
9487 This patch fixes our tests for Numpy dtype strategies on big-endian
9488 platforms, where the strategy behaved correctly but the test assumed
9489 that the native byte order was little-endian.
9490 There is no user impact unless you are running our test suite on
9492 big-endian platforms. Thanks to Graham Inggs for reporting issue
9493 #1164.
9494 3.49.0 - 2018-03-12
9496 This release deprecates passing elements=None to collection strategies,
9497 such as lists().
9498 Requiring lists(nothing()) or builds(list) instead of lists() means
9500 slightly more typing, but also improves the consistency and discover‐
9501 ability of our API - as well as showing how to compose or construct
9502 strategies in ways that still work in more complex situations.
9503 Passing a nonzero max_size to a collection strategy where the elements
9505 strategy contains no values is now deprecated, and will be an error in
9506 a future version. The equivalent with elements=None is already an er‐
9507 ror.
9508 3.48.1 - 2018-03-05
9510 This patch will minimize examples that would come out non-minimal in
9511 previous versions. Thanks to Kyle Reeve for this patch.
9512 3.48.0 - 2018-03-05
9514 This release improves some "unhappy paths" when using Hypothesis with
9515 the standard library python:unittest module:
9516 • Applying @given to a non-test method which is overridden from
9518 python:unittest.TestCase, such as setUp, raises a new health check.
9519 (issue #991)
9520 • Using subTest() within a test decorated with @given would leak inter‐
9522 mediate results when tests were run under the python:unittest test
9523 runner. Individual reporting of failing subtests is now disabled
9524 during a test using @given. (issue #1071)
9525 • @given is still not a class decorator, but the error message if you
9527 try using it on a class has been improved.
9528 As a related improvement, using django:django.test.TestCase with @given
9530 instead of hypothesis.extra.django.TestCase raises an explicit error
9531 instead of running all examples in a single database transaction.
9532 3.47.0 - 2018-03-02
9534 register_profile now accepts keyword arguments for specific settings,
9535 and the parent settings object is now optional. Using a name for a
9536 registered profile which is not a string was never suggested, but it is
9537 now also deprecated and will eventually be an error.
9538 3.46.2 - 2018-03-01
9540 This release removes an unnecessary branch from the code, and has no
9541 user-visible impact.
9542 3.46.1 - 2018-03-01
9544 This changes only the formatting of our docstrings and should have no
9545 user-visible effects.
9546 3.46.0 - 2018-02-26
9548 characters() has improved docs about what arguments are valid, and ad‐
9549 ditional validation logic to raise a clear error early (instead of e.g.
9550 silently ignoring a bad argument). Categories may be specified as the
9551 Unicode 'general category' (eg u'Nd'), or as the 'major category' (eg
9552 [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
9553 In previous versions, general categories were supported and all other
9555 input was silently ignored. Now, major categories are supported in ad‐
9556 dition to general categories (which may change the behaviour of some
9557 existing code), and all other input is deprecated.
9558 3.45.5 - 2018-02-26
9560 This patch improves strategy inference in hypothesis.extra.django to
9561 account for some validators in addition to field type - see issue #1116
9562 for ongoing work in this space.
9563 Specifically, if a CharField or TextField has an attached RegexValida‐
9565 tor, we now use from_regex() instead of text() as the underlying strat‐
9566 egy. This allows us to generate examples of the default User model,
9567 closing issue #1112.
9568 3.45.4 - 2018-02-25
9570 This patch improves some internal debugging information, fixes a typo
9571 in a validation error message, and expands the documentation for new
9572 contributors.
9573 3.45.3 - 2018-02-23
9575 This patch may improve example shrinking slightly for some strategies.
9576 3.45.2 - 2018-02-18
9578 This release makes our docstring style more consistent, thanks to
9579 flake8-docstrings. There are no user-visible changes.
9580 3.45.1 - 2018-02-17
9582 This fixes an indentation issue in docstrings for datetimes(), dates(),
9583 times(), and timedeltas().
9584 3.45.0 - 2018-02-13
9586 This release fixes builds() so that target can be used as a keyword ar‐
9587 gument for passing values to the target. The target itself can still be
9588 specified as a keyword argument, but that behavior is now deprecated.
9589 The target should be provided as the first positional argument.
9590 3.44.26 - 2018-02-06
9592 This release fixes some formatting issues in the Hypothesis source
9593 code. It should have no externally visible effects.
9594 3.44.25 - 2018-02-05
9596 This release changes the way in which Hypothesis tries to shrink the
9597 size of examples. It probably won't have much impact, but might make
9598 shrinking faster in some cases. It is unlikely but not impossible that
9599 it will change the resulting examples.
9600 3.44.24 - 2018-01-27
9602 This release fixes dependency information when installing Hypothesis
9603 from a binary "wheel" distribution.
9604 • The install_requires for enum34 is resolved at install time, rather
9606 than at build time (with potentially different results).
9607 • Django has fixed their python_requires for versions 2.0.0 onward,
9609 simplifying Python2-compatible constraints for downstream projects.
9610 3.44.23 - 2018-01-24
9612 This release improves shrinking in a class of pathological examples
9613 that you are probably never hitting in practice. If you are hitting
9614 them in practice this should be a significant speed up in shrinking. If
9615 you are not, you are very unlikely to notice any difference. You might
9616 see a slight slow down and/or slightly better falsifying examples.
9617 3.44.22 - 2018-01-23
9619 This release fixes a dependency problem. It was possible to install
9620 Hypothesis with an old version of attrs, which would throw a TypeError
9621 as soon as you tried to import hypothesis. Specifically, you need at‐
9622 trs 16.0.0 or newer.
9623 Hypothesis will now require the correct version of attrs when in‐
9625 stalling.
9626 3.44.21 - 2018-01-22
9628 This change adds some additional structural information that Hypothesis
9629 will use to guide its search.
9630 You mostly shouldn't see much difference from this. The two most likely
9632 effects you would notice are:
9633 1. Hypothesis stores slightly more examples in its database for passing
9635 tests.
9636 2. Hypothesis may find new bugs that it was previously missing, but it
9638 probably won't (this is a basic implementation of the feature that
9639 is intended to support future work. Although it is useful on its
9640 own, it's not very useful on its own).
9641 3.44.20 - 2018-01-21
9643 This is a small refactoring release that changes how Hypothesis tracks
9644 some information about the boundary of examples in its internal repre‐
9645 sentation.
9646 You are unlikely to see much difference in behaviour, but memory usage
9648 and run time may both go down slightly during normal test execution,
9649 and when failing Hypothesis might print its failing example slightly
9650 sooner.
9651 3.44.19 - 2018-01-21
9653 This changes how we compute the default average_size for all collection
9654 strategies. Previously setting a max_size without setting an aver‐
9655 age_size would have the seemingly paradoxical effect of making data
9656 generation slower, because it would raise the average size from its de‐
9657 fault. Now setting max_size will either leave the default unchanged or
9658 lower it from its default.
9659 If you are currently experiencing this problem, this may make your
9661 tests substantially faster. If you are not, this will likely have no
9662 effect on you.
9663 3.44.18 - 2018-01-20
9665 This is a small refactoring release that changes how Hypothesis detects
9666 when the structure of data generation depends on earlier values gener‐
9667 ated (e.g. when using flatmap or composite()). It should not have any
9668 observable effect on behaviour.
9669 3.44.17 - 2018-01-15
9671 This release fixes a typo in internal documentation, and has no
9672 user-visible impact.
9673 3.44.16 - 2018-01-13
9675 This release improves test case reduction for recursive data struc‐
9676 tures. Hypothesis now guarantees that whenever a strategy calls itself
9677 recursively (usually this will happen because you are using de‐
9678 ferred()), any recursive call may replace the top level value. e.g.
9679 given a tree structure, Hypothesis will always try replacing it with a
9680 subtree.
9681 Additionally this introduces a new heuristic that may in some circum‐
9683 stances significantly speed up test case reduction - Hypothesis should
9684 be better at immediately replacing elements drawn inside another strat‐
9685 egy with their minimal possible value.
9686 3.44.15 - 2018-01-13
9688 from_type() can now resolve recursive types such as binary trees (issue
9689 #1004). Detection of non-type arguments has also improved, leading to
9690 better error messages in many cases involving forward references.
9691 3.44.14 - 2018-01-08
9693 This release fixes a bug in the shrinker that prevented the optimisa‐
9694 tions in 3.44.6 from working in some cases. It would not have worked
9695 correctly when filtered examples were nested (e.g. with a set of inte‐
9696 gers in some range).
9697 This would not have resulted in any correctness problems, but shrinking
9699 may have been slower than it otherwise could be.
9700 3.44.13 - 2018-01-08
9702 This release changes the average bit length of values drawn from inte‐
9703 gers() to be much smaller. Additionally it changes the shrinking order
9704 so that now size is considered before sign - e.g. -1 will be preferred
9705 to +10.
9706 The new internal format for integers required some changes to the mini‐
9708 mizer to make work well, so you may also see some improvements to exam‐
9709 ple quality in unrelated areas.
9710 3.44.12 - 2018-01-07
9712 This changes Hypothesis's internal implementation of weighted sampling.
9713 This will affect example distribution and quality, but you shouldn't
9714 see any other effects.
9715 3.44.11 - 2018-01-06
9717 This is a change to some internals around how Hypothesis handles avoid‐
9718 ing generating duplicate examples and seeking out novel regions of the
9719 search space.
9720 You are unlikely to see much difference as a result of it, but it fixes
9722 a bug where an internal assertion could theoretically be triggered and
9723 has some minor effects on the distribution of examples so could poten‐
9724 tially find bugs that have previously been missed.
9725 3.44.10 - 2018-01-06
9727 This patch avoids creating debug statements when debugging is disabled.
9728 Profiling suggests this is a 5-10% performance improvement (issue
9729 #1040).
9730 3.44.9 - 2018-01-06
9732 This patch blacklists null characters ('\x00') in automatically created
9733 strategies for Django CharField and TextField, due to a database issue
9734 which was recently fixed upstream (Hypothesis issue #1045).
9735 3.44.8 - 2018-01-06
9737 This release makes the Hypothesis shrinker slightly less greedy in or‐
9738 der to avoid local minima - when it gets stuck, it makes a small at‐
9739 tempt to search around the final example it would previously have re‐
9740 turned to find a new starting point to shrink from. This should improve
9741 example quality in some cases, especially ones where the test data has
9742 dependencies among parts of it that make it difficult for Hypothesis to
9743 proceed.
9744 3.44.7 - 2018-01-04
9746 This release adds support for Django 2 in the hypothesis-django extra.
9747 This release drops support for Django 1.10, as it is no longer sup‐
9749 ported by the Django team.
9750 3.44.6 - 2018-01-02
9752 This release speeds up test case reduction in many examples by being
9753 better at detecting large shrinks it can use to discard redundant parts
9754 of its input. This will be particularly noticeable in examples that
9755 make use of filtering and for some integer ranges.
9756 3.44.5 - 2018-01-02
9758 Happy new year!
9759 This is a no-op release that updates the year range on all of the copy‐
9761 right headers in our source to include 2018.
9762 3.44.4 - 2017-12-23
9764 This release fixes issue #1041, which slowed tests by up to 6% due to
9765 broken caching.
9766 3.44.3 - 2017-12-21
9768 This release improves the shrinker in cases where examples drawn ear‐
9769 lier can affect how much data is drawn later (e.g. when you draw a
9770 length parameter in a composite and then draw that many elements). Ex‐
9771 amples found in cases like this should now be much closer to minimal.
9772 3.44.2 - 2017-12-20
9774 This is a pure refactoring release which changes how Hypothesis manages
9775 its set of examples internally. It should have no externally visible
9776 effects.
9777 3.44.1 - 2017-12-18
9779 This release fixes issue #997, in which under some circumstances the
9780 body of tests run under Hypothesis would not show up when run under
9781 coverage even though the tests were run and the code they called out‐
9782 side of the test file would show up normally.
9783 3.44.0 - 2017-12-17
9785 This release adds a new feature: The @reproduce_failure decorator, de‐
9786 signed to make it easy to use Hypothesis's binary format for examples
9787 to reproduce a problem locally without having to share your example
9788 database between machines.
9789 This also changes when seeds are printed:
9791 • They will no longer be printed for normal falsifying examples, as
9793 there are now adequate ways of reproducing those for all cases, so it
9794 just contributes noise.
9795 • They will once again be printed when reusing examples from the data‐
9797 base, as health check failures should now be more reliable in this
9798 scenario so it will almost always work in this case.
9799 This work was funded by Smarkets.
9801 3.43.1 - 2017-12-17
9803 This release fixes a bug with Hypothesis's database management - exam‐
9804 ples that were found in the course of shrinking were saved in a way
9805 that indicated that they had distinct causes, and so they would all be
9806 retried on the start of the next test. The intended behaviour, which is
9807 now what is implemented, is that only a bounded subset of these exam‐
9808 ples would be retried.
9809 3.43.0 - 2017-12-17
9811 HypothesisDeprecationWarning now inherits from python:FutureWarning in‐
9812 stead of python:DeprecationWarning, as recommended by PEP 565 for
9813 user-facing warnings (issue #618). If you have not changed the default
9814 warnings settings, you will now see each distinct HypothesisDepreca‐
9815 tionWarning instead of only the first.
9816 3.42.2 - 2017-12-12
9818 This patch fixes issue #1017, where instances of a list or tuple sub‐
9819 type used as an argument to a strategy would be coerced to tuple.
9820 3.42.1 - 2017-12-10
9822 This release has some internal cleanup, which makes reading the code
9823 more pleasant and may shrink large examples slightly faster.
9824 3.42.0 - 2017-12-09
9826 This release deprecates faker-extra, which was designed as a transition
9827 strategy but does not support example shrinking or coverage-guided dis‐
9828 covery.
9829 3.41.0 - 2017-12-06
9831 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
9832 pling from multi-dimensional ndarrays still results in a deprecation
9833 warning. Thanks to Charlie Tanksley for this patch.
9834 3.40.1 - 2017-12-04
9836 This release makes two changes:
9837 • It makes the calculation of some of the metadata that Hypothesis uses
9839 for shrinking occur lazily. This should speed up performance of test
9840 case generation a bit because it no longer calculates information it
9841 doesn't need.
9842 • It improves the shrinker for certain classes of nested examples. e.g.
9844 when shrinking lists of lists, the shrinker is now able to concate‐
9845 nate two adjacent lists together into a single list. As a result of
9846 this change, shrinking may get somewhat slower when the minimal exam‐
9847 ple found is large.
9848 3.40.0 - 2017-12-02
9850 This release improves how various ways of seeding Hypothesis interact
9851 with the example database:
9852 • Using the example database with seed() is now deprecated. You should
9854 set database=None if you are doing that. This will only warn if you
9855 actually load examples from the database while using @seed.
9856 • The derandomize will behave the same way as @seed.
9858 • Using --hypothesis-seed will disable use of the database.
9860 • If a test used examples from the database, it will not suggest using
9862 a seed to reproduce it, because that won't work.
9863 This work was funded by Smarkets.
9865 3.39.0 - 2017-12-01
9867 This release adds a new health check that checks if the smallest "natu‐
9868 ral" possible example of your test case is very large - this will tend
9869 to cause Hypothesis to generate bad examples and be quite slow.
9870 This work was funded by Smarkets.
9872 3.38.9 - 2017-11-29
9874 This is a documentation release to improve the documentation of shrink‐
9875 ing behaviour for Hypothesis's strategies.
9876 3.38.8 - 2017-11-29
9878 This release improves the performance of characters() when using black‐
9879 list_characters and from_regex() when using negative character classes.
9880 The problems this fixes were found in the course of work funded by
9882 Smarkets.
9883 3.38.7 - 2017-11-29
9885 This is a patch release for from_regex(), which had a bug in handling
9886 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
9887 rectly when parsing regex.
9888 3.38.6 - 2017-11-28
9890 This patch changes a few byte-string literals from double to single
9891 quotes, thanks to an update in unify. There are no user-visible
9892 changes.
9893 3.38.5 - 2017-11-23
9895 This fixes the repr of strategies using lambda that are defined inside
9896 decorators to include the lambda source.
9897 This would mostly have been visible when using the statistics function‐
9899 ality - lambdas used for e.g. filtering would have shown up with a <un‐
9900 known> as their body. This can still happen, but it should happen less
9901 often now.
9902 3.38.4 - 2017-11-22
9904 This release updates the reported statistics so that they show approxi‐
9905 mately what fraction of your test run time is spent in data generation
9906 (as opposed to test execution).
9907 This work was funded by Smarkets.
9909 3.38.3 - 2017-11-21
9911 This is a documentation release, which ensures code examples are up to
9912 date by running them as doctests in CI (issue #711).
9913 3.38.2 - 2017-11-21
9915 This release changes the behaviour of the deadline setting when used
9916 with data(): Time spent inside calls to data.draw will no longer be
9917 counted towards the deadline time.
9918 As a side effect of some refactoring required for this work, the way
9920 flaky tests are handled has changed slightly. You are unlikely to see
9921 much difference from this, but some error messages will have changed.
9922 This work was funded by Smarkets.
9924 3.38.1 - 2017-11-21
9926 This patch has a variety of non-user-visible refactorings, removing
9927 various minor warts ranging from indirect imports to typos in comments.
9928 3.38.0 - 2017-11-18
9930 This release overhauls the health check system in a variety of small
9931 ways. It adds no new features, but is nevertheless a minor release be‐
9932 cause it changes which tests are likely to fail health checks.
9933 The most noticeable effect is that some tests that used to fail health
9935 checks will now pass, and some that used to pass will fail. These
9936 should all be improvements in accuracy. In particular:
9937 • New failures will usually be because they are now taking into account
9939 things like use of data() and assume() inside the test body.
9940 • New failures may also be because for some classes of example the way
9942 data generation performance was measured was artificially faster than
9943 real data generation (for most examples that are hitting performance
9944 health checks the opposite should be the case).
9945 • Tests that used to fail health checks and now pass do so because the
9947 health check system used to run in a way that was subtly different
9948 than the main Hypothesis data generation and lacked some of its sup‐
9949 port for e.g. large examples.
9950 If your data generation is especially slow, you may also see your tests
9952 get somewhat faster, as there is no longer a separate health check
9953 phase. This will be particularly noticeable when rerunning test fail‐
9954 ures.
9955 This work was funded by Smarkets.
9957 3.37.0 - 2017-11-12
9959 This is a deprecation release for some health check related features.
9960 The following are now deprecated:
9962 • Passing HealthCheck.exception_in_generation to suppress_health_check.
9964 This no longer does anything even when passed - All errors that oc‐
9965 cur during data generation will now be immediately reraised rather
9966 than going through the health check mechanism.
9967 • Passing HealthCheck.random_module to suppress_health_check. This
9969 hasn't done anything for a long time, but was never explicitly depre‐
9970 cated. Hypothesis always seeds the random module when running @given
9971 tests, so this is no longer an error and suppressing it doesn't do
9972 anything.
9973 • Passing non-HealthCheck values in suppress_health_check. This was
9975 previously allowed but never did anything useful.
9976 In addition, passing a non-iterable value as suppress_health_check will
9978 now raise an error immediately (it would never have worked correctly,
9979 but it would previously have failed later). Some validation error mes‐
9980 sages have also been updated.
9981 This work was funded by Smarkets.
9983 3.36.1 - 2017-11-10
9985 This is a yak shaving release, mostly concerned with our own tests.
9986 While getfullargspec() was documented as deprecated in Python 3.5, it
9988 never actually emitted a warning. Our code to silence this (nonexis‐
9989 tent) warning has therefore been removed.
9990 We now run our tests with DeprecationWarning as an error, and made some
9992 minor changes to our own tests as a result. This required similar up‐
9993 stream updates to coverage and execnet (a test-time dependency via
9994 pytest-xdist).
9995 There is no user-visible change in Hypothesis itself, but we encourage
9997 you to consider enabling deprecations as errors in your own tests.
9998 3.36.0 - 2017-11-06
10000 This release adds a setting to the public API, and does some internal
10001 cleanup:
10002 • The derandomize setting is now documented (issue #890)
10004 • Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
10006 #953)
10007 • Documented the strict setting as deprecated, and updated the build so
10009 our docs always match deprecations in the code.
10010 3.35.0 - 2017-11-06
10012 This minor release supports constraining uuids() to generate a particu‐
10013 lar version of UUID (issue #721).
10014 Thanks to Dion Misic for this feature.
10016 3.34.1 - 2017-11-02
10018 This patch updates the documentation to suggest builds(callable) in‐
10019 stead of just(callable()).
10020 3.34.0 - 2017-11-02
10022 Hypothesis now emits deprecation warnings if you apply @given more than
10023 once to a target.
10024 Applying @given repeatedly wraps the target multiple times. Each wrap‐
10026 per will search the space of of possible parameters separately. This
10027 is equivalent but will be much more inefficient than doing it with a
10028 single call to @given.
10029 For example, instead of @given(booleans()) @given(integers()), you
10031 could write @given(booleans(), integers())
10032 3.33.1 - 2017-11-02
10034 This is a bugfix release:
10035 • builds() would try to infer a strategy for required positional argu‐
10037 ments of the target from type hints, even if they had been given to
10038 builds() as positional arguments (issue #946). Now it only infers
10039 missing required arguments.
10040 • An internal introspection function wrongly reported self as a re‐
10042 quired argument for bound methods, which might also have affected
10043 builds(). Now it knows better.
10044 3.33.0 - 2017-10-16
10046 This release supports strategy inference for more Django field types -
10047 you can now omit an argument for Date, Time, Duration, Slug, IP Ad‐
10048 dress, and UUID fields. (issue #642)
10049 Strategy generation for fields with grouped choices now selects choices
10051 from each group, instead of selecting from the group names.
10052 3.32.2 - 2017-10-15
10054 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
10055 an experimental basis. It has never actually worked, and the new Hy‐
10056 pothesis example database is designed to make such a tool unnecessary.
10057 3.32.1 - 2017-10-13
10059 This patch has two improvements for strategies based on enumerations.
10060 • from_type() now handles enumerations correctly, delegating to sam‐
10062 pled_from(). Previously it noted that Enum.__init__ has no required
10063 arguments and therefore delegated to builds(), which would subse‐
10064 quently fail.
10065 • When sampling from an python:enum.Flag, we also generate combinations
10067 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
10068 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
10069 3.32.0 - 2017-10-09
10071 This changes the default value of the use_coverage setting to True when
10072 running on pypy (it was already True on CPython).
10073 It was previously set to False because we expected it to be too slow,
10075 but recent benchmarking shows that actually performance of the feature
10076 on pypy is fairly acceptable - sometimes it's slower than on CPython,
10077 sometimes it's faster, but it's generally within a factor of two either
10078 way.
10079 3.31.6 - 2017-10-08
10081 This patch improves the quality of strategies inferred from Numpy
10082 dtypes:
10083 • Integer dtypes generated examples with the upper half of their
10085 (non-sign) bits set to zero. The inferred strategies can now produce
10086 any representable integer.
10087 • Fixed-width unicode- and byte-string dtypes now cap the internal ex‐
10089 ample length, which should improve example and shrink quality.
10090 • Numpy arrays can only store fixed-size strings internally, and allow
10092 shorter strings by right-padding them with null bytes. Inferred
10093 string strategies no longer generate such values, as they can never
10094 be retrieved from an array. This improves shrinking performance by
10095 skipping useless values.
10096 This has already been useful in Hypothesis - we found an overflow bug
10098 in our Pandas support, and as a result indexes() and range_indexes()
10099 now check that min_size and max_size are at least zero.
10100 3.31.5 - 2017-10-08
10102 This release fixes a performance problem in tests where the use_cover‐
10103 age setting is True.
10104 Tests experience a slow-down proportionate to the amount of code they
10106 cover. This is still the case, but the factor is now low enough that
10107 it should be unnoticeable. Previously it was large and became much
10108 larger in 3.30.4.
10109 3.31.4 - 2017-10-08
10111 from_type() failed with a very confusing error if passed a NewType() (‐
10112 issue #901). These psudeo-types are now unwrapped correctly, and
10113 strategy inference works as expected.
10114 3.31.3 - 2017-10-06
10116 This release makes some small optimisations to our use of coverage that
10117 should reduce constant per-example overhead. This is probably only no‐
10118 ticeable on examples where the test itself is quite fast. On no-op
10119 tests that don't test anything you may see up to a fourfold speed in‐
10120 crease (which is still significantly slower than without coverage). On
10121 more realistic tests the speed up is likely to be less than that.
10122 3.31.2 - 2017-09-30
10124 This release fixes some formatting and small typos/grammar issues in
10125 the documentation, specifically the page docs/settings.rst, and the in‐
10126 line docs for the various settings.
10127 3.31.1 - 2017-09-30
10129 This release improves the handling of deadlines so that they act better
10130 with the shrinking process. This fixes issue #892.
10131 This involves two changes:
10133 1. The deadline is raised during the initial generation and shrinking,
10135 and then lowered to the set value for final replay. This restricts
10136 our attention to examples which exceed the deadline by a more sig‐
10137 nificant margin, which increases their reliability.
10138 2. When despite the above a test still becomes flaky because it is sig‐
10140 nificantly faster on rerun than it was on its first run, the error
10141 message is now more explicit about the nature of this problem, and
10142 includes both the initial test run time and the new test run time.
10143 In addition, this release also clarifies the documentation of the dead‐
10145 line setting slightly to be more explicit about where it applies.
10146 This work was funded by Smarkets.
10148 3.31.0 - 2017-09-29
10150 This release blocks installation of Hypothesis on Python 3.3, which
10151 reached its end of life date on 2017-09-29.
10152 This should not be of interest to anyone but downstream maintainers -
10154 if you are affected, migrate to a secure version of Python as soon as
10155 possible or at least seek commercial support.
10156 3.30.4 - 2017-09-27
10158 This release makes several changes:
10159 1. It significantly improves Hypothesis's ability to use coverage in‐
10161 formation to find interesting examples.
10162 2. It reduces the default max_examples setting from 200 to 100. This
10164 takes advantage of the improved algorithm meaning fewer examples are
10165 typically needed to get the same testing and is sufficiently better
10166 at covering interesting behaviour, and offsets some of the perfor‐
10167 mance problems of running under coverage.
10168 3. Hypothesis will always try to start its testing with an example that
10170 is near minimized.
10171 The new algorithm for 1 also makes some changes to Hypothesis's low
10173 level data generation which apply even with coverage turned off. They
10174 generally reduce the total amount of data generated, which should im‐
10175 prove test performance somewhat. Between this and 3 you should see a
10176 noticeable reduction in test runtime (how much so depends on your tests
10177 and how much example size affects their performance. On our benchmarks,
10178 where data generation dominates, we saw up to a factor of two perfor‐
10179 mance improvement, but it's unlikely to be that large.
10180 3.30.3 - 2017-09-25
10182 This release fixes some formatting and small typos/grammar issues in
10183 the documentation, specifically the page docs/details.rst, and some in‐
10184 line docs linked from there.
10185 3.30.2 - 2017-09-24
10187 This release changes Hypothesis's caching approach for functions in hy‐
10188 pothesis.strategies. Previously it would have cached extremely aggres‐
10189 sively and cache entries would never be evicted. Now it adopts a
10190 least-frequently used, least recently used key invalidation policy, and
10191 is somewhat more conservative about which strategies it caches.
10192 Workloads which create strategies based on dynamic values, e.g. by us‐
10194 ing flatmap or composite(), will use significantly less memory.
10195 3.30.1 - 2017-09-22
10197 This release fixes a bug where when running with the use_coverage=True
10198 setting inside an existing running instance of coverage, Hypothesis
10199 would frequently put files that the coveragerc excluded in the report
10200 for the enclosing coverage.
10201 3.30.0 - 2017-09-20
10203 This release introduces two new features:
10204 • When a test fails, either with a health check failure or a falsifying
10206 example, Hypothesis will print out a seed that led to that failure,
10207 if the test is not already running with a fixed seed. You can then
10208 recreate that failure using either the @seed decorator or (if you are
10209 running pytest) with --hypothesis-seed.
10210 • pytest users can specify a seed to use for @given based tests by
10212 passing the --hypothesis-seed command line argument.
10213 This work was funded by Smarkets.
10215 3.29.0 - 2017-09-19
10217 This release makes Hypothesis coverage aware. Hypothesis now runs all
10218 test bodies under coverage, and uses this information to guide its
10219 testing.
10220 The use_coverage setting can be used to disable this behaviour if you
10222 want to test code that is sensitive to coverage being enabled (either
10223 because of performance or interaction with the trace function).
10224 The main benefits of this feature are:
10226 • Hypothesis now observes when examples it discovers cover particular
10228 lines or branches and stores them in the database for later.
10229 • Hypothesis will make some use of this information to guide its explo‐
10231 ration of the search space and improve the examples it finds (this is
10232 currently used only very lightly and will likely improve signifi‐
10233 cantly in future releases).
10234 This also has the following side-effects:
10236 • Hypothesis now has an install time dependency on the coverage pack‐
10238 age.
10239 • Tests that are already running Hypothesis under coverage will likely
10241 get faster.
10242 • Tests that are not running under coverage now run their test bodies
10244 under coverage by default.
10245 This feature is only partially supported under pypy. It is signifi‐
10247 cantly slower than on CPython and is turned off by default as a result,
10248 but it should still work correctly if you want to use it.
10249 3.28.3 - 2017-09-18
10251 This release is an internal change that affects how Hypothesis handles
10252 calculating certain properties of strategies.
10253 The primary effect of this is that it fixes a bug where use of de‐
10255 ferred() could sometimes trigger an internal assertion error. However
10256 the fix for this bug involved some moderately deep changes to how Hy‐
10257 pothesis handles certain constructs so you may notice some additional
10258 knock-on effects.
10259 In particular the way Hypothesis handles drawing data from strategies
10261 that cannot generate any values has changed to bail out sooner than it
10262 previously did. This may speed up certain tests, but it is unlikely to
10263 make much of a difference in practice for tests that were not already
10264 failing with Unsatisfiable.
10265 3.28.2 - 2017-09-18
10267 This is a patch release that fixes a bug in the hypothesis.extra.pandas
10268 documentation where it incorrectly referred to column() instead of col‐
10269 umns().
10270 3.28.1 - 2017-09-16
10272 This is a refactoring release. It moves a number of internal uses of
10273 namedtuple() over to using attrs based classes, and removes a couple of
10274 internal namedtuple classes that were no longer in use.
10275 It should have no user visible impact.
10277 3.28.0 - 2017-09-15
10279 This release adds support for testing pandas via the hypothesis.ex‐
10280 tra.pandas module.
10281 It also adds a dependency on attrs.
10283 This work was funded by Stripe.
10285 3.27.1 - 2017-09-14
10287 This release fixes some formatting and broken cross-references in the
10288 documentation, which includes editing docstrings - and thus a patch re‐
10289 lease.
10290 3.27.0 - 2017-09-13
10292 This release introduces a deadline setting to Hypothesis.
10293 When set this turns slow tests into errors. By default it is unset but
10295 will warn if you exceed 200ms, which will become the default value in a
10296 future release.
10297 This work was funded by Smarkets.
10299 3.26.0 - 2017-09-12
10301 Hypothesis now emits deprecation warnings if you are using the legacy
10302 SQLite example database format, or the tool for merging them. These
10303 were already documented as deprecated, so this doesn't change their
10304 deprecation status, only that we warn about it.
10305 3.25.1 - 2017-09-12
10307 This release fixes a bug with generating numpy datetime and timedelta
10308 types: When inferring the strategy from the dtype, datetime and
10309 timedelta dtypes with sub-second precision would always produce exam‐
10310 ples with one second resolution. Inferring a strategy from a time
10311 dtype will now always produce example with the same precision.
10312 3.25.0 - 2017-09-12
10314 This release changes how Hypothesis shrinks and replays examples to
10315 take into account that it can encounter new bugs while shrinking the
10316 bug it originally found. Previously it would end up replacing the orig‐
10317 inally found bug with the new bug and show you only that one. Now it is
10318 (often) able to recognise when two bugs are distinct and when it finds
10319 more than one will show both.
10320 3.24.2 - 2017-09-11
10322 This release removes the (purely internal and no longer useful) strat‐
10323 egy_test_suite function and the corresponding strategytests module.
10324 3.24.1 - 2017-09-06
10326 This release improves the reduction of examples involving floating
10327 point numbers to produce more human readable examples.
10328 It also has some general purpose changes to the way the minimizer works
10330 internally, which may see some improvement in quality and slow down of
10331 test case reduction in cases that have nothing to do with floating
10332 point numbers.
10333 3.24.0 - 2017-09-05
10335 Hypothesis now emits deprecation warnings if you use some_strategy.ex‐
10336 ample() inside a test function or strategy definition (this was never
10337 intended to be supported, but is sufficiently widespread that it war‐
10338 rants a deprecation path).
10339 3.23.3 - 2017-09-05
10341 This is a bugfix release for decimals() with the places argument.
10342 • No longer fails health checks (issue #725, due to internal filtering)
10344 • Specifying a min_value and max_value without any decimals with places
10346 places between them gives a more useful error message.
10347 • Works for any valid arguments, regardless of the decimal precision
10349 context.
10350 3.23.2 - 2017-09-01
10352 This is a small refactoring release that removes a now-unused parameter
10353 to an internal API. It shouldn't have any user visible effect.
10354 3.23.1 - 2017-09-01
10356 Hypothesis no longer propagates the dynamic scope of settings into
10357 strategy definitions.
10358 This release is a small change to something that was never part of the
10360 public API and you will almost certainly not notice any effect unless
10361 you're doing something surprising, but for example the following code
10362 will now give a different answer in some circumstances:
10363 import hypothesis.strategies as st
10365 from hypothesis import settings
10366 CURRENT_SETTINGS = st.builds(lambda: settings.default)
10368 (We don't actually encourage you writing code like this)
10370 Previously this would have generated the settings that were in effect
10372 at the point of definition of CURRENT_SETTINGS. Now it will generate
10373 the settings that are used for the current test.
10374 It is very unlikely to be significant enough to be visible, but you may
10376 also notice a small performance improvement.
10377 3.23.0 - 2017-08-31
10379 This release adds a unique argument to arrays() which behaves the same
10380 ways as the corresponding one for lists(), requiring all of the ele‐
10381 ments in the generated array to be distinct.
10382 3.22.2 - 2017-08-29
10384 This release fixes an issue where Hypothesis would raise a TypeError
10385 when using the datetime-related strategies if running with PYTHONOPTI‐
10386 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
10387 3.22.1 - 2017-08-28
10389 Hypothesis now transparently handles problems with an internal unicode
10390 cache, including file truncation or read-only filesystems (issue #767).
10391 Thanks to Sam Hames for the patch.
10392 3.22.0 - 2017-08-26
10394 This release provides what should be a substantial performance improve‐
10395 ment to numpy arrays generated using provided numpy support, and adds a
10396 new fill_value argument to arrays() to control this behaviour.
10397 This work was funded by Stripe.
10399 3.21.3 - 2017-08-26
10401 This release fixes some extremely specific circumstances that probably
10402 have never occurred in the wild where users of deferred() might have
10403 seen a python:RuntimeError from too much recursion, usually in cases
10404 where no valid example could have been generated anyway.
10405 3.21.2 - 2017-08-25
10407 This release fixes some minor bugs in argument validation:
10408 • hypothesis.extra.numpy dtype strategies would raise an internal
10410 error instead of an InvalidArgument exception when passed an in‐
10411 valid endianness specification.
10412 • fractions() would raise an internal error instead of an InvalidAr‐
10414 gument if passed float("nan") as one of its bounds.
10415 • The error message for passing float("nan") as a bound to various
10417 strategies has been improved.
10418 • Various bound arguments will now raise InvalidArgument in cases
10420 where they would previously have raised an internal TypeError or
10421 ValueError from the relevant conversion function.
10422 • streaming() would not have emitted a deprecation warning when
10424 called with an invalid argument.
10425 3.21.1 - 2017-08-24
10427 This release fixes a bug where test failures that were the result of an
10428 @example would print an extra stack trace before re-raising the excep‐
10429 tion.
10430 3.21.0 - 2017-08-23
10432 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
10433 sis's deprecation warnings into errors. Similar functionality can be
10434 achieved by using simplefilter('error', HypothesisDeprecationWarning).
10435 3.20.0 - 2017-08-22
10437 This release renames the relevant arguments on the datetimes(),
10438 dates(), times(), and timedeltas() strategies to min_value and
10439 max_value, to make them consistent with the other strategies in the
10440 module.
10441 The old argument names are still supported but will emit a deprecation
10443 warning when used explicitly as keyword arguments. Arguments passed po‐
10444 sitionally will go to the new argument names and are not deprecated.
10445 3.19.3 - 2017-08-22
10447 This release provides a major overhaul to the internals of how Hypothe‐
10448 sis handles shrinking.
10449 This should mostly be visible in terms of getting better examples for
10451 tests which make heavy use of composite(), data() or flatmap where the
10452 data drawn depends a lot on previous choices, especially where size pa‐
10453 rameters are affected. Previously Hypothesis would have struggled to
10454 reliably produce good examples here. Now it should do much better. Per‐
10455 formance should also be better for examples with a non-zero min_size.
10456 You may see slight changes to example generation (e.g. improved example
10458 diversity) as a result of related changes to internals, but they are
10459 unlikely to be significant enough to notice.
10460 3.19.2 - 2017-08-21
10462 This release fixes two bugs in hypothesis.extra.numpy:
10463 • unicode_string_dtypes() didn't work at all due to an incorrect dtype
10465 specifier. Now it does.
10466 • Various impossible conditions would have been accepted but would er‐
10468 ror when they fail to produced any example. Now they raise an ex‐
10469 plicit InvalidArgument error.
10470 3.19.1 - 2017-08-21
10472 This is a bugfix release for issue #739, where bounds for fractions()
10473 or floating-point decimals() were not properly converted to integers
10474 before passing them to the integers strategy. This excluded some val‐
10475 ues that should have been possible, and could trigger internal errors
10476 if the bounds lay between adjacent integers.
10477 You can now bound fractions() with two arbitrarily close fractions.
10479 It is now an explicit error to supply a min_value, max_value, and
10481 max_denominator to fractions() where the value bounds do not include a
10482 fraction with denominator at most max_denominator.
10483 3.19.0 - 2017-08-20
10485 This release adds the from_regex() strategy, which generates strings
10486 that contain a match of a regular expression.
10487 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
10489 then helping to upstream it! (issue #662)
10490 3.18.5 - 2017-08-18
10492 This is a bugfix release for integers(). Previously the strategy would
10493 hit an internal assertion if passed non-integer bounds for min_value
10494 and max_value that had no integers between them. The strategy now
10495 raises InvalidArgument instead.
10496 3.18.4 - 2017-08-18
10498 Release to fix a bug where mocks can be used as test runners under cer‐
10499 tain conditions. Specifically, if a mock is injected into a test via
10500 pytest fixtures or patch decorators, and that mock is the first argu‐
10501 ment in the list, hypothesis will think it represents self and turns
10502 the mock into a test runner. If this happens, the affected test always
10503 passes because the mock is executed instead of the test body. Some‐
10504 times, it will also fail health checks.
10505 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
10507 for this bug fix.
10508 3.18.3 - 2017-08-17
10510 This release should improve the performance of some tests which experi‐
10511 enced a slow down as a result of the 3.13.0 release.
10512 Tests most likely to benefit from this are ones that make extensive use
10514 of min_size parameters, but others may see some improvement as well.
10515 3.18.2 - 2017-08-16
10517 This release fixes a bug introduced in 3.18.0. If the arguments
10518 whitelist_characters and blacklist_characters to characters() contained
10519 overlapping elements, then an InvalidArgument exception would be
10520 raised.
10521 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
10523 3.18.1 - 2017-08-14
10525 This is a bug fix release to fix issue #780, where sets() and similar
10526 would trigger health check errors if their element strategy could only
10527 produce one element (e.g. if it was just()).
10528 3.18.0 - 2017-08-13
10530 This is a feature release:
10531 • characters() now accepts whitelist_characters, particular characters
10533 which will be added to those it produces. (issue #668)
10534 • A bug fix for the internal function _union_interval_lists(), and a
10536 rename to _union_intervals(). It now correctly handles all cases
10537 where intervals overlap, and it always returns the result as a tuple
10538 for tuples.
10539 Thanks to Alex Willmer for these.
10541 3.17.0 - 2017-08-07
10543 This release documents the previously undocumented phases feature, mak‐
10544 ing it part of the public API. It also updates how the example database
10545 is used. Principally:
10546 • A Phases.reuse argument will now correctly control whether examples
10548 from the database are run (it previously did exactly the wrong thing
10549 and controlled whether examples would be saved).
10550 • Hypothesis will no longer try to rerun all previously failing exam‐
10552 ples. Instead it will replay the smallest previously failing example
10553 and a selection of other examples that are likely to trigger any
10554 other bugs that will found. This prevents a previous failure from
10555 dominating your tests unnecessarily.
10556 • As a result of the previous change, Hypothesis will be slower about
10558 clearing out old examples from the database that are no longer fail‐
10559 ing (because it can only clear out ones that it actually runs).
10560 3.16.1 - 2017-08-07
10562 This release makes an implementation change to how Hypothesis handles
10563 certain internal constructs.
10564 The main effect you should see is improvement to the behaviour and per‐
10566 formance of collection types, especially ones with a min_size parame‐
10567 ter. Many cases that would previously fail due to being unable to gen‐
10568 erate enough valid examples will now succeed, and other cases should
10569 run slightly faster.
10570 3.16.0 - 2017-08-04
10572 This release introduces a deprecation of the timeout feature. This re‐
10573 sults in the following changes:
10574 • Creating a settings object with an explicit timeout will emit a dep‐
10576 recation warning.
10577 • If your test stops because it hits the timeout (and has not found a
10579 bug) then it will emit a deprecation warning.
10580 • There is a new value unlimited which you can import from hypothesis.
10582 settings(timeout=unlimited) will not cause a deprecation warning.
10583 • There is a new health check, hung_test, which will trigger after a
10585 test has been running for five minutes if it is not suppressed.
10586 3.15.0 - 2017-08-04
10588 This release deprecates two strategies, choices() and streaming().
10589 Both of these are somewhat confusing to use and are entirely redundant
10591 since the introduction of the data() strategy for interactive drawing
10592 in tests, and their use should be replaced with direct use of data()
10593 instead.
10594 3.14.2 - 2017-08-03
10596 This fixes a bug where Hypothesis would not work correctly on Python
10597 2.7 if you had the python:typing module backport installed.
10598 3.14.1 - 2017-08-02
10600 This raises the maximum depth at which Hypothesis starts cutting off
10601 data generation to a more reasonable value which it is harder to hit by
10602 accident.
10603 This resolves (issue #751), in which some examples which previously
10605 worked would start timing out, but it will also likely improve the data
10606 generation quality for complex data types.
10607 3.14.0 - 2017-07-23
10609 Hypothesis now understands inline type annotations (issue #293):
10610 • If the target of builds() has type annotations, a default strategy
10612 for missing required arguments is selected based on the type.
10613 Type-based strategy selection will only override a default if you
10614 pass hypothesis.infer as a keyword argument.
10615 • If @given wraps a function with type annotations, you can pass infer
10617 as a keyword argument and the appropriate strategy will be substi‐
10618 tuted.
10619 • You can check what strategy will be inferred for a type with the new
10621 from_type() function.
10622 • register_type_strategy() teaches Hypothesis which strategy to infer
10624 for custom or unknown types. You can provide a strategy, or for more
10625 complex cases a function which takes the type and returns a strategy.
10626 3.13.1 - 2017-07-20
10628 This is a bug fix release for issue #514 - Hypothesis would continue
10629 running examples after a SkipTest exception was raised, including
10630 printing a falsifying example. Skip exceptions from the standard
10631 python:unittest module, and pytest, nose, or unittest2 modules now
10632 abort the test immediately without printing output.
10633 3.13.0 - 2017-07-16
10635 This release has two major aspects to it: The first is the introduction
10636 of deferred(), which allows more natural definition of recursive (in‐
10637 cluding mutually recursive) strategies.
10638 The second is a number of engine changes designed to support this sort
10640 of strategy better. These should have a knock-on effect of also improv‐
10641 ing the performance of any existing strategies that currently generate
10642 a lot of data or involve heavy nesting by reducing their typical exam‐
10643 ple size.
10644 3.12.0 - 2017-07-07
10646 This release makes some major internal changes to how Hypothesis repre‐
10647 sents data internally, as a prelude to some major engine changes that
10648 should improve data quality. There are no API changes, but it's a sig‐
10649 nificant enough internal change that a minor version bump seemed war‐
10650 ranted.
10651 User facing impact should be fairly mild, but includes:
10653 • All existing examples in the database will probably be invalidated.
10655 Hypothesis handles this automatically, so you don't need to do any‐
10656 thing, but if you see all your examples disappear that's why.
10657 • Almost all data distributions have changed significantly. Possibly
10659 for the better, possibly for the worse. This may result in new bugs
10660 being found, but it may also result in Hypothesis being unable to
10661 find bugs it previously did.
10662 • Data generation may be somewhat faster if your existing bottleneck
10664 was in draw_bytes (which is often the case for large examples).
10665 • Shrinking will probably be slower, possibly significantly.
10667 If you notice any effects you consider to be a significant regression,
10669 please open an issue about them.
10670 3.11.6 - 2017-06-19
10672 This release involves no functionality changes, but is the first to
10673 ship wheels as well as an sdist.
10674 3.11.5 - 2017-06-18
10676 This release provides a performance improvement to shrinking. For cases
10677 where there is some non-trivial "boundary" value (e.g. the bug happens
10678 for all values greater than some other value), shrinking should now be
10679 substantially faster. Other types of bug will likely see improvements
10680 too.
10681 This may also result in some changes to the quality of the final exam‐
10683 ples - it may sometimes be better, but is more likely to get slightly
10684 worse in some edge cases. If you see any examples where this happens in
10685 practice, please report them.
10686 3.11.4 - 2017-06-17
10688 This is a bugfix release: Hypothesis now prints explicit examples when
10689 running in verbose mode. (issue #313)
10690 3.11.3 - 2017-06-11
10692 This is a bugfix release: Hypothesis no longer emits a warning if you
10693 try to use sampled_from() with python:collections.OrderedDict. (issue
10694 #688)
10695 3.11.2 - 2017-06-10
10697 This is a documentation release. Several outdated snippets have been
10698 updated or removed, and many cross-references are now hyperlinks.
10699 3.11.1 - 2017-05-28
10701 This is a minor ergonomics release. Tracebacks shown by pytest no
10702 longer include Hypothesis internals for test functions decorated with
10703 @given.
10704 3.11.0 - 2017-05-23
10706 This is a feature release, adding datetime-related strategies to the
10707 core strategies.
10708 timezones() allows you to sample pytz timezones from the Olsen data‐
10710 base. Use directly in a recipe for tz-aware datetimes, or compose with
10711 none() to allow a mix of aware and naive output.
10712 The new dates(), times(), datetimes(), and timedeltas() strategies are
10714 all constrained by objects of their type. This means that you can gen‐
10715 erate dates bounded by a single day (i.e. a single date), or datetimes
10716 constrained to the microsecond.
10717 times() and datetimes() take an optional timezones= argument, which de‐
10719 faults to none() for naive times. You can use our extra strategy based
10720 on pytz, or roll your own timezones strategy with dateutil or even the
10721 standard library.
10722 The old dates, times, and datetimes strategies in hypothesis.ex‐
10724 tra.datetimes are deprecated in favor of the new core strategies, which
10725 are more flexible and have no dependencies.
10726 3.10.0 - 2017-05-22
10728 Hypothesis now uses python:inspect.getfullargspec() internally. On
10729 Python 2, there are no visible changes.
10730 On Python 3 @given and @composite now preserve PEP 3107 annotations on
10732 the decorated function. Keyword-only arguments are now either handled
10733 correctly (e.g. @composite), or caught in validation instead of
10734 silently discarded or raising an unrelated error later (e.g. @given).
10735 3.9.1 - 2017-05-22
10737 This is a bugfix release: the default field mapping for a DateTimeField
10738 in the Django extra now respects the USE_TZ setting when choosing a
10739 strategy.
10740 3.9.0 - 2017-05-19
10742 This is feature release, expanding the capabilities of the decimals()
10743 strategy.
10744 • The new (optional) places argument allows you to generate decimals
10746 with a certain number of places (e.g. cents, thousandths, satoshis).
10747 • If allow_infinity is None, setting min_bound no longer excludes posi‐
10749 tive infinity and setting max_value no longer excludes negative in‐
10750 finity.
10751 • All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
10753 True, or if allow_nan is None and min_value or max_value is None.
10754 • min_value and max_value may be given as decimal strings, e.g.
10756 "1.234".
10757 3.8.5 - 2017-05-16
10759 Hypothesis now imports python:sqlite3 when a SQLite database is used,
10760 rather than at module load, improving compatibility with Python imple‐
10761 mentations compiled without SQLite support (such as BSD or Jython).
10762 3.8.4 - 2017-05-16
10764 This is a compatibility bugfix release. sampled_from() no longer
10765 raises a deprecation warning when sampling from an python:enum.Enum, as
10766 all enums have a reliable iteration order.
10767 3.8.3 - 2017-05-09
10769 This release removes a version check for older versions of pytest when
10770 using the Hypothesis pytest plugin. The pytest plugin will now run un‐
10771 conditionally on all versions of pytest. This breaks compatibility with
10772 any version of pytest prior to 2.7.0 (which is more than two years
10773 old).
10774 The primary reason for this change is that the version check was a fre‐
10776 quent source of breakage when pytest change their versioning scheme. If
10777 you are not working on pytest itself and are not running a very old
10778 version of it, this release probably doesn't affect you.
10779 3.8.2 - 2017-04-26
10781 This is a code reorganisation release that moves some internal test
10782 helpers out of the main source tree so as to not have changes to them
10783 trigger releases in future.
10784 3.8.1 - 2017-04-26
10786 This is a documentation release. Almost all code examples are now
10787 doctests checked in CI, eliminating stale examples.
10788 3.8.0 - 2017-04-23
10790 This is a feature release, adding the iterables() strategy, equivalent
10791 to lists(...).map(iter) but with a much more useful repr. You can use
10792 this strategy to check that code doesn't accidentally depend on se‐
10793 quence properties such as indexing support or repeated iteration.
10794 3.7.4 - 2017-04-22
10796 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
10797 ture in the same test could cause the test to fail to fill the argu‐
10798 ments, and throw a TypeError.
10799 3.7.3 - 2017-04-21
10801 This release should include no user visible changes and is purely a
10802 refactoring release. This modularises the behaviour of the core given()
10803 function, breaking it up into smaller and more accessible parts, but
10804 its actual behaviour should remain unchanged.
10805 3.7.2 - 2017-04-21
10807 This reverts an undocumented change in 3.7.1 which broke installation
10808 on debian stable: The specifier for the hypothesis[django] extra_re‐
10809 quires had introduced a wild card, which was not supported on the de‐
10810 fault version of pip.
10811 3.7.1 - 2017-04-21
10813 This is a bug fix and internal improvements release.
10814 • In particular Hypothesis now tracks a tree of where it has already
10816 explored. This allows it to avoid some classes of duplicate exam‐
10817 ples, and significantly improves the performance of shrinking failing
10818 examples by allowing it to skip some shrinks that it can determine
10819 can't possibly work.
10820 • Hypothesis will no longer seed the global random arbitrarily unless
10822 you have asked it to using random_module()
10823 • Shrinking would previously have not worked correctly in some special
10825 cases on Python 2, and would have resulted in suboptimal examples.
10826 3.7.0 - 2017-03-20
10828 This is a feature release.
10829 New features:
10831 • Rule based stateful testing now has an @invariant decorator that
10833 specifies methods that are run after init and after every step, al‐
10834 lowing you to encode properties that should be true at all times.
10835 Thanks to Tom Prince for this feature.
10836 • The decimals() strategy now supports allow_nan and allow_infinity
10838 flags.
10839 • There are significantly more strategies available for numpy, includ‐
10841 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
10842 for this feature.
10843 • When using the data() strategy you can now add a label as an argument
10845 to draw(), which will be printed along with the value when an example
10846 fails. Thanks to Peter Inglesby for this feature.
10847 Bug fixes:
10849 • Bug fix: composite() now preserves functions' docstrings.
10851 • The build is now reproducible and doesn't depend on the path you
10853 build it from. Thanks to Chris Lamb for this feature.
10854 • numpy strategies for the void data type did not work correctly.
10856 Thanks to Zac Hatfield Dodds for this fix.
10857 There have also been a number of performance optimizations:
10859 • The permutations() strategy is now significantly faster to use for
10861 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
10862 • Shrinking of failing test cases should have got significantly faster
10864 in some circumstances where it was previously struggling for a long
10865 time.
10866 • Example generation now involves less indirection, which results in a
10868 small speedup in some cases (small enough that you won't really no‐
10869 tice it except in pathological cases).
10870 3.6.1 - 2016-12-20
10872 This release fixes a dependency problem and makes some small behind the
10873 scenes improvements.
10874 • The fake-factory dependency was renamed to faker. If you were depend‐
10876 ing on it through hypothesis[django] or hypothesis[fake-factory]
10877 without pinning it yourself then it would have failed to install
10878 properly. This release changes it so that hypothesis[fakefactory]
10879 (which can now also be installed as hypothesis[faker]) will install
10880 the renamed faker package instead.
10881 • This release also removed the dependency of hypothesis[django] on hy‐
10883 pothesis[fakefactory] - it was only being used for emails. These now
10884 use a custom strategy that isn't from fakefactory. As a result you
10885 should also see performance improvements of tests which generated
10886 User objects or other things with email fields, as well as better
10887 shrinking of email addresses.
10888 • The distribution of code using nested calls to one_of() or the | op‐
10890 erator for combining strategies has been improved, as branches are
10891 now flattened to give a more uniform distribution.
10892 • Examples using composite() or .flatmap should now shrink better. In
10894 particular this will affect things which work by first generating a
10895 length and then generating that many items, which have historically
10896 not shrunk very well.
10897 3.6.0 - 2016-10-31
10899 This release reverts Hypothesis to its old pretty printing of lambda
10900 functions based on attempting to extract the source code rather than
10901 decompile the bytecode. This is unfortunately slightly inferior in
10902 some cases and may result in you occasionally seeing things like lambda
10903 x: <unknown> in statistics reports and strategy reprs.
10904 This removes the dependencies on uncompyle6, xdis and spark-parser.
10906 The reason for this is that the new functionality was based on un‐
10908 compyle6, which turns out to introduce a hidden GPLed dependency - it
10909 in turn depended on xdis, and although the library was licensed under
10910 the MIT license, it contained some GPL licensed source code and thus
10911 should have been released under the GPL.
10912 My interpretation is that Hypothesis itself was never in violation of
10914 the GPL (because the license it is under, the Mozilla Public License
10915 v2, is fully compatible with being included in a GPL licensed work),
10916 but I have not consulted a lawyer on the subject. Regardless of the an‐
10917 swer to this question, adding a GPLed dependency will likely cause a
10918 lot of users of Hypothesis to inadvertently be in violation of the GPL.
10919 As a result, if you are running Hypothesis 3.5.x you really should up‐
10921 grade to this release immediately.
10922 3.5.3 - 2016-10-05
10924 This is a bug fix release.
10925 Bugs fixed:
10927 • If the same test was running concurrently in two processes and there
10929 were examples already in the test database which no longer failed,
10930 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
10931 Python 2) because an example it was trying to read was deleted before
10932 it was read. (issue #372).
10933 • Drawing from an integers() strategy with both a min_value and a
10935 max_value would reject too many examples needlessly. Now it repeat‐
10936 edly redraws until satisfied. (pull request #366. Thanks to Calen
10937 Pennington for the contribution).
10938 3.5.2 - 2016-09-24
10940 This is a bug fix release.
10941 • The Hypothesis pytest plugin broke pytest support for doctests. Now
10943 it doesn't.
10944 3.5.1 - 2016-09-23
10946 This is a bug fix release.
10947 • Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
10949 bytes and unicode.
10950 • python:unittest.TestCase tests would not have shown up in the new
10952 statistics mode. Now they do.
10953 • Similarly, stateful tests would not have shown up in statistics and
10955 now they do.
10956 • Statistics now print with pytest node IDs (the names you'd get in
10958 pytest verbose mode).
10959 3.5.0 - 2016-09-22
10961 This is a feature release.
10962 • fractions() and decimals() strategies now support min_value and
10964 max_value parameters. Thanks go to Anne Mulhern for the development
10965 of this feature.
10966 • The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
10968 tics parameter that gives detailed statistics about the tests that
10969 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
10970 funding the development of this feature.
10971 • There is a new event() function that can be used to add custom sta‐
10973 tistics.
10974 Additionally there have been some minor bug fixes:
10976 • In some cases Hypothesis should produce fewer duplicate examples
10978 (this will mostly only affect cases with a single parameter).
10979 • pytest command line parameters are now under an option group for Hy‐
10981 pothesis (thanks to David Keijser for fixing this)
10982 • Hypothesis would previously error if you used PEP 3107 function anno‐
10984 tations on your tests under Python 3.4.
10985 • The repr of many strategies using lambdas has been improved to in‐
10987 clude the lambda body (this was previously supported in many but not
10988 all cases).
10989 3.4.2 - 2016-07-13
10991 This is a bug fix release, fixing a number of problems with the set‐
10992 tings system:
10993 • Test functions defined using @given can now be called from other
10995 threads (issue #337)
10996 • Attempting to delete a settings property would previously have
10998 silently done the wrong thing. Now it raises an AttributeError.
10999 • Creating a settings object with a custom database_file parameter was
11001 silently getting ignored and the default was being used instead. Now
11002 it's not.
11003 3.4.1 - 2016-07-07
11005 This is a bug fix release for a single bug:
11006 • On Windows when running two Hypothesis processes in parallel (e.g.
11008 using pytest-xdist) they could race with each other and one would
11009 raise an exception due to the non-atomic nature of file renaming on
11010 Windows and the fact that you can't rename over an existing file.
11011 This is now fixed.
11012 3.4.0 - 2016-05-27
11014 This release is entirely provided by Lucas Wiman:
11015 Strategies constructed by the Django extra will now respect much more
11017 of Django's validations out of the box. Wherever possible,
11018 full_clean() should succeed.
11019 In particular:
11021 • The max_length, blank and choices kwargs are now respected.
11023 • Add support for DecimalField.
11025 • If a field includes validators, the list of validators are used to
11027 filter the field strategy.
11028 3.3.0 - 2016-05-27
11030 This release went wrong and is functionally equivalent to 3.2.0. Ignore
11031 it.
11032 3.2.0 - 2016-05-19
11034 This is a small single-feature release:
11035 • All tests using @given now fix the global random seed. This removes
11037 the health check for that. If a non-zero seed is required for the fi‐
11038 nal falsifying example, it will be reported. Otherwise Hypothesis
11039 will assume randomization was not a significant factor for the test
11040 and be silent on the subject. If you use random_module() this will
11041 continue to work and will always display the seed.
11042 3.1.3 - 2016-05-01
11044 Single bug fix release
11045 • Another charmap problem. In 3.1.2 text() and characters() would break
11047 on systems which had /tmp mounted on a different partition than the
11048 Hypothesis storage directory (usually in home). This fixes that.
11049 3.1.2 - 2016-04-30
11051 Single bug fix release:
11052 • Anything which used a text() or characters() strategy was broken on
11054 Windows and I hadn't updated appveyor to use the new repository loca‐
11055 tion so I didn't notice. This is now fixed and windows support should
11056 work correctly.
11057 3.1.1 - 2016-04-29
11059 Minor bug fix release.
11060 • Fix concurrency issue when running tests that use text() from multi‐
11062 ple processes at once (issue #302, thanks to Alex Chan).
11063 • Improve performance of code using lists() with max_size (thanks to
11065 Cristi Cobzarenco).
11066 • Fix install on Python 2 with ancient versions of pip so that it in‐
11068 stalls the enum34 backport (thanks to Donald Stufft for telling me
11069 how to do this).
11070 • Remove duplicated __all__ exports from hypothesis.strategies (thanks
11072 to Piët Delport).
11073 • Update headers to point to new repository location.
11075 • Allow use of strategies that can't be used in find() (e.g. choices())
11077 in stateful testing.
11078 3.1.0 - 2016-03-06
11080 • Add a nothing() strategy that never successfully generates values.
11081 • sampled_from() and one_of() can both now be called with an empty ar‐
11083 gument list, in which case they also never generate any values.
11084 • one_of() may now be called with a single argument that is a collec‐
11086 tion of strategies as well as as varargs.
11087 • Add a runner() strategy which returns the instance of the current
11089 test object if there is one.
11090 • 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
11092 can be used as such.
11093 • Tests using RuleBasedStateMachine should now shrink significantly
11095 better.
11096 • Hypothesis now uses a pretty-printing library internally, compatible
11098 with IPython's pretty printing protocol (actually using the same
11099 code). This may improve the quality of output in some cases.
11100 • Add a 'phases' setting that allows more fine grained control over
11102 which parts of the process Hypothesis runs
11103 • Add a suppress_health_check setting which allows you to turn off spe‐
11105 cific health checks in a fine grained manner.
11106 • Fix a bug where lists of non fixed size would always draw one more
11108 element than they included. This mostly didn't matter, but if would
11109 cause problems with empty strategies or ones with side effects.
11110 • Add a mechanism to the Django model generator to allow you to explic‐
11112 itly request the default value (thanks to Jeremy Thurgood for this
11113 one).
11114 3.0.5 - 2016-02-25
11116 • Fix a bug where Hypothesis would now error on pytest development ver‐
11117 sions.
11118 3.0.4 - 2016-02-24
11120 • Fix a bug where Hypothesis would error when running on Python 2.7.3
11121 or earlier because it was trying to pass a python:bytearray object to
11122 python:struct.unpack() (which is only supported since 2.7.4).
11123 3.0.3 - 2016-02-23
11125 • Fix version parsing of pytest to work with pytest release candidates
11126 • More general handling of the health check problem where things could
11128 fail because of a cache miss - now one "free" example is generated
11129 before the start of the health check run.
11130 3.0.2 - 2016-02-18
11132 • Under certain circumstances, strategies involving text() buried in‐
11133 side some other strategy (e.g. text().filter(...) or recur‐
11134 sive(text(), ...)) would cause a test to fail its health checks the
11135 first time it ran. This was caused by having to compute some related
11136 data and cache it to disk. On travis or anywhere else where the .hy‐
11137 pothesis directory was recreated this would have caused the tests to
11138 fail their health check on every run. This is now fixed for all the
11139 known cases, although there could be others lurking.
11140 3.0.1 - 2016-02-18
11142 • Fix a case where it was possible to trigger an "Unreachable" asser‐
11143 tion when running certain flaky stateful tests.
11144 • Improve shrinking of large stateful tests by eliminating a case where
11146 it was hard to delete early steps.
11147 • Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
11149 cantly by provide a custom implementation for fixed size blocks that
11150 can bypass a lot of machinery.
11151 • Set default home directory based on the current working directory at
11153 the point Hypothesis is imported, not whenever the function first
11154 happens to be called.
11155 3.0.0 - 2016-02-17
11157 Codename: This really should have been 2.1.
11158 Externally this looks like a very small release. It has one small
11160 breaking change that probably doesn't affect anyone at all (some behav‐
11161 iour that never really worked correctly is now outright forbidden) but
11162 necessitated a major version bump and one visible new feature.
11163 Internally this is a complete rewrite. Almost nothing other than the
11165 public API is the same.
11166 New features:
11168 • Addition of data() strategy which allows you to draw arbitrary data
11170 interactively within the test.
11171 • New "exploded" database format which allows you to more easily check
11173 the example database into a source repository while supporting merg‐
11174 ing.
11175 • Better management of how examples are saved in the database.
11177 • Health checks will now raise as errors when they fail. It was too
11179 easy to have the warnings be swallowed entirely.
11180 New limitations:
11182 • choices() and streaming() strategies may no longer be used with
11184 find(). Neither may data() (this is the change that necessitated a
11185 major version bump).
11186 Feature removal:
11188 • The ForkingTestCase executor has gone away. It may return in some
11190 more working form at a later date.
11191 Performance improvements:
11193 • A new model which allows flatmap, composite strategies and stateful
11195 testing to perform much better. They should also be more reliable.
11196 • Filtering may in some circumstances have improved significantly. This
11198 will help especially in cases where you have lots of values with in‐
11199 dividual filters on them, such as lists(x.filter(...)).
11200 • Modest performance improvements to the general test runner by avoid‐
11202 ing expensive operations
11203 In general your tests should have got faster. If they've instead got
11205 significantly slower, I'm interested in hearing about it.
11206 Data distribution:
11208 The data distribution should have changed significantly. This may un‐
11210 cover bugs the previous version missed. It may also miss bugs the pre‐
11211 vious version could have uncovered. Hypothesis is now producing less
11212 strongly correlated data than it used to, but the correlations are ex‐
11213 tended over more of the structure.
11214 Shrinking:
11216 Shrinking quality should have improved. In particular Hypothesis can
11218 now perform simultaneous shrinking of separate examples within a single
11219 test (previously it was only able to do this for elements of a single
11220 collection). In some cases performance will have improved, in some
11221 cases it will have got worse but generally shouldn't have by much.
11222 Older versions
11224 2.0.0 - 2016-01-10
11225 Codename: A new beginning
11226 This release cleans up all of the legacy that accrued in the course of
11228 Hypothesis 1.0. These are mostly things that were emitting deprecation
11229 warnings in 1.19.0, but there were a few additional changes.
11230 In particular:
11232 • non-strategy values will no longer be converted to strategies when
11234 used in given or find.
11235 • FailedHealthCheck is now an error and not a warning.
11237 • Handling of non-ascii reprs in user types have been simplified by us‐
11239 ing raw strings in more places in Python 2.
11240 • given no longer allows mixing positional and keyword arguments.
11242 • given no longer works with functions with defaults.
11244 • given no longer turns provided arguments into defaults - they will
11246 not appear in the argspec at all.
11247 • the basic() strategy no longer exists.
11249 • the n_ary_tree strategy no longer exists.
11251 • the average_list_length setting no longer exists. Note: If you're us‐
11253 ing using recursive() this will cause you a significant slow down.
11254 You should pass explicit average_size parameters to collections in
11255 recursive calls.
11256 • @rule can no longer be applied to the same method twice.
11258 • Python 2.6 and 3.3 are no longer officially supported, although in
11260 practice they still work fine.
11261 This also includes two non-deprecation changes:
11263 • given's keyword arguments no longer have to be the rightmost argu‐
11265 ments and can appear anywhere in the method signature.
11266 • The max_shrinks setting would sometimes not have been respected.
11268 1.19.0 - 2016-01-09
11270 Codename: IT COMES
11271 This release heralds the beginning of a new and terrible age of Hypoth‐
11273 esis 2.0.
11274 It's primary purpose is some final deprecations prior to said release.
11276 The goal is that if your code emits no warnings under this release then
11277 it will probably run unchanged under Hypothesis 2.0 (there are some
11278 caveats to this: 2.0 will drop support for some Python versions, and if
11279 you're using internal APIs then as usual that may break without warn‐
11280 ing).
11281 It does have two new features:
11283 • New @seed() decorator which allows you to manually seed a test. This
11285 may be harmlessly combined with and overrides the derandomize set‐
11286 ting.
11287 • settings objects may now be used as a decorator to fix those settings
11289 to a particular @given test.
11290 API changes (old usage still works but is deprecated):
11292 • Settings has been renamed to settings (lower casing) in order to make
11294 the decorator usage more natural.
11295 • Functions for the storage directory that were in hypothesis.settings
11297 are now in a new hypothesis.configuration module.
11298 Additional deprecations:
11300 • the average_list_length setting has been deprecated in favour of be‐
11302 ing explicit.
11303 • the basic() strategy has been deprecated as it is impossible to sup‐
11305 port it under a Conjecture based model, which will hopefully be im‐
11306 plemented at some point in the 2.x series.
11307 • the n_ary_tree strategy (which was never actually part of the public
11309 API) has been deprecated.
11310 • Passing settings or random as keyword arguments to given is depre‐
11312 cated (use the new functionality instead)
11313 Bug fixes:
11315 • No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
11317 tion in streaming() values.
11318 • When running in health check mode with non strict, don't print quite
11320 so many errors for an exception in reify.
11321 • When an assumption made in a test or a filter is flaky, tests will
11323 now raise Flaky instead of UnsatisfiedAssumption.
11324 1.18.1 - 2015-12-22
11326 Two behind the scenes changes:
11327 • Hypothesis will no longer write generated code to the file system.
11329 This will improve performance on some systems (e.g. if you're using
11330 PythonAnywhere which is running your code from NFS) and prevent some
11331 annoying interactions with auto-restarting systems.
11332 • Hypothesis will cache the creation of some strategies. This can sig‐
11334 nificantly improve performance for code that uses flatmap or compos‐
11335 ite and thus has to instantiate strategies a lot.
11336 1.18.0 - 2015-12-21
11338 Features:
11339 • Tests and find are now explicitly seeded off the global random mod‐
11341 ule. This means that if you nest one inside the other you will now
11342 get a health check error. It also means that you can control global
11343 randomization by seeding random.
11344 • There is a new random_module() strategy which seeds the global random
11346 module for you and handles things so that you don't get a health
11347 check warning if you use it inside your tests.
11348 • floats() now accepts two new arguments: allow_nan and allow_infinity.
11350 These default to the old behaviour, but when set to False will do
11351 what the names suggest.
11352 Bug fixes:
11354 • Fix a bug where tests that used text() on Python 3.4+ would not actu‐
11356 ally be deterministic even when explicitly seeded or using the deran‐
11357 domize mode, because generation depended on dictionary iteration or‐
11358 der which was affected by hash randomization.
11359 • Fix a bug where with complicated strategies the timing of the initial
11361 health check could affect the seeding of the subsequent test, which
11362 would also render supposedly deterministic tests non-deterministic in
11363 some scenarios.
11364 • In some circumstances flatmap() could get confused by two struc‐
11366 turally similar things it could generate and would produce a flaky
11367 test where the first time it produced an error but the second time it
11368 produced the other value, which was not an error. The same bug was
11369 presumably also possible in composite().
11370 • flatmap() and composite() initial generation should now be moderately
11372 faster. This will be particularly noticeable when you have many val‐
11373 ues drawn from the same strategy in a single run, e.g. constructs
11374 like lists(s.flatmap(f)). Shrinking performance may have suffered,
11375 but this didn't actually produce an interestingly worse result in any
11376 of the standard scenarios tested.
11377 1.17.1 - 2015-12-16
11379 A small bug fix release, which fixes the fact that the 'note' function
11380 could not be used on tests which used the @example decorator to provide
11381 explicit examples.
11382 1.17.0 - 2015-12-15
11384 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
11385 because it contains the following additional change that was not in‐
11386 tended to be in a patch release (it's perfectly stable, but is a
11387 larger change that should have required a minor version bump):
11388 • Hypothesis will now perform a series of "health checks" as part of
11390 running your tests. These detect and warn about some common error
11391 conditions that people often run into which wouldn't necessarily have
11392 caused the test to fail but would cause e.g. degraded performance or
11393 confusing results.
11394 1.16.1 - 2015-12-14
11396 Note: This release has been removed.
11397 A small bugfix release that allows bdists for Hypothesis to be built
11399 under 2.7 - the compat3.py file which had Python 3 syntax wasn't in‐
11400 tended to be loaded under Python 2, but when building a bdist it was.
11401 In particular this would break running setup.py test.
11402 1.16.0 - 2015-12-08
11404 There are no public API changes in this release but it includes a be‐
11405 haviour change that I wasn't comfortable putting in a patch release.
11406 • Functions from hypothesis.strategies will no longer raise InvalidAr‐
11408 gument on bad arguments. Instead the same errors will be raised when
11409 a test using such a strategy is run. This may improve startup time in
11410 some cases, but the main reason for it is so that errors in strate‐
11411 gies won't cause errors in loading, and it can interact correctly
11412 with things like pytest.mark.skipif.
11413 • Errors caused by accidentally invoking the legacy API are now much
11415 less confusing, although still throw NotImplementedError.
11416 • hypothesis.extra.django is 1.9 compatible.
11418 • When tests are run with max_shrinks=0 this will now still rerun the
11420 test on failure and will no longer print "Trying example:" before
11421 each run. Additionally note() will now work correctly when used with
11422 max_shrinks=0.
11423 1.15.0 - 2015-11-24
11425 A release with two new features.
11426 • A 'characters' strategy for more flexible generation of text with
11428 particular character ranges and types, kindly contributed by
11429 Alexander Shorin.
11430 • Add support for preconditions to the rule based stateful testing.
11432 Kindly contributed by Christopher Armstrong
11433 1.14.0 - 2015-11-01
11435 New features:
11436 • Add 'note' function which lets you include additional information in
11438 the final test run's output.
11439 • Add 'choices' strategy which gives you a choice function that emu‐
11441 lates random.choice.
11442 • Add 'uuid' strategy that generates UUIDs'
11444 • Add 'shared' strategy that lets you create a strategy that just gen‐
11446 erates a single shared value for each test run
11447 Bugs:
11449 • Using strategies of the form streaming(x.flatmap(f)) with find or in
11451 stateful testing would have caused InvalidArgument errors when the
11452 resulting values were used (because code that expected to only be
11453 called within a test context would be invoked).
11454 1.13.0 - 2015-10-29
11456 This is quite a small release, but deprecates some public API functions
11457 and removes some internal API functionality so gets a minor version
11458 bump.
11459 • All calls to the 'strategy' function are now deprecated, even ones
11461 which pass just a SearchStrategy instance (which is still a no-op).
11462 • Never documented hypothesis.extra entry_points mechanism has now been
11464 removed ( it was previously how hypothesis.extra packages were loaded
11465 and has been deprecated and unused for some time)
11466 • Some corner cases that could previously have produced an OverflowEr‐
11468 ror when simplifying failing cases using hypothesis.extra.datetimes
11469 (or dates or times) have now been fixed.
11470 • Hypothesis load time for first import has been significantly reduced
11472 - it used to be around 250ms (on my SSD laptop) and now is around
11473 100-150ms. This almost never matters but was slightly annoying when
11474 using it in the console.
11475 • hypothesis.strategies.randoms was previously missing from __all__.
11477 1.12.0 - 2015-10-18
11479 • Significantly improved performance of creating strategies using the
11480 functions from the hypothesis.strategies module by deferring the cal‐
11481 culation of their repr until it was needed. This is unlikely to have
11482 been an performance issue for you unless you were using flatmap, com‐
11483 posite or stateful testing, but for some cases it could be quite a
11484 significant impact.
11485 • A number of cases where the repr of strategies build from lambdas is
11487 improved
11488 • Add dates() and times() strategies to hypothesis.extra.datetimes
11490 • Add new 'profiles' mechanism to the settings system
11492 • Deprecates mutability of Settings, both the Settings.default top
11494 level property and individual settings.
11495 • A Settings object may now be directly initialized from a parent Set‐
11497 tings.
11498 • @given should now give a better error message if you attempt to use
11500 it with a function that uses destructuring arguments (it still won't
11501 work, but it will error more clearly),
11502 • A number of spelling corrections in error messages
11504 • pytest should no longer display the intermediate modules Hypothesis
11506 generates when running in verbose mode
11507 • Hypothesis should now correctly handle printing objects with
11509 non-ascii reprs on python 3 when running in a locale that cannot han‐
11510 dle ascii printing to stdout.
11511 • Add a unique=True argument to lists(). This is equivalent to
11513 unique_by=lambda x: x, but offers a more convenient syntax.
11514 1.11.4 - 2015-09-27
11516 • Hide modifications Hypothesis needs to make to sys.path by undoing
11517 them after we've imported the relevant modules. This is a workaround
11518 for issues cryptography experienced on windows.
11519 • Slightly improved performance of drawing from sampled_from on large
11521 lists of alternatives.
11522 • Significantly improved performance of drawing from one_of or strate‐
11524 gies using | (note this includes a lot of strategies internally -
11525 floats() and integers() both fall into this category). There turned
11526 out to be a massive performance regression introduced in 1.10.0 af‐
11527 fecting these which probably would have made tests using Hypothesis
11528 significantly slower than they should have been.
11529 1.11.3 - 2015-09-23
11531 • Better argument validation for datetimes() strategy - previously set‐
11532 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
11533 would not have raised an InvalidArgument error and instead would have
11534 behaved confusingly.
11535 • Compatibility with being run on pytest < 2.7 (achieved by disabling
11537 the plugin).
11538 1.11.2 - 2015-09-23
11540 Bug fixes:
11541 • Settings(database=my_db) would not be correctly inherited when used
11543 as a default setting, so that newly created settings would use the
11544 database_file setting and create an SQLite example database.
11545 • Settings.default.database = my_db would previously have raised an er‐
11547 ror and now works.
11548 • Timeout could sometimes be significantly exceeded if during simplifi‐
11550 cation there were a lot of examples tried that didn't trigger the
11551 bug.
11552 • When loading a heavily simplified example using a basic() strategy
11554 from the database this could cause Python to trigger a recursion er‐
11555 ror.
11556 • Remove use of deprecated API in pytest plugin so as to not emit warn‐
11558 ing
11559 Misc:
11561 • hypothesis-pytest is now part of hypothesis core. This should have no
11563 externally visible consequences, but you should update your dependen‐
11564 cies to remove hypothesis-pytest and depend on only Hypothesis.
11565 • Better repr for hypothesis.extra.datetimes() strategies.
11567 • Add .close() method to abstract base class for Backend (it was al‐
11569 ready present in the main implementation).
11570 1.11.1 - 2015-09-16
11572 Bug fixes:
11573 • When running Hypothesis tests in parallel (e.g. using pytest-xdist)
11575 there was a race condition caused by code generation.
11576 • Example databases are now cached per thread so as to not use sqlite
11578 connections from multiple threads. This should make Hypothesis now
11579 entirely thread safe.
11580 • floats() with only min_value or max_value set would have had a very
11582 bad distribution.
11583 • Running on 3.5, Hypothesis would have emitted deprecation warnings
11585 because of use of inspect.getargspec
11586 1.11.0 - 2015-08-31
11588 • text() with a non-string alphabet would have used the repr() of the
11589 the alphabet instead of its contexts. This is obviously silly. It now
11590 works with any sequence of things convertible to unicode strings.
11591 • @given will now work on methods whose definitions contains no ex‐
11593 plicit positional arguments, only varargs (issue #118). This may
11594 have some knock on effects because it means that @given no longer
11595 changes the argspec of functions other than by adding defaults.
11596 • Introduction of new @composite feature for more natural definition of
11598 strategies you'd previously have used flatmap for.
11599 1.10.6 - 2015-08-26
11601 Fix support for fixtures on Django 1.7.
11602 1.10.4 - 2015-08-21
11604 Tiny bug fix release:
11605 • If the database_file setting is set to None, this would have resulted
11607 in an error when running tests. Now it does the same as setting data‐
11608 base to None.
11609 1.10.3 - 2015-08-19
11611 Another small bug fix release.
11612 • lists(elements, unique_by=some_function, min_size=n) would have
11614 raised a ValidationError if n > Settings.default.average_list_length
11615 because it would have wanted to use an average list length shorter
11616 than the minimum size of the list, which is impossible. Now it in‐
11617 stead defaults to twice the minimum size in these circumstances.
11618 • basic() strategy would have only ever produced at most ten distinct
11620 values per run of the test (which is bad if you e.g. have it inside a
11621 list). This was obviously silly. It will now produce a much better
11622 distribution of data, both duplicated and non duplicated.
11623 1.10.2 - 2015-08-19
11625 This is a small bug fix release:
11626 • star imports from hypothesis should now work correctly.
11628 • example quality for examples using flatmap will be better, as the way
11630 it had previously been implemented was causing problems where Hypoth‐
11631 esis was erroneously labelling some examples as being duplicates.
11632 1.10.0 - 2015-08-04
11634 This is just a bugfix and performance release, but it changes some
11635 semi-public APIs, hence the minor version bump.
11636 • Significant performance improvements for strategies which are
11638 one_of() many branches. In particular this included recursive()
11639 strategies. This should take the case where you use one recursive()
11640 strategy as the base strategy of another from unusably slow (tens of
11641 seconds per generated example) to reasonably fast.
11642 • Better handling of just() and sampled_from() for values which have an
11644 incorrect __repr__ implementation that returns non-ASCII unicode on
11645 Python 2.
11646 • Better performance for flatmap from changing the internal morpher API
11648 to be significantly less general purpose.
11649 • Introduce a new semi-public BuildContext/cleanup API. This allows
11651 strategies to register cleanup activities that should run once the
11652 example is complete. Note that this will interact somewhat weirdly
11653 with find.
11654 • Better simplification behaviour for streaming strategies.
11656 • Don't error on lambdas which use destructuring arguments in Python 2.
11658 • Add some better reprs for a few strategies that were missing good
11660 ones.
11661 • The Random instances provided by randoms() are now copyable.
11663 • Slightly more debugging information about simplify when using a debug
11665 verbosity level.
11666 • Support using given for functions with varargs, but not passing argu‐
11668 ments to it as positional.
11669 1.9.0 - 2015-07-27
11671 Codename: The great bundling.
11672 This release contains two fairly major changes.
11674 The first is the deprecation of the hypothesis-extra mechanism. From
11676 now on all the packages that were previously bundled under it other
11677 than hypothesis-pytest (which is a different beast and will remain sep‐
11678 arate). The functionality remains unchanged and you can still import
11679 them from exactly the same location, they just are no longer separate
11680 packages.
11681 The second is that this introduces a new way of building strategies
11683 which lets you build up strategies recursively from other strategies.
11684 It also contains the minor change that calling .example() on a strategy
11686 object will give you examples that are more representative of the ac‐
11687 tual data you'll get. There used to be some logic in there to make the
11688 examples artificially simple but this proved to be a bad idea.
11689 1.8.5 - 2015-07-24
11691 This contains no functionality changes but fixes a mistake made with
11692 building the previous package that would have broken installation on
11693 Windows.
11694 1.8.4 - 2015-07-20
11696 Bugs fixed:
11697 • When a call to floats() had endpoints which were not floats but
11699 merely convertible to one (e.g. integers), these would be included in
11700 the generated data which would cause it to generate non-floats.
11701 • Splitting lambdas used in the definition of flatmap, map or filter
11703 over multiple lines would break the repr, which would in turn break
11704 their usage.
11705 1.8.3 - 2015-07-20
11707 "Falsifying example" would not have been printed when the failure came
11708 from an explicit example.
11709 1.8.2 - 2015-07-18
11711 Another small bugfix release:
11712 • When using ForkingTestCase you would usually not get the falsifying
11714 example printed if the process exited abnormally (e.g. due to
11715 os._exit).
11716 • Improvements to the distribution of characters when using text() with
11718 a default alphabet. In particular produces a better distribution of
11719 ascii and whitespace in the alphabet.
11720 1.8.1 - 2015-07-17
11722 This is a small release that contains a workaround for people who have
11723 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
11724 for Hypothesis per se because that's not a thing that is actually sup‐
11725 posed to work, but Hypothesis leans more heavily on repr than is typi‐
11726 cal so it's worth having a workaround for.
11727 1.8.0 - 2015-07-16
11729 New features:
11730 • Much more sensible reprs for strategies, especially ones that come
11732 from hypothesis.strategies. These should now have as reprs python
11733 code that would produce the same strategy.
11734 • lists() accepts a unique_by argument which forces the generated lists
11736 to be only contain elements unique according to some function key
11737 (which must return a hashable value).
11738 • Better error messages from flaky tests to help you debug things.
11740 Mostly invisible implementation details that may result in finding new
11742 bugs in your code:
11743 • Sets and dictionary generation should now produce a better range of
11745 results.
11746 • floats with bounds now focus more on 'critical values', trying to
11748 produce values at edge cases.
11749 • flatmap should now have better simplification for complicated cases,
11751 as well as generally being (I hope) more reliable.
11752 Bug fixes:
11754 • You could not previously use assume() if you were using the forking
11756 executor.
11757 1.7.2 - 2015-07-10
11759 This is purely a bug fix release:
11760 • When using floats() with stale data in the database you could some‐
11762 times get values in your tests that did not respect min_value or
11763 max_value.
11764 • When getting a Flaky error from an unreliable test it would have in‐
11766 correctly displayed the example that caused it.
11767 • 2.6 dependency on backports was incorrectly specified. This would
11769 only have caused you problems if you were building a universal wheel
11770 from Hypothesis, which is not how Hypothesis ships, so unless you're
11771 explicitly building wheels for your dependencies and support Python
11772 2.6 plus a later version of Python this probably would never have af‐
11773 fected you.
11774 • If you use flatmap in a way that the strategy on the right hand side
11776 depends sensitively on the left hand side you may have occasionally
11777 seen Flaky errors caused by producing unreliable examples when mini‐
11778 mizing a bug. This use case may still be somewhat fraught to be hon‐
11779 est. This code is due a major rearchitecture for 1.8, but in the
11780 meantime this release fixes the only source of this error that I'm
11781 aware of.
11782 1.7.1 - 2015-06-29
11784 Codename: There is no 1.7.0.
11785 A slight technical hitch with a premature upload means there's was a
11787 yanked 1.7.0 release. Oops.
11788 The major feature of this release is Python 2.6 support. Thanks to Jeff
11790 Meadows for doing most of the work there.
11791 Other minor features
11793 • strategies now has a permutations() function which returns a strategy
11795 yielding permutations of values from a given collection.
11796 • if you have a flaky test it will print the exception that it last saw
11798 before failing with Flaky, even if you do not have verbose reporting
11799 on.
11800 • Slightly experimental git merge script available as "python -m hy‐
11802 pothesis.tools.mergedbs". Instructions on how to use it in the doc‐
11803 string of that file.
11804 Bug fixes:
11806 • Better performance from use of filter. In particular tests which in‐
11808 volve large numbers of heavily filtered strategies should perform a
11809 lot better.
11810 • floats() with a negative min_value would not have worked correctly
11812 (worryingly, it would have just silently failed to run any examples).
11813 This is now fixed.
11814 • tests using sampled_from would error if the number of sampled ele‐
11816 ments was smaller than min_satisfying_examples.
11817 1.6.2 - 2015-06-08
11819 This is just a few small bug fixes:
11820 • Size bounds were not validated for values for a binary() strategy
11822 when reading examples from the database.
11823 • sampled_from is now in __all__ in hypothesis.strategies
11825 • floats no longer consider negative integers to be simpler than posi‐
11827 tive non-integers
11828 • Small floating point intervals now correctly count members, so if you
11830 have a floating point interval so narrow there are only a handful of
11831 values in it, this will no longer cause an error when Hypothesis runs
11832 out of values.
11833 1.6.1 - 2015-05-21
11835 This is a small patch release that fixes a bug where 1.6.0 broke the
11836 use of flatmap with the deprecated API and assumed the passed in func‐
11837 tion returned a SearchStrategy instance rather than converting it to a
11838 strategy.
11839 1.6.0 - 2015-05-21
11841 This is a smallish release designed to fix a number of bugs and smooth
11842 out some weird behaviours.
11843 • Fix a critical bug in flatmap where it would reuse old strategies. If
11845 all your flatmap code was pure you're fine. If it's not, I'm sur‐
11846 prised it's working at all. In particular if you want to use flatmap
11847 with django models, you desperately need to upgrade to this version.
11848 • flatmap simplification performance should now be better in some cases
11850 where it previously had to redo work.
11851 • Fix for a bug where invalid unicode data with surrogates could be
11853 generated during simplification (it was already filtered out during
11854 actual generation).
11855 • The Hypothesis database is now keyed off the name of the test instead
11857 of the type of data. This makes much more sense now with the new
11858 strategies API and is generally more robust. This means you will lose
11859 old examples on upgrade.
11860 • The database will now not delete values which fail to deserialize
11862 correctly, just skip them. This is to handle cases where multiple in‐
11863 compatible strategies share the same key.
11864 • find now also saves and loads values from the database, keyed off a
11866 hash of the function you're finding from.
11867 • Stateful tests now serialize and load values from the database. They
11869 should have before, really. This was a bug.
11870 • Passing a different verbosity level into a test would not have worked
11872 entirely correctly, leaving off some messages. This is now fixed.
11873 • Fix a bug where derandomized tests with unicode characters in the
11875 function body would error on Python 2.7.
11876 1.5.0 - 2015-05-14
11878 Codename: Strategic withdrawal.
11879 The purpose of this release is a radical simplification of the API for
11881 building strategies. Instead of the old approach of @strategy.extend
11882 and things that get converted to strategies, you just build strategies
11883 directly.
11884 The old method of defining strategies will still work until Hypothesis
11886 2.0, because it's a major breaking change, but will now emit depreca‐
11887 tion warnings.
11888 The new API is also a lot more powerful as the functions for defining
11890 strategies give you a lot of dials to turn. See the updated data sec‐
11891 tion for details.
11892 Other changes:
11894 • Mixing keyword and positional arguments in a call to @given is
11896 deprecated as well.
11897 • There is a new setting called 'strict'. When set to True, Hypothe‐
11899 sis will raise warnings instead of merely printing them. Turning
11900 it on by default is inadvisable because it means that Hypothesis
11901 minor releases can break your code, but it may be useful for mak‐
11902 ing sure you catch all uses of deprecated APIs.
11903 • max_examples in settings is now interpreted as meaning the maximum
11905 number of unique (ish) examples satisfying assumptions. A new set‐
11906 ting max_iterations which defaults to a larger value has the old
11907 interpretation.
11908 • Example generation should be significantly faster due to a new
11910 faster parameter selection algorithm. This will mostly show up for
11911 simple data types - for complex ones the parameter selection is
11912 almost certainly dominated.
11913 • Simplification has some new heuristics that will tend to cut down
11915 on cases where it could previously take a very long time.
11916 • timeout would previously not have been respected in cases where
11918 there were a lot of duplicate examples. You probably wouldn't have
11919 previously noticed this because max_examples counted duplicates,
11920 so this was very hard to hit in a way that mattered.
11921 • A number of internal simplifications to the SearchStrategy API.
11923 • You can now access the current Hypothesis version as hypothe‐
11925 sis.__version__.
11926 • A top level function is provided for running the stateful tests
11928 without the TestCase infrastructure.
11929 1.4.0 - 2015-05-04
11931 Codename: What a state.
11932 The big feature of this release is the new and slightly experimental
11934 stateful testing API. You can read more about that in the appropriate
11935 section.
11936 Two minor features the were driven out in the course of developing
11938 this:
11939 • You can now set settings.max_shrinks to limit the number of times Hy‐
11941 pothesis will try to shrink arguments to your test. If this is set to
11942 <= 0 then Hypothesis will not rerun your test and will just raise the
11943 failure directly. Note that due to technical limitations if
11944 max_shrinks is <= 0 then Hypothesis will print every example it calls
11945 your test with rather than just the failing one. Note also that I
11946 don't consider settings max_shrinks to zero a sensible way to run
11947 your tests and it should really be considered a debug feature.
11948 • There is a new debug level of verbosity which is even more verbose
11950 than verbose. You probably don't want this.
11951 Breakage of semi-public SearchStrategy API:
11953 • It is now a required invariant of SearchStrategy that if u simplifies
11955 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
11956 plifying should not increase the complexity even though it is not re‐
11957 quired to decrease it. Enforcing this invariant lead to finding some
11958 bugs where simplifying of integers, floats and sets was suboptimal.
11959 • Integers in basic data are now required to fit into 64 bits. As a re‐
11961 sult python integer types are now serialized as strings, and some
11962 types have stopped using quite so needlessly large random seeds.
11963 Hypothesis Stateful testing was then turned upon Hypothesis itself,
11965 which lead to an amazing number of minor bugs being found in Hypothesis
11966 itself.
11967 Bugs fixed (most but not all from the result of stateful testing) in‐
11969 clude:
11970 • Serialization of streaming examples was flaky in a way that you would
11972 probably never notice: If you generate a template, simplify it, seri‐
11973 alize it, deserialize it, serialize it again and then deserialize it
11974 you would get the original stream instead of the simplified one.
11975 • If you reduced max_examples below the number of examples already
11977 saved in the database, you would have got a ValueError. Additionally,
11978 if you had more than max_examples in the database all of them would
11979 have been considered.
11980 • @given will no longer count duplicate examples (which it never called
11982 your function with) towards max_examples. This may result in your
11983 tests running slower, but that's probably just because they're trying
11984 more examples.
11985 • General improvements to example search which should result in better
11987 performance and higher quality examples. In particular parameters
11988 which have a history of producing useless results will be more ag‐
11989 gressively culled. This is useful both because it decreases the
11990 chance of useless examples and also because it's much faster to not
11991 check parameters which we were unlikely to ever pick!
11992 • integers_from and lists of types with only one value (e.g. [None])
11994 would previously have had a very high duplication rate so you were
11995 probably only getting a handful of examples. They now have a much
11996 lower duplication rate, as well as the improvements to search making
11997 this less of a problem in the first place.
11998 • You would sometimes see simplification taking significantly longer
12000 than your defined timeout. This would happen because timeout was only
12001 being checked after each successful simplification, so if Hypothesis
12002 was spending a lot of time unsuccessfully simplifying things it
12003 wouldn't stop in time. The timeout is now applied for unsuccessful
12004 simplifications too.
12005 • In Python 2.7, integers_from strategies would have failed during sim‐
12007 plification with an OverflowError if their starting point was at or
12008 near to the maximum size of a 64-bit integer.
12009 • flatmap and map would have failed if called with a function without a
12011 __name__ attribute.
12012 • If max_examples was less than min_satisfying_examples this would al‐
12014 ways error. Now min_satisfying_examples is capped to max_examples.
12015 Note that if you have assumptions to satisfy here this will still
12016 cause an error.
12017 Some minor quality improvements:
12019 • Lists of streams, flatmapped strategies and basic strategies should
12021 now now have slightly better simplification.
12022 1.3.0 - 2015-05-22
12024 New features:
12025 • New verbosity level API for printing intermediate results and excep‐
12027 tions.
12028 • New specifier for strings generated from a specified alphabet.
12030 • Better error messages for tests that are failing because of a lack of
12032 enough examples.
12033 Bug fixes:
12035 • Fix error where use of ForkingTestCase would sometimes result in too
12037 many open files.
12038 • Fix error where saving a failing example that used flatmap could er‐
12040 ror.
12041 • Implement simplification for sampled_from, which apparently never
12043 supported it previously. Oops.
12044 General improvements:
12046 • Better range of examples when using one_of or sampled_from.
12048 • Fix some pathological performance issues when simplifying lists of
12050 complex values.
12051 • Fix some pathological performance issues when simplifying examples
12053 that require unicode strings with high codepoints.
12054 • Random will now simplify to more readable examples.
12056 1.2.1 - 2015-04-16
12058 A small patch release for a bug in the new executors feature. Tests
12059 which require doing something to their result in order to fail would
12060 have instead reported as flaky.
12061 1.2.0 - 2015-04-15
12063 Codename: Finders keepers.
12064 A bunch of new features and improvements.
12066 • Provide a mechanism for customizing how your tests are executed.
12068 • Provide a test runner that forks before running each example. This
12070 allows better support for testing native code which might trigger a
12071 segfault or a C level assertion failure.
12072 • Support for using Hypothesis to find examples directly rather than as
12074 just as a test runner.
12075 • New streaming type which lets you generate infinite lazily loaded
12077 streams of data - perfect for if you need a number of examples but
12078 don't know how many.
12079 • Better support for large integer ranges. You can now use inte‐
12081 gers_in_range with ranges of basically any size. Previously large
12082 ranges would have eaten up all your memory and taken forever.
12083 • Integers produce a wider range of data than before - previously they
12085 would only rarely produce integers which didn't fit into a machine
12086 word. Now it's much more common. This percolates to other numeric
12087 types which build on integers.
12088 • Better validation of arguments to @given. Some situations that would
12090 previously have caused silently wrong behaviour will now raise an er‐
12091 ror.
12092 • Include +/- sys.float_info.max in the set of floating point edge
12094 cases that Hypothesis specifically tries.
12095 • Fix some bugs in floating point ranges which happen when given +/-
12097 sys.float_info.max as one of the endpoints... (really any two floats
12098 that are sufficiently far apart so that x, y are finite but y - x is
12099 infinite). This would have resulted in generating infinite values
12100 instead of ones inside the range.
12101 1.1.1 - 2015-04-07
12103 Codename: Nothing to see here
12104 This is just a patch release put out because it fixed some internal
12106 bugs that would block the Django integration release but did not actu‐
12107 ally affect anything anyone could previously have been using. It also
12108 contained a minor quality fix for floats that I'd happened to have fin‐
12109 ished in time.
12110 • Fix some internal bugs with object lifecycle management that were im‐
12112 possible to hit with the previously released versions but broke hy‐
12113 pothesis-django.
12114 • Bias floating point numbers somewhat less aggressively towards very
12116 small numbers
12117 1.1.0 - 2015-04-06
12119 Codename: No-one mention the M word.
12120 • Unicode strings are more strongly biased towards ascii characters.
12122 Previously they would generate all over the space. This is mostly so
12123 that people who try to shape their unicode strings with assume() have
12124 less of a bad time.
12125 • A number of fixes to data deserialization code that could theoreti‐
12127 cally have caused mysterious bugs when using an old version of a Hy‐
12128 pothesis example database with a newer version. To the best of my
12129 knowledge a change that could have triggered this bug has never actu‐
12130 ally been seen in the wild. Certainly no-one ever reported a bug of
12131 this nature.
12132 • Out of the box support for Decimal and Fraction.
12134 • new dictionary specifier for dictionaries with variable keys.
12136 • Significantly faster and higher quality simplification, especially
12138 for collections of data.
12139 • New filter() and flatmap() methods on Strategy for better ways of
12141 building strategies out of other strategies.
12142 • New BasicStrategy class which allows you to define your own strate‐
12144 gies from scratch without needing an existing matching strategy or
12145 being exposed to the full horror or non-public nature of the Search‐
12146 Strategy interface.
12147 1.0.0 - 2015-03-27
12149 Codename: Blast-off!
12150 There are no code changes in this release. This is precisely the 0.9.2
12152 release with some updated documentation.
12153 0.9.2 - 2015-03-26
12155 Codename: T-1 days.
12156 • floats_in_range would not actually have produced floats_in_range un‐
12158 less that range happened to be (0, 1). Fix this.
12159 0.9.1 - 2015-03-25
12161 Codename: T-2 days.
12162 • Fix a bug where if you defined a strategy using map on a lambda then
12164 the results would not be saved in the database.
12165 • Significant performance improvements when simplifying examples using
12167 lists, strings or bounded integer ranges.
12168 0.9.0 - 2015-03-23
12170 Codename: The final countdown
12171 This release could also be called 1.0-RC1.
12173 It contains a teeny tiny bugfix, but the real point of this release is
12175 to declare feature freeze. There will be zero functionality changes be‐
12176 tween 0.9.0 and 1.0 unless something goes really really wrong. No new
12177 features will be added, no breaking API changes will occur, etc. This
12178 is the final shakedown before I declare Hypothesis stable and ready to
12179 use and throw a party to celebrate.
12180 Bug bounty for any bugs found between now and 1.0: I will buy you a
12182 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
12183 we ever find ourselves in the same city at the same time.
12184 The one tiny bugfix:
12186 • Under pypy, databases would fail to close correctly when garbage col‐
12188 lected, leading to a memory leak and a confusing error message if you
12189 were repeatedly creating databases and not closing them. It is very
12190 unlikely you were doing this and the chances of you ever having no‐
12191 ticed this bug are very low.
12192 0.7.2 - 2015-03-22
12194 Codename: Hygienic macros or bust
12195 • You can now name an argument to @given 'f' and it won't break (issue
12197 #38)
12198 • strategy_test_suite is now named strategy_test_suite as the documen‐
12200 tation claims and not in fact strategy_test_suitee
12201 • Settings objects can now be used as a context manager to temporarily
12203 override the default values inside their context.
12204 0.7.1 - 2015-03-21
12206 Codename: Point releases go faster
12207 • Better string generation by parametrizing by a limited alphabet
12209 • Faster string simplification - previously if simplifying a string
12211 with high range unicode characters it would try every unicode charac‐
12212 ter smaller than that. This was pretty pointless. Now it stops after
12213 it's a short range (it can still reach smaller ones through recursive
12214 calls because of other simplifying operations).
12215 • Faster list simplification by first trying a binary chop down the
12217 middle
12218 • Simultaneous simplification of identical elements in a list. So if a
12220 bug only triggers when you have duplicates but you drew e.g. [-17,
12221 -17], this will now simplify to [0, 0].
12222 0.7.0, - 2015-03-20
12224 Codename: Starting to look suspiciously real
12225 This is probably the last minor release prior to 1.0. It consists of
12227 stability improvements, a few usability things designed to make Hypoth‐
12228 esis easier to try out, and filing off some final rough edges from the
12229 API.
12230 • Significant speed and memory usage improvements
12232 • Add an example() method to strategy objects to give an example of the
12234 sort of data that the strategy generates.
12235 • Remove .descriptor attribute of strategies
12237 • Rename descriptor_test_suite to strategy_test_suite
12239 • Rename the few remaining uses of descriptor to specifier (descriptor
12241 already has a defined meaning in Python)
12242 0.6.0 - 2015-03-13
12244 Codename: I'm sorry, were you using that API?
12245 This is primarily a "simplify all the weird bits of the API" release.
12247 As a result there are a lot of breaking changes. If you just use @given
12248 with core types then you're probably fine.
12249 In particular:
12251 • Stateful testing has been removed from the API
12253 • The way the database is used has been rendered less useful (sorry).
12255 The feature for reassembling values saved from other tests doesn't
12256 currently work. This will probably be brought back in post 1.0.
12257 • SpecificationMapper is no longer a thing. Instead there is an
12259 ExtMethod called strategy which you extend to specify how to convert
12260 other types to strategies.
12261 • Settings are now extensible so you can add your own for configuring a
12263 strategy
12264 • MappedSearchStrategy no longer needs an unpack method
12266 • Basically all the SearchStrategy internals have changed massively. If
12268 you implemented SearchStrategy directly rather than using Mapped‐
12269 SearchStrategy talk to me about fixing it.
12270 • Change to the way extra packages work. You now specify the package.
12272 This must have a load() method. Additionally any modules in the pack‐
12273 age will be loaded in under hypothesis.extra
12274 Bug fixes:
12276 • Fix for a bug where calling falsify on a lambda with a non-ascii
12278 character in its body would error.
12279 Hypothesis Extra:
12281 hypothesis-fakefactory: An extension for using faker data in hypothe‐
12283 sis. Depends
12284 on fake-factory.
12285 0.5.0 - 2015-02-10
12287 Codename: Read all about it.
12288 Core hypothesis:
12290 • Add support back in for pypy and python 3.2
12292 • @given functions can now be invoked with some arguments explicitly
12294 provided. If all arguments that hypothesis would have provided are
12295 passed in then no falsification is run.
12296 • Related to the above, this means that you can now use pytest fixtures
12298 and mark.parametrize with Hypothesis without either interfering with
12299 the other.
12300 • Breaking change: @given no longer works for functions with varargs
12302 (varkwargs are fine). This might be added back in at a later date.
12303 • Windows is now fully supported. A limited version (just the tests
12305 with none of the extras) of the test suite is run on windows with
12306 each commit so it is now a first class citizen of the Hypothesis
12307 world.
12308 • Fix a bug for fuzzy equality of equal complex numbers with different
12310 reprs (this can happen when one coordinate is zero). This shouldn't
12311 affect users - that feature isn't used anywhere public facing.
12312 • Fix generation of floats on windows and 32-bit builds of python. I
12314 was using some struct.pack logic that only worked on certain word
12315 sizes.
12316 • When a test times out and hasn't produced enough examples this now
12318 raises a Timeout subclass of Unfalsifiable.
12319 • Small search spaces are better supported. Previously something like a
12321 @given(bool, bool) would have failed because it couldn't find enough
12322 examples. Hypothesis is now aware of the fact that these are small
12323 search spaces and will not error in this case.
12324 • Improvements to parameter search in the case of hard to satisfy as‐
12326 sume. Hypothesis will now spend less time exploring parameters that
12327 are unlikely to provide anything useful.
12328 • Increase chance of generating "nasty" floats
12330 • Fix a bug that would have caused unicode warnings if you had a sam‐
12332 pled_from that was mixing unicode and byte strings.
12333 • Added a standard test suite that you can use to validate a custom
12335 strategy you've defined is working correctly.
12336 Hypothesis extra:
12338 First off, introducing Hypothesis extra packages!
12340 These are packages that are separated out from core Hypothesis because
12342 they have one or more dependencies. Every hypothesis-extra package is
12343 pinned to a specific point release of Hypothesis and will have some
12344 version requirements on its dependency. They use entry_points so you
12345 will usually not need to explicitly import them, just have them in‐
12346 stalled on the path.
12347 This release introduces two of them:
12349 hypothesis-datetime:
12351 Does what it says on the tin: Generates datetimes for Hypothesis. Just
12353 install the package and datetime support will start working.
12354 Depends on pytz for timezone support
12356 hypothesis-pytest:
12358 A very rudimentary pytest plugin. All it does right now is hook the
12360 display of falsifying examples into pytest reporting.
12361 Depends on pytest.
12363 0.4.3 - 2015-02-05
12365 Codename: TIL narrow Python builds are a thing
12366 This just fixes the one bug.
12368 • Apparently there is such a thing as a "narrow python build" and OS X
12370 ships with these by default for python 2.7. These are builds where
12371 you only have two bytes worth of unicode. As a result, generating
12372 unicode was completely broken on OS X. Fix this by only generating
12373 unicode codepoints in the range supported by the system.
12374 0.4.2 - 2015-02-04
12376 Codename: O(dear)
12377 This is purely a bugfix release:
12379 • Provide sensible external hashing for all core types. This will sig‐
12381 nificantly improve performance of tracking seen examples which hap‐
12382 pens in literally every falsification run. For Hypothesis fixing this
12383 cut 40% off the runtime of the test suite. The behaviour is quadratic
12384 in the number of examples so if you're running the default configura‐
12385 tion this will be less extreme (Hypothesis's test suite runs at a
12386 higher number of examples than default), but you should still see a
12387 significant improvement.
12388 • Fix a bug in formatting of complex numbers where the string could get
12390 incorrectly truncated.
12391 0.4.1 - 2015-02-03
12393 Codename: Cruel and unusual edge cases
12394 This release is mostly about better test case generation.
12396 Enhancements:
12398 • Has a cool release name
12400 • text_type (str in python 3, unicode in python 2) example generation
12402 now actually produces interesting unicode instead of boring ascii
12403 strings.
12404 • floating point numbers are generated over a much wider range, with
12406 particular attention paid to generating nasty numbers - nan, infin‐
12407 ity, large and small values, etc.
12408 • examples can be generated using pieces of examples previously saved
12410 in the database. This allows interesting behaviour that has previ‐
12411 ously been discovered to be propagated to other examples.
12412 • improved parameter exploration algorithm which should allow it to
12414 more reliably hit interesting edge cases.
12415 • Timeout can now be disabled entirely by setting it to any value <= 0.
12417 Bug fixes:
12419 • The descriptor on a OneOfStrategy could be wrong if you had descrip‐
12421 tors which were equal but should not be coalesced. e.g. a strategy
12422 for one_of((frozenset({int}), {int})) would have reported its de‐
12423 scriptor as {int}. This is unlikely to have caused you any problems
12424 • If you had strategies that could produce NaN (which float previously
12426 couldn't but e.g. a Just(float('nan')) could) then this would have
12427 sent hypothesis into an infinite loop that would have only been ter‐
12428 minated when it hit the timeout.
12429 • Given elements that can take a long time to minimize, minimization of
12431 floats or tuples could be quadratic or worse in the that value. You
12432 should now see much better performance for simplification, albeit at
12433 some cost in quality.
12434 Other:
12436 • A lot of internals have been been rewritten. This shouldn't affect
12438 you at all, but it opens the way for certain of hypothesis's oddities
12439 to be a lot more extensible by users. Whether this is a good thing
12440 may be up for debate...
12441 0.4.0 - 2015-01-21
12443 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
12444 stores data in a local SQLite database and will reuse it for all tests
12445 of the same type.
12446 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
12448 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
12449 ther notice. The previous license remains applicable to all code prior
12450 to 0.4.0.
12451 Enhancements:
12453 • Printing of failing examples. I was finding that the pytest runner
12455 was not doing a good job of displaying these, and that Hypothesis it‐
12456 self could do much better.
12457 • Drop dependency on six for cross-version compatibility. It was easy
12459 enough to write the shim for the small set of features that we care
12460 about and this lets us avoid a moderately complex dependency.
12461 • Some improvements to statistical distribution of selecting from small
12463 (<= 3 elements)
12464 • Improvements to parameter selection for finding examples.
12466 Bugs fixed:
12468 • could_have_produced for lists, dicts and other collections would not
12470 have examined the elements and thus when using a union of different
12471 types of list this could result in Hypothesis getting confused and
12472 passing a value to the wrong strategy. This could potentially result
12473 in exceptions being thrown from within simplification.
12474 • sampled_from would not work correctly on a single element list.
12476 • Hypothesis could get very confused by values which are equal despite
12478 having different types being used in descriptors. Hypothesis now has
12479 its own more specific version of equality it uses for descriptors and
12480 tracking. It is always more fine grained than Python equality: Things
12481 considered != are not considered equal by hypothesis, but some things
12482 that are considered == are distinguished. If your test suite uses
12483 both frozenset and set tests this bug is probably affecting you.
12484 0.3.2 - 2015-01-16
12486 • Fix a bug where if you specified floats_in_range with integer argu‐
12487 ments Hypothesis would error in example simplification.
12488 • Improve the statistical distribution of the floats you get for the
12490 floats_in_range strategy. I'm not sure whether this will affect users
12491 in practice but it took my tests for various conditions from flaky to
12492 rock solid so it at the very least improves discovery of the artifi‐
12493 cial cases I'm looking for.
12494 • Improved repr() for strategies and RandomWithSeed instances.
12496 • Add detection for flaky test cases where hypothesis managed to find
12498 an example which breaks it but on the final invocation of the test it
12499 does not raise an error. This will typically happen with too much re‐
12500 cursion errors but could conceivably happen in other circumstances
12501 too.
12502 • Provide a "derandomized" mode. This allows you to run hypothesis with
12504 zero real randomization, making your build nice and deterministic.
12505 The tests run with a seed calculated from the function they're test‐
12506 ing so you should still get a good distribution of test cases.
12507 • Add a mechanism for more conveniently defining tests which just sam‐
12509 ple from some collection.
12510 • Fix for a really subtle bug deep in the internals of the strategy ta‐
12512 ble. In some circumstances if you were to define instance strategies
12513 for both a parent class and one or more of its subclasses you would
12514 under some circumstances get the strategy for the wrong superclass of
12515 an instance. It is very unlikely anyone has ever encountered this in
12516 the wild, but it is conceivably possible given that a mix of namedtu‐
12517 ple and tuple are used fairly extensively inside hypothesis which do
12518 exhibit this pattern of strategy.
12519 0.3.1 - 2015-01-13
12521 • Support for generation of frozenset and Random values
12522 • Correct handling of the case where a called function mutates it argu‐
12524 ment. This involved introducing a notion of a strategies knowing how
12525 to copy their argument. The default method should be entirely accept‐
12526 able and the worst case is that it will continue to have the old be‐
12527 haviour if you don't mark your strategy as mutable, so this shouldn't
12528 break anything.
12529 • Fix for a bug where some strategies did not correctly implement
12531 could_have_produced. It is very unlikely that any of these would have
12532 been seen in the wild, and the consequences if they had been would
12533 have been minor.
12534 • Re-export the @given decorator from the main hypothesis namespace.
12536 It's still available at the old location too.
12537 • Minor performance optimisation for simplifying long lists.
12539 0.3.0 - 2015-01-12
12541 • Complete redesign of the data generation system. Extreme breaking
12542 change for anyone who was previously writing their own SearchStrategy
12543 implementations. These will not work any more and you'll need to mod‐
12544 ify them.
12545 • New settings system allowing more global and modular control of Veri‐
12547 fier behaviour.
12548 • Decouple SearchStrategy from the StrategyTable. This leads to much
12550 more composable code which is a lot easier to understand.
12551 • A significant amount of internal API renaming and moving. This may
12553 also break your code.
12554 • Expanded available descriptors, allowing for generating integers or
12556 floats in a specific range.
12557 • Significantly more robust. A very large number of small bug fixes,
12559 none of which anyone is likely to have ever noticed.
12560 • Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
12562 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
12563 these to remain officially supported for a very long time. I would
12564 not be surprised to add pypy support back in later but I'm not going
12565 to do so until I know someone cares about it. In the meantime it will
12566 probably still work.
12567 0.2.2 - 2015-01-08
12569 • Fix an embarrassing complete failure of the installer caused by my
12570 being bad at version control
12571 0.2.1 - 2015-01-07
12573 • Fix a bug in the new stateful testing feature where you could make
12574 __init__ a @requires method. Simplification would not always work if
12575 the prune method was able to successfully shrink the test.
12576 0.2.0 - 2015-01-07
12578 • It's aliiive.
12579 • Improve python 3 support using six.
12581 • Distinguish between byte and unicode types.
12583 • Fix issues where FloatStrategy could raise.
12585 • Allow stateful testing to request constructor args.
12587 • Fix for issue where test annotations would timeout based on when the
12589 module was loaded instead of when the test started
12590 0.1.4 - 2013-12-14
12592 • Make verification runs time bounded with a configurable timeout
12593 0.1.3 - 2013-05-03
12595 • Bugfix: Stateful testing behaved incorrectly with subclassing.
12596 • Complex number support
12598 • support for recursive strategies
12600 • different error for hypotheses with unsatisfiable assumptions
12602 0.1.2 - 2013-03-24
12604 • Bugfix: Stateful testing was not minimizing correctly and could throw
12605 exceptions.
12606 • Better support for recursive strategies.
12608 • Support for named tuples.
12610 • Much faster integer generation.
12612 0.1.1 - 2013-03-24
12614 • Python 3.x support via 2to3.
12615 • Use new style classes (oops).
12617 0.1.0 - 2013-03-23
12619 • Introduce stateful testing.
12620 • Massive rewrite of internals to add flags and strategies.
12622 0.0.5 - 2013-03-13
12624 • No changes except trying to fix packaging
12625 0.0.4 - 2013-03-13
12627 • No changes except that I checked in a failing test case for 0.0.3 so
12628 had to replace the release. Doh
12629 0.0.3 - 2013-03-13
12631 • Improved a few internals.
12632 • Opened up creating generators from instances as a general API.
12634 • Test integration.
12636 0.0.2 - 2013-03-12
12638 • Starting to tighten up on the internals.
12639 • Change API to allow more flexibility in configuration.
12641 • More testing.
12643 0.0.1 - 2013-03-10
12645 • Initial release.
12646 • Basic working prototype. Demonstrates idea, probably shouldn't be
12648 used.
12649
12651 Hypothesis development is managed by David R. MacIver and Zac Hat‐
12652 field-Dodds, respectively the first author and lead maintainer.
12653 However, these roles don't include unpaid feature development on Hy‐
12655 pothesis. Our roles as leaders of the project are:
12656 1. Helping other people do feature development on Hypothesis
12658 2. Fixing bugs and other code health issues
12660 3. Improving documentation
12662 4. General release management work
12664 5. Planning the general roadmap of the project
12666 6. Doing sponsored development on tasks that are too large or in depth
12668 for other people to take on
12669 So all new features must either be sponsored or implemented by someone
12671 else. That being said, the maintenance team takes an active role in
12672 shepherding pull requests and helping people write a new feature (see
12673 CONTRIBUTING.rst for details and these examples of how the process
12674 goes). This isn't "patches welcome", it's "we will help you write a
12675 patch".
12676 Release policy
12678 Hypothesis releases follow semantic versioning.
12679 We maintain backwards-compatibility wherever possible, and use depreca‐
12681 tion warnings to mark features that have been superseded by a newer al‐
12682 ternative. If you want to detect this, you can upgrade warnings to er‐
12683 rors in the usual ways.
12684 We use continuous deployment to ensure that you can always use our new‐
12686 est and shiniest features - every change to the source tree is automat‐
12687 ically built and published on PyPI as soon as it's merged onto master,
12688 after code review and passing our extensive test suite.
12689 Project roadmap
12691 Hypothesis does not have a long-term release plan. We respond to bug
12692 reports as they are made; new features are released as and when someone
12693 volunteers to write and maintain them.
12694
12696 For questions you are happy to ask in public, the Hypothesis community
12697 is a friendly place where I or others will be more than happy to help
12698 you out. You're also welcome to ask questions on Stack Overflow. If you
12699 do, please tag them with 'python-hypothesis' so someone sees them.
12700 For bugs and enhancements, please file an issue on the GitHub issue
12702 tracker. Note that as per the development policy, enhancements will
12703 probably not get implemented unless you're willing to pay for develop‐
12704 ment or implement them yourself (with assistance from the maintainers).
12705 Bugs will tend to get fixed reasonably promptly, though it is of course
12706 on a best effort basis.
12707 To see the versions of Python, optional dependencies, test runners, and
12709 operating systems Hypothesis supports (meaning incompatibility is
12710 treated as a bug), see supported.
12711 If you need to ask questions privately or want more of a guarantee of
12713 bugs being fixed promptly, please contact me on
12714 hypothesis-support@drmaciver.com to talk about availability of support
12715 contracts.
12716
12718 Downstream packagers often want to package Hypothesis. Here are some
12719 guidelines.
12720 The primary guideline is this: If you are not prepared to keep up with
12722 the Hypothesis release schedule, don't. You will annoy me and are doing
12723 your users a disservice.
12724 Hypothesis has a very frequent release schedule. It's rare that it goes
12726 a week without a release, and there are often multiple releases in a
12727 given week.
12728 If you are prepared to keep up with this schedule, you might find the
12730 rest of this document useful.
12731 Release tarballs
12733 These are available from the GitHub releases page. The tarballs on PyPI
12734 are intended for installation from a Python tool such as pip and should
12735 not be considered complete releases. Requests to include additional
12736 files in them will not be granted. Their absence is not a bug.
12737 Dependencies
12739 Python versions
12740 Hypothesis is designed to work with a range of Python versions - we
12741 support all versions of CPython with upstream support. We also support
12742 the latest versions of PyPy for Python 3.
12743 Other Python libraries
12745 Hypothesis has mandatory dependencies on the following libraries:
12746 • attrs
12748 • sortedcontainers
12750 Hypothesis has optional dependencies on the following libraries:
12752 extras_require = {
12754 "cli": ["click>=7.0", "black>=19.10b0", "rich>=9.0.0"],
12755 "codemods": ["libcst>=0.3.16"],
12756 "ghostwriter": ["black>=19.10b0"],
12757 "pytz": ["pytz>=2014.1"],
12758 "dateutil": ["python-dateutil>=1.4"],
12759 "lark": ["lark-parser>=0.6.5"],
12760 "numpy": ["numpy>=1.9.0"],
12761 "pandas": ["pandas>=0.25"],
12762 "pytest": ["pytest>=4.6"],
12763 "dpcontracts": ["dpcontracts>=0.4"],
12764 "redis": ["redis>=3.0.0"],
12765 # zoneinfo is an odd one: every dependency is conditional, because they're
12766 # only necessary on old versions of Python or Windows systems.
12767 "zoneinfo": [
12768 "tzdata>=2020.4 ; sys_platform == 'win32'",
12769 "backports.zoneinfo>=0.2.1 ; python_version<'3.9'",
12770 "importlib_resources>=3.3.0 ; python_version<'3.7'",
12771 ],
12772 # We only support Django versions with upstream support - see
12773 # https://www.djangoproject.com/download/#supported-versions
12774 "django": ["pytz>=2014.1", "django>=2.2"],
12775 }
12776 The way this works when installing Hypothesis normally is that these
12779 features become available if the relevant library is installed.
12780 Specifically for pytest, our plugin supports versions of pytest which
12782 have been out of upstream support for some time. Hypothesis tests can
12783 still be executed by even older versions of pytest - you just won't
12784 have the plugin to provide automatic marks, helpful usage warnings, and
12785 per-test statistics.
12786 Testing Hypothesis
12788 If you want to test Hypothesis as part of your packaging you will prob‐
12789 ably not want to use the mechanisms Hypothesis itself uses for running
12790 its tests, because it has a lot of logic for installing and testing
12791 against different versions of Python.
12792 The tests must be run with fairly recent tooling; check the
12794 tree/master/requirements/ directory for details.
12795 The organisation of the tests is described in the
12797 hypothesis-python/tests/README.rst.
12798 Examples
12800 • arch linux
12801 • fedora
12803 • gentoo
12805
12807 One of the things that is often concerning for people using randomized
12808 testing is the question of how to reproduce failing test cases.
12809 NOTE:
12811 It is better to think about the data Hypothesis generates as being
12812 arbitrary, rather than random. We deliberately generate any valid
12813 data that seems likely to cause errors, so you shouldn't rely on any
12814 expected distribution of or relationships between generated data.
12815 You can read about "swarm testing" and "coverage guided fuzzing" if
12816 you're interested, because you don't need to know for Hypothesis!
12817 Fortunately Hypothesis has a number of features to support reproducing
12819 test failures. The one you will use most commonly when developing lo‐
12820 cally is the example database, which means that you shouldn't have to
12821 think about the problem at all for local use - test failures will just
12822 automatically reproduce without you having to do anything.
12823 The example database is perfectly suitable for sharing between ma‐
12825 chines, but there currently aren't very good work flows for that, so
12826 Hypothesis provides a number of ways to make examples reproducible by
12827 adding them to the source code of your tests. This is particularly use‐
12828 ful when e.g. you are trying to run an example that has failed on your
12829 CI, or otherwise share them between machines.
12830 Providing explicit examples
12832 The simplest way to reproduce a failed test is to ask Hypothesis to run
12833 the failing example it printed. For example, if Falsifying example:
12834 test(n=1) was printed you can decorate test with @example(n=1).
12835 @example can also be used to ensure a specific example is always exe‐
12837 cuted as a regression test or to cover some edge case - basically com‐
12838 bining a Hypothesis test and a traditional parametrized test.
12839 hypothesis.example(*args, **kwargs)
12841 A decorator which ensures a specific example is always tested.
12842 Hypothesis will run all examples you've asked for first. If any of them
12844 fail it will not go on to look for more examples.
12845 It doesn't matter whether you put the example decorator before or after
12847 given. Any permutation of the decorators in the above will do the same
12848 thing.
12849 Note that examples can be positional or keyword based. If they're posi‐
12851 tional then they will be filled in from the right when calling, so ei‐
12852 ther of the following styles will work as expected:
12853 @given(text())
12855 @example("Hello world")
12856 @example(x="Some very long string")
12857 def test_some_code(x):
12858 assert True
12859 from unittest import TestCase
12862 class TestThings(TestCase):
12865 @given(text())
12866 @example("Hello world")
12867 @example(x="Some very long string")
12868 def test_some_code(self, x):
12869 assert True
12870 As with @given, it is not permitted for a single example to be a mix of
12872 positional and keyword arguments. Either are fine, and you can use one
12873 in one example and the other in another example if for some reason you
12874 really want to, but a single example must be consistent.
12875 Reproducing a test run with @seed
12877 hypothesis.seed(seed)
12878 seed: Start the test execution from a specific seed.
12879 May be any hashable object. No exact meaning for seed is pro‐
12881 vided other than that for a fixed seed value Hypothesis will try
12882 the same actions (insofar as it can given external sources of
12883 non- determinism. e.g. timing and hash randomization).
12884 Overrides the derandomize setting, which is designed to enable
12886 deterministic builds rather than reproducing observed failures.
12887 When a test fails unexpectedly, usually due to a health check failure,
12889 Hypothesis will print out a seed that led to that failure, if the test
12890 is not already running with a fixed seed. You can then recreate that
12891 failure using either the @seed decorator or (if you are running pytest)
12892 with --hypothesis-seed. For example, the following test function and
12893 RuleBasedStateMachine will each check the same examples each time they
12894 are executed, thanks to @seed():
12895 @seed(1234)
12897 @given(x=...)
12898 def test(x):
12899 ...
12900 @seed(6789)
12903 class MyModel(RuleBasedStateMachine):
12904 ...
12905 The seed will not be printed if you could simply use @example instead.
12907 Reproducing an example with @reproduce_failure
12909 Hypothesis has an opaque binary representation that it uses for all ex‐
12910 amples it generates. This representation is not intended to be stable
12911 across versions or with respect to changes in the test, but can be used
12912 to to reproduce failures with the @reproduce_failure decorator.
12913 hypothesis.reproduce_failure(version, blob)
12915 Run the example that corresponds to this data blob in order to
12916 reproduce a failure.
12917 A test with this decorator always runs only one example and al‐
12919 ways fails. If the provided example does not cause a failure,
12920 or is in some way invalid for this test, then this will fail
12921 with a DidNotReproduce error.
12922 This decorator is not intended to be a permanent addition to
12924 your test suite. It's simply some code you can add to ease re‐
12925 production of a problem in the event that you don't have access
12926 to the test database. Because of this, no compatibility guaran‐
12927 tees are made between different versions of Hypothesis - its API
12928 may change arbitrarily from version to version.
12929 The intent is that you should never write this decorator by hand, but
12931 it is instead provided by Hypothesis. When a test fails with a falsi‐
12932 fying example, Hypothesis may print out a suggestion to use @repro‐
12933 duce_failure on the test to recreate the problem as follows:
12934 >>> from hypothesis import settings, given, PrintSettings
12936 >>> import hypothesis.strategies as st
12937 >>> @given(st.floats())
12938 ... @settings(print_blob=True)
12939 ... def test(f):
12940 ... assert f == f
12941 ...
12942 >>> try:
12943 ... test()
12944 ... except AssertionError:
12945 ... pass
12946 ...
12947 Falsifying example: test(f=nan)
12948 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
12950 Adding the suggested decorator to the test should reproduce the failure
12952 (as long as everything else is the same - changing the versions of
12953 Python or anything else involved, might of course affect the behaviour
12954 of the test! Note that changing the version of Hypothesis will result
12955 in a different error - each @reproduce_failure invocation is specific
12956 to a Hypothesis version).
12957 By default these messages are not printed. If you want to see these
12959 you must set the print_blob setting to True.
12960
12962 David R. MacIver
12963
12965 2013-2021, David R. MacIver
129666.14.3 Jul 19, 2021 HYPOTHESIS(1)
