1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property-based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the
41 future.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 · Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 · If you delete an object, it is no longer visible.
53 · If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ""
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ""
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Let's see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127 Anyway, this test immediately finds a bug in the code:
129 Falsifying example: test_decode_inverts_encode(s='')
131 UnboundLocalError: local variable 'character' referenced before assignment
133 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136 If we fix that by just adding the following code to the beginning of
138 the function then Hypothesis tells us the code is correct (by doing
139 nothing as you'd expect a passing test to).
140 if not input_string:
142 return []
143 If we wanted to make sure this example was always checked we could add
145 it in explicitly by using the @example decorator:
146 from hypothesis import given, example
148 from hypothesis.strategies import text
149 @given(text())
152 @example("")
153 def test_decode_inverts_encode(s):
154 assert decode(encode(s)) == s
155 This can be useful to show other developers (or your future self) what
157 kinds of data are valid inputs, or to ensure that particular edge cases
158 such as "" are tested every time. It's also great for regression tests
159 because although Hypothesis will remember failing examples, we don't
160 recommend distributing that database.
161 It's also worth noting that both example and given support keyword
163 arguments as well as positional. The following would have worked just
164 as well:
165 @given(s=text())
167 @example(s="")
168 def test_decode_inverts_encode(s):
169 assert decode(encode(s)) == s
170 Suppose we had a more interesting bug and forgot to reset the count
172 each time. Say we missed a line in our encode method:
173 def encode(input_string):
175 count = 1
176 prev = ""
177 lst = []
178 for character in input_string:
179 if character != prev:
180 if prev:
181 entry = (prev, count)
182 lst.append(entry)
183 # count = 1 # Missing reset operation
184 prev = character
185 else:
186 count += 1
187 entry = (character, count)
188 lst.append(entry)
189 return lst
190 Hypothesis quickly informs us of the following example:
192 Falsifying example: test_decode_inverts_encode(s='001')
194 Note that the example provided is really quite simple. Hypothesis
196 doesn't just find any counter-example to your tests, it knows how to
197 simplify the examples it finds to produce small easy to understand
198 ones. In this case, two identical values are enough to set the count to
199 a number different from one, followed by another distinct value which
200 should have reset the count but in this case didn't.
201 The examples Hypothesis provides are valid Python code you can run. Any
203 arguments that you explicitly provide when calling the function are not
204 generated by Hypothesis, and if you explicitly provide all the argu‐
205 ments Hypothesis will just call the underlying function once rather
206 than running it multiple times.
207 Installing
209 Hypothesis is available on PyPI as "hypothesis". You can install it
210 with:
211 pip install hypothesis
213 You can install the dependencies for optional extensions with e.g. pip
215 install hypothesis[pandas,django].
216 If you want to install directly from the source code (e.g. because you
218 want to make changes and install the changed version), check out the
219 instructions in CONTRIBUTING.rst.
220 Running tests
222 In our example above we just let pytest discover and run our tests, but
223 we could also have run it explicitly ourselves:
224 if __name__ == "__main__":
226 test_decode_inverts_encode()
227 We could also have done this as a python:unittest.TestCase:
229 import unittest
231 class TestEncoding(unittest.TestCase):
234 @given(text())
235 def test_decode_inverts_encode(self, s):
236 self.assertEqual(decode(encode(s)), s)
237 if __name__ == "__main__":
240 unittest.main()
241 A detail: This works because Hypothesis ignores any arguments it hasn't
243 been told to provide (positional arguments start from the right), so
244 the self argument to the test is simply ignored and works as normal.
245 This also means that Hypothesis will play nicely with other ways of
246 parameterizing tests. e.g it works fine if you use pytest fixtures for
247 some arguments and Hypothesis for others.
248 Writing tests
250 A test in Hypothesis consists of two parts: A function that looks like
251 a normal test in your test framework of choice but with some additional
252 arguments, and a @given decorator that specifies how to provide those
253 arguments.
254 Here are some other examples of how you could use that:
256 from hypothesis import given
258 import hypothesis.strategies as st
259 @given(st.integers(), st.integers())
262 def test_ints_are_commutative(x, y):
263 assert x + y == y + x
264 @given(x=st.integers(), y=st.integers())
267 def test_ints_cancel(x, y):
268 assert (x + y) - y == x
269 @given(st.lists(st.integers()))
272 def test_reversing_twice_gives_same_list(xs):
273 # This will generate lists of arbitrary length (usually between 0 and
274 # 100 elements) whose elements are integers.
275 ys = list(xs)
276 ys.reverse()
277 ys.reverse()
278 assert xs == ys
279 @given(st.tuples(st.booleans(), st.text()))
282 def test_look_tuples_work_too(t):
283 # A tuple is generated as the one you provided, with the corresponding
284 # types in those positions.
285 assert len(t) == 2
286 assert isinstance(t[0], bool)
287 assert isinstance(t[1], str)
288 Note that as we saw in the above example you can pass arguments to
290 @given either as positional or as keywords.
291 Where to start
293 You should now know enough of the basics to write some tests for your
294 code using Hypothesis. The best way to learn is by doing, so go have a
295 try.
296 If you're stuck for ideas for how to use this sort of test for your
298 code, here are some good starting points:
299 1. Try just calling functions with appropriate arbitrary data and see
301 if they crash. You may be surprised how often this works. e.g. note
302 that the first bug we found in the encoding example didn't even get
303 as far as our assertion: It crashed because it couldn't handle the
304 data we gave it, not because it did the wrong thing.
305 2. Look for duplication in your tests. Are there any cases where you're
307 testing the same thing with multiple different examples? Can you
308 generalise that to a single test using Hypothesis?
309 3. This piece is designed for an F# implementation, but is still very
311 good advice which you may find helps give you good ideas for using
312 Hypothesis.
313 If you have any trouble getting started, don't feel shy about asking
315 for help.
316
318 This is an account of slightly less common Hypothesis features that you
319 don't need to get started but will nevertheless make your life easier.
320 Additional test output
322 Normally the output of a failing test will look something like:
323 Falsifying example: test_a_thing(x=1, y="foo")
325 With the repr of each keyword argument being printed.
327 Sometimes this isn't enough, either because you have a value with a
329 __repr__() method that isn't very descriptive or because you need to
330 see the output of some intermediate steps of your test. That's where
331 the note function comes in:
332 hypothesis.note(value)
334 Report this value in the final execution.
335 >>> from hypothesis import given, note, strategies as st
337 >>> @given(st.lists(st.integers()), st.randoms())
338 ... def test_shuffle_is_noop(ls, r):
339 ... ls2 = list(ls)
340 ... r.shuffle(ls2)
341 ... note("Shuffle: %r" % (ls2))
342 ... assert ls == ls2
343 ...
344 >>> try:
345 ... test_shuffle_is_noop()
346 ... except AssertionError:
347 ... print('ls != ls2')
348 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
349 Shuffle: [1, 0]
350 ls != ls2
351 The note is printed in the final run of the test in order to include
353 any additional information you might need in your test.
354 Test statistics
356 If you are using pytest you can see a number of statistics about the
357 executed tests by passing the command line argument --hypothe‐
358 sis-show-statistics. This will include some general statistics about
359 the test:
360 For example if you ran the following with --hypothesis-show-statistics:
362 from hypothesis import given, strategies as st
364 @given(st.integers())
367 def test_integers(i):
368 pass
369 You would see:
371 - during generate phase (0.06 seconds):
373 - Typical runtimes: < 1ms, ~ 47% in data generation
374 - 100 passing examples, 0 failing examples, 0 invalid examples
375 - Stopped because settings.max_examples=100
376 The final "Stopped because" line is particularly important to note: It
378 tells you the setting value that determined when the test should stop
379 trying new examples. This can be useful for understanding the behaviour
380 of your tests. Ideally you'd always want this to be max_examples.
381 In some cases (such as filtered and recursive strategies) you will see
383 events mentioned which describe some aspect of the data generation:
384 from hypothesis import given, strategies as st
386 @given(st.integers().filter(lambda x: x % 2 == 0))
389 def test_even_integers(i):
390 pass
391 You would see something like:
393 test_even_integers:
395 - during generate phase (0.08 seconds):
397 - Typical runtimes: < 1ms, ~ 57% in data generation
398 - 100 passing examples, 0 failing examples, 12 invalid examples
399 - Events:
400 * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
401 * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
402 - Stopped because settings.max_examples=100
403 You can also mark custom events in a test using the event function:
405 hypothesis.event(value)
407 Record an event that occurred this test. Statistics on number of
408 test runs with each event will be reported at the end if you run
409 Hypothesis in statistics reporting mode.
410 Events should be strings or convertible to them.
412 from hypothesis import given, event, strategies as st
414 @given(st.integers().filter(lambda x: x % 2 == 0))
417 def test_even_integers(i):
418 event("i mod 3 = %d" % (i % 3,))
419 You will then see output like:
421 test_even_integers:
423 - during generate phase (0.09 seconds):
425 - Typical runtimes: < 1ms, ~ 59% in data generation
426 - 100 passing examples, 0 failing examples, 32 invalid examples
427 - Events:
428 * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
429 * 31.06%, i mod 3 = 2
430 * 28.79%, i mod 3 = 0
431 * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
432 * 15.91%, i mod 3 = 1
433 - Stopped because settings.max_examples=100
434 Arguments to event can be any hashable type, but two events will be
436 considered the same if they are the same when converted to a string
437 with python:str.
438 Making assumptions
440 Sometimes Hypothesis doesn't give you exactly the right sort of data
441 you want - it's mostly of the right shape, but some examples won't work
442 and you don't want to care about them. You can just ignore these by
443 aborting the test early, but this runs the risk of accidentally testing
444 a lot less than you think you are. Also it would be nice to spend less
445 time on bad examples - if you're running 100 examples per test (the
446 default) and it turns out 70 of those examples don't match your needs,
447 that's a lot of wasted time.
448 hypothesis.assume(condition)
450 Calling assume is like an assert that marks the example as bad,
451 rather than failing the test.
452 This allows you to specify properties that you assume will be
454 true, and let Hypothesis try to avoid similar examples in
455 future.
456 For example suppose you had the following test:
458 @given(floats())
460 def test_negation_is_self_inverse(x):
461 assert x == -(-x)
462 Running this gives us:
464 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
466 AssertionError
467 This is annoying. We know about NaN and don't really care about it, but
469 as soon as Hypothesis finds a NaN example it will get distracted by
470 that and tell us about it. Also the test will fail and we want it to
471 pass.
472 So let's block off this particular example:
474 from math import isnan
476 @given(floats())
479 def test_negation_is_self_inverse_for_non_nan(x):
480 assume(not isnan(x))
481 assert x == -(-x)
482 And this passes without a problem.
484 In order to avoid the easy trap where you assume a lot more than you
486 intended, Hypothesis will fail a test when it can't find enough exam‐
487 ples passing the assumption.
488 If we'd written:
490 @given(floats())
492 def test_negation_is_self_inverse_for_non_nan(x):
493 assume(False)
494 assert x == -(-x)
495 Then on running we'd have got the exception:
497 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
499 How good is assume?
501 Hypothesis has an adaptive exploration strategy to try to avoid things
502 which falsify assumptions, which should generally result in it still
503 being able to find examples in hard to find situations.
504 Suppose we had the following:
506 @given(lists(integers()))
508 def test_sum_is_positive(xs):
509 assert sum(xs) > 0
510 Unsurprisingly this fails and gives the falsifying example [].
512 Adding assume(xs) to this removes the trivial empty example and gives
514 us [0].
515 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
517 of positive integers is positive.
518 The reason that this should be surprising is not that it doesn't find a
520 counter-example, but that it finds enough examples at all.
521 In order to make sure something interesting is happening, suppose we
523 wanted to try this for long lists. e.g. suppose we added an
524 assume(len(xs) > 10) to it. This should basically never find an exam‐
525 ple: a naive strategy would find fewer than one in a thousand examples,
526 because if each element of the list is negative with probability
527 one-half, you'd have to have ten of these go the right way by chance.
528 In the default configuration Hypothesis gives up long before it's tried
529 1000 examples (by default it tries 200).
530 Here's what happens if we try to run this:
532 @given(lists(integers()))
534 def test_sum_is_positive(xs):
535 assume(len(xs) > 1)
536 assume(all(x > 0 for x in xs))
537 print(xs)
538 assert sum(xs) > 0
539 In: test_sum_is_positive()
541 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
543 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
544 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
545 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
546 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
547 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
548 As you can see, Hypothesis doesn't find many examples here, but it
550 finds some - enough to keep it happy.
551 In general if you can shape your strategies better to your tests you
553 should - for example integers(1, 1000) is a lot better than assume(1 <=
554 x <= 1000), but assume will take you a long way if you can't.
555 Defining strategies
557 The type of object that is used to explore the examples given to your
558 test function is called a SearchStrategy. These are created using the
559 functions exposed in the hypothesis.strategies module.
560 Many of these strategies expose a variety of arguments you can use to
562 customize generation. For example for integers you can specify min and
563 max values of integers you want. If you want to see exactly what a
564 strategy produces you can ask for an example:
565 >>> integers(min_value=0, max_value=10).example()
567 1
568 Many strategies are built out of other strategies. For example, if you
570 want to define a tuple you need to say what goes in each element:
571 >>> from hypothesis.strategies import tuples
573 >>> tuples(integers(), integers()).example()
574 (-24597, 12566)
575 Further details are available in a separate document.
577 The gory details of given parameters
579 hypothesis.given(*_given_arguments, **_given_kwargs)
580 A decorator for turning a test function that accepts arguments
581 into a randomized test.
582 This is the main entry point to Hypothesis.
584 The @given decorator may be used to specify which arguments of a func‐
586 tion should be parametrized over. You can use either positional or key‐
587 word arguments, but not a mixture of both.
588 For example all of the following are valid uses:
590 @given(integers(), integers())
592 def a(x, y):
593 pass
594 @given(integers())
597 def b(x, y):
598 pass
599 @given(y=integers())
602 def c(x, y):
603 pass
604 @given(x=integers())
607 def d(x, y):
608 pass
609 @given(x=integers(), y=integers())
612 def e(x, **kwargs):
613 pass
614 @given(x=integers(), y=integers())
617 def f(x, *args, **kwargs):
618 pass
619 class SomeTest(TestCase):
622 @given(integers())
623 def test_a_thing(self, x):
624 pass
625 The following are not:
627 @given(integers(), integers(), integers())
629 def g(x, y):
630 pass
631 @given(integers())
634 def h(x, *args):
635 pass
636 @given(integers(), x=integers())
639 def i(x, y):
640 pass
641 @given()
644 def j(x, y):
645 pass
646 The rules for determining what are valid uses of given are as follows:
648 1. You may pass any keyword argument to given.
650 2. Positional arguments to given are equivalent to the rightmost named
652 arguments for the test function.
653 3. Positional arguments may not be used if the underlying test function
655 has varargs, arbitrary keywords, or keyword-only arguments.
656 4. Functions tested with given may not have any defaults.
658 The reason for the "rightmost named arguments" behaviour is so that
660 using @given with instance methods works: self will be passed to the
661 function as normal and not be parametrized over.
662 The function returned by given has all the same arguments as the origi‐
664 nal test, minus those that are filled in by @given. Check the notes on
665 framework compatibility to see how this affects other testing libraries
666 you may be using.
667 Targeted example generation
669 Targeted property-based testing combines the advantages of both
670 search-based and property-based testing. Instead of being completely
671 random, T-PBT uses a search-based component to guide the input genera‐
672 tion towards values that have a higher probability of falsifying a
673 property. This explores the input space more effectively and requires
674 fewer tests to find a bug or achieve a high confidence in the system
675 being tested than random PBT. (Löscher and Sagonas)
676 This is not always a good idea - for example calculating the search
678 metric might take time better spent running more uniformly-random test
679 cases - but Hypothesis has experimental support for targeted PBT you
680 may wish to try.
681 hypothesis.target(observation, *, label='')
683 Calling this function with an int or float observation gives it
684 feedback with which to guide our search for inputs that will
685 cause an error, in addition to all the usual heuristics. Obser‐
686 vations must always be finite.
687 Hypothesis will try to maximize the observed value over several
689 examples; almost any metric will work so long as it makes sense
690 to increase it. For example, -abs(error) is a metric that
691 increases as error approaches zero.
692 Example metrics:
694 · Number of elements in a collection, or tasks in a queue
696 · Mean or maximum runtime of a task (or both, if you use label)
698 · Compression ratio for data (perhaps per-algorithm or
700 per-level)
701 · Number of steps taken by a state machine
703 The optional label argument can be used to distinguish between
705 and therefore separately optimise distinct observations, such as
706 the mean and standard deviation of a dataset. It is an error to
707 call target() with any label more than once per test case.
708 NOTE:
710 The more examples you run, the better this technique works.
711 As a rule of thumb, the targeting effect is noticeable above
713 max_examples=1000, and immediately obvious by around ten
714 thousand examples per label used by your test.
715 NOTE:
717 hypothesis.target is considered experimental, and may be rad‐
718 ically changed or even removed in a future version. If you
719 find it useful, please let us know so we can share and build
720 on that success!
721 Test statistics include the best score seen for each label,
723 which can help avoid the threshold problem when the minimal
724 example shrinks right down to the threshold of failure (issue
725 #2180).
726 We recommend that users also skim the papers introducing targeted PBT;
728 from ISSTA 2017 and ICST 2018. For the curious, the initial implemen‐
729 tation in Hypothesis uses hill-climbing search via a mutating fuzzer,
730 with some tactics inspired by simulated annealing to avoid getting
731 stuck and endlessly mutating a local maximum.
732 Custom function execution
734 Hypothesis provides you with a hook that lets you control how it runs
735 examples.
736 This lets you do things like set up and tear down around each example,
738 run examples in a subprocess, transform coroutine tests into normal
739 tests, etc. For example, TransactionTestCase in the Django extra runs
740 each example in a separate database transaction.
741 The way this works is by introducing the concept of an executor. An
743 executor is essentially a function that takes a block of code and run
744 it. The default executor is:
745 def default_executor(function):
747 return function()
748 You define executors by defining a method execute_example on a class.
750 Any test methods on that class with @given used on them will use
751 self.execute_example as an executor with which to run tests. For exam‐
752 ple, the following executor runs all its code twice:
753 from unittest import TestCase
755 class TestTryReallyHard(TestCase):
758 @given(integers())
759 def test_something(self, i):
760 perform_some_unreliable_operation(i)
761 def execute_example(self, f):
763 f()
764 return f()
765 Note: The functions you use in map, etc. will run inside the executor.
767 i.e. they will not be called until you invoke the function passed to
768 execute_example.
769 An executor must be able to handle being passed a function which
771 returns None, otherwise it won't be able to run normal test cases. So
772 for example the following executor is invalid:
773 from unittest import TestCase
775 class TestRunTwice(TestCase):
778 def execute_example(self, f):
779 return f()()
780 and should be rewritten as:
782 from unittest import TestCase
784 class TestRunTwice(TestCase):
787 def execute_example(self, f):
788 result = f()
789 if callable(result):
790 result = result()
791 return result
792 An alternative hook is provided for use by test runner extensions such
794 as pytest-trio, which cannot use the execute_example method. This is
795 not recommended for end-users - it is better to write a complete test
796 function directly, perhaps by using a decorator to perform the same
797 transformation before applying @given.
798 @given(x=integers())
800 @pytest.mark.trio
801 async def test(x):
802 ...
803 # Illustrative code, inside the pytest-trio plugin
806 test.hypothesis.inner_test = lambda x: trio.run(test, x)
807 For authors of test runners however, assigning to the inner_test
809 attribute of the hypothesis attribute of the test will replace the
810 interior test.
811 NOTE:
813 The new inner_test must accept and pass through all the *args and
814 **kwargs expected by the original test.
815 If the end user has also specified a custom executor using the exe‐
817 cute_example method, it - and all other execution-time logic - will be
818 applied to the new inner test assigned by the test runner.
819 Making random code deterministic
821 While Hypothesis' example generation can be used for nondeterministic
822 tests, debugging anything nondeterministic is usually a very frustrat‐
823 ing exercise. To make things worse, our example shrinking relies on
824 the same input causing the same failure each time - though we show the
825 un-shrunk failure and a decent error message if it doesn't.
826 By default, Hypothesis will handle the global random and numpy.random
828 random number generators for you, and you can register others:
829 hypothesis.register_random(r)
831 Register the given Random instance for management by Hypothesis.
832 You can pass random.Random instances (or other objects with
834 seed, getstate, and setstate methods) to register_random(r) to
835 have their states seeded and restored in the same way as the
836 global PRNGs from the random and numpy.random modules.
837 All global PRNGs, from e.g. simulation or scheduling frameworks,
839 should be registered to prevent flaky tests. Hypothesis will
840 ensure that the PRNG state is consistent for all test runs, or
841 reproducibly varied if you choose to use the random_module()
842 strategy.
843 Inferred strategies
845 In some cases, Hypothesis can work out what to do when you omit argu‐
846 ments. This is based on introspection, not magic, and therefore has
847 well-defined limits.
848 builds() will check the signature of the target (using get‐
850 fullargspec()). If there are required arguments with type annotations
851 and no strategy was passed to builds(), from_type() is used to fill
852 them in. You can also pass the special value hypothesis.infer as a
853 keyword argument, to force this inference for arguments with a default
854 value.
855 >>> def func(a: int, b: str):
857 ... return [a, b]
858 >>> builds(func).example()
859 [-6993, '']
860 hypothesis.infer
862 @given does not perform any implicit inference for required arguments,
864 as this would break compatibility with pytest fixtures. infer can be
865 used as a keyword argument to explicitly fill in an argument from its
866 type annotation.
867 @given(a=infer)
869 def test(a: int):
870 pass
871 # is equivalent to
874 @given(a=integers())
875 def test(a):
876 pass
877 Limitations
879 Hypothesis does not inspect PEP 484 type comments at runtime. While
880 from_type() will work as usual, inference in builds() and @given will
881 only work if you manually create the __annotations__ attribute (e.g. by
882 using @annotations(...) and @returns(...) decorators).
883 The python:typing module is provisional and has a number of internal
885 changes between Python 3.5.0 and 3.6.1, including at minor versions.
886 These are all supported on a best-effort basis, but you may encounter
887 problems with an old version of the module. Please report them to us,
888 and consider updating to a newer version of Python as a workaround.
889 Type annotations in Hypothesis
891 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
892 patible tool, the type checker should automatically pick up our type
893 hints.
894 NOTE:
896 Hypothesis' type hints may make breaking changes between minor
897 releases.
898 Upstream tools and conventions about type hints remain in flux - for
900 example the python:typing module itself is provisional, and Mypy has
901 not yet reached version 1.0 - and we plan to support the latest ver‐
902 sion of this ecosystem, as well as older versions where practical.
903 We may also find more precise ways to describe the type of various
905 interfaces, or change their type and runtime behaviour together in a
906 way which is otherwise backwards-compatible. We often omit type
907 hints for deprecated features or arguments, as an additional form of
908 warning.
909 There are known issues inferring the type of examples generated by
911 deferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
912 ies(). We will fix these, and require correspondingly newer versions
913 of Mypy for type hinting, as the ecosystem improves.
914 Writing downstream type hints
916 Projects that provide Hypothesis strategies and use type hints may wish
917 to annotate their strategies too. This is a supported use-case, again
918 on a best-effort provisional basis. For example:
919 def foo_strategy() -> SearchStrategy[Foo]:
921 ...
922 class hypothesis.strategies.SearchStrategy
924 SearchStrategy is the type of all strategy objects. It is a generic
926 type, and covariant in the type of the examples it creates. For exam‐
927 ple:
928 · integers() is of type SearchStrategy[int].
930 · lists(integers()) is of type SearchStrategy[List[int]].
932 · SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
934 a subtype of Animal (as seems likely).
935 WARNING:
937 SearchStrategy should only be used in type hints. Please do not
938 inherit from, compare to, or otherwise use it in any way outside of
939 type hints. The only supported way to construct objects of this
940 type is to use the functions provided by the hypothesis.strategies
941 module!
942 The Hypothesis pytest plugin
944 Hypothesis includes a tiny plugin to improve integration with pytest,
945 which is activated by default (but does not affect other test runners).
946 It aims to improve the integration between Hypothesis and Pytest by
947 providing extra information and convenient access to config options.
948 · pytest --hypothesis-show-statistics can be used to display test and
950 data generation statistics.
951 · pytest --hypothesis-profile=<profile name> can be used to load a set‐
953 tings profile. pytest --hypothesis-verbosity=<level name> can be
954 used to override the current verbosity level.
955 · pytest --hypothesis-seed=<an int> can be used to reproduce a failure
957 with a particular seed.
958 Finally, all tests that are defined with Hypothesis automatically have
960 @pytest.mark.hypothesis applied to them. See here for information on
961 working with markers.
962 NOTE:
964 Pytest will load the plugin automatically if Hypothesis is
965 installed. You don't need to do anything at all to use it.
966 Use with external fuzzers
968 TIP:
969 Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
970 Use HypoFuzz to fuzz your whole test suite,
971 and find more bugs without more tests!
972 Sometimes, you might want to point a traditional fuzzer such as
975 python-afl, pythonfuzz, or Google's atheris (for Python and native
976 extensions) at your code. Wouldn't it be nice if you could use any of
977 your @given tests as fuzz targets, instead of converting bytestrings
978 into your objects by hand?
979 @given(st.text())
981 def test_foo(s):
982 ...
983 # This is a traditional fuzz target - call it with a bytestring,
986 # or a binary IO object, and it runs the test once.
987 fuzz_target = test_foo.hypothesis.fuzz_one_input
988 # For example:
990 fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
991 fuzz_target(io.BytesIO(...))
992 Depending on the input to fuzz_one_input, one of three things will hap‐
994 pen:
995 · If the bytestring was invalid, for example because it was too short
997 or failed a filter or assume() too many times, fuzz_one_input returns
998 None.
999 · If the bytestring was valid and the test passed, fuzz_one_input
1001 returns a canonicalised and pruned buffer which will replay that test
1002 case. This is provided as an option to improve the performance of
1003 mutating fuzzers, but can safely be ignored.
1004 · If the test failed, i.e. raised an exception, fuzz_one_input will add
1006 the pruned buffer to the Hypothesis example database and then
1007 re-raise that exception. All you need to do to reproduce, minimize,
1008 and de-duplicate all the failures found via fuzzing is run your test
1009 suite!
1010 Note that the interpretation of both input and output bytestrings is
1012 specific to the exact version of Hypothesis you are using and the
1013 strategies given to the test, just like the example database and
1014 @reproduce_failure decorator.
1015 Interaction with settings
1017 fuzz_one_input uses just enough of Hypothesis' internals to drive your
1018 test function with a fuzzer-provided bytestring, and most settings
1019 therefore have no effect in this mode. We recommend running your tests
1020 the usual way before fuzzing to get the benefits of healthchecks, as
1021 well as afterwards to replay, shrink, deduplicate, and report whatever
1022 errors were discovered.
1023 · The database setting is used by fuzzing mode - adding failures to the
1025 database to be replayed when you next run your tests is our preferred
1026 reporting mechanism and reponse to the 'fuzzer taming' problem.
1027 · The verbosity and stateful_step_count settings work as usual.
1029 The deadline, derandomize, max_examples, phases, print_blob,
1031 report_multiple_bugs, and suppress_health_check settings do not affect
1032 fuzzing mode.
1033
1035 Hypothesis tries to have good defaults for its behaviour, but sometimes
1036 that's not enough and you need to tweak it.
1037 The mechanism for doing this is the settings object. You can set up a
1039 @given based test to use this using a settings decorator:
1040 @given invocation is as follows:
1042 from hypothesis import given, settings
1044 @given(integers())
1047 @settings(max_examples=500)
1048 def test_this_thoroughly(x):
1049 pass
1050 This uses a settings object which causes the test to receive a much
1052 larger set of examples than normal.
1053 This may be applied either before or after the given and the results
1055 are the same. The following is exactly equivalent:
1056 from hypothesis import given, settings
1058 @settings(max_examples=500)
1061 @given(integers())
1062 def test_this_thoroughly(x):
1063 pass
1064 Available settings
1066 class hypothesis.settings(parent=None, *, max_examples=not_set, deran‐
1067 domize=not_set, database=not_set, verbosity=not_set, phases=not_set,
1068 stateful_step_count=not_set, report_multiple_bugs=not_set, sup‐
1069 press_health_check=not_set, deadline=not_set, print_blob=not_set)
1070 A settings object controls a variety of parameters that are used
1071 in falsification. These may control both the falsification
1072 strategy and the details of the data that is generated.
1073 Default values are picked up from the settings.default object
1075 and changes made there will be picked up in newly created set‐
1076 tings.
1077 database
1079 An instance of ExampleDatabase that will be used to save
1080 examples to and load previous examples from. May be None
1081 in which case no storage will be used.
1082 See the example database documentation for a list of
1084 built-in example database implementations, and how to
1085 define custom implementations.
1086 default value: (dynamically calculated)
1088 deadline
1090 If set, a duration (as timedelta, or integer or float
1091 number of milliseconds) that each individual example
1092 (i.e. each time your test function is called, not the
1093 whole decorated test) within a test is not allowed to
1094 exceed. Tests which take longer than that may be con‐
1095 verted into errors (but will not necessarily be if close
1096 to the deadline, to allow some variability in test run
1097 time).
1098 Set this to None to disable this behaviour entirely.
1100 default value: timedelta(milliseconds=200)
1102 derandomize
1104 If True, seed Hypothesis' random number generator using a
1105 hash of the test function, so that every run will test
1106 the same set of examples until you update Hypothesis,
1107 Python, or the test function.
1108 This allows you to check for regressions and look for
1110 bugs using separate settings profiles - for example run‐
1111 ning quick deterministic tests on every commit, and a
1112 longer non-deterministic nightly testing run.
1113 default value: False
1115 max_examples
1117 Once this many satisfying examples have been considered
1118 without finding any counter-example, falsification will
1119 terminate.
1120 The default value is chosen to suit a workflow where the
1122 test will be part of a suite that is regularly executed
1123 locally or on a CI server, balancing total running time
1124 against the chance of missing a bug.
1125 If you are writing one-off tests, running tens of thou‐
1127 sands of examples is quite reasonable as Hypothesis may
1128 miss uncommon bugs with default settings. For very com‐
1129 plex code, we have observed Hypothesis finding novel bugs
1130 after several million examples while testing SymPy. If
1131 you are running more than 100k examples for a test, con‐
1132 sider using our integration for coverage-guided fuzzing -
1133 it really shines when given minutes or hours to run.
1134 default value: 100
1136 phases Control which phases should be run. See the full documen‐
1138 tation for more details
1139 default value: (Phase.explicit, Phase.reuse, Phase.gener‐
1141 ate, Phase.target, Phase.shrink)
1142 print_blob
1144 If set to True, Hypothesis will print code for failing
1145 examples that can be used with @reproduce_failure to
1146 reproduce the failing example. The default is True if
1147 the CI or TF_BUILD env vars are set, False otherwise.
1148 default value: (dynamically calculated)
1150 report_multiple_bugs
1152 Because Hypothesis runs the test many times, it can some‐
1153 times find multiple bugs in a single run. Reporting all
1154 of them at once is usually very useful, but replacing the
1155 exceptions can occasionally clash with debuggers. If
1156 disabled, only the exception with the smallest minimal
1157 example is raised.
1158 default value: True
1160 stateful_step_count
1162 Number of steps to run a stateful program for before giv‐
1163 ing up on it breaking.
1164 default value: 50
1166 suppress_health_check
1168 A list of HealthCheck items to disable.
1169 default value: ()
1171 verbosity
1173 Control the verbosity level of Hypothesis messages
1174 default value: Verbosity.normal
1176 Controlling what runs
1178 Hypothesis divides tests into five logically distinct phases:
1179 1. Running explicit examples provided with the @example decorator.
1181 2. Rerunning a selection of previously failing examples to reproduce a
1183 previously seen error
1184 3. Generating new examples.
1186 4. Mutating examples for targeted property-based testing.
1188 5. Attempting to shrink an example found in previous phases (other than
1190 phase 1 - explicit examples cannot be shrunk). This turns poten‐
1191 tially large and complicated examples which may be hard to read into
1192 smaller and simpler ones.
1193 The phases setting provides you with fine grained control over which of
1195 these run, with each phase corresponding to a value on the Phase enum:
1196 class hypothesis.Phase
1198 1. Phase.explicit controls whether explicit examples are run.
1200 2. Phase.reuse controls whether previous examples will be reused.
1202 3. Phase.generate controls whether new examples will be generated.
1204 4. Phase.target controls whether examples will be mutated for target‐
1206 ing.
1207 5. Phase.shrink controls whether examples will be shrunk.
1209 The phases argument accepts a collection with any subset of these. e.g.
1211 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1212 ples and shrink them, but will not run explicit examples or reuse pre‐
1213 vious failures, while settings(phases=[Phase.explicit]) will only run
1214 the explicit examples.
1215 Seeing intermediate result
1217 To see what's going on while Hypothesis runs your tests, you can turn
1218 up the verbosity setting.
1219 >>> from hypothesis import find, settings, Verbosity
1221 >>> from hypothesis.strategies import lists, integers
1222 >>> @given(lists(integers()))
1223 ... @settings(verbosity=Verbosity.verbose)
1224 ... def f(x): assert not any(x)
1225 ... f()
1226 Trying example: []
1227 Falsifying example: [-1198601713, -67, 116, -29578]
1228 Shrunk example to [-1198601713]
1229 Shrunk example to [-1198601600]
1230 Shrunk example to [-1191228800]
1231 Shrunk example to [-8421504]
1232 Shrunk example to [-32896]
1233 Shrunk example to [-128]
1234 Shrunk example to [64]
1235 Shrunk example to [32]
1236 Shrunk example to [16]
1237 Shrunk example to [8]
1238 Shrunk example to [4]
1239 Shrunk example to [3]
1240 Shrunk example to [2]
1241 Shrunk example to [1]
1242 [1]
1243 The four levels are quiet, normal, verbose and debug. normal is the
1245 default, while in quiet mode Hypothesis will not print anything out,
1246 not even the final falsifying example. debug is basically verbose but a
1247 bit more so. You probably don't want it.
1248 If you are using pytest, you may also need to disable output capturing
1250 for passing tests.
1251 Building settings objects
1253 Settings can be created by calling settings with any of the available
1254 settings values. Any absent ones will be set to defaults:
1255 >>> from hypothesis import settings
1257 >>> settings().max_examples
1258 100
1259 >>> settings(max_examples=10).max_examples
1260 10
1261 You can also pass a 'parent' settings object as the first argument, and
1263 any settings you do not specify as keyword arguments will be copied
1264 from the parent settings:
1265 >>> parent = settings(max_examples=10)
1267 >>> child = settings(parent, deadline=None)
1268 >>> parent.max_examples == child.max_examples == 10
1269 True
1270 >>> parent.deadline
1271 200
1272 >>> child.deadline is None
1273 True
1274 Default settings
1276 At any given point in your program there is a current default settings,
1277 available as settings.default. As well as being a settings object in
1278 its own right, all newly created settings objects which are not explic‐
1279 itly based off another settings are based off the default, so will
1280 inherit any values that are not explicitly set from it.
1281 You can change the defaults by using profiles.
1283 Settings profiles
1285 Depending on your environment you may want different default settings.
1286 For example: during development you may want to lower the number of
1287 examples to speed up the tests. However, in a CI environment you may
1288 want more examples so you are more likely to find bugs.
1289 Hypothesis allows you to define different settings profiles. These pro‐
1291 files can be loaded at any time.
1292 static settings.register_profile(name, parent=None, **kwargs)
1294 Registers a collection of values to be used as a settings pro‐
1295 file.
1296 Settings profiles can be loaded by name - for example, you might
1298 create a 'fast' profile which runs fewer examples, keep the
1299 'default' profile, and create a 'ci' profile that increases the
1300 number of examples and uses a different database to store fail‐
1301 ures.
1302 The arguments to this method are exactly as for settings:
1304 optional parent settings, and keyword arguments for each setting
1305 that will be set differently to parent (or settings.default, if
1306 parent is None).
1307 static settings.get_profile(name)
1309 Return the profile with the given name.
1310 static settings.load_profile(name)
1312 Loads in the settings defined in the profile provided.
1313 If the profile does not exist, InvalidArgument will be raised.
1315 Any setting not defined in the profile will be the library
1316 defined default for that setting.
1317 Loading a profile changes the default settings but will not change the
1319 behaviour of tests that explicitly change the settings.
1320 >>> from hypothesis import settings
1322 >>> settings.register_profile("ci", max_examples=1000)
1323 >>> settings().max_examples
1324 100
1325 >>> settings.load_profile("ci")
1326 >>> settings().max_examples
1327 1000
1328 Instead of loading the profile and overriding the defaults you can
1330 retrieve profiles for specific tests.
1331 >>> settings.get_profile("ci").max_examples
1333 1000
1334 Optionally, you may define the environment variable to load a profile
1336 for you. This is the suggested pattern for running your tests on CI.
1337 The code below should run in a conftest.py or any setup/initialization
1338 section of your test suite. If this variable is not defined the
1339 Hypothesis defined defaults will be loaded.
1340 >>> import os
1342 >>> from hypothesis import settings, Verbosity
1343 >>> settings.register_profile("ci", max_examples=1000)
1344 >>> settings.register_profile("dev", max_examples=10)
1345 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1346 >>> settings.load_profile(os.getenv(u'HYPOTHESIS_PROFILE', 'default'))
1347 If you are using the hypothesis pytest plugin and your profiles are
1349 registered by your conftest you can load one with the command line
1350 option --hypothesis-profile.
1351 $ pytest tests --hypothesis-profile <profile-name>
1353
1355 Most things should be easy to generate and everything should be possi‐
1356 ble.
1357 To support this principle Hypothesis provides strategies for most
1359 built-in types with arguments to constrain or adjust the output, as
1360 well as higher-order strategies that can be composed to generate more
1361 complex types.
1362 This document is a guide to what strategies are available for generat‐
1364 ing data and how to build them. Strategies have a variety of other
1365 important internal features, such as how they simplify, but the data
1366 they can generate is the only public part of their API.
1367 Core strategies
1369 Functions for building strategies are all available in the hypothe‐
1370 sis.strategies module. The salient functions from it are as follows:
1371 hypothesis.strategies.binary(*, min_size=0, max_size=None)
1373 Generates python:bytes.
1374 The generated python:bytes will have a length of at least
1376 min_size and at most max_size. If max_size is None there is no
1377 upper limit.
1378 Examples from this strategy shrink towards smaller strings and
1380 lower byte values.
1381 hypothesis.strategies.booleans()
1383 Returns a strategy which generates instances of python:bool.
1384 Examples from this strategy will shrink towards False (i.e.
1386 shrinking will replace True with False where possible).
1387 hypothesis.strategies.builds(target, /, *args, **kwargs)
1389 Generates values by drawing from args and kwargs and passing
1390 them to the callable (provided as the first positional argument)
1391 in the appropriate argument position.
1392 e.g. builds(target, integers(), flag=booleans()) would draw an
1394 integer i and a boolean b and call target(i, flag=b).
1395 If the callable has type annotations, they will be used to infer
1397 a strategy for required arguments that were not passed to
1398 builds. You can also tell builds to infer a strategy for an
1399 optional argument by passing the special value hypothesis.infer
1400 as a keyword argument to builds, instead of a strategy for that
1401 argument to the callable.
1402 If the callable is a class defined with attrs, missing required
1404 arguments will be inferred from the attribute on a best-effort
1405 basis, e.g. by checking attrs standard validators. Dataclasses
1406 are handled natively by the inference from type hints.
1407 Examples from this strategy shrink by shrinking the argument
1409 values to the callable.
1410 hypothesis.strategies.characters(*, whitelist_categories=None, black‐
1412 list_categories=None, blacklist_characters=None, min_codepoint=None,
1413 max_codepoint=None, whitelist_characters=None)
1414 Generates characters, length-one python:strings, following spec‐
1415 ified filtering rules.
1416 · When no filtering rules are specified, any character can be
1418 produced.
1419 · If min_codepoint or max_codepoint is specified, then only
1421 characters having a codepoint in that range will be produced.
1422 · If whitelist_categories is specified, then only characters
1424 from those Unicode categories will be produced. This is a fur‐
1425 ther restriction, characters must also satisfy min_codepoint
1426 and max_codepoint.
1427 · If blacklist_categories is specified, then any character from
1429 those categories will not be produced. Any overlap between
1430 whitelist_categories and blacklist_categories will raise an
1431 exception, as each character can only belong to a single
1432 class.
1433 · If whitelist_characters is specified, then any additional
1435 characters in that list will also be produced.
1436 · If blacklist_characters is specified, then any characters in
1438 that list will be not be produced. Any overlap between
1439 whitelist_characters and blacklist_characters will raise an
1440 exception.
1441 The _codepoint arguments must be integers between zero and
1443 python:sys.maxunicode. The _characters arguments must be col‐
1444 lections of length-one unicode strings, such as a unicode
1445 string.
1446 The _categories arguments must be used to specify either the
1448 one-letter Unicode major category or the two-letter Unicode
1449 general category. For example, ('Nd', 'Lu') signifies "Number,
1450 decimal digit" and "Letter, uppercase". A single letter ('major
1451 category') can be given to match all corresponding categories,
1452 for example 'P' for characters in any punctuation category.
1453 Examples from this strategy shrink towards the codepoint for
1455 '0', or the first allowable codepoint after it if '0' is
1456 excluded.
1457 hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magni‐
1459 tude=None, allow_infinity=None, allow_nan=None)
1460 Returns a strategy that generates complex numbers.
1461 This strategy draws complex numbers with constrained magnitudes.
1463 The min_magnitude and max_magnitude parameters should be
1464 non-negative Real numbers; a value of None corresponds an infi‐
1465 nite upper bound.
1466 If min_magnitude is nonzero or max_magnitude is finite, it is an
1468 error to enable allow_nan. If max_magnitude is finite, it is an
1469 error to enable allow_infinity.
1470 The magnitude contraints are respected up to a relative error of
1472 (around) floating-point epsilon, due to implementation via the
1473 system sqrt function.
1474 Examples from this strategy shrink by shrinking their real and
1476 imaginary parts, as floats().
1477 If you need to generate complex numbers with particular real and
1479 imaginary parts or relationships between parts, consider using
1480 builds(complex, ...) or @composite respectively.
1481 hypothesis.strategies.composite(f)
1483 Defines a strategy that is built out of potentially arbitrarily
1484 many other strategies.
1485 This is intended to be used as a decorator. See the full docu‐
1487 mentation for more details about how to use this function.
1488 Examples from this strategy shrink by shrinking the output of
1490 each draw call.
1491 hypothesis.strategies.data()
1493 This isn't really a normal strategy, but instead gives you an
1494 object which can be used to draw data interactively from other
1495 strategies.
1496 See the rest of the documentation for more complete information.
1498 Examples from this strategy do not shrink (because there is only
1500 one), but the result of calls to each draw() call shrink as they
1501 normally would.
1502 class hypothesis.strategies.DataObject(data)
1504 This type only exists so that you can write type hints for tests
1505 using the data() strategy. Do not use it directly!
1506 hypothesis.strategies.dates(min_value=datetime.date.min,
1508 max_value=datetime.date.max)
1509 A strategy for dates between min_value and max_value.
1510 Examples from this strategy shrink towards January 1st 2000.
1512 hypothesis.strategies.datetimes(min_value=datetime.datetime.min,
1514 max_value=datetime.datetime.max, *, timezones=none(), allow_imagi‐
1515 nary=True)
1516 A strategy for generating datetimes, which may be time‐
1517 zone-aware.
1518 This strategy works by drawing a naive datetime between
1520 min_value and max_value, which must both be naive (have no time‐
1521 zone).
1522 timezones must be a strategy that generates either None, for
1524 naive datetimes, or tzinfo objects for 'aware' datetimes. You
1525 can construct your own, though we recommend using the dateutil
1526 package and hypothesis.extra.dateutil.timezones() strategy, and
1527 also provide hypothesis.extra.pytz.timezones().
1528 You may pass allow_imaginary=False to filter out "imaginary"
1530 datetimes which did not (or will not) occur due to daylight sav‐
1531 ings, leap seconds, timezone and calendar adjustments, etc.
1532 Imaginary datetimes are allowed by default, because malformed
1533 timestamps are a common source of bugs. Note that because pytz
1534 predates PEP 495, this does not work correctly with timezones
1535 that use a negative DST offset (such as "Europe/Dublin").
1536 Examples from this strategy shrink towards midnight on January
1538 1st 2000, local time.
1539 hypothesis.strategies.decimals(min_value=None, max_value=None, *,
1541 allow_nan=None, allow_infinity=None, places=None)
1542 Generates instances of python:decimal.Decimal, which may be:
1543 · A finite rational number, between min_value and max_value.
1545 · Not a Number, if allow_nan is True. None means "allow NaN,
1547 unless min_value and max_value are not None".
1548 · Positive or negative infinity, if max_value and min_value
1550 respectively are None, and allow_infinity is not False. None
1551 means "allow infinity, unless excluded by the min and max val‐
1552 ues".
1553 Note that where floats have one NaN value, Decimals have four:
1555 signed, and either quiet or signalling. See the decimal module
1556 docs for more information on special values.
1557 If places is not None, all finite values drawn from the strategy
1559 will have that number of digits after the decimal place.
1560 Examples from this strategy do not have a well defined shrink
1562 order but try to maximize human readability when shrinking.
1563 hypothesis.strategies.deferred(definition)
1565 A deferred strategy allows you to write a strategy that refer‐
1566 ences other strategies that have not yet been defined. This
1567 allows for the easy definition of recursive and mutually recur‐
1568 sive strategies.
1569 The definition argument should be a zero-argument function that
1571 returns a strategy. It will be evaluated the first time the
1572 strategy is used to produce an example.
1573 Example usage:
1575 >>> import hypothesis.strategies as st
1577 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1578 >>> x.example()
1579 (((False, (True, True)), (False, True)), (True, True))
1580 >>> x.example()
1581 True
1582 Mutual recursion also works fine:
1584 >>> a = st.deferred(lambda: st.booleans() | b)
1586 >>> b = st.deferred(lambda: st.tuples(a, a))
1587 >>> a.example()
1588 True
1589 >>> b.example()
1590 (False, (False, ((False, True), False)))
1591 Examples from this strategy shrink as they normally would from
1593 the strategy returned by the definition.
1594 hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class
1596 'dict'>, min_size=0, max_size=None)
1597 Generates dictionaries of type dict_class with keys drawn from
1598 the keys argument and values drawn from the values argument.
1599 The size parameters have the same interpretation as for lists().
1601 Examples from this strategy shrink by trying to remove keys from
1603 the generated dictionary, and by shrinking each generated key
1604 and value.
1605 hypothesis.strategies.emails()
1607 A strategy for generating email addresses as unicode strings.
1608 The address format is specified in RFC 5322#section-3.4.1. Val‐
1609 ues shrink towards shorter local-parts and host domains.
1610 This strategy is useful for generating "user data" for tests, as
1612 mishandling of email addresses is a common source of bugs.
1613 hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)
1615 Generates a dictionary of the same type as mapping with a fixed
1616 set of keys mapping to strategies. mapping must be a dict sub‐
1617 class.
1618 Generated values have all keys present in mapping, with the cor‐
1620 responding values drawn from mapping[key]. If mapping is an
1621 instance of OrderedDict the keys will also be in the same order,
1622 otherwise the order is arbitrary.
1623 If optional is passed, the generated value may or may not con‐
1625 tain each key from optional and a value drawn from the corre‐
1626 sponding strategy.
1627 Examples from this strategy shrink by shrinking each individual
1629 value in the generated dictionary, and omitting optional
1630 key-value pairs.
1631 hypothesis.strategies.floats(min_value=None, max_value=None, *,
1633 allow_nan=None, allow_infinity=None, width=64, exclude_min=False,
1634 exclude_max=False)
1635 Returns a strategy which generates floats.
1636 · If min_value is not None, all values will be >= min_value (or
1638 > min_value if exclude_min).
1639 · If max_value is not None, all values will be <= max_value (or
1641 < max_value if exclude_max).
1642 · If min_value or max_value is not None, it is an error to
1644 enable allow_nan.
1645 · If both min_value and max_value are not None, it is an error
1647 to enable allow_infinity.
1648 Where not explicitly ruled out by the bounds, all of infinity,
1650 -infinity and NaN are possible values generated by this strat‐
1651 egy.
1652 The width argument specifies the maximum number of bits of pre‐
1654 cision required to represent the generated float. Valid values
1655 are 16, 32, or 64. Passing width=32 will still use the builtin
1656 64-bit float class, but always for values which can be exactly
1657 represented as a 32-bit float.
1658 The exclude_min and exclude_max argument can be used to generate
1660 numbers from open or half-open intervals, by excluding the
1661 respective endpoints. Excluding either signed zero will also
1662 exclude the other. Attempting to exclude an endpoint which is
1663 None will raise an error; use allow_infinity=False to generate
1664 finite floats. You can however use e.g. min_value=-math.inf,
1665 exclude_min=True to exclude only one infinite endpoint.
1666 Examples from this strategy have a complicated and hard to
1668 explain shrinking behaviour, but it tries to improve "human
1669 readability". Finite numbers will be preferred to infinity and
1670 infinity will be preferred to NaN.
1671 hypothesis.strategies.fractions(min_value=None, max_value=None, *,
1673 max_denominator=None)
1674 Returns a strategy which generates Fractions.
1675 If min_value is not None then all generated values are no less
1677 than min_value. If max_value is not None then all generated
1678 values are no greater than max_value. min_value and max_value
1679 may be anything accepted by the Fraction constructor.
1680 If max_denominator is not None then the denominator of any gen‐
1682 erated values is no greater than max_denominator. Note that
1683 max_denominator must be None or a positive integer.
1684 Examples from this strategy shrink towards smaller denominators,
1686 then closer to zero.
1687 hypothesis.strategies.from_regex(regex, *, fullmatch=False)
1689 Generates strings that contain a match for the given regex (i.e.
1690 ones for which python:re.search() will return a non-None
1691 result).
1692 regex may be a pattern or compiled regex. Both byte-strings and
1694 unicode strings are supported, and will generate examples of the
1695 same type.
1696 You can use regex flags such as python:re.IGNORECASE or
1698 python:re.DOTALL to control generation. Flags can be passed
1699 either in compiled regex or inside the pattern with a (?iLmsux)
1700 group.
1701 Some regular expressions are only partly supported - the under‐
1703 lying strategy checks local matching and relies on filtering to
1704 resolve context-dependent expressions. Using too many of these
1705 constructs may cause health-check errors as too many examples
1706 are filtered out. This mainly includes (positive or negative)
1707 lookahead and lookbehind groups.
1708 If you want the generated string to match the whole regex you
1710 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1711 gle character string, while "." will return any string, and
1712 r"\A.$" will return a single character optionally followed by a
1713 "\n". Alternatively, passing fullmatch=True will ensure that
1714 the whole string is a match, as if you had used the \A and \Z
1715 markers.
1716 Examples from this strategy shrink towards shorter strings and
1718 lower character values, with exact behaviour that may depend on
1719 the pattern.
1720 hypothesis.strategies.from_type(thing)
1722 Looks up the appropriate search strategy for the given type.
1723 from_type is used internally to fill in missing arguments to
1725 builds() and can be used interactively to explore what strate‐
1726 gies are available or to debug type resolution.
1727 You can use register_type_strategy() to handle your custom
1729 types, or to globally redefine certain strategies - for example
1730 excluding NaN from floats, or use timezone-aware instead of
1731 naive time and datetime strategies.
1732 The resolution logic may be changed in a future version, but
1734 currently tries these five options:
1735 1. If thing is in the default lookup mapping or user-registered
1737 lookup, return the corresponding strategy. The default
1738 lookup covers all types with Hypothesis strategies, including
1739 extras where possible.
1740 2. If thing is from the python:typing module, return the corre‐
1742 sponding strategy (special logic).
1743 3. If thing has one or more subtypes in the merged lookup,
1745 return the union of the strategies for those types that are
1746 not subtypes of other elements in the lookup.
1747 4. Finally, if thing has type annotations for all required argu‐
1749 ments, and is not an abstract class, it is resolved via
1750 builds().
1751 5. Because abstract types cannot be instantiated, we treat
1753 abstract types as the union of their concrete subclasses.
1754 Note that this lookup works via inheritance but not via reg‐
1755 ister, so you may still need to use register_type_strategy().
1756 There is a valuable recipe for leveraging from_type() to gener‐
1758 ate "everything except" values from a specified type. I.e.
1759 def everything_except(excluded_types):
1761 return (
1762 from_type(type).flatmap(from_type)
1763 .filter(lambda x: not isinstance(x, excluded_types))
1764 )
1765 For example, everything_except(int) returns a strategy that can
1767 generate anything that from_type() can ever generate, except for
1768 instances of python:int, and excluding instances of types added
1769 via register_type_strategy().
1770 This is useful when writing tests which check that invalid input
1772 is rejected in a certain way.
1773 hypothesis.strategies.frozensets(elements, *, min_size=0,
1775 max_size=None)
1776 This is identical to the sets function but instead returns
1777 frozensets.
1778 hypothesis.strategies.functions(*, like=lambda : ..., returns=none(),
1780 pure=False)
1781 A strategy for functions, which can be used in callbacks.
1782 The generated functions will mimic the interface of like, which
1784 must be a callable (including a class, method, or function).
1785 The return value for the function is drawn from the returns
1786 argument, which must be a strategy.
1787 If pure=True, all arguments passed to the generated function
1789 must be hashable, and if passed identical arguments the original
1790 return value will be returned again - not regenerated, so beware
1791 mutable values.
1792 If pure=False, generated functions do not validate their argu‐
1794 ments, and may return a different value if called again with the
1795 same arguments.
1796 Generated functions can only be called within the scope of the
1798 @given which created them. This strategy does not support
1799 .example().
1800 hypothesis.strategies.integers(min_value=None, max_value=None)
1802 Returns a strategy which generates integers.
1803 If min_value is not None then all values will be >= min_value.
1805 If max_value is not None then all values will be <= max_value
1806 Examples from this strategy will shrink towards zero, and nega‐
1808 tive values will also shrink towards positive (i.e. -n may be
1809 replaced by +n).
1810 hypothesis.strategies.ip_addresses(*, v=None, network=None)
1812 Generate IP addresses - v=4 for IPv4Addresses, v=6 for
1813 IPv6Addresses, or leave unspecified to allow both versions.
1814 network may be an IPv4Network or IPv6Network, or a string repre‐
1816 senting a network such as "127.0.0.0/24" or "2001:db8::/32". As
1817 well as generating addresses within a particular routable net‐
1818 work, this can be used to generate addresses from a reserved
1819 range listed in the IANA registries.
1820 If you pass both v and network, they must be for the same ver‐
1822 sion.
1823 hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None,
1825 unique_by=None, unique=False)
1826 This has the same behaviour as lists, but returns iterables
1827 instead.
1828 Some iterables cannot be indexed (e.g. sets) and some do not
1830 have a fixed length (e.g. generators). This strategy produces
1831 iterators, which cannot be indexed and do not have a fixed
1832 length. This ensures that you do not accidentally depend on
1833 sequence behaviour.
1834 hypothesis.strategies.just(value)
1836 Return a strategy which only generates value.
1837 Note: value is not copied. Be wary of using mutable values.
1839 If value is the result of a callable, you can use
1841 builds(callable) instead of just(callable()) to get a fresh
1842 value each time.
1843 Examples from this strategy do not shrink (because there is only
1845 one).
1846 hypothesis.strategies.lists(elements, *, min_size=0, max_size=None,
1848 unique_by=None, unique=False)
1849 Returns a list containing values drawn from elements with length
1850 in the interval [min_size, max_size] (no bounds in that direc‐
1851 tion if these are None). If max_size is 0, only the empty list
1852 will be drawn.
1853 If unique is True (or something that evaluates to True), we com‐
1855 pare direct object equality, as if unique_by was lambda x: x.
1856 This comparison only works for hashable types.
1857 If unique_by is not None it must be a callable or tuple of
1859 callables returning a hashable type when given a value drawn
1860 from elements. The resulting list will satisfy the condition
1861 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1862 If unique_by is a tuple of callables the uniqueness will be
1864 respective to each callable.
1865 For example, the following will produce two columns of integers
1867 with both columns being unique respectively.
1868 >>> twoints = st.tuples(st.integers(), st.integers())
1870 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1871 Examples from this strategy shrink by trying to remove elements
1873 from the list, and by shrinking each individual element of the
1874 list.
1875 hypothesis.strategies.none()
1877 Return a strategy which only generates None.
1878 Examples from this strategy do not shrink (because there is only
1880 one).
1881 hypothesis.strategies.nothing()
1883 This strategy never successfully draws a value and will always
1884 reject on an attempt to draw.
1885 Examples from this strategy do not shrink (because there are
1887 none).
1888 hypothesis.strategies.one_of(*args)
1890 Return a strategy which generates values from any of the argu‐
1891 ment strategies.
1892 This may be called with one iterable argument instead of multi‐
1894 ple strategy arguments, in which case one_of(x) and one_of(*x)
1895 are equivalent.
1896 Examples from this strategy will generally shrink to ones that
1898 come from strategies earlier in the list, then shrink according
1899 to behaviour of the strategy that produced them. In order to get
1900 good shrinking behaviour, try to put simpler strategies first.
1901 e.g. one_of(none(), text()) is better than one_of(text(),
1902 none()).
1903 This is especially important when using recursive strategies.
1905 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1906 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1907 st.none()) will shrink very badly indeed.
1908 hypothesis.strategies.permutations(values)
1910 Return a strategy which returns permutations of the ordered col‐
1911 lection values.
1912 Examples from this strategy shrink by trying to become closer to
1914 the original order of values.
1915 hypothesis.strategies.random_module()
1917 The Hypothesis engine handles PRNG state for the stdlib and
1918 Numpy random modules internally, always seeding them to zero and
1919 restoring the previous state after the test.
1920 If having a fixed seed would unacceptably weaken your tests, and
1922 you cannot use a random.Random instance provided by randoms(),
1923 this strategy calls python:random.seed() with an arbitrary inte‐
1924 ger and passes you an opaque object whose repr displays the seed
1925 value for debugging. If numpy.random is available, that state
1926 is also managed.
1927 Examples from these strategy shrink to seeds closer to zero.
1929 hypothesis.strategies.randoms(*, note_method_calls=False, use_true_ran‐
1931 dom=None)
1932 Generates instances of random.Random. The generated Random
1933 instances are of a special HypothesisRandom subclass.
1934 · If note_method_calls is set to True, Hypothesis will print the
1936 randomly drawn values in any falsifying test case. This can be
1937 helpful for debugging the behaviour of randomized algorithms.
1938 · If use_true_random is set to True then values will be drawn
1940 from their usual distribution, otherwise they will actually be
1941 Hypothesis generated values (and will be shrunk accordingly
1942 for any failing test case). Setting use_true_random=False will
1943 tend to expose bugs that would occur with very low probability
1944 when it is set to True, and this flag should only be set to
1945 True when your code relies on the distribution of values for
1946 correctness.
1947 hypothesis.strategies.recursive(base, extend, *, max_leaves=100)
1949 base: A strategy to start from.
1950 extend: A function which takes a strategy and returns a new
1952 strategy.
1953 max_leaves: The maximum number of elements to be drawn from base
1955 on a given run.
1956 This returns a strategy S such that S = extend(base | S). That
1958 is, values may be drawn from base, or from any strategy reach‐
1959 able by mixing applications of | and extend.
1960 An example may clarify: recursive(booleans(), lists) would
1962 return a strategy that may return arbitrarily nested and mixed
1963 lists of booleans. So e.g. False, [True], [False, []], and
1964 [[[[True]]]] are all valid values to be drawn from that strat‐
1965 egy.
1966 Examples from this strategy shrink by trying to reduce the
1968 amount of recursion and by shrinking according to the shrinking
1969 behaviour of base and the result of extend.
1970 hypothesis.strategies.register_type_strategy(custom_type, strategy)
1972 Add an entry to the global type-to-strategy lookup.
1973 This lookup is used in builds() and @given.
1975 builds() will be used automatically for classes with type anno‐
1977 tations on __init__ , so you only need to register a strategy if
1978 one or more arguments need to be more tightly defined than their
1979 type-based default, or if you want to supply a strategy for an
1980 argument with a default value.
1981 strategy may be a search strategy, or a function that takes a
1983 type and returns a strategy (useful for generic types).
1984 Note that you may not register a parametrised generic type (such
1986 as MyCollection[int]) directly, because the resolution logic
1987 does not handle this case correctly. Instead, you may register
1988 a function for MyCollection and inspect the type parameters
1989 within that function.
1990 hypothesis.strategies.runner(*, default=not_set)
1992 A strategy for getting "the current test runner", whatever that
1993 may be. The exact meaning depends on the entry point, but it
1994 will usually be the associated 'self' value for it.
1995 If there is no current test runner and a default is provided,
1997 return that default. If no default is provided, raises Invali‐
1998 dArgument.
1999 Examples from this strategy do not shrink (because there is only
2001 one).
2002 hypothesis.strategies.sampled_from(elements)
2004 Returns a strategy which generates any value present in ele‐
2005 ments.
2006 Note that as with just(), values will not be copied and thus you
2008 should be careful of using mutable data.
2009 sampled_from supports ordered collections, as well as Enum
2011 objects. Flag objects may also generate any combination of
2012 their members.
2013 Examples from this strategy shrink by replacing them with values
2015 earlier in the list. So e.g. sampled_from([10, 1]) will shrink
2016 by trying to replace 1 values with 10, and sampled_from([1, 10])
2017 will shrink by trying to replace 10 values with 1.
2018 hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)
2020 This has the same behaviour as lists, but returns sets instead.
2021 Note that Hypothesis cannot tell if values are drawn from ele‐
2023 ments are hashable until running the test, so you can define a
2024 strategy for sets of an unhashable type but it will fail at test
2025 time.
2026 Examples from this strategy shrink by trying to remove elements
2028 from the set, and by shrinking each individual element of the
2029 set.
2030 hypothesis.strategies.shared(base, *, key=None)
2032 Returns a strategy that draws a single shared value per run,
2033 drawn from base. Any two shared instances with the same key will
2034 share the same value, otherwise the identity of this strategy
2035 will be used. That is:
2036 >>> s = integers() # or any other strategy
2038 >>> x = shared(s)
2039 >>> y = shared(s)
2040 In the above x and y may draw different (or potentially the
2042 same) values. In the following they will always draw the same:
2043 >>> x = shared(s, key="hi")
2045 >>> y = shared(s, key="hi")
2046 Examples from this strategy shrink as per their base strategy.
2048 hypothesis.strategies.slices(size)
2050 Generates slices that will select indices up to the supplied
2051 size
2052 Generated slices will have start and stop indices that range
2054 from -size to size - 1 and will step in the appropriate direc‐
2055 tion. Slices should only produce an empty selection if the start
2056 and end are the same.
2057 Examples from this strategy shrink toward 0 and smaller values
2059 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
2061 gories='Cs'), *, min_size=0, max_size=None)
2062 Generates strings with characters drawn from alphabet, which
2063 should be a collection of length one strings or a strategy gen‐
2064 erating such strings.
2065 The default alphabet strategy can generate the full unicode
2067 range but excludes surrogate characters because they are invalid
2068 in the UTF-8 encoding. You can use characters() without argu‐
2069 ments to find surrogate-related bugs such as bpo-34454.
2070 min_size and max_size have the usual interpretations. Note that
2072 Python measures string length by counting codepoints: U+00C5 Å
2073 is a single character, while U+0041 U+030A Å is two - the A,
2074 and a combining ring above.
2075 Examples from this strategy shrink towards shorter strings, and
2077 with the characters in the text shrinking as per the alphabet
2078 strategy. This strategy does not normalize() examples, so gen‐
2079 erated strings may be in any or none of the 'normal forms'.
2080 hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min,
2082 max_value=datetime.timedelta.max)
2083 A strategy for timedeltas between min_value and max_value.
2084 Examples from this strategy shrink towards zero.
2086 hypothesis.strategies.times(min_value=datetime.time.min,
2088 max_value=datetime.time.max, *, timezones=none())
2089 A strategy for times between min_value and max_value.
2090 The timezones argument is handled as for datetimes().
2092 Examples from this strategy shrink towards midnight, with the
2094 timezone component shrinking as for the strategy that provided
2095 it.
2096 hypothesis.strategies.timezone_keys(*, allow_prefix=True)
2098 A strategy for IANA timezone names.
2099 As well as timezone names like "UTC", "Australia/Sydney", or
2101 "America/New_York", this strategy can generate:
2102 · Aliases such as "Antarctica/McMurdo", which links to
2104 "Pacific/Auckland".
2105 · Deprecated names such as "Antarctica/South_Pole", which also
2107 links to "Pacific/Auckland". Note that most but not all dep‐
2108 recated timezone names are also aliases.
2109 · Timezone names with the "posix/" or "right/" prefixes, unless
2111 allow_prefix=False.
2112 These strings are provided separately from Tzinfo objects - such
2114 as ZoneInfo instances from the timezones() strategy - to facili‐
2115 tate testing of timezone logic without needing workarounds to
2116 access non-canonical names.
2117 NOTE:
2119 The python:zoneinfo module is new in Python 3.9, so you will
2120 need to install the backports.zoneinfo module on earlier ver‐
2121 sions, and the importlib_resources backport on Python 3.6.
2122 pip install hypothesis[zoneinfo] will install these condi‐
2124 tional dependencies if and only if they are needed.
2125 On Windows, you may need to access IANA timezone data via the
2127 tzdata package. For non-IANA timezones, such as Windows-native
2128 names or GNU TZ strings, we recommend using sampled_from() with
2129 the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().
2130 hypothesis.strategies.timezones(*, no_cache=False)
2132 A strategy for python:zoneinfo.ZoneInfo objects.
2133 If no_cache=True, the generated instances are constructed using
2135 ZoneInfo.no_cache instead of the usual constructor. This may
2136 change the semantics of your datetimes in surprising ways, so
2137 only use it if you know that you need to!
2138 NOTE:
2140 The python:zoneinfo module is new in Python 3.9, so you will
2141 need to install the backports.zoneinfo module on earlier ver‐
2142 sions, and the importlib_resources backport on Python 3.6.
2143 pip install hypothesis[zoneinfo] will install these condi‐
2145 tional dependencies if and only if they are needed.
2146 hypothesis.strategies.tuples(*args)
2148 Return a strategy which generates a tuple of the same length as
2149 args by generating the value at index i from args[i].
2150 e.g. tuples(integers(), integers()) would generate a tuple of
2152 length two with both values an integer.
2153 Examples from this strategy shrink by shrinking their component
2155 parts.
2156 hypothesis.strategies.uuids(*, version=None)
2158 Returns a strategy that generates UUIDs.
2159 If the optional version argument is given, value is passed
2161 through to UUID and only UUIDs of that version will be gener‐
2162 ated.
2163 All returned values from this will be unique, so e.g. if you do
2165 lists(uuids()) the resulting list will never contain duplicates.
2166 Examples from this strategy don't have any meaningful shrink
2168 order.
2169 Provisional strategies
2171 This module contains various provisional APIs and strategies.
2172 It is intended for internal use, to ease code reuse, and is not stable.
2174 Point releases may move or break the contents at any time!
2175 Internet strategies should conform to RFC 3986 or the authoritative
2177 definitions it links to. If not, report the bug!
2178 hypothesis.provisional.domains(*, max_length=255, max_ele‐
2180 ment_length=63)
2181 Generate RFC 1035 compliant fully qualified domain names.
2182 hypothesis.provisional.urls()
2184 A strategy for RFC 3986, generating http/https URLs.
2185 Shrinking
2187 When using strategies it is worth thinking about how the data shrinks.
2188 Shrinking is the process by which Hypothesis tries to produce human
2189 readable examples when it finds a failure - it takes a complex example
2190 and turns it into a simpler one.
2191 Each strategy defines an order in which it shrinks - you won't usually
2193 need to care about this much, but it can be worth being aware of as it
2194 can affect what the best way to write your own strategies is.
2195 The exact shrinking behaviour is not a guaranteed part of the API, but
2197 it doesn't change that often and when it does it's usually because we
2198 think the new way produces nicer examples.
2199 Possibly the most important one to be aware of is one_of(), which has a
2201 preference for values produced by strategies earlier in its argument
2202 list. Most of the others should largely "do the right thing" without
2203 you having to think about it.
2204 Adapting strategies
2206 Often it is the case that a strategy doesn't produce exactly what you
2207 want it to and you need to adapt it. Sometimes you can do this in the
2208 test, but this hurts reuse because you then have to repeat the adaption
2209 in every test.
2210 Hypothesis gives you ways to build strategies from other strategies
2212 given functions for transforming the data.
2213 Mapping
2215 map is probably the easiest and most useful of these to use. If you
2216 have a strategy s and a function f, then an example s.map(f).example()
2217 is f(s.example()), i.e. we draw an example from s and then apply f to
2218 it.
2219 e.g.:
2221 >>> lists(integers()).map(sorted).example()
2223 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2224 Note that many things that you might use mapping for can also be done
2226 with builds().
2227 Filtering
2229 filter lets you reject some examples. s.filter(f).example() is some
2230 example of s such that f(example) is truthy.
2231 >>> integers().filter(lambda x: x > 11).example()
2233 26126
2234 >>> integers().filter(lambda x: x > 11).example()
2235 23324
2236 It's important to note that filter isn't magic and if your condition is
2238 too hard to satisfy then this can fail:
2239 >>> integers().filter(lambda x: False).example()
2241 Traceback (most recent call last):
2242 ...
2243 hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries
2244 In general you should try to use filter only to avoid corner cases that
2246 you don't want rather than attempting to cut out a large chunk of the
2247 search space.
2248 A technique that often works well here is to use map to first transform
2250 the data and then use filter to remove things that didn't work out. So
2251 for example if you wanted pairs of integers (x,y) such that x < y you
2252 could do the following:
2253 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2255 [-8543729478746591815, 3760495307320535691]
2256 Chaining strategies together
2258 Finally there is flatmap. flatmap draws an example, then turns that
2259 example into a strategy, then draws an example from that strategy.
2260 It may not be obvious why you want this at first, but it turns out to
2262 be quite useful because it lets you generate different types of data
2263 with relationships to each other.
2264 For example suppose we wanted to generate a list of lists of the same
2266 length:
2267 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2269 ... lambda n: lists(lists(integers(), min_size=n, max_size=n)))
2270 >>> rectangle_lists.example()
2271 []
2272 >>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
2273 [[], [], [], [], [], [], [], [], [], []]
2274 >>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
2275 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2276 >>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
2277 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2278 In this example we first choose a length for our tuples, then we build
2280 a strategy which generates lists containing lists precisely of that
2281 length. The finds show what simple examples for this look like.
2282 Most of the time you probably don't want flatmap, but unlike filter and
2284 map which are just conveniences for things you could just do in your
2285 tests, flatmap allows genuinely new data generation that you wouldn't
2286 otherwise be able to easily do.
2287 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2289 don't know Haskell, ignore everything in these parentheses. You do not
2290 need to understand anything about monads to use this, or anything else
2291 in Hypothesis).
2292 Recursive data
2294 Sometimes the data you want to generate has a recursive definition.
2295 e.g. if you wanted to generate JSON data, valid JSON is:
2296 1. Any float, any boolean, any unicode string.
2298 2. Any list of valid JSON data
2300 3. Any dictionary mapping unicode strings to valid JSON data.
2302 The problem is that you cannot call a strategy recursively and expect
2304 it to not just blow up and eat all your memory. The other problem here
2305 is that not all unicode strings display consistently on different
2306 machines, so we'll restrict them in our doctest.
2307 The way Hypothesis handles this is with the recursive() strategy which
2309 you pass in a base case and a function that, given a strategy for your
2310 data type, returns a new strategy for it. So for example:
2311 >>> from string import printable; from pprint import pprint
2313 >>> json = recursive(none() | booleans() | floats() | text(printable),
2314 ... lambda children: lists(children, 1) | dictionaries(text(printable), children, min_size=1))
2315 >>> pprint(json.example())
2316 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2317 >>> pprint(json.example())
2318 {'de(l': None,
2319 'nK': {'(Rt)': None,
2320 '+hoZh1YU]gy8': True,
2321 '8z]EIFA06^li^': 'LFE{Q',
2322 '9,': 'l{cA=/'}}
2323 That is, we start with our leaf data and then we augment it by allowing
2325 lists and dictionaries of anything we can generate as JSON data.
2326 The size control of this works by limiting the maximum number of values
2328 that can be drawn from the base strategy. So for example if we wanted
2329 to only generate really small JSON we could do this as:
2330 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2332 >>> small_lists.example()
2333 True
2334 >>> small_lists.example()
2335 [False]
2336 Composite strategies
2338 The @composite decorator lets you combine other strategies in more or
2339 less arbitrary ways. It's probably the main thing you'll want to use
2340 for complicated custom strategies.
2341 The composite decorator works by converting a function that returns one
2343 example into a function that returns a strategy that produces such
2344 examples - which you can pass to @given, modify with .map or .filter,
2345 and generally use like any other strategy.
2346 It does this by giving you a special function draw as the first argu‐
2348 ment, which can be used just like the corresponding method of the
2349 data() strategy within a test. In fact, the implementation is almost
2350 the same - but defining a strategy with @composite makes code reuse
2351 easier, and usually improves the display of failing examples.
2352 For example, the following gives you a list and an index into it:
2354 >>> @composite
2356 ... def list_and_index(draw, elements=integers()):
2357 ... xs = draw(lists(elements, min_size=1))
2358 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2359 ... return (xs, i)
2360 draw(s) is a function that should be thought of as returning s.exam‐
2362 ple(), except that the result is reproducible and will minimize cor‐
2363 rectly. The decorated function has the initial argument removed from
2364 the list, but will accept all the others in the expected order.
2365 Defaults are preserved.
2366 >>> list_and_index()
2368 list_and_index()
2369 >>> list_and_index().example()
2370 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2371 >>> list_and_index(booleans())
2373 list_and_index(elements=booleans())
2374 >>> list_and_index(booleans()).example()
2375 ([True, False], 0)
2376 Note that the repr will work exactly like it does for all the built-in
2378 strategies: it will be a function that you can call to get the strategy
2379 in question, with values provided only if they do not match the
2380 defaults.
2381 You can use assume inside composite functions:
2383 @composite
2385 def distinct_strings_with_common_characters(draw):
2386 x = draw(text(min_size=1))
2387 y = draw(text(alphabet=x))
2388 assume(x != y)
2389 return (x, y)
2390 This works as assume normally would, filtering out any examples for
2392 which the passed in argument is falsey.
2393 Take care that your function can cope with adversarial draws, or
2395 explicitly rejects them using the .filter() method or assume() - our
2396 mutation and shrinking logic can do some strange things, and a naive
2397 implementation might lead to serious performance problems. For exam‐
2398 ple:
2399 @composite
2401 def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
2402 # The bad way: if Hypothesis keeps generating e.g. zero,
2403 # we'll keep looping for a very long time.
2404 result = set()
2405 while len(result) < size:
2406 result.add(draw(elements))
2407 # The good way: use a filter, so Hypothesis can tell what's valid!
2408 for _ in range(size):
2409 result.add(draw(elements.filter(lambda x: x not in result)))
2410 return result
2411 If @composite is used to decorate a method or classmethod, the draw
2413 argument must come before self or cls. While we therefore recommend
2414 writing strategies as standalone functions and using the
2415 register_type_strategy() function to associate them with a class, meth‐
2416 ods are supported and the @composite decorator may be applied either
2417 before or after @classmethod or @staticmethod. See issue #2578 and
2418 pull request #2634 for more details.
2419 Drawing interactively in tests
2421 There is also the data() strategy, which gives you a means of using
2422 strategies interactively. Rather than having to specify everything up
2423 front in @given you can draw from strategies in the body of your test.
2424 This is similar to @composite, but even more powerful as it allows you
2426 to mix test code with example generation. The downside of this power
2427 is that data() is incompatible with explicit @example(...)s - and the
2428 mixed code is often harder to debug when something goes wrong.
2429 If you need values that are affected by previous draws but which don't
2431 depend on the execution of your test, stick to the simpler @composite.
2432 @given(data())
2434 def test_draw_sequentially(data):
2435 x = data.draw(integers())
2436 y = data.draw(integers(min_value=x))
2437 assert x < y
2438 If the test fails, each draw will be printed with the falsifying exam‐
2440 ple. e.g. the above is wrong (it has a boundary condition error), so
2441 will print:
2442 Falsifying example: test_draw_sequentially(data=data(...))
2444 Draw 1: 0
2445 Draw 2: 0
2446 As you can see, data drawn this way is simplified as usual.
2448 Optionally, you can provide a label to identify values generated by
2450 each call to data.draw(). These labels can be used to identify values
2451 in the output of a falsifying example.
2452 For instance:
2454 @given(data())
2456 def test_draw_sequentially(data):
2457 x = data.draw(integers(), label="First number")
2458 y = data.draw(integers(min_value=x), label="Second number")
2459 assert x < y
2460 will produce the output:
2462 Falsifying example: test_draw_sequentially(data=data(...))
2464 Draw 1 (First number): 0
2465 Draw 2 (Second number): 0
2466
2468 Hypothesis has minimal dependencies, to maximise compatibility and make
2469 installing Hypothesis as easy as possible.
2470 Our integrations with specific packages are therefore provided by extra
2472 modules that need their individual dependencies installed in order to
2473 work. You can install these dependencies using the setuptools extra
2474 feature as e.g. pip install hypothesis[django]. This will check
2475 installation of compatible versions.
2476 You can also just install hypothesis into a project using them, ignore
2478 the version constraints, and hope for the best.
2479 In general "Which version is Hypothesis compatible with?" is a hard
2481 question to answer and even harder to regularly test. Hypothesis is
2482 always tested against the latest compatible version and each package
2483 will note the expected compatibility range. If you run into a bug with
2484 any of these please specify the dependency version.
2485 There are separate pages for django and numpy.
2487 hypothesis[cli]
2489 $ hypothesis --help
2490 Usage: hypothesis [OPTIONS] COMMAND [ARGS]...
2491 Options:
2493 --version Show the version and exit.
2494 -h, --help Show this message and exit.
2495 Commands:
2497 fuzz [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
2498 write `hypothesis write` writes property-based tests for you!
2499 This module requires the click package, and provides Hypothesis' com‐
2501 mand-line interface, for e.g. 'ghostwriting' tests via the terminal.
2502 It's also where HypoFuzz adds the hypothesis fuzz command (learn more
2503 about that here).
2504 hypothesis[lark]
2506 This extra can be used to generate strings matching any context-free
2507 grammar, using the Lark parser library.
2508 It currently only supports Lark's native EBNF syntax, but we plan to
2510 extend this to support other common syntaxes such as ANTLR and RFC 5234
2511 ABNF. Lark already supports loading grammars from nearley.js, so you
2512 may not have to write your own at all.
2513 Note that as Lark is at version 0.x, this module may break API compati‐
2515 bility in minor releases if supporting the latest version of Lark would
2516 otherwise be infeasible. We may also be quite aggressive in bumping
2517 the minimum version of Lark, unless someone volunteers to either fund
2518 or do the maintainence.
2519 hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None)
2521 A strategy for strings accepted by the given context-free gram‐
2522 mar.
2523 grammar must be a Lark object, which wraps an EBNF specifica‐
2525 tion. The Lark EBNF grammar reference can be found here.
2526 from_lark will automatically generate strings matching the non‐
2528 terminal start symbol in the grammar, which was supplied as an
2529 argument to the Lark class. To generate strings matching a dif‐
2530 ferent symbol, including terminals, you can override this by
2531 passing the start argument to from_lark. Note that Lark may
2532 remove unreachable productions when the grammar is compiled, so
2533 you should probably pass the same value for start to both.
2534 Currently from_lark does not support grammars that need custom
2536 lexing. Any lexers will be ignored, and any undefined terminals
2537 from the use of %declare will result in generation errors. To
2538 define strategies for such terminals, pass a dictionary mapping
2539 their name to a corresponding strategy as the explicit argument.
2540 The hypothesmith project includes a strategy for Python source,
2542 based on a grammar and careful post-processing.
2543 Example grammars, which may provide a useful starting point for your
2545 tests, can be found in the Lark repository and in this third-party col‐
2546 lection.
2547 hypothesis[pytz]
2549 This module provides pytz timezones.
2550 You can use this strategy to make hypothesis.strategies.datetimes() and
2552 hypothesis.strategies.times() produce timezone-aware values.
2553 hypothesis.extra.pytz.timezones()
2555 Any timezone in the Olsen database, as a pytz tzinfo object.
2556 This strategy minimises to UTC, or the smallest possible fixed
2558 offset, and is designed for use with hypothesis.strategies.date‐
2559 times().
2560 hypothesis[dateutil]
2562 This module provides dateutil timezones.
2563 You can use this strategy to make datetimes() and times() produce time‐
2565 zone-aware values.
2566 hypothesis.extra.dateutil.timezones()
2568 Any timezone from dateutil.
2569 This strategy minimises to UTC, or the timezone with the small‐
2571 est offset from UTC as of 2000-01-01, and is designed for use
2572 with datetimes().
2573 Note that the timezones generated by the strategy may vary
2575 depending on the configuration of your machine. See the dateutil
2576 documentation for more information.
2577
2579 Writing tests with Hypothesis frees you from the tedium of deciding on
2580 and writing out specific inputs to test. Now, the hypothe‐
2581 sis.extra.ghostwriter module can write your test functions for you too!
2582 The idea is to provide an easy way to start property-based testing, and
2584 a seamless transition to more complex test code - because ghostwritten
2585 tests are source code that you could have written for yourself.
2586 So just pick a function you'd like tested, and feed it to one of the
2588 functions below. They follow imports, use but do not require type
2589 annotations, and generally do their best to write you a useful test.
2590 You can also use our command-line interface:
2591 $ hypothesis write --help
2593 Usage: hypothesis write [OPTIONS] FUNC...
2594 `hypothesis write` writes property-based tests for you!
2596 Type annotations are helpful but not required for our advanced
2598 introspection and templating logic. Try running the examples below to see
2599 how it works:
2600 hypothesis write gzip
2602 hypothesis write numpy.matmul
2603 hypothesis write re.compile --except re.error
2604 hypothesis write --equivalent ast.literal_eval eval
2605 hypothesis write --roundtrip json.dumps json.loads
2606 hypothesis write --style=unittest --idempotent sorted
2607 hypothesis write --binary-op operator.add
2608 Options:
2610 --roundtrip start by testing write/read or encode/decode!
2611 --equivalent very useful when optimising or refactoring code
2612 --idempotent
2613 --binary-op
2614 --style [pytest|unittest] pytest-style function, or unittest-style method?
2615 -e, --except OBJ_NAME dotted name of exception(s) to ignore
2616 -h, --help Show this message and exit.
2617 NOTE:
2619 The ghostwriter requires black, but the generated code only requires
2620 Hypothesis itself.
2621 NOTE:
2623 Legal questions? While the ghostwriter fragments and logic is under
2624 the MPL-2.0 license like the rest of Hypothesis, the output from the
2625 ghostwriter is made available under the Creative Commons Zero (CC0)
2626 public domain dedication, so you can use it without any restric‐
2627 tions.
2628 hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(),
2630 style='pytest')
2631 Guess which ghostwriters to use, for a module or collection of
2632 functions.
2633 As for all ghostwriters, the except_ argument should be an
2635 python:Exception or tuple of exceptions, and style may be either
2636 "pytest" to write test functions or "unittest" to write test
2637 methods and TestCase.
2638 After finding the public functions attached to any modules, the
2640 magic ghostwriter looks for pairs of functions to pass to
2641 roundtrip(), then checks for binary_operation() and ufunc()
2642 functions, and any others are passed to fuzz().
2643 For example, try hypothesis write gzip on the command line!
2645 hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest')
2647 Write source code for a property-based test of func.
2648 The resulting test checks that valid input only leads to
2650 expected exceptions. For example:
2651 from re import compile, error
2653 from hypothesis.extra import ghostwriter
2654 ghostwriter.fuzz(compile, except_=error)
2656 Gives:
2658 # This test code was written by the `hypothesis.extra.ghostwriter` module
2660 # and is provided under the Creative Commons Zero public domain dedication.
2661 import re
2662 from hypothesis import given, reject, strategies as st
2663 # TODO: replace st.nothing() with an appropriate strategy
2665 @given(pattern=st.nothing(), flags=st.just(0))
2667 def test_fuzz_compile(pattern, flags):
2668 try:
2669 re.compile(pattern=pattern, flags=flags)
2670 except re.error:
2671 reject()
2672 Note that it includes all the required imports. Because the
2674 pattern parameter doesn't have annotations or a default argu‐
2675 ment, you'll need to specify a strategy - for example text() or
2676 binary(). After that, you have a test!
2677 hypothesis.extra.ghostwriter.idempotent(func, *, except_=(),
2679 style='pytest')
2680 Write source code for a property-based test of func.
2681 The resulting test checks that if you call func on it's own out‐
2683 put, the result does not change. For example:
2684 from typing import Sequence
2686 from hypothesis.extra import ghostwriter
2687 def timsort(seq: Sequence[int]) -> Sequence[int]:
2689 return sorted(seq)
2690 ghostwriter.idempotent(timsort)
2692 Gives:
2694 # This test code was written by the `hypothesis.extra.ghostwriter` module
2696 # and is provided under the Creative Commons Zero public domain dedication.
2697 from hypothesis import given, strategies as st
2699 @given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
2701 def test_idempotent_timsort(seq):
2702 result = timsort(seq=seq)
2703 repeat = timsort(seq=result)
2704 assert result == repeat, (result, repeat)
2705 hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(),
2707 style='pytest')
2708 Write source code for a property-based test of funcs.
2709 The resulting test checks that if you call the first function,
2711 pass the result to the second (and so on), the final result is
2712 equal to the first input argument.
2713 This is a very powerful property to test, especially when the
2715 config options are varied along with the object to round-trip.
2716 For example, try ghostwriting a test for python:json.dumps() -
2717 would you have thought of all that?
2718 hypothesis write --roundtrip json.dumps json.loads
2720 hypothesis.extra.ghostwriter.equivalent(*funcs, except_=(),
2722 style='pytest')
2723 Write source code for a property-based test of funcs.
2724 The resulting test checks that calling each of the functions
2726 gives the same result. This can be used as a classic 'oracle',
2727 such as testing a fast sorting algorithm against the
2728 python:sorted() builtin, or for differential testing where none
2729 of the compared functions are fully trusted but any difference
2730 indicates a bug (e.g. running a function on different numbers of
2731 threads, or simply multiple times).
2732 The functions should have reasonably similar signatures, as only
2734 the common parameters will be passed the same arguments - any
2735 other parameters will be allowed to vary.
2736 hypothesis.extra.ghostwriter.binary_operation(func, *, associa‐
2738 tive=True, commutative=True, identity=infer, distributes_over=None,
2739 except_=(), style='pytest')
2740 Write property tests for the binary operation func.
2741 While binary operations are not particularly common, they have
2743 such nice properties to test that it seems a shame not to demon‐
2744 strate them with a ghostwriter. For an operator f, test that:
2745 · if associative, f(a, f(b, c)) == f(f(a, b), c)
2747 · if commutative, f(a, b) == f(b, a)
2749 · if identity is not None, f(a, identity) == a
2751 · if distributes_over is +, f(a, b) + f(a, c) == f(a, b+c)
2753 For example:
2755 ghostwriter.binary_operation(
2757 operator.mul,
2758 identity=1,
2759 distributes_over=operator.add,
2760 style="unittest",
2761 )
2762 hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest')
2764 Write a property-based test for the array unfunc func.
2765 The resulting test checks that your ufunc or gufunc has the
2767 expected broadcasting and dtype casting behaviour. You will
2768 probably want to add extra assertions, but as with the other
2769 ghostwriters this gives you a great place to start.
2770 hypothesis write numpy.matmul
2772
2774 Hypothesis offers a number of features specific for Django testing,
2775 available in the hypothesis[django] extra. This is tested against each
2776 supported series with mainstream or extended support - if you're still
2777 getting security patches, you can test with Hypothesis.
2778 class hypothesis.extra.django.TestCase
2780 Using it is quite straightforward: All you need to do is subclass
2782 hypothesis.extra.django.TestCase or
2783 hypothesis.extra.django.TransactionTestCase and you can use @given as
2784 normal, and the transactions will be per example rather than per test
2785 function as they would be if you used @given with a normal django test
2786 suite (this is important because your test function will be called mul‐
2787 tiple times and you don't want them to interfere with each other). Test
2788 cases on these classes that do not use @given will be run as normal.
2789 class hypothesis.extra.django.TransactionTestCase
2791 We recommend avoiding TransactionTestCase unless you really have to run
2793 each test case in a database transaction. Because Hypothesis runs this
2794 in a loop, the performance problems it normally has are significantly
2795 exacerbated and your tests will be really slow. If you are using
2796 TransactionTestCase, you may need to use @settings(sup‐
2797 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2798 example generation.
2799 Having set up a test class, you can now pass @given a strategy for
2801 Django models:
2802 For example, using the trivial django project we have for testing:
2804 >>> from hypothesis.extra.django import from_model
2806 >>> from toystore.models import Customer
2807 >>> c = from_model(Customer).example()
2808 >>> c
2809 <Customer: Customer object>
2810 >>> c.email
2811 'jaime.urbina@gmail.com'
2812 >>> c.name
2813 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2814 >>> c.age
2815 -873375803
2816 Hypothesis has just created this with whatever the relevant type of
2818 data is.
2819 Obviously the customer's age is implausible, which is only possible
2821 because we have not used (eg) MinValueValidator to set the valid range
2822 for this field (or used a PositiveSmallIntegerField, which would only
2823 need a maximum value validator).
2824 If you do have validators attached, Hypothesis will only generate exam‐
2826 ples that pass validation. Sometimes that will mean that we fail a
2827 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2828 egy to skip validation at the strategy level:
2829 NOTE:
2831 Inference from validators will be much more powerful when issue
2832 #1116 is implemented, but there will always be some edge cases that
2833 require you to pass an explicit strategy.
2834 >>> from hypothesis.strategies import integers
2836 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2837 >>> c
2838 <Customer: Customer object>
2839 >>> c.age
2840 5
2841 Tips and tricks
2843 Custom field types
2844 If you have a custom Django field type you can register it with Hypoth‐
2845 esis's model deriving functionality by registering a default strategy
2846 for it:
2847 >>> from toystore.models import CustomishField, Customish
2849 >>> from_model(Customish).example()
2850 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2851 customish for model Customish
2852 >>> from hypothesis.extra.django import register_field_strategy
2853 >>> from hypothesis.strategies import just
2854 >>> register_field_strategy(CustomishField, just("hi"))
2855 >>> x = from_model(Customish).example()
2856 >>> x.customish
2857 'hi'
2858 Note that this mapping is on exact type. Subtypes will not inherit it.
2860 Generating child models
2862 For the moment there's no explicit support in hypothesis-django for
2863 generating dependent models. i.e. a Company model will generate no
2864 Shops. However if you want to generate some dependent models as well,
2865 you can emulate this by using the flatmap function as follows:
2866 from hypothesis.strategies import lists, just
2868 def generate_with_shops(company):
2871 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
2872 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
2875 Let's unpack what this is doing:
2877 The way flatmap works is that we draw a value from the original strat‐
2879 egy, then apply a function to it which gives us a new strategy. We then
2880 draw a value from that strategy. So in this case we're first drawing a
2881 company, and then we're drawing a list of shops belonging to that com‐
2882 pany: The just strategy is a strategy such that drawing it always pro‐
2883 duces the individual value, so from_model(Shop, company=just(company))
2884 is a strategy that generates a Shop belonging to the original company.
2885 So the following code would give us a list of shops all belonging to
2887 the same company:
2888 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
2890 The only difference from this and the above is that we want the com‐
2892 pany, not the shops. This is where the inner map comes in. We build the
2893 list of shops and then throw it away, instead returning the company we
2894 started for. This works because the models that Hypothesis generates
2895 are saved in the database, so we're essentially running the inner
2896 strategy purely for the side effect of creating those children in the
2897 database.
2898 Generating primary key values
2900 If your model includes a custom primary key that you want to generate
2901 using a strategy (rather than a default auto-increment primary key)
2902 then Hypothesis has to deal with the possibility of a duplicate primary
2903 key.
2904 If a model strategy generates a value for the primary key field,
2906 Hypothesis will create the model instance with update_or_create(),
2907 overwriting any existing instance in the database for this test case
2908 with the same primary key.
2909 On the subject of MultiValueField
2911 Django forms feature the MultiValueField which allows for several
2912 fields to be combined under a single named field, the default example
2913 of this is the SplitDateTimeField.
2914 class CustomerForm(forms.Form):
2916 name = forms.CharField()
2917 birth_date_time = forms.SplitDateTimeField()
2918 from_form supports MultiValueField subclasses directly, however if you
2920 want to define your own strategy be forewarned that Django binds data
2921 for a MultiValueField in a peculiar way. Specifically each sub-field is
2922 expected to have its own entry in data addressed by the field name
2923 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
2924 ValueField, so form data for the example above might look like this:
2925 {
2927 "name": "Samuel John",
2928 "birth_date_time_0": "2018-05-19", # the date, as the first sub-field
2929 "birth_date_time_1": "15:18:00", # the time, as the second sub-field
2930 }
2931 Thus, if you want to define your own strategies for such a field you
2933 must address your sub-fields appropriately:
2934 from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))
2936
2938 numpy
2939 Hypothesis offers a number of strategies for NumPy testing, available
2940 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
2941 package.
2942 The centerpiece is the arrays() strategy, which generates arrays with
2944 any dtype, shape, and contents you can specify or give a strategy for.
2945 To make this as useful as possible, strategies are provided to generate
2946 array shapes and generate all kinds of fixed-size or compound dtypes.
2947 hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0,
2949 max_size=None, min_value=None, max_value=None, allow_nan=None,
2950 allow_infinity=None, exclude_min=None, exclude_max=None)
2951 Creates a strategy which can generate any value of the given
2952 dtype.
2953 Compatible **kwargs are passed to the inferred strategy function
2955 for integers, floats, and strings. This allows you to customise
2956 the min and max values, control the length or contents of
2957 strings, or exclude non-finite numbers. This is particularly
2958 useful when kwargs are passed through from arrays() which allow
2959 a variety of numeric dtypes, as it seamlessly handles the width
2960 or representable bounds for you. See issue #2552 for more
2961 detail.
2962 hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None,
2964 fill=None, unique=False)
2965 Returns a strategy for generating numpy:numpy.ndarrays.
2966 · dtype may be any valid input to dtype (this includes dtype
2968 objects), or a strategy that generates such values.
2969 · shape may be an integer >= 0, a tuple of such integers, or a
2971 strategy that generates such values.
2972 · elements is a strategy for generating values to put in the
2974 array. If it is None a suitable value will be inferred based
2975 on the dtype, which may give any legal value (including eg NaN
2976 for floats). If a mapping, it will be passed as **kwargs to
2977 from_dtype()
2978 · fill is a strategy that may be used to generate a single back‐
2980 ground value for the array. If None, a suitable default will
2981 be inferred based on the other arguments. If set to nothing()
2982 then filling behaviour will be disabled entirely and every
2983 element will be generated independently.
2984 · unique specifies if the elements of the array should all be
2986 distinct from one another. Note that in this case multiple NaN
2987 values may still be allowed. If fill is also set, the only
2988 valid values for it to return are NaN values (anything for
2989 which numpy:numpy.isnan returns True. So e.g. for complex num‐
2990 bers (nan+1j) is also a valid fill). Note that if unique is
2991 set to True the generated values must be hashable.
2992 Arrays of specified dtype and shape are generated for example
2994 like this:
2995 >>> import numpy as np
2997 >>> arrays(np.int8, (2, 3)).example()
2998 array([[-8, 6, 3],
2999 [-6, 4, 6]], dtype=int8)
3000 · See What you can generate and how.
3002 >>> import numpy as np
3004 >>> from hypothesis.strategies import floats
3005 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
3006 array([ 0.88974794, 0.77387938, 0.1977879 ])
3007 Array values are generated in two parts:
3009 1. Some subset of the coordinates of the array are populated
3011 with a value drawn from the elements strategy (or its
3012 inferred form).
3013 2. If any coordinates were not assigned in the previous step, a
3015 single value is drawn from the fill strategy and is assigned
3016 to all remaining places.
3017 You can set fill to nothing() if you want to disable this behav‐
3019 iour and draw a value for every element.
3020 If fill is set to None then it will attempt to infer the correct
3022 behaviour automatically: If unique is True, no filling will
3023 occur by default. Otherwise, if it looks safe to reuse the val‐
3024 ues of elements across multiple coordinates (this will be the
3025 case for any inferred strategy, and for most of the builtins,
3026 but is not the case for mutable values or strategies built with
3027 flatmap, map, composite, etc) then it will use the elements
3028 strategy as the fill, else it will default to having no fill.
3029 Having a fill helps Hypothesis craft high quality examples, but
3031 its main importance is when the array generated is large:
3032 Hypothesis is primarily designed around testing small examples.
3033 If you have arrays with hundreds or more elements, having a fill
3034 value is essential if you want your tests to run in reasonable
3035 time.
3036 hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None,
3038 min_side=1, max_side=None)
3039 Return a strategy for array shapes (tuples of int >= 1).
3040 hypothesis.extra.numpy.scalar_dtypes()
3042 Return a strategy that can return any non-flexible scalar dtype.
3043 hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?',
3045 sizes=8, 16, 32, 64)
3046 Return a strategy for unsigned integer dtypes.
3047 endianness may be < for little-endian, > for big-endian, = for
3049 native byte order, or ? to allow either byte order. This argu‐
3050 ment only applies to dtypes of more than one byte.
3051 sizes must be a collection of integer sizes in bits. The
3053 default (8, 16, 32, 64) covers the full range of sizes.
3054 hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=8, 16,
3056 32, 64)
3057 Return a strategy for signed integer dtypes.
3058 endianness and sizes are treated as for
3060 unsigned_integer_dtypes().
3061 hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=16, 32,
3063 64)
3064 Return a strategy for floating-point dtypes.
3065 sizes is the size in bits of floating-point number. Some
3067 machines support 96- or 128-bit floats, but these are not gener‐
3068 ated by default.
3069 Larger floats (96 and 128 bit real parts) are not supported on
3071 all platforms and therefore disabled by default. To generate
3072 these dtypes, include these values in the sizes argument.
3073 hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?',
3075 sizes=64, 128)
3076 Return a strategy for complex-number dtypes.
3077 sizes is the total size in bits of a complex number, which con‐
3079 sists of two floats. Complex halfs (a 16-bit real part) are not
3080 supported by numpy and will not be generated by this strategy.
3081 hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y',
3083 min_period='ns', endianness='?')
3084 Return a strategy for datetime64 dtypes, with various precisions
3085 from year to attosecond.
3086 hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y',
3088 min_period='ns', endianness='?')
3089 Return a strategy for timedelta64 dtypes, with various preci‐
3090 sions from year to attosecond.
3091 hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1,
3093 max_len=16)
3094 Return a strategy for generating bytestring dtypes, of various
3095 lengths and byteorder.
3096 While Hypothesis' string strategies can generate empty strings,
3098 string dtypes with length 0 indicate that size is still to be
3099 determined, so the minimum length for string dtypes is 1.
3100 hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?',
3102 min_len=1, max_len=16)
3103 Return a strategy for generating unicode string dtypes, of vari‐
3104 ous lengths and byteorder.
3105 While Hypothesis' string strategies can generate empty strings,
3107 string dtypes with length 0 indicate that size is still to be
3108 determined, so the minimum length for string dtypes is 1.
3109 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
3111 *, min_size=1, max_size=5, allow_subarrays=False)
3112 Return a strategy for generating array (compound) dtypes, with
3113 members drawn from the given subtype strategy.
3114 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
3116 *, max_leaves=10, max_itemsize=None)
3117 Return the most-general dtype strategy.
3118 Elements drawn from this strategy may be simple (from the sub‐
3120 type_strategy), or several such values drawn from array_dtypes()
3121 with allow_subarrays=True. Subdtypes in an array dtype may be
3122 nested to any depth, subject to the max_leaves argument.
3123 hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0,
3125 max_size=None)
3126 Return a strategy for generating permissible tuple-values for
3127 the axis argument for a numpy sequential function (e.g.
3128 numpy:numpy.sum()), given an array of the specified dimensional‐
3129 ity.
3130 All tuples will have an length >= min_size and <= max_size. The
3132 default value for max_size is ndim.
3133 Examples from this strategy shrink towards an empty tuple, which
3135 render most sequential functions as no-ops.
3136 The following are some examples drawn from this strategy.
3138 >>> [valid_tuple_axes(3).example() for i in range(4)]
3140 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3141 valid_tuple_axes can be joined with other strategies to generate
3143 any type of valid axis object, i.e. integers, tuples, and None:
3144 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3146 hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0,
3148 max_dims=None, min_side=1, max_side=None)
3149 Return a strategy for generating shapes that are broadcast-com‐
3150 patible with the provided shape.
3151 Examples from this strategy shrink towards a shape with length
3153 min_dims. The size of an aligned dimension shrinks towards size
3154 1. The size of an unaligned dimension shrink towards min_side.
3155 · shape a tuple of integers
3157 · min_dims The smallest length that the generated shape can pos‐
3159 sess.
3160 · max_dims The largest length that the generated shape can pos‐
3162 sess. The default-value for max_dims is min(32,
3163 max(len(shape), min_dims) + 2).
3164 · min_side The smallest size that an unaligned dimension can
3166 possess.
3167 · max_side The largest size that an unaligned dimension can pos‐
3169 sess. The default value is 2 +
3170 'size-of-largest-aligned-dimension'.
3171 The following are some examples drawn from this strategy.
3173 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3175 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3176 hypothesis.extra.numpy.mutually_broadcastable_shapes(*,
3178 num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0,
3179 max_dims=None, min_side=1, max_side=None)
3180 Return a strategy for generating a specified number of shapes,
3181 N, that are mutually-broadcastable with one another and with the
3182 provided "base-shape".
3183 The strategy will generate a named-tuple of:
3185 · input_shapes: the N generated shapes
3187 · result_shape: the resulting shape, produced by broadcasting
3189 the N shapes with the base-shape
3190 Each shape produced from this strategy shrinks towards a shape
3192 with length min_dims. The size of an aligned dimension shrinks
3193 towards being having a size of 1. The size of an unaligned
3194 dimension shrink towards min_side.
3195 · num_shapes The number of mutually broadcast-compatible shapes
3197 to generate.
3198 · base-shape The shape against which all generated shapes can
3200 broadcast. The default shape is empty, which corresponds to a
3201 scalar and thus does not constrain broadcasting at all.
3202 · min_dims The smallest length that any generated shape can pos‐
3204 sess.
3205 · max_dims The largest length that any generated shape can pos‐
3207 sess. It cannot exceed 32, which is the greatest supported
3208 dimensionality for a numpy array. The default-value for
3209 max_dims is 2 + max(len(shape), min_dims), capped at 32.
3210 · min_side The smallest size that an unaligned dimension can
3212 possess.
3213 · max_side The largest size that an unaligned dimension can pos‐
3215 sess. The default value is 2 +
3216 'size-of-largest-aligned-dimension'.
3217 The following are some examples drawn from this strategy.
3219 >>> # Draw three shapes, and each shape is broadcast-compatible with `(2, 3)`
3221 >>> for _ in range(5):
3222 ... mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3)).example()
3223 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3224 BroadcastableShapes(input_shapes=((3,), (1,), (2, 1)), result_shape=(2, 3))
3225 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3226 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
3227 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
3228 Use with Generalised Universal Function signatures
3230 A universal function (or ufunc for short) is a function that
3232 operates on ndarrays in an element-by-element fashion, support‐
3233 ing array broadcasting, type casting, and several other standard
3234 features. A generalised ufunc operates on sub-arrays rather
3235 than elements, based on the "signature" of the function. Com‐
3236 pare e.g. numpy:numpy.add (ufunc) to numpy:numpy.matmul
3237 (gufunc).
3238 To generate shapes for a gufunc, you can pass the signature
3240 argument instead of num_shapes. This must be a gufunc signature
3241 string; which you can write by hand or access as e.g. np.mat‐
3242 mul.signature on generalised ufuncs.
3243 In this case, the side arguments are applied to the 'core dimen‐
3245 sions' as well, ignoring any frozen dimensions. base_shape and
3246 the dims arguments are applied to the 'loop dimensions', and if
3247 necessary, the dimensionality of each shape is silently capped
3248 to respect the 32-dimension limit.
3249 The generated result_shape is the real result shape of applying
3251 the gufunc to arrays of the generated input_shapes, even where
3252 this is different to broadcasting the loop dimensions.
3253 gufunc-compatible shapes shrink their loop dimensions as above,
3255 towards omitting optional core dimensions, and smaller-size core
3256 dimensions.
3257 >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
3259 >>> for _ in range(3):
3260 ... mutually_broadcastable_shapes(signature=np.matmul.signature).example()
3261 BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
3262 BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
3263 BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))
3264 hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0,
3266 max_dims=None, allow_newaxis=False, allow_ellipsis=True)
3267 The basic_indices strategy generates basic indexes for arrays
3268 of the specified shape, which may include dimensions of size
3269 zero.
3270 It generates tuples containing some mix of integers,
3272 python:slice objects, ... (Ellipsis), and numpy:numpy.newaxis;
3273 which when used to index a shape-shaped array will produce
3274 either a scalar or a shared-memory view. When a length-one
3275 tuple would be generated, this strategy may instead return the
3276 element which will index the first axis, e.g. 5 instead of (5,).
3277 · shape: the array shape that will be indexed, as a tuple of
3279 integers >= 0. This must be at least two-dimensional for a
3280 tuple to be a valid basic index; for one-dimensional arrays
3281 use slices() instead.
3282 · min_dims: the minimum dimensionality of the resulting view
3284 from use of the generated index. When min_dims == 0, scalars
3285 and zero-dimensional arrays are both allowed.
3286 · max_dims: the maximum dimensionality of the resulting view.
3288 If not specified, it defaults to max(len(shape), min_dims) +
3289 2.
3290 · allow_ellipsis: whether ...` is allowed in the index.
3292 · allow_newaxis: whether numpy:numpy.newaxis is allowed in the
3294 index.
3295 Note that the length of the generated tuple may be anywhere
3297 between zero and min_dims. It may not match the length of
3298 shape, or even the dimensionality of the array view resulting
3299 from its use!
3300 hypothesis.extra.numpy.integer_array_indices(shape, *,
3302 result_shape=array_shapes(), dtype='int')
3303 Return a search strategy for tuples of integer-arrays that, when
3304 used to index into an array of shape shape, given an array whose
3305 shape was drawn from result_shape.
3306 Examples from this strategy shrink towards the tuple of
3308 index-arrays:
3309 len(shape) * (np.zeros(drawn_result_shape, dtype), )
3311 · shape a tuple of integers that indicates the shape of the
3313 array, whose indices are being generated.
3314 · result_shape a strategy for generating tuples of integers,
3316 which describe the shape of the resulting index arrays. The
3317 default is array_shapes(). The shape drawn from this strategy
3318 determines the shape of the array that will be produced when
3319 the corresponding example from integer_array_indices is used
3320 as an index.
3321 · dtype the integer data type of the generated index-arrays.
3323 Negative integer indices can be generated if a signed integer
3324 type is specified.
3325 Recall that an array can be indexed using a tuple of inte‐
3327 ger-arrays to access its members in an arbitrary order, produc‐
3328 ing an array with an arbitrary shape. For example:
3329 >>> from numpy import array
3331 >>> x = array([-0, -1, -2, -3, -4])
3332 >>> ind = (array([[4, 0], [0, 1]]),) # a tuple containing a 2D integer-array
3333 >>> x[ind] # the resulting array is commensurate with the indexing array(s)
3334 array([[-4, 0],
3335 [0, -1]])
3336 Note that this strategy does not accommodate all variations of
3338 so-called 'advanced indexing', as prescribed by NumPy's nomen‐
3339 clature. Combinations of basic and advanced indexes are too
3340 complex to usefully define in a standard strategy; we leave
3341 application-specific strategies to the user. Advanced-boolean
3342 indexing can be defined as arrays(shape=..., dtype=bool), and is
3343 similarly left to the user.
3344 pandas
3346 Hypothesis provides strategies for several of the core pandas data
3347 types: pandas.Index, pandas.Series and pandas.DataFrame.
3348 The general approach taken by the pandas module is that there are mul‐
3350 tiple strategies for generating indexes, and all of the other strate‐
3351 gies take the number of entries they contain from their index strategy
3352 (with sensible defaults). So e.g. a Series is specified by specifying
3353 its numpy.dtype (and/or a strategy for generating elements for it).
3354 hypothesis.extra.pandas.indexes(*, elements=None, dtype=None,
3356 min_size=0, max_size=None, unique=True)
3357 Provides a strategy for producing a pandas.Index.
3358 Arguments:
3360 · elements is a strategy which will be used to generate the
3362 individual values of the index. If None, it will be inferred
3363 from the dtype. Note: even if the elements strategy produces
3364 tuples, the generated value will not be a MultiIndex, but
3365 instead be a normal index whose elements are tuples.
3366 · dtype is the dtype of the resulting index. If None, it will be
3368 inferred from the elements strategy. At least one of dtype or
3369 elements must be provided.
3370 · min_size is the minimum number of elements in the index.
3372 · max_size is the maximum number of elements in the index. If
3374 None then it will default to a suitable small size. If you
3375 want larger indexes you should pass a max_size explicitly.
3376 · unique specifies whether all of the elements in the resulting
3378 index should be distinct.
3379 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
3381 Provides a strategy which generates an Index whose values are 0,
3382 1, ..., n for some n.
3383 Arguments:
3385 · min_size is the smallest number of elements the index can
3387 have.
3388 · max_size is the largest number of elements the index can have.
3390 If None it will default to some suitable value based on
3391 min_size.
3392 hypothesis.extra.pandas.series(*, elements=None, dtype=None,
3394 index=None, fill=None, unique=False)
3395 Provides a strategy for producing a pandas.Series.
3396 Arguments:
3398 · elements: a strategy that will be used to generate the indi‐
3400 vidual values in the series. If None, we will attempt to infer
3401 a suitable default from the dtype.
3402 · dtype: the dtype of the resulting series and may be any value
3404 that can be passed to numpy.dtype. If None, will use pandas's
3405 standard behaviour to infer it from the type of the elements
3406 values. Note that if the type of values that comes out of your
3407 elements strategy varies, then so will the resulting dtype of
3408 the series.
3409 · index: If not None, a strategy for generating indexes for the
3411 resulting Series. This can generate either pandas.Index
3412 objects or any sequence of values (which will be passed to the
3413 Index constructor).
3414 You will probably find it most convenient to use the indexes()
3416 or range_indexes() function to produce values for this argu‐
3417 ment.
3418 Usage:
3420 >>> series(dtype=int).example()
3422 0 -2001747478
3423 1 1153062837
3424 class hypothesis.extra.pandas.column(name=None, elements=None,
3426 dtype=None, fill=None, unique=False)
3427 Data object for describing a column in a DataFrame.
3428 Arguments:
3430 · name: the column name, or None to default to the column posi‐
3432 tion. Must be hashable, but can otherwise be any value sup‐
3433 ported as a pandas column name.
3434 · elements: the strategy for generating values in this column,
3436 or None to infer it from the dtype.
3437 · dtype: the dtype of the column, or None to infer it from the
3439 element strategy. At least one of dtype or elements must be
3440 provided.
3441 · fill: A default value for elements of the column. See arrays()
3443 for a full explanation.
3444 · unique: If all values in this column should be distinct.
3446 hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, ele‐
3448 ments=None, fill=None, unique=False)
3449 A convenience function for producing a list of column objects of
3450 the same general shape.
3451 The names_or_number argument is either a sequence of values, the
3453 elements of which will be used as the name for individual column
3454 objects, or a number, in which case that many unnamed columns
3455 will be created. All other arguments are passed through verbatim
3456 to create the columns.
3457 hypothesis.extra.pandas.data_frames(columns=None, *, rows=None,
3459 index=None)
3460 Provides a strategy for producing a pandas.DataFrame.
3461 Arguments:
3463 · columns: An iterable of column objects describing the shape of
3465 the generated DataFrame.
3466 · rows: A strategy for generating a row object. Should generate
3468 either dicts mapping column names to values or a sequence map‐
3469 ping column position to the value in that position (note that
3470 unlike the pandas.DataFrame constructor, single values are not
3471 allowed here. Passing e.g. an integer is an error, even if
3472 there is only one column).
3473 At least one of rows and columns must be provided. If both are
3475 provided then the generated rows will be validated against the
3476 columns and an error will be raised if they don't match.
3477 Caveats on using rows:
3479 · In general you should prefer using columns to rows, and only
3481 use rows if the columns interface is insufficiently flexible
3482 to describe what you need - you will get better performance
3483 and example quality that way.
3484 · If you provide rows and not columns, then the shape and
3486 dtype of the resulting DataFrame may vary. e.g. if you have
3487 a mix of int and float in the values for one column in your
3488 row entries, the column will sometimes have an integral
3489 dtype and sometimes a float.
3490 · index: If not None, a strategy for generating indexes for the
3492 resulting DataFrame. This can generate either pandas.Index
3493 objects or any sequence of values (which will be passed to the
3494 Index constructor).
3495 You will probably find it most convenient to use the indexes()
3497 or range_indexes() function to produce values for this argu‐
3498 ment.
3499 Usage:
3501 The expected usage pattern is that you use column and columns()
3503 to specify a fixed shape of the DataFrame you want as follows.
3504 For example the following gives a two column data frame:
3505 >>> from hypothesis.extra.pandas import column, data_frames
3507 >>> data_frames([
3508 ... column('A', dtype=int), column('B', dtype=float)]).example()
3509 A B
3510 0 2021915903 1.793898e+232
3511 1 1146643993 inf
3512 2 -2096165693 1.000000e+07
3513 If you want the values in different columns to interact in some
3515 way you can use the rows argument. For example the following
3516 gives a two column DataFrame where the value in the first column
3517 is always at most the value in the second:
3518 >>> from hypothesis.extra.pandas import column, data_frames
3520 >>> import hypothesis.strategies as st
3521 >>> data_frames(
3522 ... rows=st.tuples(st.floats(allow_nan=False),
3523 ... st.floats(allow_nan=False)).map(sorted)
3524 ... ).example()
3525 0 1
3526 0 -3.402823e+38 9.007199e+15
3527 1 -1.562796e-298 5.000000e-01
3528 You can also combine the two:
3530 >>> from hypothesis.extra.pandas import columns, data_frames
3532 >>> import hypothesis.strategies as st
3533 >>> data_frames(
3534 ... columns=columns(["lo", "hi"], dtype=float),
3535 ... rows=st.tuples(st.floats(allow_nan=False),
3536 ... st.floats(allow_nan=False)).map(sorted)
3537 ... ).example()
3538 lo hi
3539 0 9.314723e-49 4.353037e+45
3540 1 -9.999900e-01 1.000000e+07
3541 2 -2.152861e+134 -1.069317e-73
3542 (Note that the column dtype must still be specified and will not
3544 be inferred from the rows. This restriction may be lifted in
3545 future).
3546 Combining rows and columns has the following behaviour:
3548 · The column names and dtypes will be used.
3550 · If the column is required to be unique, this will be enforced.
3552 · Any values missing from the generated rows will be provided
3554 using the column's fill.
3555 · Any values in the row not present in the column specification
3557 (if dicts are passed, if there are keys with no corresponding
3558 column name, if sequences are passed if there are too many
3559 items) will result in InvalidArgument being raised.
3560 Supported versions
3562 There is quite a lot of variation between pandas versions. We only com‐
3563 mit to supporting the latest version of pandas, but older minor ver‐
3564 sions are supported on a "best effort" basis. Hypothesis is currently
3565 tested against and confirmed working with every Pandas minor version
3566 from 0.25 through to 1.1.
3567 Releases that are not the latest patch release of their minor version
3569 are not tested or officially supported, but will probably also work
3570 unless you hit a pandas bug.
3571
3573 Hypothesis tries to detect common mistakes and things that will cause
3574 difficulty at run time in the form of a number of 'health checks'.
3575 These include detecting and warning about:
3577 · Strategies with very slow data generation
3579 · Strategies which filter out too much
3581 · Recursive strategies which branch too much
3583 · Tests that are unlikely to complete in a reasonable amount of time.
3585 If any of these scenarios are detected, Hypothesis will emit a warning
3587 about them.
3588 The general goal of these health checks is to warn you about things
3590 that you are doing that might appear to work but will either cause
3591 Hypothesis to not work correctly or to perform badly.
3592 To selectively disable health checks, use the suppress_health_check
3594 setting. The argument for this parameter is a list with elements drawn
3595 from any of the class-level attributes of the HealthCheck class. Using
3596 a value of HealthCheck.all() will disable all health checks.
3597 class hypothesis.HealthCheck(value)
3599 Arguments for suppress_health_check.
3600 Each member of this enum is a type of health check to suppress.
3602 data_too_large = 1
3604 Check for when the typical size of the examples you are
3605 generating exceeds the maximum allowed size too often.
3606 filter_too_much = 2
3608 Check for when the test is filtering out too many exam‐
3609 ples, either through use of assume() or filter(), or
3610 occasionally for Hypothesis internal reasons.
3611 too_slow = 3
3613 Check for when your data generation is extremely slow and
3614 likely to hurt testing.
3615 return_value = 5
3617 Checks if your tests return a non-None value (which will
3618 be ignored and is unlikely to do what you want).
3619 large_base_example = 7
3621 Checks if the natural example to shrink towards is very
3622 large.
3623 not_a_test_method = 8
3625 Checks if @given has been applied to a method defined by
3626 python:unittest.TestCase (i.e. not a test).
3627 Deprecations
3629 We also use a range of custom exception and warning types, so you can
3630 see exactly where an error came from - or turn only our warnings into
3631 errors.
3632 class hypothesis.errors.HypothesisDeprecationWarning
3634 A deprecation warning issued by Hypothesis.
3635 Actually inherits from FutureWarning, because DeprecationWarning
3637 is hidden by the default warnings filter.
3638 You can configure the Python python:warnings to handle these
3640 warnings differently to others, either turning them into errors
3641 or suppressing them entirely. Obviously we would prefer the
3642 former!
3643 Deprecated features will be continue to emit warnings for at least six
3645 months, and then be removed in the following major release. Note how‐
3646 ever that not all warnings are subject to this grace period; sometimes
3647 we strengthen validation by adding a warning and these may become
3648 errors immediately at a major release.
3649
3651 When Hypothesis finds a bug it stores enough information in its data‐
3652 base to reproduce it. This enables you to have a classic testing work‐
3653 flow of find a bug, fix a bug, and be confident that this is actually
3654 doing the right thing because Hypothesis will start by retrying the
3655 examples that broke things last time.
3656 Limitations
3658 The database is best thought of as a cache that you never need to
3659 invalidate: Information may be lost when you upgrade a Hypothesis ver‐
3660 sion or change your test, so you shouldn't rely on it for correctness -
3661 if there's an example you want to ensure occurs each time then there's
3662 a feature for including them in your source code - but it helps the
3663 development workflow considerably by making sure that the examples
3664 you've just found are reproduced.
3665 The database also records examples that exercise less-used parts of
3667 your code, so the database may update even when no failing examples
3668 were found.
3669 Upgrading Hypothesis and changing your tests
3671 The design of the Hypothesis database is such that you can put arbi‐
3672 trary data in the database and not get wrong behaviour. When you
3673 upgrade Hypothesis, old data might be invalidated, but this should hap‐
3674 pen transparently. It can never be the case that e.g. changing the
3675 strategy that generates an argument gives you data from the old strat‐
3676 egy.
3677 ExampleDatabase implementations
3679 Hypothesis' default database setting creates a
3680 DirectoryBasedExampleDatabase in your current working directory, under
3681 .hypothesis/examples. If this location is unusable, e.g. because you
3682 do not have read or write permissions, Hypothesis will emit a warning
3683 and fall back to an InMemoryExampleDatabase.
3684 Hypothesis provides the following ExampleDatabase implementations:
3686 class hypothesis.database.InMemoryExampleDatabase
3688 A non-persistent example database, implemented in terms of a
3689 dict of sets.
3690 This can be useful if you call a test function several times in
3692 a single session, or for testing other database implementations,
3693 but because it does not persist between runs we do not recommend
3694 it for general use.
3695 class hypothesis.database.DirectoryBasedExampleDatabase(path)
3697 Use a directory to store Hypothesis examples as files.
3698 Each test corresponds to a directory, and each example to a file
3700 within that directory. While the contents are fairly opaque, a
3701 DirectoryBasedExampleDatabase can be shared by checking the
3702 directory into version control, for example with the following
3703 .gitignore:
3704 # Ignore files cached by Hypothesis...
3706 .hypothesis/*
3707 # except for the examples directory
3708 !.hypothesis/examples/
3709 Note however that this only makes sense if you also pin to an
3711 exact version of Hypothesis, and we would usually recommend
3712 implementing a shared database with a network datastore - see
3713 ExampleDatabase, and the MultiplexedDatabase helper.
3714 class hypothesis.database.ReadOnlyDatabase(db)
3716 A wrapper to make the given database read-only.
3717 The implementation passes through fetch, and turns save, delete,
3719 and move into silent no-ops.
3720 Note that this disables Hypothesis' automatic discarding of
3722 stale examples. It is designed to allow local machines to
3723 access a shared database (e.g. from CI servers), without propa‐
3724 gating changes back from a local or in-development branch.
3725 class hypothesis.database.MultiplexedDatabase(*dbs)
3727 A wrapper around multiple databases.
3728 Each save, fetch, move, or delete operation will be run against
3730 all of the wrapped databases. fetch does not yield duplicate
3731 values, even if the same value is present in two or more of the
3732 wrapped databases.
3733 This combines well with a ReadOnlyDatabase, as follows:
3735 local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
3737 shared = CustomNetworkDatabase()
3738 settings.register_profile("ci", database=shared)
3740 settings.register_profile(
3741 "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
3742 )
3743 settings.load_profile("ci" if os.environ.get("CI") else "dev")
3744 So your CI system or fuzzing runs can populate a central shared
3746 database; while local runs on development machines can reproduce
3747 any failures from CI but will only cache their own failures
3748 locally and cannot remove examples from the shared database.
3749 class hypothesis.extra.redis.RedisExampleDatabase(redis, *,
3751 expire_after=datetime.timedelta(days=8), key_prefix=b'hypothesis-exam‐
3752 ple:')
3753 Store Hypothesis examples as sets in the given redis.Redis data‐
3754 store.
3755 This is particularly useful for shared databases, as per the
3757 recipe for a MultiplexedDatabase.
3758 NOTE:
3760 If a test has not been run for expire_after, those examples
3761 will be allowed to expire. The default time-to-live persists
3762 examples between weekly runs.
3763 Defining your own ExampleDatabase
3765 You can define your ExampleDatabase, for example to use a shared datas‐
3766 tore, with just a few methods:
3767 class hypothesis.database.ExampleDatabase(*args, **kwargs)
3769 An abstract base class for storing examples in Hypothesis'
3770 internal format.
3771 An ExampleDatabase maps each bytes key to many distinct bytes
3773 values, like a Mapping[bytes, AbstractSet[bytes]].
3774 abstract save(key, value)
3776 Save value under key.
3777 If this value is already present for this key, silently
3779 do nothing.
3780 abstract fetch(key)
3782 Return an iterable over all values matching this key.
3783 abstract delete(key, value)
3785 Remove this value from this key.
3786 If this value is not present, silently do nothing.
3788 move(src, dest, value)
3790 Move value from key src to key dest. Equivalent to
3791 delete(src, value) followed by save(src, value), but may
3792 have a more efficient implementation.
3793 Note that value will be inserted at dest regardless of
3795 whether it is currently present at src.
3796
3798 With @given, your tests are still something that you mostly write your‐
3799 self, with Hypothesis providing some data. With Hypothesis's stateful
3800 testing, Hypothesis instead tries to generate not just data but entire
3801 tests. You specify a number of primitive actions that can be combined
3802 together, and then Hypothesis will try to find sequences of those
3803 actions that result in a failure.
3804 NOTE:
3806 This style of testing is often called model-based testing, but in
3807 Hypothesis is called stateful testing (mostly for historical reasons
3808 - the original implementation of this idea in Hypothesis was more
3809 closely based on ScalaCheck's stateful testing where the name is
3810 more apt). Both of these names are somewhat misleading: You don't
3811 really need any sort of formal model of your code to use this, and
3812 it can be just as useful for pure APIs that don't involve any state
3813 as it is for stateful ones.
3814 It's perhaps best to not take the name of this sort of testing too
3816 seriously. Regardless of what you call it, it is a powerful form of
3817 testing which is useful for most non-trivial APIs.
3818 You may not need state machines
3820 The basic idea of stateful testing is to make Hypothesis choose actions
3821 as well as values for your test, and state machines are a great declar‐
3822 ative way to do just that.
3823 For simpler cases though, you might not need them at all - a standard
3825 test with @given might be enough, since you can use data() in branches
3826 or loops. In fact, that's how the state machine explorer works inter‐
3827 nally. For more complex workloads though, where a higher level API
3828 comes into it's own, keep reading!
3829 Rule-based state machines
3831 class hypothesis.stateful.RuleBasedStateMachine
3832 A RuleBasedStateMachine gives you a structured way to define
3833 state machines.
3834 The idea is that a state machine carries a bunch of types of
3836 data divided into Bundles, and has a set of rules which may read
3837 data from bundles (or just from normal strategies) and push data
3838 onto bundles. At any given point a random applicable rule will
3839 be executed.
3840 A rule is very similar to a normal @given based test in that it takes
3842 values drawn from strategies and passes them to a user defined test
3843 function. The key difference is that where @given based tests must be
3844 independent, rules can be chained together - a single test run may
3845 involve multiple rule invocations, which may interact in various ways.
3846 Rules can take normal strategies as arguments, or a specific kind of
3848 strategy called a Bundle. A Bundle is a named collection of generated
3849 values that can be reused by other operations in the test. They are
3850 populated with the results of rules, and may be used as arguments to
3851 rules, allowing data to flow from one rule to another, and rules to
3852 work on the results of previous computations or actions.
3853 You can think of each value that gets added to any Bundle as being
3855 assigned to a new variable. Drawing a value from the bundle strategy
3856 means choosing one of the corresponding variables and using that value,
3857 and consumes() as a del statement for that variable. If you can
3858 replace use of Bundles with instance attributes of the class that is
3859 often simpler, but often Bundles are strictly more powerful.
3860 The following rule based state machine example is a simplified version
3862 of a test for Hypothesis's example database implementation. An example
3863 database maps keys to sets of values, and in this test we compare one
3864 implementation of it to a simplified in memory model of its behaviour,
3865 which just stores the same values in a Python dict. The test then runs
3866 operations against both the real database and the in-memory representa‐
3867 tion of it and looks for discrepancies in their behaviour.
3868 import shutil
3870 import tempfile
3871 from collections import defaultdict
3873 import hypothesis.strategies as st
3874 from hypothesis.database import DirectoryBasedExampleDatabase
3875 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
3876 class DatabaseComparison(RuleBasedStateMachine):
3879 def __init__(self):
3880 super(DatabaseComparison, self).__init__()
3881 self.tempd = tempfile.mkdtemp()
3882 self.database = DirectoryBasedExampleDatabase(self.tempd)
3883 self.model = defaultdict(set)
3884 keys = Bundle("keys")
3886 values = Bundle("values")
3887 @rule(target=keys, k=st.binary())
3889 def add_key(self, k):
3890 return k
3891 @rule(target=values, v=st.binary())
3893 def add_value(self, v):
3894 return v
3895 @rule(k=keys, v=values)
3897 def save(self, k, v):
3898 self.model[k].add(v)
3899 self.database.save(k, v)
3900 @rule(k=keys, v=values)
3902 def delete(self, k, v):
3903 self.model[k].discard(v)
3904 self.database.delete(k, v)
3905 @rule(k=keys)
3907 def values_agree(self, k):
3908 assert set(self.database.fetch(k)) == self.model[k]
3909 def teardown(self):
3911 shutil.rmtree(self.tempd)
3912 TestDBComparison = DatabaseComparison.TestCase
3915 In this we declare two bundles - one for keys, and one for values. We
3917 have two trivial rules which just populate them with data (k and v),
3918 and three non-trivial rules: save saves a value under a key and delete
3919 removes a value from a key, in both cases also updating the model of
3920 what should be in the database. values_agree then checks that the con‐
3921 tents of the database agrees with the model for a particular key.
3922 We can then integrate this into our test suite by getting a unittest
3924 TestCase from it:
3925 TestTrees = DatabaseComparison.TestCase
3927 # Or just run with pytest's unittest support
3929 if __name__ == "__main__":
3930 unittest.main()
3931 This test currently passes, but if we comment out the line where we
3933 call self.model[k].discard(v), we would see the following output when
3934 run under pytest:
3935 AssertionError: assert set() == {b''}
3937 ------------ Hypothesis ------------
3939 state = DatabaseComparison()
3941 var1 = state.add_key(k=b'')
3942 var2 = state.add_value(v=var1)
3943 state.save(k=var1, v=var2)
3944 state.delete(k=var1, v=var2)
3945 state.values_agree(k=var1)
3946 state.teardown()
3947 Note how it's printed out a very short program that will demonstrate
3949 the problem. The output from a rule based state machine should gener‐
3950 ally be pretty close to Python code - if you have custom repr implemen‐
3951 tations that don't return valid Python then it might not be, but most
3952 of the time you should just be able to copy and paste the code into a
3953 test to reproduce it.
3954 You can control the detailed behaviour with a settings object on the
3956 TestCase (this is a normal hypothesis settings object using the
3957 defaults at the time the TestCase class was first referenced). For
3958 example if you wanted to run fewer examples with larger programs you
3959 could change the settings to:
3960 DatabaseComparison.TestCase.settings = settings(
3962 max_examples=50, stateful_step_count=100
3963 )
3964 Which doubles the number of steps each program runs and halves the num‐
3966 ber of test cases that will be run.
3967 Rules
3969 As said earlier, rules are the most common feature used in RuleBased‐
3970 StateMachine. They are defined by applying the rule() decorator on a
3971 function. Note that RuleBasedStateMachine must have at least one rule
3972 defined and that a single function cannot be used to define multiple
3973 rules (this to avoid having multiple rules doing the same things). Due
3974 to the stateful execution method, rules generally cannot take arguments
3975 from other sources such as fixtures or pytest.mark.parametrize - con‐
3976 sider providing them via a strategy such as sampled_from() instead.
3977 hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)
3979 Decorator for RuleBasedStateMachine. Any name present in target
3980 or targets will define where the end result of this function
3981 should go. If both are empty then the end result will be dis‐
3982 carded.
3983 target must be a Bundle, or if the result should go to multiple
3985 bundles you can pass a tuple of them as the targets argument.
3986 It is invalid to use both arguments for a single rule. If the
3987 result should go to exactly one of several bundles, define a
3988 separate rule for each case.
3989 kwargs then define the arguments that will be passed to the
3991 function invocation. If their value is a Bundle, or if it is
3992 consumes(b) where b is a Bundle, then values that have previ‐
3993 ously been produced for that bundle will be provided. If con‐
3994 sumes is used, the value will also be removed from the bundle.
3995 Any other kwargs should be strategies and values from them will
3997 be provided.
3998 hypothesis.stateful.consumes(bundle)
4000 When introducing a rule in a RuleBasedStateMachine, this func‐
4001 tion can be used to mark bundles from which each value used in a
4002 step with the given rule should be removed. This function
4003 returns a strategy object that can be manipulated and combined
4004 like any other.
4005 For example, a rule declared with
4007 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
4009 sumes(b3)))
4010 will consume a value from Bundle b2 and several values from Bun‐
4012 dle b3 to populate value2 and value3 each time it is executed.
4013 hypothesis.stateful.multiple(*args)
4015 This function can be used to pass multiple results to the tar‐
4016 get(s) of a rule. Just use return multiple(result1, result2,
4017 ...) in your rule.
4018 It is also possible to use return multiple() with no arguments
4020 in order to end a rule without passing any result.
4021 Initializes
4023 Initializes are a special case of rules that are guaranteed to be run
4024 at most once at the beginning of a run (i.e. before any normal rule is
4025 called). Note if multiple initialize rules are defined, they may be
4026 called in any order, and that order will vary from run to run.
4027 Initializes are typically useful to populate bundles:
4029 hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)
4031 Decorator for RuleBasedStateMachine.
4032 An initialize decorator behaves like a rule, but the decorated
4034 method is called at most once in a run. All initialize decorated
4035 methods will be called before any rule decorated methods, in an
4036 arbitrary order.
4037 import hypothesis.strategies as st
4039 from hypothesis.stateful import RuleBasedStateMachine, Bundle, rule, initialize
4040 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
4042 class NumberModifier(RuleBasedStateMachine):
4045 folders = Bundle("folders")
4047 files = Bundle("files")
4048 @initialize(target=folders)
4050 def init_folders(self):
4051 return "/"
4052 @rule(target=folders, name=name_strategy)
4054 def create_folder(self, parent, name):
4055 return "%s/%s" % (parent, name)
4056 @rule(target=files, name=name_strategy)
4058 def create_file(self, parent, name):
4059 return "%s/%s" % (parent, name)
4060 Preconditions
4062 While it's possible to use assume() in RuleBasedStateMachine rules, if
4063 you use it in only a few rules you can quickly run into a situation
4064 where few or none of your rules pass their assumptions. Thus, Hypothe‐
4065 sis provides a precondition() decorator to avoid this problem. The
4066 precondition() decorator is used on rule-decorated functions, and must
4067 be given a function that returns True or False based on the RuleBased‐
4068 StateMachine instance.
4069 hypothesis.stateful.precondition(precond)
4071 Decorator to apply a precondition for rules in a RuleBased‐
4072 StateMachine. Specifies a precondition for a rule to be consid‐
4073 ered as a valid step in the state machine. The given function
4074 will be called with the instance of RuleBasedStateMachine and
4075 should return True or False. Usually it will need to look at
4076 attributes on that instance.
4077 For example:
4079 class MyTestMachine(RuleBasedStateMachine):
4081 state = 1
4082 @precondition(lambda self: self.state != 0)
4084 @rule(numerator=integers())
4085 def divide_with(self, numerator):
4086 self.state = numerator / self.state
4087 This is better than using assume in your rule since more valid
4089 rules should be able to be run.
4090 from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
4092 class NumberModifier(RuleBasedStateMachine):
4095 num = 0
4097 @rule()
4099 def add_one(self):
4100 self.num += 1
4101 @precondition(lambda self: self.num != 0)
4103 @rule()
4104 def divide_with_one(self):
4105 self.num = 1 / self.num
4106 By using precondition() here instead of assume(), Hypothesis can filter
4108 the inapplicable rules before running them. This makes it much more
4109 likely that a useful sequence of steps will be generated.
4110 Note that currently preconditions can't access bundles; if you need to
4112 use preconditions, you should store relevant data on the instance
4113 instead.
4114 Invariants
4116 Often there are invariants that you want to ensure are met after every
4117 step in a process. It would be possible to add these as rules that are
4118 run, but they would be run zero or multiple times between other rules.
4119 Hypothesis provides a decorator that marks a function to be run after
4120 every step.
4121 hypothesis.stateful.invariant()
4123 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
4124 chine. The decorated function will be run after every rule and
4125 can raise an exception to indicate failed invariants.
4126 For example:
4128 class MyTestMachine(RuleBasedStateMachine):
4130 state = 1
4131 @invariant()
4133 def is_nonzero(self):
4134 assert self.state != 0
4135 from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
4137 class NumberModifier(RuleBasedStateMachine):
4140 num = 0
4142 @rule()
4144 def add_two(self):
4145 self.num += 2
4146 if self.num > 50:
4147 self.num += 1
4148 @invariant()
4150 def divide_with_one(self):
4151 assert self.num % 2 == 0
4152 NumberTest = NumberModifier.TestCase
4155 Invariants can also have precondition()s applied to them, in which case
4157 they will only be run if the precondition function returns true.
4158 Note that currently invariants can't access bundles; if you need to use
4160 invariants, you should store relevant data on the instance instead.
4161 More fine grained control
4163 If you want to bypass the TestCase infrastructure you can invoke these
4164 manually. The stateful module exposes the function
4165 run_state_machine_as_test, which takes an arbitrary function returning
4166 a RuleBasedStateMachine and an optional settings parameter and does the
4167 same as the class based runTest provided.
4168 This is not recommended as it bypasses some important internal func‐
4170 tions, including reporting of statistics such as runtimes and event()
4171 calls. It was originally added to support custom __init__ methods, but
4172 you can now use initialize() rules instead.
4173
4175 Hypothesis does its level best to be compatible with everything you
4176 could possibly need it to be compatible with. Generally you should just
4177 try it and expect it to work. If it doesn't, you can be surprised and
4178 check this document for the details.
4179 Hypothesis versions
4181 Backwards compatibility is better than backporting fixes, so we use
4182 semantic versioning and only support the most recent version of Hypoth‐
4183 esis. See support for more information.
4184 Documented APIs will not break except between major version bumps. All
4186 APIs mentioned in this documentation are public unless explicitly noted
4187 as provisional, in which case they may be changed in minor releases.
4188 Undocumented attributes, modules, and behaviour may include breaking
4189 changes in patch releases.
4190 Python versions
4192 Hypothesis is supported and tested on CPython 3.6+, i.e. all versions
4193 of CPython with upstream support,
4194 Hypothesis also supports the latest PyPy for Python 3.6. 32-bit builds
4196 of CPython also work, though they are currently only tested on Windows.
4197 In general Hypothesis does not officially support anything except the
4199 latest patch release of any version of Python it supports. Earlier
4200 releases should work and bugs in them will get fixed if reported, but
4201 they're not tested in CI and no guarantees are made.
4202 Operating systems
4204 In theory Hypothesis should work anywhere that Python does. In practice
4205 it is only known to work and regularly tested on OS X, Windows and
4206 Linux, and you may experience issues running it elsewhere.
4207 If you're using something else and it doesn't work, do get in touch and
4209 I'll try to help, but unless you can come up with a way for me to run a
4210 CI server on that operating system it probably won't stay fixed due to
4211 the inevitable march of time.
4212 Testing frameworks
4214 In general Hypothesis goes to quite a lot of effort to generate things
4215 that look like normal Python test functions that behave as closely to
4216 the originals as possible, so it should work sensibly out of the box
4217 with every test framework.
4218 If your testing relies on doing something other than calling a function
4220 and seeing if it raises an exception then it probably won't work out of
4221 the box. In particular things like tests which return generators and
4222 expect you to do something with them (e.g. nose's yield based tests)
4223 will not work. Use a decorator or similar to wrap the test to take this
4224 form, or ask the framework maintainer to support our hooks for insert‐
4225 ing such a wrapper later.
4226 In terms of what's actually known to work:
4228 · Hypothesis integrates as smoothly with pytest and unittest as we
4230 can make it, and this is verified as part of the CI.
4231 · pytest fixtures work in the usual way for tests that have been
4233 decorated with @given - just avoid passing a strategy for each
4234 argument that will be supplied by a fixture. However, each fix‐
4235 ture will run once for the whole function, not once per example.
4236 Decorating a fixture function with @given is meaningless.
4237 · The python:unittest.mock.patch() decorator works with @given, but
4239 we recommend using it as a context manager within the decorated
4240 test to ensure that the mock is per-test-case and avoid poor
4241 interactions with Pytest fixtures.
4242 · Nose works fine with Hypothesis, and this is tested as part of the
4244 CI. yield based tests simply won't work.
4245 · Integration with Django's testing requires use of the hypothe‐
4247 sis-django package. The issue is that in Django's tests' normal
4248 mode of execution it will reset the database once per test rather
4249 than once per example, which is not what you want.
4250 · Coverage works out of the box with Hypothesis; our own test suite
4252 has 100% branch coverage.
4253 Optional packages
4255 The supported versions of optional packages, for strategies in hypothe‐
4256 sis.extra, are listed in the documentation for that extra. Our general
4257 goal is to support all versions that are supported upstream.
4258 Regularly verifying this
4260 Everything mentioned above as explicitly supported is checked on every
4261 commit with GitHub Actions. Our continuous delivery pipeline runs all
4262 of these checks before publishing each release, so when we say they're
4263 supported we really mean it.
4264
4266 This is a collection of examples of how to use Hypothesis in interest‐
4267 ing ways. It's small for now but will grow over time.
4268 All of these examples are designed to be run under pytest, and nose
4270 should work too.
4271 How not to sort by a partial order
4273 The following is an example that's been extracted and simplified from a
4274 real bug that occurred in an earlier version of Hypothesis. The real
4275 bug was a lot harder to find.
4276 Suppose we've got the following type:
4278 class Node(object):
4280 def __init__(self, label, value):
4281 self.label = label
4282 self.value = tuple(value)
4283 def __repr__(self):
4285 return "Node(%r, %r)" % (self.label, self.value)
4286 def sorts_before(self, other):
4288 if len(self.value) >= len(other.value):
4289 return False
4290 return other.value[: len(self.value)] == self.value
4291 Each node is a label and a sequence of some data, and we have the rela‐
4293 tionship sorts_before meaning the data of the left is an initial seg‐
4294 ment of the right. So e.g. a node with value [1, 2] will sort before a
4295 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
4296 before the other.
4297 We have a list of nodes, and we want to topologically sort them with
4299 respect to this ordering. That is, we want to arrange the list so that
4300 if x.sorts_before(y) then x appears earlier in the list than y. We
4301 naively think that the easiest way to do this is to extend the partial
4302 order defined here to a total order by breaking ties arbitrarily and
4303 then using a normal sorting algorithm. So we define the following code:
4304 from functools import total_ordering
4306 @total_ordering
4309 class TopoKey(object):
4310 def __init__(self, node):
4311 self.value = node
4312 def __lt__(self, other):
4314 if self.value.sorts_before(other.value):
4315 return True
4316 if other.value.sorts_before(self.value):
4317 return False
4318 return self.value.label < other.value.label
4320 def sort_nodes(xs):
4323 xs.sort(key=TopoKey)
4324 This takes the order defined by sorts_before and extends it by breaking
4326 ties by comparing the node labels.
4327 But now we want to test that it works.
4329 First we write a function to verify that our desired outcome holds:
4331 def is_prefix_sorted(xs):
4333 for i in range(len(xs)):
4334 for j in range(i + 1, len(xs)):
4335 if xs[j].sorts_before(xs[i]):
4336 return False
4337 return True
4338 This will return false if it ever finds a pair in the wrong order and
4340 return true otherwise.
4341 Given this function, what we want to do with Hypothesis is assert that
4343 for all sequences of nodes, the result of calling sort_nodes on it is
4344 sorted.
4345 First we need to define a strategy for Node:
4347 from hypothesis import settings, strategies
4349 import hypothesis.strategies as s
4350 NodeStrategy = s.builds(Node, s.integers(), s.lists(s.booleans(), max_size=10))
4352 We want to generate short lists of values so that there's a decent
4354 chance of one being a prefix of the other (this is also why the choice
4355 of bool as the elements). We then define a strategy which builds a node
4356 out of an integer and one of those short lists of booleans.
4357 We can now write a test:
4359 from hypothesis import given
4361 @given(s.lists(NodeStrategy))
4364 def test_sorting_nodes_is_prefix_sorted(xs):
4365 sort_nodes(xs)
4366 assert is_prefix_sorted(xs)
4367 this immediately fails with the following example:
4369 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
4371 The reason for this is that because False is not a prefix of (True,
4373 True) nor vice versa, sorting things the first two nodes are equal
4374 because they have equal labels. This makes the whole order non-transi‐
4375 tive and produces basically nonsense results.
4376 But this is pretty unsatisfying. It only works because they have the
4378 same label. Perhaps we actually wanted our labels to be unique. Let's
4379 change the test to do that.
4380 def deduplicate_nodes_by_label(nodes):
4382 table = {node.label: node for node in nodes}
4383 return list(table.values())
4384 We define a function to deduplicate nodes by labels, and can now map
4386 that over a strategy for lists of nodes to give us a strategy for lists
4387 of nodes with unique labels:
4388 @given(s.lists(NodeStrategy).map(deduplicate_nodes_by_label))
4390 def test_sorting_nodes_is_prefix_sorted(xs):
4391 sort_nodes(xs)
4392 assert is_prefix_sorted(xs)
4393 Hypothesis quickly gives us an example of this still being wrong:
4395 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]
4397 Now this is a more interesting example. None of the nodes will sort
4399 equal. What is happening here is that the first node is strictly less
4400 than the last node because (False,) is a prefix of (False, False). This
4401 is in turn strictly less than the middle node because neither is a pre‐
4402 fix of the other and -2 < -1. The middle node is then less than the
4403 first node because -1 < 0.
4404 So, convinced that our implementation is broken, we write a better one:
4406 def sort_nodes(xs):
4408 for i in range(1, len(xs)):
4409 j = i - 1
4410 while j >= 0:
4411 if xs[j].sorts_before(xs[j + 1]):
4412 break
4413 xs[j], xs[j + 1] = xs[j + 1], xs[j]
4414 j -= 1
4415 This is just insertion sort slightly modified - we swap a node back‐
4417 wards until swapping it further would violate the order constraints.
4418 The reason this works is because our order is a partial order already
4419 (this wouldn't produce a valid result for a general topological sorting
4420 - you need the transitivity).
4421 We now run our test again and it passes, telling us that this time
4423 we've successfully managed to sort some nodes without getting it com‐
4424 pletely wrong. Go us.
4425 Time zone arithmetic
4427 This is an example of some tests for pytz which check that various
4428 timezone conversions behave as you would expect them to. These tests
4429 should all pass, and are mostly a demonstration of some useful sorts of
4430 thing to test with Hypothesis, and how the datetimes() strategy works.
4431 >>> from datetime import timedelta
4433 >>> from hypothesis.extra.pytz import timezones
4434 >>> from hypothesis.strategies import datetimes
4435 >>> # The datetimes strategy is naive by default, so tell it to use timezones
4437 >>> aware_datetimes = datetimes(timezones=timezones())
4438 >>> @given(aware_datetimes, timezones(), timezones())
4440 ... def test_convert_via_intermediary(dt, tz1, tz2):
4441 ... """Test that converting between timezones is not affected
4442 ... by a detour via another timezone.
4443 ... """
4444 ... assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
4445 >>> @given(aware_datetimes, timezones())
4447 ... def test_convert_to_and_fro(dt, tz2):
4448 ... """If we convert to a new timezone and back to the old one
4449 ... this should leave the result unchanged.
4450 ... """
4451 ... tz1 = dt.tzinfo
4452 ... assert dt == dt.astimezone(tz2).astimezone(tz1)
4453 >>> @given(aware_datetimes, timezones())
4455 ... def test_adding_an_hour_commutes(dt, tz):
4456 ... """When converting between timezones it shouldn't matter
4457 ... if we add an hour here or add an hour there.
4458 ... """
4459 ... an_hour = timedelta(hours=1)
4460 ... assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
4461 >>> @given(aware_datetimes, timezones())
4463 ... def test_adding_a_day_commutes(dt, tz):
4464 ... """When converting between timezones it shouldn't matter
4465 ... if we add a day here or add a day there.
4466 ... """
4467 ... a_day = timedelta(days=1)
4468 ... assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
4469 >>> # And we can check that our tests pass
4471 >>> test_convert_via_intermediary()
4472 >>> test_convert_to_and_fro()
4473 >>> test_adding_an_hour_commutes()
4474 >>> test_adding_a_day_commutes()
4475 Condorcet's paradox
4477 A classic paradox in voting theory, called Condorcet's paradox, is that
4478 majority preferences are not transitive. That is, there is a population
4479 and a set of three candidates A, B and C such that the majority of the
4480 population prefer A to B, B to C and C to A.
4481 Wouldn't it be neat if we could use Hypothesis to provide an example of
4483 this?
4484 Well as you can probably guess from the presence of this section, we
4486 can! The main trick is to decide how we want to represent the result
4487 of an election - for this example, we'll use a list of "votes", where
4488 each vote is a list of candidates in the voters preferred order. With‐
4489 out further ado, here is the code:
4490 from hypothesis import given, assume
4492 from hypothesis.strategies import lists, permutations
4493 from collections import Counter
4494 # We need at least three candidates and at least three voters to have a
4496 # paradox; anything less can only lead to victories or at worst ties.
4497 @given(lists(permutations(["A", "B", "C"]), min_size=3))
4498 def test_elections_are_transitive(election):
4499 all_candidates = {"A", "B", "C"}
4500 # First calculate the pairwise counts of how many prefer each candidate
4502 # to the other
4503 counts = Counter()
4504 for vote in election:
4505 for i in range(len(vote)):
4506 for j in range(i + 1, len(vote)):
4507 counts[(vote[i], vote[j])] += 1
4508 # Now look at which pairs of candidates one has a majority over the
4510 # other and store that.
4511 graph = {}
4512 for i in all_candidates:
4513 for j in all_candidates:
4514 if counts[(i, j)] > counts[(j, i)]:
4515 graph.setdefault(i, set()).add(j)
4516 # Now for each triple assert that it is transitive.
4518 for x in all_candidates:
4519 for y in graph.get(x, ()):
4520 for z in graph.get(y, ()):
4521 assert x not in graph.get(z, ())
4522 The example Hypothesis gives me on my first run (your mileage may of
4524 course vary) is:
4525 [["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]
4527 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
4529 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
4530 and 2) prefer C to A. This is in fact basically the canonical example
4531 of the voting paradox.
4532 Fuzzing an HTTP API
4534 Hypothesis's support for testing HTTP services is somewhat nascent.
4535 There are plans for some fully featured things around this, but right
4536 now they're probably quite far down the line.
4537 But you can do a lot yourself without any explicit support! Here's a
4539 script I wrote to throw arbitrary data against the API for an entirely
4540 fictitious service called Waspfinder (this is only lightly obfuscated
4541 and you can easily figure out who I'm actually talking about, but I
4542 don't want you to run this code and hammer their API without their per‐
4543 mission).
4544 All this does is use Hypothesis to generate arbitrary JSON data match‐
4546 ing the format their API asks for and check for 500 errors. More
4547 advanced tests which then use the result and go on to do other things
4548 are definitely also possible. The swagger-conformance package provides
4549 an excellent example of this!
4550 import unittest
4552 from hypothesis import given, assume, settings, strategies as st
4553 from collections import namedtuple
4554 import requests
4555 import os
4556 import random
4557 import time
4558 import math
4559 Goal = namedtuple("Goal", ("slug",))
4562 # We just pass in our API credentials via environment variables.
4565 waspfinder_token = os.getenv("WASPFINDER_TOKEN")
4566 waspfinder_user = os.getenv("WASPFINDER_USER")
4567 assert waspfinder_token is not None
4568 assert waspfinder_user is not None
4569 GoalData = st.fixed_dictionaries(
4571 {
4572 "title": st.text(),
4573 "goal_type": st.sampled_from(
4574 ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
4575 ),
4576 "goaldate": st.one_of(st.none(), st.floats()),
4577 "goalval": st.one_of(st.none(), st.floats()),
4578 "rate": st.one_of(st.none(), st.floats()),
4579 "initval": st.floats(),
4580 "panic": st.floats(),
4581 "secret": st.booleans(),
4582 "datapublic": st.booleans(),
4583 }
4584 )
4585 needs2 = ["goaldate", "goalval", "rate"]
4588 class WaspfinderTest(unittest.TestCase):
4591 @given(GoalData)
4592 def test_create_goal_dry_run(self, data):
4593 # We want slug to be unique for each run so that multiple test runs
4594 # don't interfere with each other. If for some reason some slugs trigger
4595 # an error and others don't we'll get a Flaky error, but that's OK.
4596 slug = hex(random.getrandbits(32))[2:]
4597 # Use assume to guide us through validation we know about, otherwise
4599 # we'll spend a lot of time generating boring examples.
4600 # Title must not be empty
4602 assume(data["title"])
4603 # Exactly two of these values should be not None. The other will be
4605 # inferred by the API.
4606 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
4608 for v in data.values():
4609 if isinstance(v, float):
4610 assume(not math.isnan(v))
4611 data["slug"] = slug
4612 # The API nicely supports a dry run option, which means we don't have
4614 # to worry about the user account being spammed with lots of fake goals
4615 # Otherwise we would have to make sure we cleaned up after ourselves
4616 # in this test.
4617 data["dryrun"] = True
4618 data["auth_token"] = waspfinder_token
4619 for d, v in data.items():
4620 if v is None:
4621 data[d] = "null"
4622 else:
4623 data[d] = str(v)
4624 result = requests.post(
4625 "https://waspfinder.example.com/api/v1/users/"
4626 "%s/goals.json" % (waspfinder_user,),
4627 data=data,
4628 )
4629 # Let's not hammer the API too badly. This will of course make the
4631 # tests even slower than they otherwise would have been, but that's
4632 # life.
4633 time.sleep(1.0)
4634 # For the moment all we're testing is that this doesn't generate an
4636 # internal error. If we didn't use the dry run option we could have
4637 # then tried doing more with the result, but this is a good start.
4638 self.assertNotEqual(result.status_code, 500)
4639 if __name__ == "__main__":
4642 unittest.main()
4643
4645 The Hypothesis community is small for the moment but is full of excel‐
4646 lent people who can answer your questions and help you out. Please do
4647 join us. The two major places for community discussion are:
4648 · The mailing list.
4650 · An IRC channel, #hypothesis on freenode, which is more active than
4652 the mailing list.
4653 Feel free to use these to ask for help, provide feedback, or discuss
4655 anything remotely Hypothesis related at all. If you post a question on
4656 Stack Overflow, please use the python-hypothesis tag!
4657 Please note that the Hypothesis code of conduct applies in all Hypothe‐
4659 sis community spaces.
4660 If you would like to cite Hypothesis, please consider our suggested
4662 citation.
4663 If you like repo badges, we suggest the following badge, which you can
4665 add with reStructuredText or Markdown, respectively: [image]
4666 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
4668 :alt: Tested with Hypothesis
4669 :target: https://hypothesis.readthedocs.io
4670 [](https://hypothesis.readthedocs.io/)
4672 Finally, we have a beautiful logo which appears online, and often on
4674 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
4675 [image]
4676 As well as being beautiful, dragonflies actively hunt down bugs for a
4678 living! You can find the images and a usage guide in the brand direc‐
4679 tory on GitHub, or find us at conferences where we often have stickers
4680 and sometimes other swag.
4681
4683 What is Hypothesis for?
4684 From the perspective of a user, the purpose of Hypothesis is to make it
4686 easier for you to write better tests.
4687 From my perspective as the author, that is of course also a purpose of
4689 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
4690 mania for a moment), the larger purpose of Hypothesis is to drag the
4691 world kicking and screaming into a new and terrifying age of high qual‐
4692 ity software.
4693 Software is, as they say, eating the world. Software is also terrible.
4695 It's buggy, insecure and generally poorly thought out. This combination
4696 is clearly a recipe for disaster.
4697 And the state of software testing is even worse. Although it's fairly
4699 uncontroversial at this point that you should be testing your code, can
4700 you really say with a straight face that most projects you've worked on
4701 are adequately tested?
4702 A lot of the problem here is that it's too hard to write good tests.
4704 Your tests encode exactly the same assumptions and fallacies that you
4705 had when you wrote the code, so they miss exactly the same bugs that
4706 you missed when you wrote the code.
4707 Meanwhile, there are all sorts of tools for making testing better that
4709 are basically unused. The original Quickcheck is from 1999 and the
4710 majority of developers have not even heard of it, let alone used it.
4711 There are a bunch of half-baked implementations for most languages, but
4712 very few of them are worth using.
4713 The goal of Hypothesis is to bring advanced testing techniques to the
4715 masses, and to provide an implementation that is so high quality that
4716 it is easier to use them than it is not to use them. Where I can, I
4717 will beg, borrow and steal every good idea I can find that someone has
4718 had to make software testing better. Where I can't, I will invent new
4719 ones.
4720 Quickcheck is the start, but I also plan to integrate ideas from fuzz
4722 testing (a planned future feature is to use coverage information to
4723 drive example selection, and the example saving database is already
4724 inspired by the workflows people use for fuzz testing), and am open to
4725 and actively seeking out other suggestions and ideas.
4726 The plan is to treat the social problem of people not using these ideas
4728 as a bug to which there is a technical solution: Does property-based
4729 testing not match your workflow? That's a bug, let's fix it by figur‐
4730 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
4731 tom data for your application? That's a bug. Let's fix it by figuring
4732 out how to make it easier, or how to take something you're already
4733 using to specify your data and derive a generator from that automati‐
4734 cally. Find the explanations of these advanced ideas hopelessly obtuse
4735 and hard to follow? That's a bug. Let's provide you with an easy API
4736 that lets you test your code better without a PhD in software verifica‐
4737 tion.
4738 Grand ambitions, I know, and I expect ultimately the reality will be
4740 somewhat less grand, but so far in about three months of development,
4741 Hypothesis has become the most solid implementation of Quickcheck ever
4742 seen in a mainstream language (as long as we don't count Scala as main‐
4743 stream yet), and at the same time managed to significantly push forward
4744 the state of the art, so I think there's reason to be optimistic.
4745
4747 This is a page for listing people who are using Hypothesis and how
4748 excited they are about that. If that's you and your name is not on the
4749 list, this file is in Git and I'd love it if you sent me a pull request
4750 to fix that.
4751 Stripe
4753 At Stripe we use Hypothesis to test every piece of our machine learning
4754 model training pipeline (powered by scikit). Before we migrated, our
4755 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
4756 resentative at all of our actual very complex data. Because we needed
4757 to craft examples for each test, we took the easy way out and lived
4758 with extremely low test coverage.
4759 Hypothesis changed all that. Once we had our strategies for generating
4761 Dataframes of features it became trivial to slightly customize each
4762 strategy for new tests. Our coverage is now close to 90%.
4763 Full-stop, property-based testing is profoundly more powerful - and has
4765 caught or prevented far more bugs - than our old style of example-based
4766 testing.
4767 Kristian Glass - Director of Technology at LaterPay GmbH
4769 Hypothesis has been brilliant for expanding the coverage of our test
4770 cases, and also for making them much easier to read and understand, so
4771 we're sure we're testing the things we want in the way we want.
4772 Seth Morton
4774 When I first heard about Hypothesis, I knew I had to include it in my
4775 two open-source Python libraries, natsort and fastnumbers . Quite
4776 frankly, I was a little appalled at the number of bugs and "holes" I
4777 found in the code. I can now say with confidence that my libraries are
4778 more robust to "the wild." In addition, Hypothesis gave me the confi‐
4779 dence to expand these libraries to fully support Unicode input, which I
4780 never would have had the stomach for without such thorough testing
4781 capabilities. Thanks!
4782 Sixty North
4784 At Sixty North we use Hypothesis for testing Segpy an open source
4785 Python library for shifting data between Python data structures and SEG
4786 Y files which contain geophysical data from the seismic reflection sur‐
4787 veys used in oil and gas exploration.
4788 This is our first experience of property-based testing – as opposed to
4790 example-based testing. Not only are our tests more powerful, they are
4791 also much better explanations of what we expect of the production code.
4792 In fact, the tests are much closer to being specifications. Hypothesis
4793 has located real defects in our code which went undetected by tradi‐
4794 tional test cases, simply because Hypothesis is more relentlessly devi‐
4795 ous about test case generation than us mere humans! We found Hypothe‐
4796 sis particularly beneficial for Segpy because SEG Y is an antiquated
4797 format that uses legacy text encodings (EBCDIC) and even a legacy
4798 floating point format we implemented from scratch in Python.
4799 Hypothesis is sure to find a place in most of our future Python code‐
4801 bases and many existing ones too.
4802 mulkieran
4804 Just found out about this excellent QuickCheck for Python implementa‐
4805 tion and ran up a few tests for my bytesize package last night. Refuted
4806 a few hypotheses in the process.
4807 Looking forward to using it with a bunch of other projects as well.
4809 Adam Johnson
4811 I have written a small library to serialize dicts to MariaDB's dynamic
4812 columns binary format, mariadb-dyncol. When I first developed it, I
4813 thought I had tested it really well - there were hundreds of test
4814 cases, some of them even taken from MariaDB's test suite itself. I was
4815 ready to release.
4816 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
4818 Wow! It found bug after bug after bug. Even after a first release, I
4819 thought of a way to make the tests do more validation, which revealed a
4820 further round of bugs! Most impressively, Hypothesis found a compli‐
4821 cated off-by-one error in a condition with 4095 versus 4096 bytes of
4822 data - something that I would never have found.
4823 Long live Hypothesis! (Or at least, property-based testing).
4825 Josh Bronson
4827 Adopting Hypothesis improved bidict's test coverage and significantly
4828 increased our ability to make changes to the code with confidence that
4829 correct behavior would be preserved. Thank you, David, for the great
4830 testing tool.
4831 Cory Benfield
4833 Hypothesis is the single most powerful tool in my toolbox for working
4834 with algorithmic code, or any software that produces predictable output
4835 from a wide range of sources. When using it with Priority, Hypothesis
4836 consistently found errors in my assumptions and extremely subtle bugs
4837 that would have taken months of real-world use to locate. In some
4838 cases, Hypothesis found subtle deviations from the correct output of
4839 the algorithm that may never have been noticed at all.
4840 When it comes to validating the correctness of your tools, nothing
4842 comes close to the thoroughness and power of Hypothesis.
4843 Jon Moore
4845 One extremely satisfied user here. Hypothesis is a really solid imple‐
4846 mentation of property-based testing, adapted well to Python, and with
4847 good features such as failure-case shrinkers. I first used it on a
4848 project where we needed to verify that a vendor's Python and non-Python
4849 implementations of an algorithm matched, and it found about a dozen
4850 cases that previous example-based testing and code inspections had not.
4851 Since then I've been evangelizing for it at our firm.
4852 Russel Winder
4854 I am using Hypothesis as an integral part of my Python workshops. Test‐
4855 ing is an integral part of Python programming and whilst unittest and,
4856 better, pytest can handle example-based testing, property-based testing
4857 is increasingly far more important than example-base testing, and
4858 Hypothesis fits the bill.
4859 Wellfire Interactive
4861 We've been using Hypothesis in a variety of client projects, from test‐
4862 ing Django-related functionality to domain-specific calculations. It
4863 both speeds up and simplifies the testing process since there's so much
4864 less tedious and error-prone work to do in identifying edge cases. Test
4865 coverage is nice but test depth is even nicer, and it's much easier to
4866 get meaningful test depth using Hypothesis.
4867 Cody Kochmann
4869 Hypothesis is being used as the engine for random object generation
4870 with my open source function fuzzer battle_tested which maps all behav‐
4871 iors of a function allowing you to minimize the chance of unexpected
4872 crashes when running code in production.
4873 With how efficient Hypothesis is at generating the edge cases that
4875 cause unexpected behavior occur, battle_tested is able to map out the
4876 entire behavior of most functions in less than a few seconds.
4877 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
4879 ing it.
4880 Merchise Autrement
4882 Just minutes after our first use of hypothesis we uncovered a subtle
4883 bug in one of our most used library. Since then, we have increasingly
4884 used hypothesis to improve the quality of our testing in libraries and
4885 applications as well.
4886 Florian Kromer
4888 At Roboception GmbH I use Hypothesis to implement fully automated
4889 stateless and stateful reliability tests for the 3D sensor rc_visard
4890 and robotic software components .
4891 Thank you very much for creating the (probably) most powerful prop‐
4893 erty-based testing framework.
4894 Reposit Power
4896 With a micro-service architecture, testing between services is made
4897 easy using Hypothesis in integration testing. Ensuring everything is
4898 running smoothly is vital to help maintain a secure network of Virtual
4899 Power Plants.
4900 It allows us to find potential bugs and edge cases with relative ease
4902 and minimal overhead. As our architecture relies on services communi‐
4903 cating effectively, Hypothesis allows us to strictly test for the kind
4904 of data which moves around our services, particularly our backend
4905 Python applications.
4906 Your name goes here
4908 I know there are many more, because I keep finding out about new people
4909 I'd never even heard of using Hypothesis. If you're looking to way to
4910 give back to a tool you love, adding your name here only takes a moment
4911 and would really help a lot. As per instructions at the top, just send
4912 me a pull request and I'll add you to the list.
4913
4915 The following is a non-exhaustive list of open source projects I know
4916 are using Hypothesis. If you're aware of any others please add them to
4917 the list! The only inclusion criterion right now is that if it's a
4918 Python library then it should be available on PyPI.
4919 You can find hundreds more from the Hypothesis page at libraries.io,
4921 and thousands on GitHub. Hypothesis has over 100,000 downloads per
4922 week, and was used by more than 4% of Python users surveyed by the PSF
4923 in 2018.
4924 · aur
4926 · argon2_cffi
4928 · attrs
4930 · axelrod
4932 · bidict
4934 · binaryornot
4936 · brotlipy
4938 · chardet
4940 · cmph-cffi
4942 · cryptography
4944 · dbus-signature-pyparsing
4946 · dry-python/returns
4948 · fastnumbers
4950 · flocker
4952 · flownetpy
4954 · funsize
4956 · fusion-index
4958 · hyper-h2
4960 · into-dbus-python
4962 · justbases
4964 · justbytes
4966 · loris
4968 · mariadb-dyncol
4970 · mercurial
4972 · natsort
4974 · poliastro
4976 · pretext
4978 · priority
4980 · PyCEbox
4982 · PyPy
4984 · pyrsistent
4986 · python-humble-utils
4988 · pyudev
4990 · qutebrowser
4992 · RubyMarshal
4994 · Segpy
4996 · simoa
4998 · srt
5000 · tchannel
5002 · vdirsyncer
5004 · wcag-contrast-ratio
5006 · yacluster
5008 · yturl
5010
5012 Hypothesis has been eagerly used and extended by the open source commu‐
5013 nity. This page lists extensions and applications; you can find more
5014 or newer packages by searching PyPI by keyword or filter by classifier,
5015 or search libraries.io.
5016 If there's something missing which you think should be here, let us
5018 know!
5019 NOTE:
5021 Being listed on this page does not imply that the Hypothesis main‐
5022 tainers endorse a package.
5023 External strategies
5025 Some packages provide strategies directly:
5026 · hypothesis-fspaths - strategy to generate filesystem paths.
5028 · hypothesis-geojson - strategy to generate GeoJson.
5030 · hypothesis-geometry - strategies to generate geometric objects.
5032 · hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
5034 · hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
5036 · hypothesis-ros - strategies to generate messages and parameters for
5038 the Robot Operating System.
5039 · hypothesis-csv - strategy to generate CSV files.
5041 · hypothesis-networkx - strategy to generate networkx graphs.
5043 · hypothesis-bio - strategies for bioinformatics data, such as DNA,
5045 codons, FASTA, and FASTQ formats.
5046 · hypothesmith - strategy to generate syntatically-valid Python code.
5048 Others provide a function to infer a strategy from some other schema:
5050 · hypothesis-jsonschema - infer strategies from JSON schemas.
5052 · lollipop-hypothesis - infer strategies from lollipop schemas.
5054 · hypothesis-drf - infer strategies from a djangorestframework seri‐
5056 aliser.
5057 · hypothesis-graphql - infer strategies from GraphQL schemas.
5059 · hypothesis-mongoengine - infer strategies from a mongoengine model.
5061 · hypothesis-pb - infer strategies from Protocol Buffer schemas.
5063 Other cool things
5065 schemathesis is a tool for testing web applications built with Open API
5066 / Swagger specifications. It reads the schema and generates test cases
5067 which will ensure that the application is compliant with its schema.
5068 The application under test could be written in any language, the only
5069 thing you need is a valid API schema in a supported format. Includes
5070 CLI and convenient pytest integration. Powered by Hypothesis and
5071 hypothesis-jsonschema, inspired by the earlier swagger-conformance
5072 library.
5073 Trio is an async framework with "an obsessive focus on usability and
5075 correctness", so naturally it works with Hypothesis! pytest-trio
5076 includes a custom hook that allows @given(...) to work with Trio-style
5077 async test functions, and hypothesis-trio includes stateful testing
5078 extensions to support concurrent programs.
5079 pymtl3 is "an open-source Python-based hardware generation, simulation,
5081 and verification framework with multi-level hardware modeling support",
5082 which ships with Hypothesis integrations to check that all of those
5083 levels are eqivalent, from function-level to register-transfer level
5084 and even to hardware.
5085 libarchimedes makes it easy to use Hypothesis in the Hy language, a
5087 Lisp embedded in Python.
5088 battle_tested is a fuzzing tool that will show you how your code can
5090 fail - by trying all kinds of inputs and reporting whatever happens.
5091 pytest-subtesthack functions as a workaround for issue #377.
5093 returns uses Hypothesis to verify that Higher Kinded Types correctly
5095 implement functor, applicative, monad, and other laws; allowing a
5096 declarative approach to be combined with traditional pythonic code.
5097 Writing an extension
5099 See CONTRIBUTING.rst for more information.
5100 New strategies can be added to Hypothesis, or published as an external
5102 package on PyPI - either is fine for most strategies. If in doubt, ask!
5103 It's generally much easier to get things working outside, because
5105 there's more freedom to experiment and fewer requirements in stability
5106 and API style. We're happy to review and help with external packages as
5107 well as pull requests!
5108 If you're thinking about writing an extension, please name it hypothe‐
5110 sis-{something} - a standard prefix makes the community more visible
5111 and searching for extensions easier. And make sure you use the Frame‐
5112 work :: Hypothesis trove classifier!
5113 On the other hand, being inside gets you access to some deeper imple‐
5115 mentation features (if you need them) and better long-term guarantees
5116 about maintenance. We particularly encourage pull requests for new
5117 composable primitives that make implementing other strategies easier,
5118 or for widely used types in the standard library. Strategies for other
5119 things are also welcome; anything with external dependencies just goes
5120 in hypothesis.extra.
5121 Registering strategies via setuptools entry points
5123 If you would like to ship Hypothesis strategies for a custom type -
5124 either as part of the upstream library, or as a third-party extension,
5125 there's a catch: from_type() only works after the corresponding call to
5126 register_type_strategy(). This means that either
5127 · you have to try importing Hypothesis to register the strategy when
5129 your library is imported, though that's only useful at test time, or
5130 · the user has to call a 'register the strategies' helper that you pro‐
5132 vide before running their tests
5133 Entry points are Python's standard way of automating the latter: when
5135 you register a "hypothesis" entry point in your setup.py, we'll import
5136 and run it automatically when hypothesis is imported. Nothing happens
5137 unless Hypothesis is already in use, and it's totally seamless for
5138 downstream users!
5139 Let's look at an example. You start by adding a function somewhere in
5141 your package that does all the Hypothesis-related setup work:
5142 # mymodule.py
5144 class MyCustomType:
5147 def __init__(self, x: int):
5148 assert x >= 0, f"got {x}, but only positive numbers are allowed"
5149 self.x = x
5150 def _hypothesis_setup_hook():
5153 import hypothesis.strategies as st
5154 st.register_type_strategy(MyCustomType, st.integers(min_value=0))
5156 and then tell setuptools that this is your "hypothesis" entry point:
5158 # setup.py
5160 # You can list a module to import by dotted name
5162 entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}
5163 # Or name a specific function too, and Hypothesis will call it for you
5165 entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}
5166 And that's all it takes!
5168 NOTE:
5170 On Python 3.6 and 3.7, where the importlib.metadata module is not in
5171 the standard library, loading entry points requires either the
5172 importlib_metadata (preferred) or setuptools (fallback) package to
5173 be installed.
5174 Interaction with pytest-cov
5176 Because pytest does not load plugins from entrypoints in any particular
5177 order, using the Hypothesis entrypoint may import your module before
5178 pytest-cov starts. This is a known issue, but there are workarounds.
5179 You can use coverage run pytest ... instead of pytest --cov ..., opting
5181 out of the pytest plugin entirely. Alternatively, you can ensure that
5182 Hypothesis is loaded after coverage measurement is started by disabling
5183 the entrypoint, and loading our pytest plugin from your conftest.py
5184 instead:
5185 echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
5187 pytest -p "no:hypothesispytest" ...
5188
5190 This is a record of all past Hypothesis releases and what went into
5191 them, in reverse chronological order. All previous releases should
5192 still be available on PyPI.
5193 Hypothesis 5.x
5195 5.43.9 - 2021-01-02
5196 This patch fixes issue #2722, where certain orderings of regis‐
5197 ter_type_strategy(), ForwardRef, and from_type() could trigger an
5198 internal error.
5199 5.43.8 - 2021-01-02
5201 This patch makes some strategies for collections with a uniqueness con‐
5202 straint much more efficient, including dictionaries(keys=sam‐
5203 pled_from(...), values=..) and lists(tuples(sampled_from(...), ...),
5204 unique_by=lambda x: x[0]). (related to issue #2036)
5205 5.43.7 - 2021-01-02
5207 This patch extends our faster special case for sampled_from() elements
5208 in unique lists() to account for chains of .map(...) and .filter(...)
5209 calls (issue #2036).
5210 5.43.6 - 2021-01-02
5212 This patch improves the type annotations on assume() and @repro‐
5213 duce_failure().
5214 5.43.5 - 2021-01-01
5216 This patch updates our copyright headers to include 2021. Happy new
5217 year!
5218 5.43.4 - 2020-12-24
5220 This change fixes a documentation error in the database setting.
5221 The previous documentation suggested that callers could specify a data‐
5223 base path string, or the special string ":memory:", but this setting
5224 has never actually allowed string arguments.
5225 Permitted values are None, and instances of ExampleDatabase.
5227 5.43.3 - 2020-12-11
5229 This patch fixes issue #2696, an internal error triggered when the
5230 @example decorator was used and the verbosity setting was quiet.
5231 5.43.2 - 2020-12-10
5233 This patch improves the error message from the data_frames() strategy
5234 when both the rows and columns arguments are given, but there is a
5235 missing entry in rows and the corresponding column has no fill value (‐
5236 issue #2678).
5237 5.43.1 - 2020-12-10
5239 This patch improves the error message if builds() is passed an Enum
5240 which cannot be called without arguments, to suggest using sam‐
5241 pled_from() (issue #2693).
5242 5.43.0 - 2020-12-09
5244 This release adds new timezones() and timezone_keys() strategies (issue
5245 #2630) based on the new python:zoneinfo module in Python 3.9.
5246 pip install hypothesis[zoneinfo] will ensure that you have the appro‐
5248 priate backports installed if you need them.
5249 5.42.3 - 2020-12-09
5251 This patch fixes an internal error in datetimes() with allow_imagi‐
5252 nary=False where the timezones argument can generate tzinfo=None (issue
5253 #2662).
5254 5.42.2 - 2020-12-09
5256 This patch teaches hypothesis.extra.django.from_field() to infer more
5257 efficient strategies by inspecting (not just filtering by) field val‐
5258 idators for numeric and string fields (issue #1116).
5259 5.42.1 - 2020-12-09
5261 This patch refactors hypothesis.settings to use type-annotated keyword
5262 arguments instead of **kwargs, which makes tab-completion much more
5263 useful - as well as type-checkers like mypy.
5264 5.42.0 - 2020-12-09
5266 This patch teaches the magic() ghostwriter to recognise "en/de" func‐
5267 tion roundtrips other than the common encode/decode pattern, such as
5268 encrypt/decrypt or, encipher/decipher.
5269 5.41.5 - 2020-12-05
5271 This patch adds a performance optimisation to avoid saving redundant
5272 seeds when using the .fuzz_one_input hook.
5273 5.41.4 - 2020-11-28
5275 This patch fixes issue #2657, where passing unicode patterns compiled
5276 with python:re.IGNORECASE to from_regex() could trigger an internal
5277 error when casefolding a character creates a longer string (e.g.
5278 "\u0130".lower() -> "i\u0370").
5279 5.41.3 - 2020-11-18
5281 This patch adds a final fallback clause to our plugin logic to fail
5282 with a warning rather than error on Python < 3.8 when neither the
5283 importlib_metadata (preferred) or setuptools (fallback) packages are
5284 available.
5285 5.41.2 - 2020-11-08
5287 This patch fixes urls() strategy ensuring that ~ (tilde) is treated as
5288 one of the url-safe characters (issue #2658).
5289 5.41.1 - 2020-11-03
5291 This patch improves our CLI help and documentation.
5292 5.41.0 - 2020-10-30
5294 Hypothesis now shrinks examples where the error is raised while drawing
5295 from a strategy. This makes complicated custom strategies much easier
5296 to debug, at the cost of a slowdown for use-cases where you catch and
5297 ignore such errors.
5298 5.40.0 - 2020-10-30
5300 This release teaches from_type() how to handle ChainMap, Counter,
5301 Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).
5302 5.39.0 - 2020-10-30
5304 from_type() now knows how to resolve PEP 585 parameterized standard
5305 collection types, which are new in Python 3.9 (issue #2629).
5306 5.38.1 - 2020-10-26
5308 This patch fixes builds(), so that when passed infer for an argument
5309 with a non-Optional type annotation and a default value of None to
5310 build a class which defines an explicit __signature__ attribute, either
5311 None or that type may be generated.
5312 This is unlikely to happen unless you are using pydantic (issue #2648).
5314 5.38.0 - 2020-10-24
5316 This release improves our support for @st.composite on a python:class‐
5317 method or python:staticmethod (issue #2578).
5318 5.37.5 - 2020-10-24
5320 This patch fixes from_type() with Iterable[T] (issue #2645).
5321 5.37.4 - 2020-10-20
5323 This patch teaches the magic() ghostwriter to recognise that pairs of
5324 functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().
5325 5.37.3 - 2020-10-15
5327 This patch improves builds() and from_type() support for explicitly
5328 defined __signature__ attributes, from version 5.8.3, to support
5329 generic types from the python:typing module.
5330 Thanks to Rónán Carrigan for identifying and fixing this problem!
5332 5.37.2 - 2020-10-14
5334 This patch fixes from_lark() with version 0.10.1+ of the lark-parser
5335 package.
5336 5.37.1 - 2020-10-07
5338 This patch fixes some broken links in the lark extra documentation.
5339 5.37.0 - 2020-10-03
5341 This release adds a new RedisExampleDatabase, along with the ReadOnly‐
5342 Database and MultiplexedDatabase helpers, to support team workflows
5343 where failing examples can be seamlessly shared between everyone on the
5344 team - and your CI servers or buildbots.
5345 5.36.2 - 2020-10-02
5347 This patch ensures that if the "hypothesis" entry point is callable, we
5348 call it after importing it. You can still use non-callable entry
5349 points (like modules), which are only imported.
5350 We also prefer importlib.metadata or the backport over pkg_resources,
5352 which makes import hypothesis around 200 milliseconds faster (issue
5353 #2571).
5354 5.36.1 - 2020-09-25
5356 This patch adds some helpful suggestions to error messages you might
5357 see while learning to use the @example() decorator (issue #2611) or the
5358 one_of() strategy.
5359 5.36.0 - 2020-09-24
5361 This release upgrades the from_dtype() strategy to pass optional
5362 **kwargs to the inferred strategy, and upgrades the arrays() strategy
5363 to accept an elements=kwargs dict to pass through to from_dtype().
5364 arrays(floating_dtypes(), shape, elements={"min_value": -10,
5366 "max_value": 10}) is a particularly useful pattern, as it allows for
5367 any floating dtype without triggering the roundoff warning for smaller
5368 types or sacrificing variety for larger types (issue #2552).
5369 5.35.4 - 2020-09-21
5371 This patch reformats our code with the latest black to take advantage
5372 of the support for magic trailing commas.
5373 5.35.3 - 2020-09-15
5375 This release significantly improves the performance of Hypothesis's
5376 internal implementation of automaton learning. However this code does
5377 not run as part of the user-accessible API so this has no user-visible
5378 impact.
5379 5.35.2 - 2020-09-14
5381 This patch ensures that, when the generate phases is disabled, we can
5382 replay up to max_examples examples from the database - which is very
5383 useful when using Hypothesis with a fuzzer.
5384 Thanks to Afrida Tabassum for fixing issue #2585!
5386 5.35.1 - 2020-09-14
5388 This patch changes some internal python:struct.Struct.format strings
5389 from bytes to str, to avoid python:BytesWarning when running python
5390 -bb.
5391 Thanks to everyone involved in pytest-xdist issue 596, bpo-16349,
5393 bpo-21071, and bpo-41777 for their work on this - it was a remarkably
5394 subtle issue!
5395 5.35.0 - 2020-09-11
5397 The target() function now accepts integers as well as floats.
5398 5.34.1 - 2020-09-11
5400 This patch adds explicit Optional annotations to our public API, to
5401 better support users who run mypy with --strict or
5402 no_implicit_optional=True.
5403 Thanks to Krzysztof Przybyła for bringing this to our attention and
5405 writing the patch!
5406 5.34.0 - 2020-09-11
5408 This release drops support for Python 3.5, which reached end of life
5409 upstream on 2020-09-13.
5410 5.33.2 - 2020-09-09
5412 This patch fixes a problem with builds() that was not able to generate
5413 valid data for annotated classes with constructors.
5414 Thanks to Nikita Sobolev for fixing issue #2603!
5416 5.33.1 - 2020-09-07
5418 This patch improves the error message from the hypothesis write command
5419 if black (required for the ghostwriter) is not installed.
5420 Thanks to Nikita Sobolev for fixing issue #2604!
5422 5.33.0 - 2020-09-06
5424 When reporting failing examples, or tried examples in verbose mode,
5425 Hypothesis now identifies which were from @example(...) explicit exam‐
5426 ples.
5427 5.32.1 - 2020-09-06
5429 This patch contains some internal refactoring. Thanks to Felix Sheldon
5430 for fixing issue #2516!
5431 5.32.0 - 2020-09-04
5433 An array drawn from arrays() will own its own memory; previously most
5434 arrays returned by this strategy were views.
5435 5.31.0 - 2020-09-04
5437 builds() will use the __signature__ attribute of the target, if it
5438 exists, to retrieve type hints. Previously python:typ‐
5439 ing.get_type_hints(), was used by default. If argument names varied
5440 between the __annotations__ and __signature__, they would not be sup‐
5441 plied to the target.
5442 This was particularily an issue for pydantic models which use an alias
5444 generator.
5445 5.30.1 - 2020-09-04
5447 This patch makes the ghostwriter much more robust when passed unusual
5448 modules.
5449 · improved support for non-resolvable type annotations
5451 · magic() can now write equivalent() tests
5453 · running magic() on modules where some names in __all__ are undefined
5455 skips such names, instead of raising an error
5456 · magic() now knows to skip mocks
5458 · improved handling of import-time errors found by the ghostwriter CLI
5460 5.30.0 - 2020-08-30
5462 register_type_strategy() now supports python:typing.TypeVar, which was
5463 previously hard-coded, and allows a variety of types to be generated
5464 for an unconstrained TypeVar instead of just text().
5465 Thanks again to Nikita Sobolev for all your work on advanced types!
5467 5.29.4 - 2020-08-28
5469 This release fixes some hard to trigger bugs in Hypothesis's automata
5470 learning code. This code is only run as part of the Hypothesis build
5471 process, and not for user code, so this release has no user visible
5472 impact.
5473 5.29.3 - 2020-08-27
5475 This patch adds type annotations to the hypothesis.database module.
5476 There is no runtime change, but your typechecker might notice.
5477 5.29.2 - 2020-08-27
5479 This patch tracks some additional information in Hypothesis internals,
5480 and has no user-visible impact.
5481 5.29.1 - 2020-08-27
5483 This release fixes a bug in some Hypothesis internal support code for
5484 learning automata. This mostly doesn't have any user visible impact,
5485 although it slightly affects the learned shrink passes so shrinking may
5486 be subtly different.
5487 5.29.0 - 2020-08-24
5489 This release adds support for entry-points, which allows for smoother
5490 integration of third-party Hypothesis extensions and external
5491 libraries. Unless you're publishing a library with Hypothesis integra‐
5492 tion, you'll probably only ever use this indirectly!
5493 5.28.0 - 2020-08-24
5495 from_type() can now resolve TypeVar instances when the bound is a For‐
5496 wardRef, so long as that name is in fact defined in the same module as
5497 the typevar (no TYPE_CHECKING tricks, sorry). This feature requires
5498 Python 3.7 or later.
5499 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
5501 5.27.0 - 2020-08-20
5503 This patch adds two new ghostwriters to test binary operations, like
5504 python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().
5505 5.26.1 - 2020-08-19
5507 This release improves the performance of some methods in Hypothesis's
5508 internal automaton library. These are currently only lightly used by
5509 user code, but this may result in slightly faster shrinking.
5510 5.26.0 - 2020-08-17
5512 register_type_strategy() no longer accepts parametrised user-defined
5513 generic types, because the resolution logic was quite badly broken (‐
5514 issue #2537).
5515 Instead of registering a strategy for e.g. MyCollection[int], you
5517 should register a function for MyCollection and inspect the type param‐
5518 eters within that function.
5519 Thanks to Nikita Sobolev for the bug report, design assistance, and
5521 pull request to implement this feature!
5522 5.25.0 - 2020-08-16
5524 Tired of writing tests? Or new to Hypothesis and not sure where to
5525 start?
5526 This release is for you! With our new Ghostwriter functions and
5528 hypothesis write ... command-line interface, you can stop writing tests
5529 entirely... or take the source code Hypothesis writes for you as a
5530 starting point.
5531 This has been in the works for months, from issue #2118 to versions
5533 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who
5534 reviewed pull requests or commented on demos, and to Timothy Crosley's
5535 hypothesis-auto project for inspiration.
5536 5.24.4 - 2020-08-14
5538 This patch adds yet more internal functions to support a new feature
5539 we're working on, like version 5.18.3 and version 5.23.6. We promise
5540 it's worth the wait!
5541 5.24.3 - 2020-08-13
5543 This release fixes a small internal bug in Hypothesis's internal autom‐
5544 aton library. Fortunately this bug was currently impossible to hit in
5545 user facing code, so this has no user visible impact.
5546 5.24.2 - 2020-08-12
5548 This release improves shrink quality by allowing Hypothesis to automat‐
5549 ically learn new shrink passes for difficult to shrink tests.
5550 The automatic learning is not currently accessible in user code (it
5552 still needs significant work on robustness and performance before it is
5553 ready for that), but this release includes learned passes that should
5554 improve shrinking quality for tests which use any of the text(),
5555 floats(), datetimes(), emails(), and complex_numbers() strategies.
5556 5.24.1 - 2020-08-12
5558 This patch updates some docstrings, without changing runtime behaviour.
5559 5.24.0 - 2020-08-10
5561 The functions() strategy has a new argument pure=True, which ensures
5562 that the same return value is generated for identical calls to the gen‐
5563 erated function (issue #2538).
5564 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
5566 5.23.12 - 2020-08-10
5568 This release removes a number of Hypothesis's internal "shrink passes"
5569 - transformations it makes to a generated test case during shrinking -
5570 which appeared to be redundant with other transformations.
5571 It is unlikely that you will see much impact from this. If you do, it
5573 will likely show up as a change in shrinking performance (probably
5574 slower, maybe faster), or possibly in worse shrunk results. If you
5575 encounter the latter, please let us know.
5576 5.23.11 - 2020-08-04
5578 This release fixes a bug in some internal Hypothesis support code. It
5579 has no user visible impact.
5580 5.23.10 - 2020-08-04
5582 This release improves the quality of shrunk test cases in some special
5583 cases. Specifically, it should get shrinking unstuck in some scenarios
5584 which require simultaneously changing two parts of the generated test
5585 case.
5586 5.23.9 - 2020-08-03
5588 This release improves the performance of some internal support code. It
5589 has no user visible impact, as that code is not currently run during
5590 normal Hypothesis operation.
5591 5.23.8 - 2020-07-31
5593 This release adds a heuristic to detect when shrinking has finished
5594 despite the fact that there are many more possible transformations to
5595 try. This will be particularly useful for tests where the minimum fail‐
5596 ing test case is very large despite there being many smaller test cases
5597 possible, where it is likely to speed up shrinking dramatically.
5598 In some cases it is likely that this will result in worse shrunk test
5600 cases. In those cases rerunning the test will result in further shrink‐
5601 ing.
5602 5.23.7 - 2020-07-29
5604 This release makes some performance improvements to shrinking. They
5605 should only be noticeable for tests that are currently particularly
5606 slow to shrink.
5607 5.23.6 - 2020-07-29
5609 This patch adds some more internal functions to support a new feature
5610 we're working on, like version 5.18.3. There is still no user-visible
5611 change... yet.
5612 5.23.5 - 2020-07-29
5614 This release makes some changes to internal support code that is not
5615 currently used in production Hypothesis. It has no user visible effect
5616 at present.
5617 5.23.4 - 2020-07-29
5619 This release improves shrinking quality in some special cases.
5620 5.23.3 - 2020-07-27
5622 This release fixes issue #2507, where lazy evaluation meant that the
5623 values drawn from a sampled_from() strategy could depend on mutations
5624 of the sampled sequence that happened after the strategy was con‐
5625 structed.
5626 5.23.2 - 2020-07-27
5628 This patch fixes issue #2462, a bug in our handling of unittest.Test‐
5629 Case.subTest(). Thanks to Israel Fruchter for fixing this at the
5630 EuroPython sprints!
5631 5.23.1 - 2020-07-26
5633 This release improves the behaviour of the characters() strategy when
5634 shrinking, by changing which characters are considered smallest to pre‐
5635 fer more "normal" ascii characters where available.
5636 5.23.0 - 2020-07-26
5638 The default print_blob setting is now smarter. It defaults to True in
5639 CI and False for local development.
5640 Thanks to Hugo van Kemenade for implementing this feature at the
5642 EuroPython sprints!
5643 5.22.0 - 2020-07-25
5645 The slices() strategy can now generate slices for empty sequences,
5646 slices with negative start and stop indices (from the end of the
5647 sequence), and step=None in place of step=1.
5648 Thanks to Sangarshanan for implementing this feature at the EuroPython
5650 sprints!
5651 5.21.0 - 2020-07-23
5653 This release ensures that tests which raise RecursionError are not
5654 reported as flaky simply because we run them from different initial
5655 stack depths (issue #2494).
5656 5.20.4 - 2020-07-23
5658 This release improves the performance of the sample method on objects
5659 obtained from randoms() when use_true_random=False. This should mostly
5660 only be noticeable when the sample size is a large fraction of the pop‐
5661 ulation size, but may also help avoid health check failures in some
5662 other cases.
5663 5.20.3 - 2020-07-21
5665 This release makes some internal changes for testing purposes and
5666 should have no user visible effect.
5667 5.20.2 - 2020-07-18
5669 This release fixes a small caching bug in Hypothesis internals that may
5670 under some circumstances have resulted in a less diverse set of test
5671 cases being generated than was intended.
5672 Fixing this problem revealed some performance problems that could occur
5674 during targeted property based testing, so this release also fixes
5675 those. Targeted property-based testing should now be significantly
5676 faster in some cases, but this may be at the cost of reduced effective‐
5677 ness.
5678 5.20.1 - 2020-07-17
5680 This patch updates our formatting to use isort 5. There is no
5681 user-visible change.
5682 5.20.0 - 2020-07-17
5684 The basic_indices() strategy can now generate bare indexers in place of
5685 length-one tuples. Thanks to Andrea for this patch!
5686 5.19.3 - 2020-07-15
5688 This patch removes an internal use of distutils in order to avoid this
5689 setuptools warning for some users.
5690 5.19.2 - 2020-07-13
5692 This patch contains a small internal refactoring with no user-visible
5693 impact.
5694 Thanks to Andrea for writing this at the SciPy 2020 Sprints!
5696 5.19.1 - 2020-07-12
5698 This release slightly improves shrinking behaviour. This should mainly
5699 only impact stateful tests, but may have some minor positive impact on
5700 shrinking collections (lists, sets, etc).
5701 5.19.0 - 2020-06-30
5703 This release improves the randoms() strategy by adding support for Ran‐
5704 dom instances where Hypothesis generates the random values rather than
5705 having them be "truly" random.
5706 5.18.3 - 2020-06-27
5708 This patch adds some internal functions to support a new feature we're
5709 working on. There is no user-visible change... yet.
5710 5.18.2 - 2020-06-26
5712 This patch improves our docs for the derandomize setting.
5713 5.18.1 - 2020-06-25
5715 This release consists of some internal refactoring to the shrinker in
5716 preparation for future work. It has no user visible impact.
5717 5.18.0 - 2020-06-22
5719 This release teaches Hypothesis to shorten tracebacks for explicit
5720 examples, as we already do for generated examples, so that you can
5721 focus on your code rather than ours.
5722 If you have multiple failing explicit examples, they will now all be
5724 reported. To report only the first failure, you can use the
5725 report_multiple_bugs=False setting as for generated examples.
5726 5.17.0 - 2020-06-22
5728 This patch adds strategy inference for the Literal, NewType, Type,
5729 DefaultDict, and TypedDict types from the typing_extensions backport on
5730 PyPI.
5731 5.16.3 - 2020-06-21
5733 This patch precomputes some of the setup logic for our experimental
5734 external fuzzer integration and sets deadline=None in fuzzing mode,
5735 saving around 150us on each iteration.
5736 This is around two-thirds the runtime to fuzz an empty test with
5738 @given(st.none()), and nice to have even as a much smaller fraction of
5739 the runtime for non-trivial tests.
5740 5.16.2 - 2020-06-19
5742 This patch fixes an internal error when warning about the use of func‐
5743 tion-scoped fixtures for parametrised tests where the parametrised
5744 value contained a % character. Thanks to Bryant for reporting and fix‐
5745 ing this bug!
5746 5.16.1 - 2020-06-10
5748 If you pass a python:list or python:tuple where a strategy was
5749 expected, the error message now mentions sampled_from() as an example
5750 strategy.
5751 Thanks to the enthusiastic participants in the PyCon Mentored Sprints
5753 who suggested adding this hint.
5754 5.16.0 - 2020-05-27
5756 functions() can now infer the appropriate returns strategy if you pass
5757 a like function with a return-type annotation. Before, omitting the
5758 returns argument would generate functions that always returned None.
5759 5.15.1 - 2020-05-21
5761 Fix from_type() with generic types under Python 3.9.
5762 5.15.0 - 2020-05-19
5764 This patch fixes an error that happens when multiple threads create new
5765 strategies.
5766 5.14.0 - 2020-05-13
5768 Passing min_magnitude=None to complex_numbers() is now deprecated - you
5769 can explicitly pass min_magnitude=0, or omit the argument entirely.
5770 5.13.1 - 2020-05-13
5772 This patch fixes an internal error in from_type() for python:typ‐
5773 ing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and
5774 fixing issue #2427!
5775 5.13.0 - 2020-05-12
5777 This release upgrades the test statistics available via the --hypothe‐
5778 sis-show-statistics option to include separate information on each of
5779 the phases (issue #1555).
5780 5.12.2 - 2020-05-12
5782 This patch teaches the from_type() internals to return slightly more
5783 efficient strategies for some generic sets and mappings.
5784 5.12.1 - 2020-05-12
5786 This patch adds a # noqa comment for flake8 3.8.0, which disagrees with
5787 mypy about how to write the type of ....
5788 5.12.0 - 2020-05-10
5790 This release limits the maximum duration of the shrinking phase to five
5791 minutes, so that Hypothesis does not appear to hang when making very
5792 slow progress shrinking a failing example (issue #2340).
5793 If one of your tests triggers this logic, we would really appreciate a
5795 bug report to help us improve the shrinker for difficult but realistic
5796 workloads.
5797 5.11.0 - 2020-05-07
5799 This release improves the interaction between assume() and the @exam‐
5800 ple() decorator, so that the following test no longer fails with Unsat‐
5801 isfiedAssumption (issue #2125):
5802 @given(value=floats(0, 1))
5804 @example(value=0.56789) # used to make the test fail!
5805 @pytest.mark.parametrize("threshold", [0.5, 1])
5806 def test_foo(threshold, value):
5807 assume(value < threshold)
5808 ...
5809 5.10.5 - 2020-05-04
5811 If you have django installed but don't use it, this patch will make
5812 import hypothesis a few hundred milliseconds faster (e.g. 0.704s ->
5813 0.271s).
5814 Thanks to importtime-waterfall for highlighting this problem and Jake
5816 Vanderplas for the solution - it's impossible to misuse code from a
5817 module you haven't imported!
5818 5.10.4 - 2020-04-24
5820 This patch improves the internals of builds() type inference, to handle
5821 recursive forward references in certain dataclasses. This is useful
5822 for e.g. hypothesmith's forthcoming LibCST mode.
5823 5.10.3 - 2020-04-22
5825 This release reverses the order in which some operations are tried dur‐
5826 ing shrinking. This should generally be a slight performance improve‐
5827 ment, but most tests are unlikely to notice much difference.
5828 5.10.2 - 2020-04-22
5830 This patch fixes issue #2406, where use of pandas:pandas.Timestamp
5831 objects as bounds for the datetimes() strategy caused an internal
5832 error. This bug was introduced in version 5.8.1.
5833 5.10.1 - 2020-04-19
5835 This release is a small internal refactoring to how shrinking interacts
5836 with targeted property-based testing that should have no user user vis‐
5837 ible impact.
5838 5.10.0 - 2020-04-18
5840 This release improves our support for datetimes and times around DST
5841 transitions.
5842 times() and datetimes() are now sometimes generated with fold=1, indi‐
5844 cating that they represent the second occurrence of a given wall-time
5845 when clocks are set backwards. This may be set even when there is no
5846 transition, in which case the fold value should be ignored.
5847 For consistency, timezones provided by the pytz package can now gener‐
5849 ate imaginary times (such as the hour skipped over when clocks 'spring
5850 forward' to daylight saving time, or during some historical timezone
5851 transitions). All other timezones have always supported generation of
5852 imaginary times.
5853 If you prefer the previous behaviour, datetimes() now takes an argument
5855 allow_imaginary which defaults to True but can be set to False for any
5856 timezones strategy.
5857 5.9.1 - 2020-04-16
5859 This patch fixes the rendering of binary() docstring by using the
5860 proper backticks syntax.
5861 5.9.0 - 2020-04-15
5863 Failing tests which use target() now report the highest score observed
5864 for each target alongside the failing example(s), even without explic‐
5865 itly showing test statistics.
5866 This improves the debugging workflow for tests of accuracy, which
5868 assert that the total imprecision is within some error budget - for
5869 example, abs(a - b) < 0.5. Previously, shrinking to a minimal failing
5870 example could often make errors seem smaller or more subtle than they
5871 really are (see the threshold problem, and issue #2180).
5872 5.8.6 - 2020-04-15
5874 This patch improves the docstring of binary(), the python:repr() of
5875 sampled_from() on an python:enum.Enum subclass, and a warning in our
5876 pytest plugin. There is no change in runtime behaviour.
5877 5.8.5 - 2020-04-15
5879 This release (potentially very significantly) improves the performance
5880 of failing tests in some rare cases, mostly only relevant when using
5881 targeted property-based testing, by stopping further optimisation of
5882 unrelated test cases once a failing example is found.
5883 5.8.4 - 2020-04-14
5885 This release fixes issue #2395, where under some circumstances targeted
5886 property-based testing could cause Hypothesis to get caught in an infi‐
5887 nite loop.
5888 5.8.3 - 2020-04-12
5890 This patch teaches builds() and from_type() to use the __signature__
5891 attribute of classes where it has been set, improving our support for
5892 Pydantic models (in pydantic >= 1.5).
5893 5.8.2 - 2020-04-12
5895 This release improves the performance of the part of the core engine
5896 that deliberately generates duplicate values.
5897 5.8.1 - 2020-04-12
5899 This patch improves dates() shrinking, to simplify year, month, and day
5900 like datetimes() rather than minimizing the number of days since
5901 2000-01-01.
5902 5.8.0 - 2020-03-24
5904 This release adds a .hypothesis.fuzz_one_input attribute to @given
5905 tests, for easy integration with external fuzzers such as python-afl
5906 (supporting issue #171).
5907 5.7.2 - 2020-03-24
5909 This patch fixes issue #2341, ensuring that the printed output from a
5910 stateful test cannot use variable names before they are defined.
5911 5.7.1 - 2020-03-23
5913 This patch fixes issue #2375, preventing incorrect failure when a func‐
5914 tion scoped fixture is overridden with a higher scoped fixture.
5915 5.7.0 - 2020-03-19
5917 This release allows the array_dtypes() strategy to generate Numpy
5918 dtypes which have field titles in addition to field names. We expect
5919 this to expose latent bugs where code expects that set(dtype.names) ==
5920 set(dtype.fields), though the latter may include titles.
5921 5.6.1 - 2020-03-18
5923 This makes model a positional-only argument to from_model(), to support
5924 models with a field literally named "model" (issue #2369).
5925 5.6.0 - 2020-02-29
5927 This release adds an explicit warning for tests that are both decorated
5928 with @given(...) and request a function-scoped pytest fixture, because
5929 such fixtures are only executed once for all Hypothesis test cases and
5930 that often causes trouble (issue #377).
5931 It's very difficult to fix this on the pytest side, so since 2015 our
5933 advice has been "just don't use function-scoped fixtures with Hypothe‐
5934 sis". Now we detect and warn about the issue at runtime!
5935 5.5.5 - 2020-02-29
5937 This release cleans up the internal machinery for stateful, after we
5938 dropped the legacy APIs in Hypothesis 5.0 (issue #2218). There is no
5939 user-visible change.
5940 5.5.4 - 2020-02-16
5942 This patch fixes issue #2351, arrays() would raise a confusing error if
5943 we inferred a strategy for datetime64 or timedelta64 values with vary‐
5944 ing time units.
5945 We now infer an internally-consistent strategy for such arrays, and
5947 have a more helpful error message if an inconsistent strategy is
5948 explicitly specified.
5949 5.5.3 - 2020-02-14
5951 This patch improves the signature of builds() by specifying target as a
5952 positional-only argument on Python 3.8 (see PEP 570). The semantics of
5953 builds() have not changed at all - this just clarifies the documenta‐
5954 tion.
5955 5.5.2 - 2020-02-13
5957 This release makes Hypothesis faster at generating test cases that con‐
5958 tain duplicated values in their inputs.
5959 5.5.1 - 2020-02-07
5961 This patch has some tiny internal code clean-ups, with no user-visible
5962 change.
5963 5.5.0 - 2020-02-07
5965 Our style guide suggests that optional parameters should usually be
5966 keyword-only arguments (see PEP 3102) to prevent confusion based on
5967 positional arguments - for example, hypothesis.strategies.floats()
5968 takes up to four boolean flags and many of the Numpy strategies have
5969 both dims and side bounds.
5970 This release converts most optional parameters in our API to use key‐
5972 word-only arguments - and adds a compatibility shim so you get warnings
5973 rather than errors everywhere (issue #2130).
5974 5.4.2 - 2020-02-06
5976 This patch fixes compatibility with Python 3.5.2 (issue #2334). Note
5977 that we only test the latest patch of each minor version, though as in
5978 this case we usually accept pull requests for older patch versions.
5979 5.4.1 - 2020-02-01
5981 This patch improves the repr of from_type(), so that in most cases it
5982 will display the strategy it resolves to rather than from_type(...).
5983 The latter form will continue to be used where resolution is not imme‐
5984 diately successful, e.g. invalid arguments or recursive type defini‐
5985 tions involving forward references.
5986 5.4.0 - 2020-01-30
5988 This release removes support for Python 3.5.0 and 3.5.1, where the
5989 python:typing module was quite immature (e.g. missing overload() and
5990 Type).
5991 Note that Python 3.5 will reach its end-of-life in September 2020, and
5993 new releases of Hypothesis may drop support somewhat earlier.
5994 NOTE:
5996 pip install hypothesis should continue to give you the latest com‐
5997 patible version. If you have somehow ended up with an incompatible
5998 version, you need to update your packaging stack to pip >= 9.0 and
5999 setuptools >= 24.2 - see here for details. Then pip uninstall
6000 hypothesis && pip install hypothesis will get you back to a compati‐
6001 ble version.
6002 5.3.1 - 2020-01-26
6004 This patch does some minor internal cleanup; there is no user-visible
6005 change.
6006 5.3.0 - 2020-01-21
6008 The standard library ipaddress module is new in Python 3, and this
6009 release adds the new ip_addresses() strategy to generate IPv4Addresses
6010 and/or IPv6Addresses (depending on the v and network arguments).
6011 If you use them in type annotations, from_type() now has strategies
6013 registered for ipaddress address, network, and interface types.
6014 The provisional strategies for IP address strings are therefore depre‐
6016 cated.
6017 5.2.1 - 2020-01-21
6019 This patch reverts version 5.2, due to a strange issue where indexing
6020 an array of strings can raise an error instead of returning an item
6021 which contains certain surrogate characters.
6022 5.2.0 - 2020-01-19
6024 This release allows from_dtype() to generate Unicode strings which can‐
6025 not be encoded in UTF-8, but are valid in Numpy arrays (which use
6026 UTF-32).
6027 5.1.6 - 2020-01-19
6029 This patch fixes issue #2320, where from_type(Set[Hashable]) could
6030 raise an internal error because Decimal("snan") is of a hashable type,
6031 but raises an error when hashed. We now ensure that set elements and
6032 dict keys in generic types can actually be hashed.
6033 5.1.5 - 2020-01-12
6035 This patch fixes an internal error when running in an IPython repl or
6036 Jupyter notebook on Windows (issue #2319), and an internal error on
6037 Python 3.5.1 (issue #2318).
6038 5.1.4 - 2020-01-11
6040 This patch fixes a bug where errors in third-party extensions such as
6041 hypothesis-trio or hypothesis-jsonschema were incorrectly considered to
6042 be Hypothesis internal errors, which could result in confusing error
6043 messages.
6044 Thanks to Vincent Michel for reporting and fixing the bug!
6046 5.1.3 - 2020-01-11
6048 This release converts the type hint comments on our public API to PEP
6049 484 type annotations.
6050 Thanks to Ivan Levkivskyi for com2ann - with the refactoring tools from
6052 5.0.1 it made this process remarkably easy!
6053 5.1.2 - 2020-01-09
6055 This patch makes multiple() iterable, so that output like a, b =
6056 state.some_rule() is actually executable and can be used to reproduce
6057 failing examples.
6058 Thanks to Vincent Michel for reporting and fixing issue #2311!
6060 5.1.1 - 2020-01-06
6062 This patch contains many small refactorings to replace our Python 2
6063 compatibility functions with their native Python 3 equivalents. Since
6064 Hypothesis is now Python 3 only, there is no user-visible change.
6065 5.1.0 - 2020-01-03
6067 This release teaches from_type() how to generate python:datetime.time‐
6068 zone. As a result, you can now generate python:datetime.tzinfo objects
6069 without having pytz installed.
6070 If your tests specifically require pytz timezones, you should be using
6072 hypothesis.extra.pytz.timezones() instead of st.from_type(tzinfo).
6073 5.0.1 - 2020-01-01
6075 This patch contains mostly-automated refactorings to remove code that
6076 we only needed to support Python 2. Since Hypothesis is now Python 3
6077 only (hurray!), there is no user-visible change.
6078 Our sincere thanks to the authors of autoflake, black, isort, and
6080 pyupgrade, who have each and collectively made this kind of update
6081 enormously easier.
6082 5.0.0 - 2020-01-01
6084 Welcome to the next major version of Hypothesis!
6085 There are no new features here, as we release those in minor versions.
6087 Instead, 5.0 is a chance for us to remove deprecated features (many
6088 already converted into no-ops), and turn a variety of warnings into
6089 errors.
6090 If you were running on the last version of Hypothesis 4.x without any
6092 Hypothesis deprecation warnings, this will be a very boring upgrade.
6093 In fact, nothing will change for you at all.
6094 NOTE:
6096 This release drops support for Python 2, which has passed its end of
6097 life date. The Python 3 Statement outlines our reasons, and lists
6098 many other packages that have made the same decision.
6099 pip install hypothesis should continue to give you the latest com‐
6101 patible version. If you have somehow ended up with Hypothesis 5.0
6102 on Python 2, you need to update your packaging stack to pip >= 9.0
6103 and setuptools >= 24.2 - see here for details. Then pip uninstall
6104 hypothesis && pip install hypothesis will get you back to a compati‐
6105 ble version.
6106 Strategies
6108 · integers() bounds must be equal to an integer, though they can still
6109 be other types.
6110 · If fractions() is passed a max_denominator, the bounds must have at
6112 most that denominator.
6113 · floats() bounds must be exactly representable as a floating-point
6115 number with the given width. If not, the error message includes the
6116 nearest such number.
6117 · sampled_from([]) is now an error.
6119 · The values from the elements and fill strategies for hypothe‐
6121 sis.extra.numpy.arrays() must be losslessly representable in an array
6122 of the given dtype.
6123 · The min_size and max_size arguments to all collection strategies must
6125 be of type python:int (or max_size may be None).
6126 Miscellaneous
6128 · The .example() method of strategies (intended for interactive explo‐
6129 ration) no longer takes a random argument.
6130 · It is now an error to apply @example, @seed, or @reproduce_failure
6132 without also applying @given.
6133 · You may pass either the target or targets argument to stateful rules,
6135 but not both.
6136 · deadline must be None (to disable), a timedelta, or an integer or
6138 float number of milliseconds.
6139 · Both of derandomize and print_blob must be either True or False,
6141 where they previously accepted other values.
6142 · stateful_step_count must be at least one.
6144 · max_examples must be at least one. To disable example generation,
6146 use the phases setting.
6147 Removals
6149 · hypothesis.stateful.GenericStateMachine in favor of hypothesis.state‐
6150 ful.RuleBasedStateMachine
6151 · hypothesis.extra.django.models.models in favor of hypothe‐
6153 sis.extra.django.from_model() and hypothesis.extra.django.mod‐
6154 els.add_default_field_mapping in favor of hypothe‐
6155 sis.extra.django.register_field_strategy()
6156 · hypothesis.HealthCheck.hung_test, without replacement
6158 · hypothesis.settings.buffer, without replacement
6160 · hypothesis.PrintSettings, because hypothesis.settings.print_blob
6162 takes True or False
6163 · hypothesis.settings.timeout, in favor of hypothesis.settings.deadline
6165 · hypothesis.unlimited without replacement (only only useful as argu‐
6167 ment to timeout)
6168 Hypothesis 4.x
6170 4.57.1 - 2019-12-29
6171 This patch improves the type hints and documentation for the django
6172 extra. There is no runtime change.
6173 4.57.0 - 2019-12-28
6175 This release improves support for the SupportsOp protocols from the
6176 python:typing module when using on from_type() as outlined in issue
6177 #2292. The following types now generate much more varied strategies
6178 when called with from_type():
6179 · python:typing.SupportsAbs
6181 · python:typing.SupportsBytes
6183 · python:typing.SupportsComplex
6185 · python:typing.SupportsInt
6187 · python:typing.SupportsFloat
6189 · python:typing.SupportsRound
6191 Note that using from_type() with one of the above strategies will not
6193 ensure that the the specified function will execute successfully (ie :
6194 the strategy returned for from_type(typing.SupportsAbs) may include
6195 NaNs or things which cause the python:abs() function to error. )
6196 Thanks to Lea Provenzano for this patch.
6198 4.56.3 - 2019-12-22
6200 This release fixes a small internal bug in shrinking which could have
6201 caused it to perform slightly more tests than were necessary. Fixing
6202 this shouldn't have much effect but it will make shrinking slightly
6203 faster.
6204 4.56.2 - 2019-12-21
6206 This release removes an internal heuristic that was no longer providing
6207 much benefit. It is unlikely that there will be any user visible
6208 effect.
6209 4.56.1 - 2019-12-19
6211 This release further improves the optimisation algorithm for targeted
6212 property-based testing.
6213 4.56.0 - 2019-12-18
6215 This release enables deprecation warnings even when the verbosity set‐
6216 ting is quiet, in preparation for Hypothesis 5.0 (issue #2218).
6217 Warnings can still be filtered by the standard mechanisms provided in
6219 the standard-library python:warnings module.
6220 4.55.4 - 2019-12-18
6222 This release improves Hypothesis's management of the set of test cases
6223 it tracks between runs. It will only do anything if you have Phase.tar‐
6224 get enabled and an example database set. In those circumstances it
6225 should result in a more thorough and faster set of examples that are
6226 tried on each run.
6227 4.55.3 - 2019-12-18
6229 This release makes Hypothesis better at generating test cases where
6230 generated values are duplicated in different parts of the test case.
6231 This will be especially noticeable with reasonably complex values, as
6232 it was already able to do this for simpler ones such as integers or
6233 floats.
6234 4.55.2 - 2019-12-17
6236 This release expands the set of test cases that Hypothesis saves in its
6237 database for future runs to include a representative set of "struc‐
6238 turally different" test cases - e.g. it might try to save test cases
6239 where a given list is empty or not.
6240 Currently this is unlikely to have much user visible impact except to
6242 produce slightly more consistent behaviour between consecutive runs of
6243 a test suite. It is mostly groundwork for future improvements which
6244 will exploit this functionality more effectively.
6245 4.55.1 - 2019-12-16
6247 This patch fixes issue #2257, where from_type() could incorrectly gen‐
6248 erate bytestrings when passed a generic python:typing.Sequence such as
6249 Sequence[set].
6250 4.55.0 - 2019-12-16
6252 This release adds database support for targeted property-based testing,
6253 so the best examples based on the targeting will be saved and reused
6254 between runs. This is mostly laying groundwork for future features in
6255 this area, but will also make targeted property-based tests more useful
6256 during development, where the same tests tend to get run over and over
6257 again.
6258 If max_examples is large, this may increase memory usage significantly
6260 under some circumstances, but these should be relatively rare.
6261 This release also adds a dependency on the sortedcontainers package.
6263 4.54.2 - 2019-12-16
6265 This release improves the optimisation algorithm for targeted prop‐
6266 erty-based testing, so that it will find higher quality results more
6267 reliably. Specifically, in cases where it would previously have got
6268 near a local optimum, it will now tend to achieve the locally optimal
6269 value.
6270 4.54.1 - 2019-12-16
6272 This release is mostly internal changes in support of better testing of
6273 the core engine. You are unlikely to see much effect, although some
6274 internal heuristics have changed slightly.
6275 4.54.0 - 2019-12-15
6277 This release adds a dedicated phase for targeted property-based test‐
6278 ing, and (somewhat) improves the targeting algorithm so that it will
6279 find higher quality results more reliably. This comes at a cost of
6280 making it more likely to get stuck in a local optimum.
6281 4.53.3 - 2019-12-15
6283 This patch fixes from_type() with python:typing.Hashable and
6284 python:typing.Sized, which previously failed with an internal error on
6285 Python 3.7 or later.
6286 Thanks to Lea Provenzano for both reporting issue #2272 and writing the
6288 patch!
6289 4.53.2 - 2019-12-11
6291 This release reorganises a number of the Hypothesis internal modules
6292 into a package structure. If you are only depending on the public API
6293 it should have no effect. If you are depending on the internal API
6294 (which you shouldn't be, and which we don't guarantee compatibility on)
6295 you may have to rename some imports.
6296 4.53.1 - 2019-12-09
6298 This release changes the size distribution of the number of steps run
6299 in stateful testing: It will now almost always run the maximum number
6300 of steps permitted.
6301 4.53.0 - 2019-12-09
6303 statistics now include the best score seen for each label, which can
6304 help avoid the threshold problem when the minimal example shrinks
6305 right down to the threshold of failure (issue #2180).
6306 4.52.0 - 2019-12-09
6308 This release changes the stateful_step_count setting to raise an error
6309 if set to 0. This is a backwards compatible change because a value of 0
6310 would never have worked and attempting to run it would have resulted in
6311 an internal assertion error.
6312 4.51.1 - 2019-12-09
6314 This release makes a small internal change to the distribution of test
6315 cases. It is unlikely to have much user visible impact.
6316 4.51.0 - 2019-12-07
6318 This release deprecates use of @example, @seed, or @reproduce_failure
6319 without @given.
6320 Thanks to Nick Anyos for the patch!
6322 4.50.8 - 2019-12-05
6324 This patch makes certain uses of Bundles more efficient in stateful
6325 testing (issue #2078).
6326 4.50.7 - 2019-12-05
6328 This release refactors some of Hypothesis's internal interfaces for
6329 representing data generation. It should have no user visible effect.
6330 4.50.6 - 2019-12-02
6332 This patch removes some old debugging helpers in our Numpy extra which
6333 have not been needed since issue #1963 and issue #2245.
6334 4.50.5 - 2019-12-01
6336 This patch fixes issue #2229, where Numpy arrays of unsized strings
6337 would only ever have strings of size one due to an interaction between
6338 our generation logic and Numpy's allocation strategy.
6339 4.50.4 - 2019-12-01
6341 This patch fixes a rare internal error in strategies for a list of
6342 unique items sampled from a short non-unique sequence (issue #2247).
6343 The bug was discovered via hypothesis-jsonschema.
6344 4.50.3 - 2019-12-01
6346 This release improves the error message when @settings tries to inherit
6347 settings from a parent argument that isn't a settings instance.
6348 4.50.2 - 2019-11-29
6350 This release improves Hypothesis's "Falsifying example" output, by
6351 breaking output across multiple lines where necessary, and by removing
6352 irrelevant information from the stateful testing output.
6353 4.50.1 - 2019-11-29
6355 This patch adds flake8-comprehensions to our linter suite. There is no
6356 user-visible change - expect perhaps via some strange microbenchmarks -
6357 but certain parts of the code now have a clear and more consistent
6358 style.
6359 4.50.0 - 2019-11-28
6361 This release fixes some cases where we might previously have failed to
6362 run the validation logic for some strategies. As a result tests which
6363 would previously have been silently testing significantly less than
6364 they should may now start to raise InvalidArgument now that these
6365 errors are caught.
6366 4.49.0 - 2019-11-28
6368 This release significantly improves the data distribution in rule based
6369 stateful testing, by using a technique called Swarm Testing (Groce,
6370 Alex, et al. "Swarm testing." Proceedings of the 2012 International
6371 Symposium on Software Testing and Analysis. ACM, 2012.) to select
6372 which rules are run in any given test case. This should allow it to
6373 find many issues that it would previously have missed.
6374 This change is likely to be especially beneficial for stateful tests
6376 with large numbers of rules.
6377 4.48.1 - 2019-11-28
6379 This release adds some heuristics to test case generation that try to
6380 ensure that test cases generated early on will be relatively small.
6381 This fixes a bug introduced in Hypothesis 4.42.0 which would cause
6383 occasional too_slow failures on some tests.
6384 4.48.0 - 2019-11-28
6386 This release revokes the deprecation of find, as we've now rebuilt it
6387 on top of @given, which means it has minimal maintenance burden and
6388 we're happy to support it.
6389 4.47.5 - 2019-11-28
6391 This release rebuilds find() on top of @given in order to have more
6392 code in common. It should have minimal user visible effect.
6393 4.47.4 - 2019-11-27
6395 This patch removes an internal compatibility shim that we no longer
6396 need.
6397 4.47.3 - 2019-11-26
6399 This patch fixes several typos in our docstrings and comments, with no
6400 change in behaviour. Thanks to Dmitry Dygalo for identifying and fix‐
6401 ing them!
6402 4.47.2 - 2019-11-25
6404 This release fixes an internal issue where Hypothesis would sometimes
6405 generate test cases that were above its intended maximum size. This
6406 would only have happened rarely and probably would not have caused
6407 major problems when it did.
6408 Users of the new targeted property-based testing might see minor
6410 impact (possibly slightly faster tests and slightly worse target
6411 scores), but only in the unlikely event that they were hitting this
6412 problem. Other users should not see any effect at all.
6413 4.47.1 - 2019-11-24
6415 This release removes some unused code from the core engine. There is
6416 no user-visible change.
6417 4.47.0 - 2019-11-24
6419 This release commonizes some code between running explicit examples and
6420 normal test execution. The main user visible impact of this is that
6421 deadlines are now enforced when running explicit examples.
6422 4.46.1 - 2019-11-23
6424 This patch ensures that a KeyboardInterrupt received during example
6425 generation is not treated as a mystery test failure but instead propa‐
6426 gates to the top level, not recording the interrupted generation in the
6427 conjecture data tree. Thanks to Anne Archibald for identifying and
6428 fixing the problem.
6429 4.46.0 - 2019-11-22
6431 This release changes the behaviour of floats() when excluding signed
6432 zeros - floats(max_value=0.0, exclude_max=True) can no longer generate
6433 -0.0 nor the much rarer floats(min_value=-0.0, exclude_min=True) gener‐
6434 ate +0.0.
6435 The correct interaction between signed zeros and exclusive endpoints
6437 was unclear; we now enforce the invariant that floats() will never gen‐
6438 erate a value equal to an excluded endpoint (issue #2201).
6439 If you prefer the old behaviour, you can pass floats(max_value=-0.0) or
6441 floats(min_value=0.0) which is exactly equivalent and has not changed.
6442 If you had two endpoints equal to zero, we recommend clarifying your
6443 tests by using just() or sampled_from() instead of floats().
6444 4.45.1 - 2019-11-20
6446 This patch improves the error message when invalid arguments are passed
6447 to rule() or invariant() (issue #2149).
6448 Thanks to Benjamin Palmer for this bugfix!
6450 4.45.0 - 2019-11-20
6452 This release supports python:typing.Final and python:typing.TypedDict
6453 in from_type().
6454 4.44.5 - 2019-11-20
6456 This patch disables our pytest plugin when running on versions of
6457 pytest before 4.3, the oldest our plugin supports. Note that at time
6458 of writing the Pytest developers only support 4.6 and later!
6459 Hypothesis tests using @given() work on any test runner, but our inte‐
6461 grations to e.g. avoid example database collisions when combined with
6462 @pytest.mark.parametrize eventually drop support for obsolete versions.
6463 4.44.4 - 2019-11-20
6465 This patch adds some internal comments and clarifications to the
6466 Hypothesis implementation. There is no user-visible change.
6467 4.44.3 - 2019-11-20
6469 This patch avoids importing test runners such as pytest, unittest2, or
6470 nose solely to access their special "skip test" exception types - if
6471 the module is not in sys.modules, the exception can't be raised anyway.
6472 This fixes a problem where importing an otherwise unused module could
6474 cause spurious errors due to import-time side effects (and possibly
6475 -Werror).
6476 4.44.2 - 2019-11-12
6478 This release fixes @given to only complain about missing keyword-only
6479 arguments if the associated test function is actually called.
6480 This matches the behaviour of other InvalidArgument errors produced by
6482 @given.
6483 4.44.1 - 2019-11-11
6485 This patch allows Hypothesis to run in environments that do not specify
6486 a __file__, such as a python:zipapp (issue #2196).
6487 4.44.0 - 2019-11-11
6489 This release adds a signature argument to mutually_broad‐
6490 castable_shapes() (issue #2174), which allows us to generate shapes
6491 which are valid for functions like numpy:numpy.matmul that require
6492 shapes which are not simply broadcastable.
6493 Thanks to everyone who has contributed to this feature over the last
6495 year, and a particular shout-out to Zac Hatfield-Dodds and Ryan Sok‐
6496 laski for mutually_broadcastable_shapes() and to Ryan Turner for the
6497 downstream hypothesis-gufunc project.
6498 4.43.9 - 2019-11-11
6500 This patch fixes issue #2108, where the first test using data() to draw
6501 from characters() or text() would be flaky due to unreliable test tim‐
6502 ings.
6503 Time taken by lazy instantiation of strategies is now counted towards
6505 drawing from the strategy, rather than towards the deadline for the
6506 test function.
6507 4.43.8 - 2019-11-08
6509 This release ensures that the strategies passed to @given are properly
6510 validated when applied to a test method inside a test class.
6511 This should result in clearer error messages when some of those strate‐
6513 gies are invalid.
6514 4.43.7 - 2019-11-08
6516 This release changes how Hypothesis manages its search space in cases
6517 where it generates redundant data. This should cause it to generate
6518 significantly fewer duplicated examples (especially with short integer
6519 ranges), and may cause it to produce more useful examples in some cases
6520 (especially ones where there is a significant amount of filtering).
6521 4.43.6 - 2019-11-07
6523 This patch refactors width handling in floats(); you may notice small
6524 performance improvements but the main purpose is to enable work on
6525 issue #1704 (improving shrinking of bounded floats).
6526 4.43.5 - 2019-11-06
6528 This patch removes an unused internal flag. There is no user-visible
6529 change.
6530 4.43.4 - 2019-11-05
6532 This patch corrects the exception type and error message you get if you
6533 attempt to use data() to draw from something which is not a strategy.
6534 This never worked, but the error is more helpful now.
6535 4.43.3 - 2019-11-05
6537 We've adopted flake8-bugbear to check for a few more style issues, and
6538 this patch implements the minor internal cleanups it suggested. There
6539 is no user-visible change.
6540 4.43.2 - 2019-11-05
6542 This patch fixes the formatting of some documentation, but there is no
6543 change to any executed code.
6544 4.43.1 - 2019-11-04
6546 Python 3.8's new python:typing.Literal type - see PEP 586 for details -
6547 is now supported in from_type().
6548 4.43.0 - 2019-11-04
6550 This release adds the strategy mutually_broadcastable_shapes(), which
6551 generates multiple array shapes that are mutually broadcast-compatible
6552 with an optional user-specified base-shape.
6553 This is a generalisation of broadcastable_shapes(). It relies heavily
6555 on non-public internals for performance when generating and shrinking
6556 examples. We intend to support generating shapes matching a ufunc sig‐
6557 nature in a future version (issue #2174).
6558 Thanks to Ryan Soklaski, Zac Hatfield-Dodds, and @rdturnermtl who con‐
6560 tributed to this new feature.
6561 4.42.10 - 2019-11-03
6563 This release fixes from_type() when used with bounded or constrained
6564 python:typing.TypeVar objects (issue #2094).
6565 Previously, distinct typevars with the same constraints would be
6567 treated as all single typevar, and in cases where a typevar bound was
6568 resolved to a union of subclasses this could result in mixed types
6569 being generated for that typevar.
6570 4.42.9 - 2019-11-03
6572 This patch ensures that the default value broadcastable_shapes()
6573 chooses for max_dims is always valid (at most 32), even if you pass
6574 min_dims=32.
6575 4.42.8 - 2019-11-02
6577 This patch ensures that we only add profile information to the pytest
6578 header if running either pytest or Hypothesis in verbose mode, matching
6579 the builtin cache plugin (issue #2155).
6580 4.42.7 - 2019-11-02
6582 This patch makes stateful step printing expand the result of a step
6583 into multiple variables when you return multiple() (issue #2139).
6584 Thanks to Joseph Weston for reporting and fixing this bug!
6585 4.42.6 - 2019-11-02
6587 This release fixes a bug (issue #2166) where a Unicode character info
6588 cache file was generated but never used on subsequent test runs, caus‐
6589 ing tests to run more slowly than they should have.
6590 Thanks to Robert Knight for this bugfix!
6592 4.42.5 - 2019-11-01
6594 This patch corrects some internal documentation. There is no user-vis‐
6595 ible change.
6596 4.42.4 - 2019-11-01
6598 This release fixes a bug (issue #2160) where decorators applied after
6599 @settings and before @given were ignored.
6600 Thanks to Tom Milligan for this bugfix!
6602 4.42.3 - 2019-10-30
6604 This release updates Hypothesis's formatting to the new version of
6605 black, and has absolutely no user visible effect.
6606 4.42.2 - 2019-10-30
6608 This release fixes a bug in recursive() which would have meant that in
6609 practice max_leaves was treated as if it was lower than it actually is
6610 - specifically it would be capped at the largest power of two smaller
6611 than it. It is now handled correctly.
6612 4.42.1 - 2019-10-30
6614 Python 3.8's new python:typing.SupportsIndex type - see PEP 357 for
6615 details - is now supported in from_type().
6616 Thanks to Grigorios Giannakopoulos for the patch!
6618 4.42.0 - 2019-10-27
6620 This release significantly simplifies Hypothesis's internal logic for
6621 data generation, by removing a number of heuristics of questionable or
6622 unproven value.
6623 The results of this change will vary significantly from test to test.
6625 Most test suites will see significantly faster data generation and
6626 lower memory usage. The "quality" of the generated data may go up or
6627 down depending on your particular test suites.
6628 If you see any significant regressions in Hypothesis's ability to find
6630 bugs in your code as a result of this release, please file an issue to
6631 let us know.
6632 Users of the new targeted property-based testing functionality are
6634 reasonably likely to see improvements in data generation, as this
6635 release changes the search algorithm for targeted property based test‐
6636 ing to one that is more likely to be productive than the existing
6637 approach.
6638 4.41.3 - 2019-10-21
6640 This patch is to ensure that our internals remain comprehensible to
6641 mypy 0.740 - there is no user-visible change.
6642 4.41.2 - 2019-10-17
6644 This patch changes some internal hashes to SHA384, to better support
6645 users subject to FIPS-140. There is no user-visible API change.
6646 Thanks to Paul Kehrer for this contribution!
6648 4.41.1 - 2019-10-16
6650 This release makes --hypothesis-show-statistics much more useful for
6651 tests using a RuleBasedStateMachine, by simplifying the reprs so that
6652 events are aggregated correctly.
6653 4.41.0 - 2019-10-16
6655 This release upgrades the fixed_dictionaries() strategy to support
6656 optional keys (issue #1913).
6657 4.40.2 - 2019-10-16
6659 This release makes some minor internal changes in support of improving
6660 the Hypothesis test suite. It should not have any user visible impact.
6661 4.40.1 - 2019-10-14
6663 This release changes how Hypothesis checks if a parameter to a test
6664 function is a mock object. It is unlikely to have any noticeable
6665 effect, but may result in a small performance improvement, especially
6666 for test functions where a mock object is being passed as the first
6667 argument.
6668 4.40.0 - 2019-10-09
6670 This release fixes a bug where our example database logic did not dis‐
6671 tinguish between failing examples based on arguments from a
6672 @pytest.mark.parametrize(...). This could in theory cause data loss if
6673 a common failure overwrote a rare one, and in practice caused occa‐
6674 sional file-access collisions in highly concurrent workloads (e.g. dur‐
6675 ing a 300-way parametrize on 16 cores).
6676 For internal reasons this also involves bumping the minimum supported
6678 version of pytest to 4.3
6679 Thanks to Peter C Kroon for the Hacktoberfest patch!
6681 4.39.3 - 2019-10-09
6683 This patch improves our type hints on the emails(), functions(), inte‐
6684 gers(), iterables(), and slices() strategies, as well as the .filter()
6685 method.
6686 There is no runtime change, but if you use mypy or a similar
6688 type-checker on your tests the results will be a bit more precise.
6689 4.39.2 - 2019-10-09
6691 This patch improves the performance of unique collections such as
6692 sets() of just() or booleans() strategies. They were already pretty
6693 good though, so you're unlikely to notice much!
6694 4.39.1 - 2019-10-09
6696 If a value in a dict passed to fixed_dictionaries() is not a strategy,
6697 Hypothesis now tells you which one.
6698 4.39.0 - 2019-10-07
6700 This release adds the basic_indices() strategy, to generate basic
6701 indexes for arrays of the specified shape (issue #1930).
6702 It generates tuples containing some mix of integers, python:slice
6704 objects, ... (Ellipsis), and numpy:numpy.newaxis; which when used to
6705 index an array of the specified shape produce either a scalar or a
6706 shared-memory view of the array. Note that the index tuple may be
6707 longer or shorter than the array shape, and may produce a view with
6708 another dimensionality again!
6709 Thanks to Lampros Mountrakis, Ryan Soklaski, and Zac Hatfield-Dodds for
6711 their collaboration on this surprisingly subtle strategy!
6712 4.38.3 - 2019-10-04
6714 This patch defers creation of the .hypothesis directory until we have
6715 something to store in it, meaning that it will appear when Hypothesis
6716 is used rather than simply installed.
6717 Thanks to Peter C Kroon for the Hacktoberfest patch!
6719 4.38.2 - 2019-10-02
6721 This patch bumps our dependency on attrs to >=19.2.0; but there are no
6722 user-visible changes to Hypothesis.
6723 4.38.1 - 2019-10-01
6725 This is a comment-only patch which tells mypy 0.730 to ignore some
6726 internal compatibility shims we use to support older Pythons.
6727 4.38.0 - 2019-10-01
6729 This release adds the hypothesis.target() function, which implements
6730 experimental support for targeted property-based testing (issue #1779).
6731 By calling target() in your test function, Hypothesis can do a
6733 hill-climbing search for bugs. If you can calculate a suitable metric
6734 such as the load factor or length of a queue, this can help you find
6735 bugs with inputs that are highly improbably from unguided generation -
6736 however good our heuristics, example diversity, and deduplication logic
6737 might be. After all, those features are at work in targeted PBT too!
6738 4.37.0 - 2019-09-28
6740 This release emits a warning if you use the .example() method of a
6741 strategy in a non-interactive context.
6742 given() is a much better choice for writing tests, whether you care
6744 about performance, minimal examples, reproducing failures, or even just
6745 the variety of inputs that will be tested!
6746 4.36.2 - 2019-09-20
6748 This patch disables part of the typing-based inference for the attrs
6749 package under Python 3.5.0, which has some incompatible internal
6750 details (issue #2095).
6751 4.36.1 - 2019-09-17
6753 This patch fixes a bug in strategy inference for attrs classes where
6754 Hypothesis would fail to infer a strategy for attributes of a generic
6755 type such as Union[int, str] or List[bool] (issue #2091).
6756 Thanks to Jonathan Gayvallet for the bug report and this patch!
6758 4.36.0 - 2019-09-09
6760 This patch deprecates min_len or max_len of 0 in byte_string_dtypes()
6761 and unicode_string_dtypes(). The lower limit is now 1.
6762 Numpy uses a length of 0 in these dtypes to indicate an undetermined
6764 size, chosen from the data at array creation. However, as the arrays()
6765 strategy creates arrays before filling them, strings were truncated to
6766 1 byte.
6767 4.35.1 - 2019-09-09
6769 This patch improves the messaging that comes from invalid size argu‐
6770 ments to collection strategies such as lists().
6771 4.35.0 - 2019-09-04
6773 This release improves the from_lark() strategy, tightening argument
6774 validation and adding the explicit argument to allow use with terminals
6775 that use @declare instead of a string or regular expression.
6776 This feature is required to handle features such as indent and dedent
6778 tokens in Python code, which can be generated with the hypothesmith
6779 package.
6780 4.34.0 - 2019-08-23
6782 The from_type() strategy now knows to look up the subclasses of
6783 abstract types, which cannot be instantiated directly.
6784 This is very useful for hypothesmith to support libCST.
6786 4.33.1 - 2019-08-21
6788 This patch works around a crash when an incompatible version of Numpy
6789 is installed under PyPy 5.10 (Python 2.7).
6790 If you are still using Python 2, please upgrade to Python 3 as soon as
6792 possible - it will be unsupported at the end of this year.
6793 4.33.0 - 2019-08-20
6795 This release improves the domains() strategy, as well as the urls() and
6796 the emails() strategies which use it. These strategies now use the
6797 full IANA list of Top Level Domains and are correct as per RFC 1035.
6798 Passing tests using these strategies may now fail.
6800 Thanks to TechDragon for this improvement.
6802 4.32.3 - 2019-08-05
6804 This patch tidies up the repr of several settings-related objects, at
6805 runtime and in the documentation, and deprecates the undocumented edge
6806 case that phases=None was treated like phases=tuple(Phase).
6807 It also fixes from_lark() with lark 0.7.2 and later.
6809 4.32.2 - 2019-07-30
6811 This patch updates some internal comments for mypy 0.720. There is no
6812 user-visible impact.
6813 4.32.1 - 2019-07-29
6815 This release changes how the shrinker represents its progress inter‐
6816 nally. For large generated test cases this should result in signifi‐
6817 cantly less memory usage and possibly faster shrinking. Small generated
6818 test cases may be slightly slower to shrink but this shouldn't be very
6819 noticeable.
6820 4.32.0 - 2019-07-28
6822 This release makes arrays() more pedantic about elements strategies
6823 that cannot be exactly represented as array elements.
6824 In practice, you will see new warnings if you were using a float16 or
6826 float32 dtype without passing floats() the width=16 or width=32 argu‐
6827 ments respectively.
6828 The previous behaviour could lead to silent truncation, and thus some
6830 elements being equal to an explicitly excluded bound (issue #1899).
6831 4.31.1 - 2019-07-28
6833 This patch changes an internal use of MD5 to SHA hashes, to better sup‐
6834 port users subject to FIPS-140. There is no user-visible or API
6835 change.
6836 Thanks to Alex Gaynor for this patch.
6838 4.31.0 - 2019-07-24
6840 This release simplifies the logic of the print_blob setting by removing
6841 the option to set it to PrintSettings.INFER. As a result the
6842 print_blob setting now takes a single boolean value, and the use of
6843 PrintSettings is deprecated.
6844 4.28.2 - 2019-07-14
6846 This patch improves the docstrings of several Hypothesis strategies, by
6847 clarifying markup and adding cross-references. There is no runtime
6848 change.
6849 Thanks to Elizabeth Williams and Serah Njambi Rono for their contribu‐
6851 tions at the SciPy 2019 sprints!
6852 4.28.1 - 2019-07-12
6854 This patch improves the behaviour of the text() strategy when passed an
6855 alphabet which is not a strategy. The value is now interpreted as
6856 whitelist_characters to characters() instead of a sequence for sam‐
6857 pled_from(), which standardises the distribution of examples and the
6858 shrinking behaviour.
6859 You can get the previous behaviour by using lists(sampled_from(alpha‐
6861 bet)).map("".map) instead.
6862 4.28.0 - 2019-07-11
6864 This release deprecates find(). The .example() method is a better
6865 replacement if you want an example, and for the rare occasions where
6866 you want the minimal example you can get it from @given.
6867 @given has steadily outstripped find() in both features and performance
6869 over recent years, and as we do not have the resources to maintain and
6870 test both we think it is better to focus on just one.
6871 4.27.0 - 2019-07-08
6873 This release refactors the implementation of the .example() method, to
6874 more accurately represent the data which will be generated by @given.
6875 As a result, calling s.example() on an empty strategy s (such as noth‐
6877 ing()) now raises Unsatisfiable instead of the old NoExamples excep‐
6878 tion.
6879 4.26.4 - 2019-07-07
6881 This patch ensures that the Pandas extra will keep working when Python
6882 3.8 removes abstract base classes from the top-level python:collections
6883 namespace. This also fixes the relevant warning in Python 3.7, but
6884 there is no other difference in behaviour and you do not need to do
6885 anything.
6886 4.26.3 - 2019-07-05
6888 This release fixes issue #2027, by changing the way Hypothesis tries
6889 to generate distinct examples to be more efficient.
6890 This may result in slightly different data distribution, and should
6892 improve generation performance in general, but should otherwise have
6893 minimal user impact.
6894 4.26.2 - 2019-07-04
6896 This release fixes issue #1864, where some simple tests would perform
6897 very slowly, because they would run many times with each subsequent run
6898 being progressively slower. They will now stop after a more reasonable
6899 number of runs without hitting this problem.
6900 Unless you are hitting exactly this issue, it is unlikely that this
6902 release will have any effect, but certain classes of custom generators
6903 that are currently very slow may become a bit faster, or start to trig‐
6904 ger health check failures.
6905 4.26.1 - 2019-07-04
6907 This release adds the strategy integer_array_indices(), which generates
6908 tuples of Numpy arrays that can be used for advanced indexing to select
6909 an array of a specified shape.
6910 4.26.0 - 2019-07-04
6912 This release significantly improves the performance of drawing unique
6913 collections whose elements are drawn from sampled_from() strategies.
6914 As a side effect, this detects an error condition that would previously
6916 have passed silently: When the min_size argument on a collection with
6917 distinct elements is greater than the number of elements being sampled,
6918 this will now raise an error.
6919 4.25.1 - 2019-07-03
6921 This release removes some defunct internal functionality that was only
6922 being used for testing. It should have no user visible impact.
6923 4.25.0 - 2019-07-03
6925 This release deprecates and disables the buffer_size setting, which
6926 should have been treated as a private implementation detail all along.
6927 We recommend simply deleting this settings argument.
6928 4.24.6 - 2019-06-26
6930 This patch makes datetimes() more efficient, as it now handles short
6931 months correctly by construction instead of filtering.
6932 4.24.5 - 2019-06-23
6934 This patch improves the development experience by simplifying the
6935 tracebacks you will see when e.g. you have used the .map(...) method of
6936 a strategy and the mapped function raises an exception.
6937 No new exceptions can be raised, nor existing exceptions change any‐
6939 thing but their traceback. We're simply using if-statements rather
6940 than exceptions for control flow in a certain part of the internals!
6941 4.24.4 - 2019-06-21
6943 This patch fixes issue #2014, where our compatibility layer broke with
6944 version 3.7.4 of the typing module backport on PyPI.
6945 This issue only affects Python 2. We remind users that Hypothesis,
6947 like many other packages, will drop Python 2 support on 2020-01-01 and
6948 already has several features that are only available on Python 3.
6949 4.24.3 - 2019-06-07
6951 This patch improves the implementation of an internal wrapper on Python
6952 3.8 beta1 (and will break on the alphas; but they're not meant to be
6953 stable). On other versions, there is no change at all.
6954 Thanks to Daniel Hahler for the patch, and Victor Stinner for his work
6956 on bpo-37032 that made it possible.
6957 4.24.2 - 2019-06-06
6959 Deprecation messages for functions in hypothesis.extra.django.models
6960 now explicitly name the deprecated function to make it easier to track
6961 down usages. Thanks to Kristian Glass for this contribution!
6962 4.24.1 - 2019-06-04
6964 This patch fixes issue #1999, a spurious bug raised when a @st.compos‐
6965 ite function was passed a keyword-only argument.
6966 Thanks to Jim Nicholls for his fantastic bug report.
6968 4.24.0 - 2019-05-29
6970 This release deprecates GenericStateMachine, in favor of RuleBased‐
6971 StateMachine. Rule-based stateful testing is significantly faster,
6972 especially during shrinking.
6973 If your use-case truly does not fit rule-based stateful testing, we
6975 recommend writing a custom test function which drives your specific
6976 control-flow using data().
6977 4.23.9 - 2019-05-28
6979 This patch fixes a very rare example database issue with file permis‐
6980 sions.
6981 When running a test that uses both @given and pytest.mark.parametrize,
6983 using pytest-xdist on Windows, with failing examples in the database,
6984 two attempts to read a file could overlap and we caught FileNotFound
6985 but not other OSErrors.
6986 4.23.8 - 2019-05-26
6988 This patch has a minor cleanup of the internal engine. There is no
6989 user-visible impact.
6990 4.23.7 - 2019-05-26
6992 This patch clarifies some error messages when the test function signa‐
6993 ture is incompatible with the arguments to @given, especially when the
6994 @settings() decorator is also used (issue #1978).
6995 4.23.6 - 2019-05-19
6997 This release adds the pyupgrade fixer to our code style, for consistent
6998 use of dict and set literals and comprehensions.
6999 4.23.5 - 2019-05-16
7001 This release slightly simplifies a small part of the core engine.
7002 There is no user-visible change.
7003 4.23.4 - 2019-05-09
7005 Fixes a minor formatting issue the docstring of from_type()
7006 4.23.3 - 2019-05-09
7008 Adds a recipe to the docstring of from_type() that describes a means
7009 for drawing values for "everything except" a specified type. This
7010 recipe is especially useful for writing tests that perform input-type
7011 validation.
7012 4.23.2 - 2019-05-08
7014 This patch uses autoflake to remove some pointless pass statements,
7015 which improves our workflow but has no user-visible impact.
7016 4.23.1 - 2019-05-08
7018 This patch fixes an OverflowError in from_type(xrange) on Python 2.
7019 It turns out that not only do the start and stop values have to fit in
7021 a C long, but so does stop - start. We now handle this even on 32bit
7022 platforms, but remind users that Python2 will not be supported after
7023 2019 without specific funding.
7024 4.23.0 - 2019-05-08
7026 This release implements the slices() strategy, to generate slices of a
7027 length-size sequence.
7028 Thanks to Daniel J. West for writing this patch at the PyCon 2019
7030 sprints!
7031 4.22.3 - 2019-05-07
7033 This patch exposes DataObject, solely to support more precise type
7034 hints. Objects of this type are provided by data(), and can be used to
7035 draw examples from strategies intermixed with your test code.
7036 4.22.2 - 2019-05-07
7038 This patch fixes the very rare issue #1798 in array_dtypes(), which
7039 caused an internal error in our tests.
7040 4.22.1 - 2019-05-07
7042 This patch fixes a rare bug in from_type(range).
7043 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
7045 Sprints.
7046 4.22.0 - 2019-05-07
7048 The unique_by argument to lists now accepts a tuple of callables such
7049 that every element of the generated list will be unique with respect to
7050 each callable in the tuple (issue #1916).
7051 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
7053 4.21.1 - 2019-05-06
7055 This patch cleans up the internals of one_of(). You may see a slight
7056 change to the distribution of examples from this strategy but there is
7057 no change to the public API.
7058 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
7060 sprints!
7061 4.21.0 - 2019-05-05
7063 The from_type() strategy now supports python:slice objects.
7064 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
7066 Mentored Sprints.
7067 4.20.0 - 2019-05-05
7069 This release improves the array_shapes() strategy, to choose an appro‐
7070 priate default for max_side based on the min_side, and max_dims based
7071 on the min_dims. An explicit error is raised for dimensions greater
7072 than 32, which are not supported by Numpy, as for other invalid combi‐
7073 nations of arguments.
7074 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
7076 tored Sprints.
7077 4.19.0 - 2019-05-05
7079 The from_type() strategy now supports python:range objects (or xrange
7080 on Python 2).
7081 Thanks to Katrina Durance for writing this feature at the PyCon 2019
7083 Mentored Sprints.
7084 4.18.3 - 2019-04-30
7086 This release fixes a very rare edge case in the test-case mutator,
7087 which could cause an internal error with certain unusual tests.
7088 4.18.2 - 2019-04-30
7090 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
7091 changed the representation of code objects to support positional-only
7092 arguments. Note however that Hypothesis does not (yet) support such
7093 functions as e.g. arguments to builds() or inputs to @given.
7094 Thanks to Paul Ganssle for identifying and fixing this bug.
7096 4.18.1 - 2019-04-29
7098 This patch improves the performance of unique collections such as
7099 sets() when the elements are drawn from a sampled_from() strategy (‐
7100 issue #1115).
7101 4.18.0 - 2019-04-24
7103 This release adds the functions() strategy, which can be used to imi‐
7104 tate your 'real' function for callbacks.
7105 4.17.2 - 2019-04-19
7107 This release refactors stateful rule selection to share the new machin‐
7108 ery with sampled_from() instead of using the original independent
7109 implementation.
7110 4.17.1 - 2019-04-16
7112 This patch allows Hypothesis to try a few more examples after finding
7113 the first bug, in hopes of reporting multiple distinct bugs. The
7114 heuristics described in issue #847 ensure that we avoid wasting time on
7115 fruitless searches, while still surfacing each bug as soon as possible.
7116 4.17.0 - 2019-04-16
7118 This release adds the strategy broadcastable_shapes(), which generates
7119 array shapes that are broadcast-compatible with a provided shape.
7120 4.16.0 - 2019-04-12
7122 This release allows register_type_strategy() to be used with
7123 python:typing.NewType instances. This may be useful to e.g. provide
7124 only positive integers for from_type(UserId) with a UserId = New‐
7125 Type('UserId', int) type.
7126 Thanks to PJCampi for suggesting and writing the patch!
7128 4.15.0 - 2019-04-09
7130 This release supports passing a timedelta as the deadline setting, so
7131 you no longer have to remember that the number is in milliseconds (‐
7132 issue #1900).
7133 Thanks to Damon Francisco for this change!
7135 4.14.7 - 2019-04-09
7137 This patch makes the type annotations on hypothesis.extra.dateutil com‐
7138 patible with mypy 0.700.
7139 4.14.6 - 2019-04-07
7141 This release fixes a bug introduced in Hypothesis 4.14.3 that would
7142 sometimes cause sampled_from(...).filter(...) to encounter an internal
7143 assertion failure when there are three or fewer elements, and every
7144 element is rejected by the filter.
7145 4.14.5 - 2019-04-05
7147 This patch takes the previous efficiency improvements to sam‐
7148 pled_from(...).filter(...) strategies that reject most elements, and
7149 generalises them to also apply to sampled_from(...).filter(...).fil‐
7150 ter(...) and longer chains of filters.
7151 4.14.4 - 2019-04-05
7153 This release fixes a bug that prevented random_module() from correctly
7154 restoring the previous state of the random module.
7155 The random state was instead being restored to a temporary determinis‐
7157 tic state, which accidentally caused subsequent tests to see the same
7158 random values across multiple test runs.
7159 4.14.3 - 2019-04-03
7161 This patch adds an internal special case to make sampled_from(...).fil‐
7162 ter(...) much more efficient when the filter rejects most elements (‐
7163 issue #1885).
7164 4.14.2 - 2019-03-31
7166 This patch improves the error message if the function f in s.flatmap(f)
7167 does not return a strategy.
7168 Thanks to Kai Chen for this change!
7170 4.14.1 - 2019-03-30
7172 This release modifies how Hypothesis selects operations to run during
7173 shrinking, by causing it to deprioritise previously useless classes of
7174 shrink until others have reached a fixed point.
7175 This avoids certain pathological cases where the shrinker gets very
7177 close to finishing and then takes a very long time to finish the last
7178 small changes because it tries many useless shrinks for each useful one
7179 towards the end. It also should cause a more modest improvement (prob‐
7180 ably no more than about 30%) in shrinking performance for most tests.
7181 4.14.0 - 2019-03-19
7183 This release blocks installation of Hypothesis on Python 3.4, which
7184 reached its end of life date on 2019-03-18.
7185 This should not be of interest to anyone but downstream maintainers -
7187 if you are affected, migrate to a secure version of Python as soon as
7188 possible or at least seek commercial support.
7189 4.13.0 - 2019-03-19
7191 This release makes it an explicit error to call floats(min_value=inf,
7192 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
7193 are no possible values that can be generated (issue #1859).
7194 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
7196 -0. and we could thus generate either if comparing by value, violating
7197 the sequence ordering of floats is a special case we don't want or
7198 need.
7199 4.12.1 - 2019-03-18
7201 This release should significantly reduce the amount of memory that
7202 Hypothesis uses for representing large test cases, by storing informa‐
7203 tion in a more compact representation and only unpacking it lazily when
7204 it is first needed.
7205 4.12.0 - 2019-03-18
7207 This update adds the report_multiple_bugs setting, which you can use to
7208 disable multi-bug reporting and only raise whichever bug had the small‐
7209 est minimal example. This is occasionally useful when using a debugger
7210 or tools that annotate tracebacks via introspection.
7211 4.11.7 - 2019-03-18
7213 This change makes a tiny improvement to the core engine's bookkeeping.
7214 There is no user-visible change.
7215 4.11.6 - 2019-03-15
7217 This release changes some of Hypothesis's internal shrinking behaviour
7218 in order to reduce memory usage and hopefully improve performance.
7219 4.11.5 - 2019-03-13
7221 This release adds a micro-optimisation to how Hypothesis handles debug
7222 reporting internally. Hard to shrink test may see a slight performance
7223 improvement, but in most common scenarios it is unlikely to be notice‐
7224 able.
7225 4.11.4 - 2019-03-13
7227 This release removes some redundant code that was no longer needed but
7228 was still running a significant amount of computation and allocation on
7229 the hot path. This should result in a modest speed improvement for
7230 most tests, especially those with large test cases.
7231 4.11.3 - 2019-03-13
7233 This release adds a micro-optimisation to how Hypothesis caches test
7234 cases. This will cause a small improvement in speed and memory usage
7235 for large test cases, but in most common scenarios it is unlikely to be
7236 noticeable.
7237 4.11.2 - 2019-03-13
7239 This release removes some internal code that populates a field that is
7240 no longer used anywhere. This should result in some modest performance
7241 and speed improvements and no other user visible effects.
7242 4.11.1 - 2019-03-13
7244 This is a formatting-only patch, enabled by a new version of isort.
7245 4.11.0 - 2019-03-12
7247 This release deprecates sampled_from() with empty sequences. This
7248 returns nothing(), which gives a clear error if used directly... but
7249 simply vanishes if combined with another strategy.
7250 Tests that silently generate less than expected are a serious problem
7252 for anyone relying on them to find bugs, and we think reliability more
7253 important than convenience in this case.
7254 4.10.0 - 2019-03-11
7256 This release improves Hypothesis's to detect flaky tests, by noticing
7257 when the behaviour of the test changes between runs. In particular
7258 this will notice many new cases where data generation depends on exter‐
7259 nal state (e.g. external sources of randomness) and flag those as flaky
7260 sooner and more reliably.
7261 The basis of this feature is a considerable reengineering of how
7263 Hypothesis stores its history of test cases, so on top of this its mem‐
7264 ory usage should be considerably reduced.
7265 4.9.0 - 2019-03-09
7267 This release adds the strategy valid_tuple_axes(), which generates
7268 tuples of axis-indices that can be passed to the axis argument in
7269 NumPy's sequential functions (e.g. numpy:numpy.sum()).
7270 Thanks to Ryan Soklaski for this strategy.
7272 4.8.0 - 2019-03-06
7274 This release significantly tightens validation in hypothesis.settings.
7275 max_examples, buffer_size, and stateful_step_count must be positive
7276 integers; deadline must be a positive number or None; and derandomize
7277 must be either True or False.
7278 As usual, this replaces existing errors with a more helpful error and
7280 starts new validation checks as deprecation warnings.
7281 4.7.19 - 2019-03-04
7283 This release makes some micro-optimisations to certain calculations
7284 performed in the shrinker. These should particularly speed up large
7285 test cases where the shrinker makes many small changes. It will also
7286 reduce the amount allocated, but most of this is garbage that would
7287 have been immediately thrown away, so you probably won't see much
7288 effect specifically from that.
7289 4.7.18 - 2019-03-03
7291 This patch removes some overhead from arrays() with a constant shape
7292 and dtype. The resulting performance improvement is modest, but worth‐
7293 wile for small arrays.
7294 4.7.17 - 2019-03-01
7296 This release makes some micro-optimisations within Hypothesis's inter‐
7297 nal representation of test cases. This should cause heavily nested
7298 test cases to allocate less during generation and shrinking, which
7299 should speed things up slightly.
7300 4.7.16 - 2019-02-28
7302 This changes the order in which Hypothesis runs certain operations dur‐
7303 ing shrinking. This should significantly decrease memory usage and
7304 speed up shrinking of large examples.
7305 4.7.15 - 2019-02-28
7307 This release allows Hypothesis to calculate a number of attributes of
7308 generated test cases lazily. This should significantly reduce memory
7309 usage and modestly improve performance, especially for large test
7310 cases.
7311 4.7.14 - 2019-02-28
7313 This release reduces the number of operations the shrinker will try
7314 when reordering parts of a test case. This should in some circum‐
7315 stances significantly speed up shrinking. It may result in different
7316 final test cases, and if so usually slightly worse ones, but it should
7317 not generally have much impact on the end result as the operations
7318 removed were typically useless.
7319 4.7.13 - 2019-02-27
7321 This release changes how Hypothesis reorders examples within a test
7322 case during shrinking. This should make shrinking considerably faster.
7323 4.7.12 - 2019-02-27
7325 This release slightly improves the shrinker's ability to replace parts
7326 of a test case with their minimal version, by allowing it to do so in
7327 bulk rather than one at a time. Where this is effective, shrinker per‐
7328 formance should be modestly improved.
7329 4.7.11 - 2019-02-25
7331 This release makes some micro-optimisations to common operations per‐
7332 formed during shrinking. Shrinking should now be slightly faster,
7333 especially for large examples with relatively fast test functions.
7334 4.7.10 - 2019-02-25
7336 This release is a purely internal refactoring of Hypothesis's API for
7337 representing test cases. There should be no user visible effect.
7338 4.7.9 - 2019-02-24
7340 This release changes certain shrink passes to make them more efficient
7341 when they aren't making progress.
7342 4.7.8 - 2019-02-23
7344 This patch removes some unused code, which makes the internals a bit
7345 easier to understand. There is no user-visible impact.
7346 4.7.7 - 2019-02-23
7348 This release reduces the number of operations the shrinker will try
7349 when reordering parts of a test case. This should in some circum‐
7350 stances significantly speed up shrinking. It may result in different
7351 final test cases, and if so usually slightly worse ones, but it should
7352 not generally have much impact on the end result as the operations
7353 removed were typically useless.
7354 4.7.6 - 2019-02-23
7356 This patch removes some unused code from the shrinker. There is no
7357 user-visible change.
7358 4.7.5 - 2019-02-23
7360 This release changes certain shrink passes to make them adaptive - that
7361 is, in cases where they are successfully making progress they may now
7362 do so significantly faster.
7363 4.7.4 - 2019-02-22
7365 This is a docs-only patch, noting that because the lark-parser is under
7366 active development at version 0.x, hypothesis[lark] APIs may break in
7367 minor releases if necessary to keep up with the upstream package.
7368 4.7.3 - 2019-02-22
7370 This changes Hypothesis to no longer import various test frameworks by
7371 default (if they are installed). which will speed up the initial
7372 import hypothesis call.
7373 4.7.2 - 2019-02-22
7375 This release changes Hypothesis's internal representation of a test
7376 case to calculate some expensive structural information on demand
7377 rather than eagerly. This should reduce memory usage a fair bit, and
7378 may make generation somewhat faster.
7379 4.7.1 - 2019-02-21
7381 This release refactors the internal representation of previously run
7382 test cases. The main thing you should see as a result is that Hypothe‐
7383 sis becomes somewhat less memory hungry.
7384 4.7.0 - 2019-02-21
7386 This patch allows array_shapes() to generate shapes with side-length or
7387 even dimension zero, though the minimum still defaults to one. These
7388 shapes are rare and have some odd behavior, but are particularly impor‐
7389 tant to test for just that reason!
7390 In a related bigfix, arrays() now supports generating zero-dimensional
7392 arrays with dtype=object and a strategy for iterable elements. Previ‐
7393 ously, the array element would incorrectly be set to the first item in
7394 the generated iterable.
7395 Thanks to Ryan Turner for continuing to improve our Numpy support.
7397 4.6.1 - 2019-02-19
7399 This release is a trivial micro-optimisation inside Hypothesis which
7400 should result in it using significantly less memory.
7401 4.6.0 - 2019-02-18
7403 This release changes some inconsistent behavior of arrays() from the
7404 Numpy extra when asked for an array of shape=(). arrays() will now
7405 always return a Numpy ndarray, and the array will always be of the
7406 requested dtype.
7407 Thanks to Ryan Turner for this change.
7409 4.5.12 - 2019-02-18
7411 This release fixes a minor typo in an internal comment. There is no
7412 user-visible change.
7413 4.5.11 - 2019-02-15
7415 This release fixes issue #1813, a bug introduced in 3.59.1, which
7416 caused random_module() to no longer affect the body of the test:
7417 Although Hypothesis would claim to be seeding the random module in fact
7418 tests would always run with a seed of zero.
7419 4.5.10 - 2019-02-14
7421 This patch fixes an off-by-one error in the maximum length of emails().
7422 Thanks to Krzysztof Jurewicz for pull request #1812.
7423 4.5.9 - 2019-02-14
7425 This patch removes some unused code from the shrinker. There is no
7426 user-visible change.
7427 4.5.8 - 2019-02-12
7429 This release fixes an internal IndexError in Hypothesis that could
7430 sometimes be triggered during shrinking.
7431 4.5.7 - 2019-02-11
7433 This release modifies the shrinker to interleave different types of
7434 reduction operations, e.g. switching between deleting data and lowering
7435 scalar values rather than trying entirely deletions then entirely low‐
7436 ering.
7437 This may slow things down somewhat in the typical case, but has the
7439 major advantage that many previously difficult to shrink examples
7440 should become much faster, because the shrinker will no longer tend to
7441 stall when trying some ineffective changes to the shrink target but
7442 will instead interleave it with other more effective operations.
7443 4.5.6 - 2019-02-11
7445 This release makes a number of internal changes to the implementation
7446 of hypothesis.extra.lark.from_lark(). These are primarily intended as
7447 a refactoring, but you may see some minor improvements to performance
7448 when generating large strings, and possibly to shrink quality.
7449 4.5.5 - 2019-02-10
7451 This patch prints an explanatory note when issue #1798 is triggered,
7452 because the error message from Numpy is too terse to locate the prob‐
7453 lem.
7454 4.5.4 - 2019-02-08
7456 In Python 2, long integers are not allowed in the shape argument to
7457 arrays(). Thanks to Ryan Turner for fixing this.
7458 4.5.3 - 2019-02-08
7460 This release makes a small internal refactoring to clarify how Hypothe‐
7461 sis instructs tests to stop running when appropriate. There is no
7462 user-visible change.
7463 4.5.2 - 2019-02-06
7465 This release standardises all of the shrinker's internal operations on
7466 running in a random order.
7467 The main effect you will see from this that it should now be much less
7469 common for the shrinker to stall for a long time before making further
7470 progress. In some cases this will correspond to shrinking more slowly,
7471 but on average it should result in faster shrinking.
7472 4.5.1 - 2019-02-05
7474 This patch updates some docstrings, but has no runtime changes.
7475 4.5.0 - 2019-02-03
7477 This release adds exclude_min and exclude_max arguments to floats(), so
7478 that you can easily generate values from open or half-open intervals (‐
7479 issue #1622).
7480 4.4.6 - 2019-02-03
7482 This patch fixes a bug where from_regex() could throw an internal error
7483 if the python:re.IGNORECASE flag was used (issue #1786).
7484 4.4.5 - 2019-02-02
7486 This release removes two shrink passes that Hypothesis runs late in the
7487 process. These were very expensive when the test function was slow and
7488 often didn't do anything useful.
7489 Shrinking should get faster for most failing tests. If you see any
7491 regression in example quality as a result of this release, please let
7492 us know.
7493 4.4.4 - 2019-02-02
7495 This release modifies the way that Hypothesis deletes data during
7496 shrinking. It will primarily be noticeable for very large examples,
7497 which should now shrink faster.
7498 The shrinker is now also able to perform some deletions that it could
7500 not previously, but this is unlikely to be very noticeable.
7501 4.4.3 - 2019-01-25
7503 This release fixes an open file leak that used to cause ResourceWarn‐
7504 ings.
7505 4.4.2 - 2019-01-24
7507 This release changes Hypothesis's internal approach to caching the
7508 results of executing test cases. The result should be that it is now
7509 significantly less memory hungry, especially when shrinking large test
7510 cases.
7511 Some tests may get slower or faster depending on whether the new or old
7513 caching strategy was well suited to them, but any change in speed in
7514 either direction should be minor.
7515 4.4.1 - 2019-01-24
7517 This patch tightens up some of our internal heuristics to deal with
7518 shrinking floating point numbers, which will now run in fewer circum‐
7519 stances.
7520 You are fairly unlikely to see much difference from this, but if you do
7522 you are likely to see shrinking become slightly faster and/or producing
7523 slightly worse results.
7524 4.4.0 - 2019-01-24
7526 This release adds the from_form() function, which allows automatic
7527 testing against Django forms. (issue #35)
7528 Thanks to Paul Stiverson for this feature, which resolves our oldest
7530 open issue!
7531 4.3.0 - 2019-01-24
7533 This release deprecates HealthCheck.hung_test and disables the associ‐
7534 ated runtime check for tests that ran for more than five minutes. Such
7535 a check is redundant now that we enforce the deadline and max_examples
7536 setting, which can be adjusted independently.
7537 4.2.0 - 2019-01-23
7539 This release adds a new module, hypothesis.extra.lark, which you can
7540 use to generate strings matching a context-free grammar.
7541 In this initial version, only lark-parser EBNF grammars are supported,
7543 by the new hypothesis.extra.lark.from_lark() function.
7544 4.1.2 - 2019-01-23
7546 This patch fixes a very rare overflow bug (issue #1748) which could
7547 raise an InvalidArgument error in complex_numbers() even though the
7548 arguments were valid.
7549 4.1.1 - 2019-01-23
7551 This release makes some improvements to internal code organisation and
7552 documentation and has no impact on behaviour.
7553 4.1.0 - 2019-01-22
7555 This release adds register_random(), which registers random.Random
7556 instances or compatible objects to be seeded and reset by Hypothesis to
7557 ensure that test cases are deterministic.
7558 We still recommend explicitly passing a random.Random instance from
7560 randoms() if possible, but registering a framework-global state for
7561 Hypothesis to manage is better than flaky tests!
7562 4.0.2 - 2019-01-22
7564 This patch fixes issue #1387, where bounded integers() with a very
7565 large range would almost always generate very large numbers. Now, we
7566 usually use the same tuned distribution as unbounded integers().
7567 4.0.1 - 2019-01-16
7569 This release randomizes the order in which the shrinker tries some of
7570 its initial normalization operations. You are unlikely to see much
7571 difference as a result unless your generated examples are very large.
7572 In this case you may see some performance improvements in shrinking.
7573 4.0.0 - 2019-01-14
7575 Welcome to the next major version of Hypothesis!
7576 There are no new features here, as we release those in minor versions.
7578 Instead, 4.0 is a chance for us to remove deprecated features (many
7579 already converted into no-ops), and turn a variety of warnings into
7580 errors.
7581 If you were running on the last version of Hypothesis 3.x without any
7583 Hypothesis deprecation warnings (or using private APIs), this will be a
7584 very boring upgrade. In fact, nothing will change for you at all. Per
7585 our deprecation policy, warnings added in the last six months (after
7586 2018-07-05) have not been converted to errors.
7587 Removals
7589 · hypothesis.extra.datetime has been removed, replaced by the core date
7590 and time strategies.
7591 · hypothesis.extra.fakefactory has been removed, replaced by general
7593 expansion of Hypothesis' strategies and the third-party ecosystem.
7594 · The SQLite example database backend has been removed.
7596 Settings
7598 · The deadline is now enforced by default, rather than just emitting a
7599 warning when the default (200 milliseconds per test case) deadline is
7600 exceeded.
7601 · The database_file setting has been removed; use database.
7603 · The perform_health_check setting has been removed; use sup‐
7605 press_health_check.
7606 · The max_shrinks setting has been removed; use phases to disable
7608 shrinking.
7609 · The min_satisfying_examples, max_iterations, strict, timeout, and
7611 use_coverage settings have been removed without user-configurable
7612 replacements.
7613 Strategies
7615 · The elements argument is now required for collection strategies.
7616 · The average_size argument was a no-op and has been removed.
7618 · Date and time strategies now only accept min_value and max_value for
7620 bounds.
7621 · builds() now requires that the thing to build is passed as the first
7623 positional argument.
7624 · Alphabet validation for text() raises errors, not warnings, as does
7626 category validation for characters().
7627 · The choices() strategy has been removed. Instead, you can use data()
7629 with sampled_from(), so choice(elements) becomes data.draw(sam‐
7630 pled_from(elements)).
7631 · The streaming() strategy has been removed. Instead, you can use
7633 data() and replace iterating over the stream with data.draw() calls.
7634 · sampled_from() and permutations() raise errors instead of warnings if
7636 passed a collection that is not a sequence.
7637 Miscellaneous
7639 · Applying @given to a test function multiple times was really ineffi‐
7640 cient, and now it's also an error.
7641 · Using the .example() method of a strategy (intended for interactive
7643 exploration) within another strategy or a test function always weak‐
7644 ened data generation and broke shrinking, and now it's an error too.
7645 · The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
7647 ported, as the database_file setting has been removed.
7648 · The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
7650 ported. You can use the --hypothesis-verbosity pytest argument
7651 instead, or write your own setup code using the settings profile sys‐
7652 tem to replace it.
7653 · Using @seed or derandomize=True now forces database=None to ensure
7655 results are in fact reproducible. If database is not None, doing so
7656 also emits a HypothesisWarning.
7657 · Unused exception types have been removed from hypothesis.errors;
7659 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
7660 ple, Timeout, and WrongFormat.
7661 Hypothesis 3.x
7663 3.88.3 - 2019-01-11
7664 This changes the order that the shrinker tries certain operations in
7665 its "emergency" phase which runs late in the process. The new order
7666 should be better at avoiding long stalls where the shrinker is failing
7667 to make progress, which may be helpful if you have difficult to shrink
7668 test cases. However this will not be noticeable in the vast majority
7669 of use cases.
7670 3.88.2 - 2019-01-11
7672 This is a pure refactoring release that extracts some logic from the
7673 core Hypothesis engine into its own class and file. It should have no
7674 user visible impact.
7675 3.88.1 - 2019-01-11
7677 This patch fixes some markup in our documentation.
7678 3.88.0 - 2019-01-10
7680 Introduces hypothesis.stateful.multiple(), which allows rules in rule
7681 based state machines to send multiple results at once to their target
7682 Bundle, or none at all.
7683 3.87.0 - 2019-01-10
7685 This release contains a massive cleanup of the Hypothesis for Django
7686 extra:
7687 · hypothesis.extra.django.models.models() is deprecated in favor of
7689 hypothesis.extra.django.from_model().
7690 · hypothesis.extra.django.models.add_default_field_mapping() is depre‐
7692 cated in favor of hypothesis.extra.django.register_field_strategy().
7693 · from_model() does not infer a strategy for nullable fields or fields
7695 with a default unless passed infer, like builds(). models.models()
7696 would usually but not always infer, and a special default_value
7697 marker object was required to disable inference.
7698 3.86.9 - 2019-01-09
7700 This release improves some internal logic about when a test case in
7701 Hypothesis's internal representation could lead to a valid test case.
7702 In some circumstances this can lead to a significant speed up during
7703 shrinking. It may have some minor negative impact on the quality of
7704 the final result due to certain shrink passes now having access to less
7705 information about test cases in some circumstances, but this should
7706 rarely matter.
7707 3.86.8 - 2019-01-09
7709 This release has no user visible changes but updates our URLs to use
7710 HTTPS.
7711 3.86.7 - 2019-01-08
7713 Hypothesis can now automatically generate values for Django models with
7714 a URLfield, thanks to a new provisional strategy for URLs (issue
7715 #1388).
7716 3.86.6 - 2019-01-07
7718 This release is a pure refactoring that extracts some internal code
7719 into its own file. It should have no user visible effect.
7720 3.86.5 - 2019-01-06
7722 This is a docs-only patch, which fixes some typos and removes a few
7723 hyperlinks for deprecated features.
7724 3.86.4 - 2019-01-04
7726 This release changes the order in which the shrinker tries to delete
7727 data. For large and slow tests this may significantly improve the per‐
7728 formance of shrinking.
7729 3.86.3 - 2019-01-04
7731 This release fixes a bug where certain places Hypothesis internal
7732 errors could be raised during shrinking when a user exception occurred
7733 that suppressed an exception Hypothesis uses internally in its genera‐
7734 tion.
7735 The two known ways to trigger this problem were:
7737 · Errors raised in stateful tests' teardown function.
7739 · Errors raised in finally blocks that wrapped a call to data.draw.
7741 These cases will now be handled correctly.
7743 3.86.2 - 2019-01-04
7745 This patch is a docs-only change to fix a broken hyperlink.
7746 3.86.1 - 2019-01-04
7748 This patch fixes issue #1732, where integers() would always return long
7749 values on Python 2.
7750 3.86.0 - 2019-01-03
7752 This release ensures that infinite numbers are never generated by
7753 floats() with allow_infinity=False, which could previously happen in
7754 some cases where one bound was also provided.
7755 The trivially inconsistent min_value=inf, allow_infinity=False now
7757 raises an InvalidArgumentError, as does the inverse with max_value.
7758 You can still use just(inf) to generate inf without violating other
7759 constraints.
7760 3.85.3 - 2019-01-02
7762 Happy new year everyone! This release has no user visible changes but
7763 updates our copyright headers to include 2019.
7764 3.85.2 - 2018-12-31
7766 This release makes a small change to the way the shrinker works. You
7767 may see some improvements to speed of shrinking on especially large and
7768 hard to shrink examples, but most users are unlikely to see much dif‐
7769 ference.
7770 3.85.1 - 2018-12-30
7772 This patch fixes issue #1700, where a line that contained a Unicode
7773 character before a lambda definition would cause an internal exception.
7774 3.85.0 - 2018-12-29
7776 Introduces the hypothesis.stateful.consumes() function. When defining a
7777 rule in stateful testing, it can be used to mark bundles from which
7778 values should be consumed, i. e. removed after use in the rule. This
7779 has been proposed in issue #136.
7780 Thanks to Jochen Müller for this long-awaited feature.
7782 3.84.6 - 2018-12-28
7784 This patch makes a small internal change to fix an issue in Hypothe‐
7785 sis's own coverage tests (issue #1718).
7786 There is no user-visible change.
7788 3.84.5 - 2018-12-21
7790 This patch refactors the hypothesis.strategies module, so that private
7791 names should no longer appear in tab-completion lists. We previously
7792 relied on __all__ for this, but not all editors respect it.
7793 3.84.4 - 2018-12-21
7795 This is a follow-up patch to ensure that the deprecation date is auto‐
7796 matically recorded for any new deprecations. There is no user-visible
7797 effect.
7798 3.84.3 - 2018-12-20
7800 This patch updates the Hypothesis pytest plugin to avoid a recently
7801 deprecated hook interface. There is no user-visible change.
7802 3.84.2 - 2018-12-19
7804 This patch fixes the internals for integers() with one bound. Values
7805 from this strategy now always shrink towards zero instead of towards
7806 the bound, and should shrink much more efficiently too. On Python 2,
7807 providing a bound incorrectly excluded long integers, which can now be
7808 generated.
7809 3.84.1 - 2018-12-18
7811 This patch adds information about when features were deprecated, but
7812 this is only recorded internally and has no user-visible effect.
7813 3.84.0 - 2018-12-18
7815 This release changes the stateful testing backend from find() to use
7816 @given (issue #1300). This doesn't change how you create stateful
7817 tests, but does make them run more like other Hypothesis tests.
7818 @reproduce_failure and @seed now work for stateful tests.
7820 Stateful tests now respect the deadline and suppress_health_check set‐
7822 tings, though they are disabled by default. You can enable them by
7823 using @settings(...) as a class decorator with whatever arguments you
7824 prefer.
7825 3.83.2 - 2018-12-17
7827 Hypothesis has adopted Black as our code formatter (issue #1686).
7828 There are no functional changes to the source, but it's prettier!
7829 3.83.1 - 2018-12-13
7831 This patch increases the variety of examples generated by from_type().
7832 3.83.0 - 2018-12-12
7834 Our pytest plugin now warns you when strategy functions have been col‐
7835 lected as tests, which may happen when e.g. using the @composite deco‐
7836 rator when you should be using @given(st.data()) for inline draws.
7837 Such functions always pass when treated as tests, because the lazy cre‐
7838 ation of strategies mean that the function body is never actually exe‐
7839 cuted!
7840 3.82.6 - 2018-12-11
7842 Hypothesis can now show statistics when running under pytest-xdist.
7843 Previously, statistics were only reported when all tests were run in a
7844 single process (issue #700).
7845 3.82.5 - 2018-12-08
7847 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
7848 to integers() could cause errors on Python 3 due to internal rounding.
7849 3.82.4 - 2018-12-08
7851 Hypothesis now seeds and resets the global state of np.random for each
7852 test case, to ensure that tests are reproducible.
7853 This matches and complements the existing handling of the python:random
7855 module - Numpy simply maintains an independent PRNG for performance
7856 reasons.
7857 3.82.3 - 2018-12-08
7859 This is a no-op release to add the new Framework :: Hypothesis trove
7860 classifier to hypothesis on PyPI.
7861 You can use it as a filter to find Hypothesis-related packages such as
7863 extensions as they add the tag over the coming weeks, or simply visit
7864 our curated list.
7865 3.82.2 - 2018-12-08
7867 The Hypothesis for Pandas extension is now listed in setup.py, so you
7868 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
7869 tion.
7870 3.82.1 - 2018-10-29
7872 This patch fixes from_type() on Python 2 for classes where cls.__init__
7873 is object.__init__. Thanks to ccxcz for reporting issue #1656.
7874 3.82.0 - 2018-10-29
7876 The alphabet argument for text() now uses its default value of charac‐
7877 ters(blacklist_categories=('Cs',)) directly, instead of hiding that
7878 behind alphabet=None and replacing it within the function. Passing
7879 None is therefore deprecated.
7880 3.81.0 - 2018-10-27
7882 GenericStateMachine and RuleBasedStateMachine now raise an explicit
7883 error when instances of settings are assigned to the classes' settings
7884 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
7885 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
7886 tor to handle this automatically.
7887 3.80.0 - 2018-10-25
7889 Since version 3.68.0, arrays() checks that values drawn from the ele‐
7890 ments and fill strategies can be safely cast to the dtype of the array,
7891 and emits a warning otherwise.
7892 This release expands the checks to cover overflow for finite complex64
7894 elements and string truncation caused by too-long elements or trailing
7895 null characters (issue #1591).
7896 3.79.4 - 2018-10-25
7898 Tests using @given now shrink errors raised from pytest helper func‐
7899 tions, instead of reporting the first example found.
7900 This was previously fixed in version 3.56.0, but only for stateful
7902 testing.
7903 3.79.3 - 2018-10-23
7905 Traceback elision is now disabled on Python 2, to avoid an import-time
7906 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
7907 3.79.2: issue #1648).
7908 We encourage all users to upgrade to Python 3 before the end of 2019.
7910 3.79.2 - 2018-10-23
7912 This patch shortens tracebacks from Hypothesis, so you can see exactly
7913 happened in your code without having to skip over irrelevant details
7914 about our internals (issue #848).
7915 In the example test (see pull request #1582), this reduces tracebacks
7917 from nine frames to just three - and for a test with multiple errors,
7918 from seven frames per error to just one!
7919 If you do want to see the internal details, you can disable frame eli‐
7921 sion by setting verbosity to debug.
7922 3.79.1 - 2018-10-22
7924 The abstract number classes Number, Complex, Real, Rational, and Inte‐
7925 gral are now supported by the from_type() strategy. Previously, you
7926 would have to use register_type_strategy() before they could be
7927 resolved (issue #1636)
7928 3.79.0 - 2018-10-18
7930 This release adds a CLI flag for verbosity --hypothesis-verbosity to
7931 the Hypothesis pytest plugin, applied after loading the profile speci‐
7932 fied by --hypothesis-profile. Valid options are the names of verbosity
7933 settings, quiet, normal, verbose or debug.
7934 Thanks to Bex Dunn for writing this patch at the PyCon Australia
7936 sprints!
7937 The pytest header now correctly reports the current profile if
7939 --hypothesis-profile has been used.
7940 Thanks to Mathieu Paturel for the contribution at the Canberra Python
7942 Hacktoberfest.
7943 3.78.0 - 2018-10-16
7945 This release has deprecated the generation of integers, floats and
7946 fractions when the conversion of the upper and/ or lower bound is not
7947 100% exact, e.g. when an integer gets passed a bound that is not a
7948 whole number. (issue #1625)
7949 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
7951 3.77.0 - 2018-10-16
7953 This minor release adds functionality to settings allowing it to be
7954 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
7955 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
7957 3.76.1 - 2018-10-16
7959 This patch fixes some warnings added by recent releases of pydocstyle
7960 and mypy.
7961 3.76.0 - 2018-10-11
7963 This release deprecates using floats for min_size and max_size.
7964 The type hint for average_size arguments has been changed from
7966 Optional[int] to None, because non-None values are always ignored and
7967 deprecated.
7968 3.75.4 - 2018-10-10
7970 This patch adds more internal comments to the core engine's
7971 sequence-length shrinker. There should be no user-visible change.
7972 3.75.3 - 2018-10-09
7974 This patch adds additional comments to some of the core engine's inter‐
7975 nal data structures. There is no user-visible change.
7976 3.75.2 - 2018-10-09
7978 This patch avoids caching a trivial case, fixing issue #493.
7979 3.75.1 - 2018-10-09
7981 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
7982 for this contribution!
7983 3.75.0 - 2018-10-08
7985 This release deprecates the use of min_size=None, setting the default
7986 min_size to 0 (issue #1618).
7987 3.74.3 - 2018-10-08
7989 This patch makes some small internal changes to comply with a new lint
7990 setting in the build. There should be no user-visible change.
7991 3.74.2 - 2018-10-03
7993 This patch fixes issue #1153, where time spent reifying a strategy was
7994 also counted in the time spent generating the first example. Strate‐
7995 gies are now fully constructed and validated before the timer is
7996 started.
7997 3.74.1 - 2018-10-03
7999 This patch fixes some broken formatting and links in the documentation.
8000 3.74.0 - 2018-10-01
8002 This release checks that the value of the print_blob setting is a
8003 PrintSettings instance.
8004 Being able to specify a boolean value was not intended, and is now dep‐
8006 recated. In addition, specifying True will now cause the blob to
8007 always be printed, instead of causing it to be suppressed.
8008 Specifying any value that is not a PrintSettings or a boolean is now an
8010 error.
8011 3.73.5 - 2018-10-01
8013 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
8014 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
8015 eter names min_value and max_value instead of the deprecated names
8016 3.73.4 - 2018-09-30
8018 This patch ensures that Hypothesis deprecation warnings display the
8019 code that emitted them when you're not running in -Werror mode (issue
8020 #652).
8021 3.73.3 - 2018-09-27
8023 Tracebacks involving @composite are now slightly shorter due to some
8024 internal refactoring.
8025 3.73.2 - 2018-09-26
8027 This patch fixes errors in the internal comments for one of the
8028 shrinker passes. There is no user-visible change.
8029 3.73.1 - 2018-09-25
8031 This patch substantially improves the distribution of data generated
8032 with recursive(), and fixes a rare internal error (issue #1502).
8033 3.73.0 - 2018-09-24
8035 This release adds the fulfill() function, which is designed for testing
8036 code that uses dpcontracts 0.4 or later for input validation. This
8037 provides some syntactic sugar around use of assume(), to automatically
8038 filter out and retry calls that cause a precondition check to fail (‐
8039 issue #1474).
8040 3.72.0 - 2018-09-24
8042 This release makes setting attributes of the hypothesis.settings class
8043 an explicit error. This has never had any effect, but could mislead
8044 users who confused it with the current settings instance hypothe‐
8045 sis.settings.default (which is also immutable). You can change the
8046 global settings with settings profiles.
8047 3.71.11 - 2018-09-24
8049 This patch factors out some common code in the shrinker for iterating
8050 over pairs of data blocks. There should be no user-visible change.
8051 3.71.10 - 2018-09-18
8053 This patch allows from_type() to handle the empty tuple type, typ‐
8054 ing.Tuple[()].
8055 3.71.9 - 2018-09-17
8057 This patch updates some internal comments for mypy. There is no
8058 user-visible effect, even for Mypy users.
8059 3.71.8 - 2018-09-17
8061 This patch fixes a rare bug that would cause a particular shrinker pass
8062 to raise an IndexError, if a shrink improvement changed the underlying
8063 data in an unexpected way.
8064 3.71.7 - 2018-09-17
8066 This release fixes the broken cross-references in our docs, and adds a
8067 CI check so we don't add new ones.
8068 3.71.6 - 2018-09-16
8070 This patch fixes two bugs (issue #944 and issue #1521), where messages
8071 about @seed did not check the current verbosity setting, and the wrong
8072 settings were active while executing explicit examples.
8073 3.71.5 - 2018-09-15
8075 This patch fixes a DeprecationWarning added in Python 3.8 (issue
8076 #1576).
8077 Thanks to tirkarthi for this contribution!
8079 3.71.4 - 2018-09-14
8081 This is a no-op release, which implements automatic DOI minting and
8082 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
8083 zon 2020 programme for providing this service!
8084 Check our CITATION file for details, or head right on over to
8086 doi.org/10.5281/zenodo.1412597
8087 3.71.3 - 2018-09-10
8089 This release adds the test name to some deprecation warnings, for eas‐
8090 ier debugging.
8091 Thanks to Sanyam Khurana for the patch!
8093 3.71.2 - 2018-09-10
8095 This release makes Hypothesis's memory usage substantially smaller for
8096 tests with many examples, by bounding the number of past examples it
8097 keeps around.
8098 You will not see much difference unless you are running tests with
8100 max_examples set to well over 1000, but if you do have such tests then
8101 you should see memory usage mostly plateau where previously it would
8102 have grown linearly with time.
8103 3.71.1 - 2018-09-09
8105 This patch adds internal comments to some tree traversals in the core
8106 engine. There is no user-visible change.
8107 3.71.0 - 2018-09-08
8109 This release deprecates the coverage-guided testing functionality, as
8110 it has proven brittle and does not really pull its weight.
8111 We intend to replace it with something more useful in the future, but
8113 the feature in its current form does not seem to be worth the cost of
8114 using, and whatever replaces it will likely look very different.
8115 3.70.4 - 2018-09-08
8117 This patch changes the behaviour of reproduce_failure() so that blobs
8118 are only printed in quiet mode when the print_blob setting is set to
8119 ALWAYS.
8120 Thanks to Cameron McGill for writing this patch at the PyCon Australia
8122 sprints!
8123 3.70.3 - 2018-09-03
8125 This patch removes some unnecessary code from the internals. There is
8126 no user-visible change.
8127 3.70.2 - 2018-09-03
8129 This patch fixes an internal bug where a corrupted argument to @repro‐
8130 duce_failure could raise the wrong type of error. Thanks again to
8131 Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
8132 tently provides excellent bug reports including issue #1558.
8133 3.70.1 - 2018-09-03
8135 This patch updates hypothesis to report its version and settings when
8136 run with pytest. (issue #1223).
8137 Thanks to Jack Massey for this feature.
8139 3.70.0 - 2018-09-01
8141 This release adds a fullmatch argument to from_regex(). When full‐
8142 match=True, the whole example will match the regex pattern as for
8143 python:re.fullmatch().
8144 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
8146 sprints!
8147 3.69.12 - 2018-08-30
8149 This release reverts the changes to logging handling in 3.69.11, which
8150 broke test that use the pytest caplog fixture internally because all
8151 logging was disabled (issue #1546).
8152 3.69.11 - 2018-08-29
8154 This patch will hide all logging messages produced by test cases before
8155 the final, minimal, failing test case (issue #356).
8156 Thanks to Gary Donovan for writing this patch at the PyCon Australia
8158 sprints!
8159 3.69.10 - 2018-08-29
8161 This patch fixes a bug that prevents coverage from reporting unexecuted
8162 Python files (issue #1085).
8163 Thanks to Gary Donovan for writing this patch at the PyCon Australia
8165 sprints!
8166 3.69.9 - 2018-08-28
8168 This patch improves the packaging of the Python package by adding
8169 LICENSE.txt to the sdist (issue #1311), clarifying the minimum sup‐
8170 ported versions of pytz and dateutil (issue #1383), and adds keywords
8171 to the metadata (issue #1520).
8172 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
8174 tralia sprints!
8175 3.69.8 - 2018-08-28
8177 This is an internal change which replaces pickle with json to prevent
8178 possible security issues.
8179 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
8181 sprints!
8182 3.69.7 - 2018-08-28
8184 This patch ensures that note() prints the note for every test case when
8185 the verbosity setting is Verbosity.verbose. At normal verbosity it
8186 only prints from the final test case.
8187 Thanks to Tom McDermott for writing this patch at the PyCon Australia
8189 sprints!
8190 3.69.6 - 2018-08-27
8192 This patch improves the testing of some internal caching. It should
8193 have no user-visible effect.
8194 3.69.5 - 2018-08-27
8196 This change performs a small rename and refactoring in the core engine.
8197 There is no user-visible change.
8198 3.69.4 - 2018-08-27
8200 This change improves the core engine's ability to avoid unnecessary
8201 work, by consulting its cache of previously-tried inputs in more cases.
8202 3.69.3 - 2018-08-27
8204 This patch handles passing an empty python:enum.Enum to from_type() by
8205 returning nothing(), instead of raising an internal python:Assertion‐
8206 Error.
8207 Thanks to Paul Amazona for writing this patch at the PyCon Australia
8209 sprints!
8210 3.69.2 - 2018-08-23
8212 This patch fixes a small mistake in an internal comment. There is no
8213 user-visible change.
8214 3.69.1 - 2018-08-21
8216 This change fixes a small bug in how the core engine consults its cache
8217 of previously-tried inputs. There is unlikely to be any user-visible
8218 change.
8219 3.69.0 - 2018-08-20
8221 This release improves argument validation for stateful testing.
8222 · If the target or targets of a rule() are invalid, we now raise a use‐
8224 ful validation error rather than an internal exception.
8225 · Passing both the target and targets arguments is deprecated - append
8227 the target bundle to the targets tuple of bundles instead.
8228 · Passing the name of a Bundle rather than the Bundle itself is also
8230 deprecated.
8231 3.68.3 - 2018-08-20
8233 This is a docs-only patch, fixing some typos and formatting issues.
8234 3.68.2 - 2018-08-19
8236 This change fixes a small bug in how the core engine caches the results
8237 of previously-tried inputs. The effect is unlikely to be noticeable,
8238 but it might avoid unnecesary work in some cases.
8239 3.68.1 - 2018-08-18
8241 This patch documents the from_dtype() function, which infers a strategy
8242 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
8243 directly when creating e.g. Pandas objects.
8244 3.68.0 - 2018-08-15
8246 arrays() now checks that integer and float values drawn from elements
8247 and fill strategies can be safely cast to the dtype of the array, and
8248 emits a warning otherwise (issue #1385).
8249 Elements in the resulting array could previously violate constraints on
8251 the elements strategy due to floating-point overflow or truncation of
8252 integers to fit smaller types.
8253 3.67.1 - 2018-08-14
8255 This release contains a tiny refactoring of the internals. There is no
8256 user-visible change.
8257 3.67.0 - 2018-08-10
8259 This release adds a width argument to floats(), to generate lower-pre‐
8260 cision floating point numbers for e.g. Numpy arrays.
8261 The generated examples are always instances of Python's native float
8263 type, which is 64bit, but passing width=32 will ensure that all values
8264 can be exactly represented as 32bit floats. This can be useful to
8265 avoid overflow (to +/- infinity), and for efficiency of generation and
8266 shrinking.
8267 Half-precision floats (width=16) are also supported, but require Numpy
8269 if you are running Python 3.5 or earlier.
8270 3.66.33 - 2018-08-10
8272 This release fixes a bug in floats(), where setting allow_infin‐
8273 ity=False and exactly one of min_value and max_value would allow infi‐
8274 nite values to be generated.
8275 3.66.32 - 2018-08-09
8277 This release adds type hints to the example() and seed() decorators,
8278 and fixes the type hint on register_type_strategy(). The second argu‐
8279 ment to register_type_strategy() must either be a SearchStrategy, or a
8280 callable which takes a type and returns a SearchStrategy.
8281 3.66.31 - 2018-08-08
8283 Another set of changes designed to improve the performance of shrinking
8284 on large examples. In particular the shrinker should now spend consid‐
8285 erably less time running useless shrinks.
8286 3.66.30 - 2018-08-06
8288 "Bug fixes and performance improvements".
8289 This release is a fairly major overhaul of the shrinker designed to
8291 improve its behaviour on large examples, especially around stateful
8292 testing. You should hopefully see shrinking become much faster, with
8293 little to no quality degradation (in some cases quality may even
8294 improve).
8295 3.66.29 - 2018-08-05
8297 This release fixes two very minor bugs in the core engine:
8298 · it fixes a corner case that was missing in 3.66.28, which should
8300 cause shrinking to work slightly better.
8301 · it fixes some logic for how shrinking interacts with the database
8303 that was causing Hypothesis to be insufficiently aggressive about
8304 clearing out old keys.
8305 3.66.28 - 2018-08-05
8307 This release improves how Hypothesis handles reducing the size of inte‐
8308 gers' representation. This change should mostly be invisible as it's
8309 purely about the underlying representation and not the generated value,
8310 but it may result in some improvements to shrink performance.
8311 3.66.27 - 2018-08-05
8313 This release changes the order in which Hypothesis chooses parts of the
8314 test case to shrink. For typical usage this should be a significant
8315 performance improvement on large examples. It is unlikely to have a
8316 major impact on example quality, but where it does change the result it
8317 should usually be an improvement.
8318 3.66.26 - 2018-08-05
8320 This release improves the debugging information that the shrinker emits
8321 about the operations it performs, giving better summary statistics
8322 about which passes resulted in test executions and whether they were
8323 successful.
8324 3.66.25 - 2018-08-05
8326 This release fixes several bugs that were introduced to the shrinker in
8327 3.66.24 which would have caused it to behave significantly less well
8328 than advertised. With any luck you should actually see the promised
8329 benefits now.
8330 3.66.24 - 2018-08-03
8332 This release changes how Hypothesis deletes data when shrinking in
8333 order to better handle deletion of large numbers of contiguous
8334 sequences. Most tests should see little change, but this will hopefully
8335 provide a significant speed up for stateful testing.
8336 3.66.23 - 2018-08-02
8338 This release makes some internal changes to enable further improvements
8339 to the shrinker. You may see some changes in the final shrunk examples,
8340 but they are unlikely to be significant.
8341 3.66.22 - 2018-08-01
8343 This release adds some more internal caching to the shrinker. This
8344 should cause a significant speed up for shrinking, especially for
8345 stateful testing and large example sizes.
8346 3.66.21 - 2018-08-01
8348 This patch is for downstream packagers - our tests now pass under
8349 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
8350 of Hypothesis itself.
8351 3.66.20 - 2018-08-01
8353 This release removes some functionality from the shrinker that was tak‐
8354 ing a considerable amount of time and does not appear to be useful any
8355 more due to a number of quality improvements in the shrinker.
8356 You may see some degradation in shrink quality as a result of this, but
8358 mostly shrinking should just get much faster.
8359 3.66.19 - 2018-08-01
8361 This release slightly changes the format of some debugging information
8362 emitted during shrinking, and refactors some of the internal interfaces
8363 around that.
8364 3.66.18 - 2018-07-31
8366 This release is a very small internal refactoring which should have no
8367 user visible impact.
8368 3.66.17 - 2018-07-31
8370 This release fixes a bug that could cause an IndexError to be raised
8371 from inside Hypothesis during shrinking. It is likely that it was
8372 impossible to trigger this bug in practice - it was only made visible
8373 by some currently unreleased work.
8374 3.66.16 - 2018-07-31
8376 This release is a very small internal refactoring which should have no
8377 user visible impact.
8378 3.66.15 - 2018-07-31
8380 This release makes Hypothesis's shrinking faster by removing some
8381 redundant work that it does when minimizing values in its internal rep‐
8382 resentation.
8383 3.66.14 - 2018-07-30
8385 This release expands the deprecation of timeout from 3.16.0 to also
8386 emit the deprecation warning in find or stateful testing.
8387 3.66.13 - 2018-07-30
8389 This release adds an additional shrink pass that is able to reduce the
8390 size of examples in some cases where the transformation is non-obvious.
8391 In particular this will improve the quality of some examples which
8392 would have regressed in 3.66.12.
8393 3.66.12 - 2018-07-28
8395 This release changes how we group data together for shrinking. It
8396 should result in improved shrinker performance, especially in stateful
8397 testing.
8398 3.66.11 - 2018-07-28
8400 This patch modifies how which rule to run is selected during rule based
8401 stateful testing. This should result in a slight performance increase
8402 during generation and a significant performance and quality improvement
8403 when shrinking.
8404 As a result of this change, some state machines which would previously
8406 have thrown an InvalidDefinition are no longer detected as invalid.
8407 3.66.10 - 2018-07-28
8409 This release weakens some minor functionality in the shrinker that had
8410 only modest benefit and made its behaviour much harder to reason about.
8411 This is unlikely to have much user visible effect, but it is possible
8413 that in some cases shrinking may get slightly slower. It is primarily
8414 to make it easier to work on the shrinker and pave the way for future
8415 work.
8416 3.66.9 - 2018-07-26
8418 This release improves the information that Hypothesis emits about its
8419 shrinking when verbosity is set to debug.
8420 3.66.8 - 2018-07-24
8422 This patch includes some minor fixes in the documentation, and updates
8423 the minimum version of pytest to 3.0 (released August 2016).
8424 3.66.7 - 2018-07-24
8426 This release fixes a bug where difficult to shrink tests could some‐
8427 times trigger an internal assertion error inside the shrinker.
8428 3.66.6 - 2018-07-23
8430 This patch ensures that Hypothesis fully supports Python 3.7, by
8431 upgrading from_type() (issue #1264) and fixing some minor issues in our
8432 test suite (issue #1148).
8433 3.66.5 - 2018-07-22
8435 This patch fixes the online docs for various extras, by ensuring that
8436 their dependencies are installed on readthedocs.io (issue #1326).
8437 3.66.4 - 2018-07-20
8439 This release improves the shrinker's ability to reorder examples.
8440 For example, consider the following test:
8442 import hypothesis.strategies as st
8444 from hypothesis import given
8445 @given(st.text(), st.text())
8448 def test_non_equal(x, y):
8449 assert x != y
8450 Previously this could have failed with either of x="", y="0" or x="0",
8452 y="". Now it should always fail with x="", y="0".
8453 This will allow the shrinker to produce more consistent results, espe‐
8455 cially in cases where test cases contain some ordered collection whose
8456 actual order does not matter.
8457 3.66.3 - 2018-07-20
8459 This patch fixes inference in the builds() strategy with subtypes of
8460 python:typing.NamedTuple, where the __init__ method is not useful for
8461 introspection. We now use the field types instead - thanks to James
8462 Uther for identifying this bug.
8463 3.66.2 - 2018-07-19
8465 This release improves the shrinker's ability to handle situations where
8466 there is an additive constraint between two values.
8467 For example, consider the following test:
8469 import hypothesis.strategies as st
8471 from hypothesis import given
8472 @given(st.integers(), st.integers())
8475 def test_does_not_exceed_100(m, n):
8476 assert m + n < 100
8477 Previously this could have failed with almost any pair (m, n) with 0 <=
8479 m <= n and m + n == 100. Now it should almost always fail with m=0,
8480 n=100.
8481 This is a relatively niche specialisation, but can be useful in situa‐
8483 tions where e.g. a bug is triggered by an integer overflow.
8484 3.66.1 - 2018-07-09
8486 This patch fixes a rare bug where an incorrect percentage drawtime
8487 could be displayed for a test, when the system clock was changed during
8488 a test running under Python 2 (we use python:time.monotonic() where it
8489 is available to avoid such problems). It also fixes a possible
8490 zero-division error that can occur when the underlying C library dou‐
8491 ble-rounds an intermediate value in python:math.fsum() and gets the
8492 least significant bit wrong.
8493 3.66.0 - 2018-07-05
8495 This release improves validation of the alphabet argument to the text()
8496 strategy. The following misuses are now deprecated, and will be an
8497 error in a future version:
8498 · passing an unordered collection (such as set('abc')), which violates
8500 invariants about shrinking and reproducibility
8501 · passing an alphabet sequence with elements that are not strings
8503 · passing an alphabet sequence with elements that are not of length
8505 one, which violates any size constraints that may apply
8506 Thanks to Sushobhit for adding these warnings (issue #1329).
8508 3.65.3 - 2018-07-04
8510 This release fixes a mostly theoretical bug where certain usage of the
8511 internal API could trigger an assertion error inside Hypothesis. It is
8512 unlikely that this problem is even possible to trigger through the pub‐
8513 lic API.
8514 3.65.2 - 2018-07-04
8516 This release fixes dependency information for coverage. Previously
8517 Hypothesis would allow installing coverage with any version, but it
8518 only works with coverage 4.0 or later.
8519 We now specify the correct metadata in our setup.py, so Hypothesis will
8521 only allow installation with compatible versions of coverage.
8522 3.65.1 - 2018-07-03
8524 This patch ensures that stateful tests which raise an error from a
8525 pytest helper still print the sequence of steps taken to reach that
8526 point (issue #1372). This reporting was previously broken because the
8527 helpers inherit directly from python:BaseException, and therefore
8528 require special handling to catch without breaking e.g. the use of
8529 ctrl-C to quit the test.
8530 3.65.0 - 2018-06-30
8532 This release deprecates the max_shrinks setting in favor of an internal
8533 heuristic. If you need to avoid shrinking examples, use the phases
8534 setting instead. (issue #1235)
8535 3.64.2 - 2018-06-27
8537 This release fixes a bug where an internal assertion error could some‐
8538 times be triggered while shrinking a failing test.
8539 3.64.1 - 2018-06-27
8541 This patch fixes type-checking errors in our vendored pretty-printer,
8542 which were ignored by our mypy config but visible for anyone else
8543 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
8544 3.64.0 - 2018-06-26
8546 This release adds an interface which can be used to insert a wrapper
8547 between the original test function and @given (issue #1257). This will
8548 be particularly useful for test runner extensions such as pytest-trio,
8549 but is not recommended for direct use by other users of Hypothesis.
8550 3.63.0 - 2018-06-26
8552 This release adds a new mechanism to infer strategies for classes
8553 defined using attrs, based on the the type, converter, or validator of
8554 each attribute. This inference is now built in to builds() and
8555 from_type().
8556 On Python 2, from_type() no longer generates instances of int when
8558 passed long, or vice-versa.
8559 3.62.0 - 2018-06-26
8561 This release adds PEP 484 type hints to Hypothesis on a provisional
8562 basis, using the comment-based syntax for Python 2 compatibility. You
8563 can read more about our type hints here.
8564 It also adds the py.typed marker specified in PEP 561. After you pip
8566 install hypothesis, mypy 0.590 or later will therefore type-check your
8567 use of our public interface!
8568 3.61.0 - 2018-06-24
8570 This release deprecates the use of settings as a context manager, the
8571 use of which is somewhat ambiguous.
8572 Users should define settings with global state or with the @set‐
8574 tings(...) decorator.
8575 3.60.1 - 2018-06-20
8577 Fixed a bug in generating an instance of a Django model from a strategy
8578 where the primary key is generated as part of the strategy. See details
8579 here.
8580 Thanks to Tim Martin for this contribution.
8582 3.60.0 - 2018-06-20
8584 This release adds the @initialize decorator for stateful testing (orig‐
8585 inally discussed in issue #1216). All @initialize rules will be called
8586 once each in an arbitrary order before any normal rule is called.
8587 3.59.3 - 2018-06-19
8589 This is a no-op release to take into account some changes to the
8590 release process. It should have no user visible effect.
8591 3.59.2 - 2018-06-18
8593 This adds support for partially sorting examples which cannot be fully
8594 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
8595 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
8596 0].
8597 Thanks to Luke for contributing.
8599 3.59.1 - 2018-06-16
8601 This patch uses python:random.getstate() and python:random.setstate()
8602 to restore the PRNG state after @given runs deterministic tests. With‐
8603 out restoring state, you might have noticed problems such as issue
8604 #1266. The fix also applies to stateful testing (issue #702).
8605 3.59.0 - 2018-06-14
8607 This release adds the emails() strategy, which generates unicode
8608 strings representing an email address.
8609 Thanks to Sushobhit for moving this to the public API (issue #162).
8611 3.58.1 - 2018-06-13
8613 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
8614 1 2 3.
8615 Thanks to Luke for contributing.
8617 3.58.0 - 2018-06-13
8619 This adds a new extra timezones() strategy that generates dateutil
8620 timezones.
8621 Thanks to Conrad for contributing.
8623 3.57.0 - 2018-05-20
8625 Using an unordered collection with the permutations() strategy has been
8626 deprecated because the order in which e.g. a set shrinks is arbitrary.
8627 This may cause different results between runs.
8628 3.56.10 - 2018-05-16
8630 This release makes hypothesis.settings.define_setting a private method,
8631 which has the effect of hiding it from the documentation.
8632 3.56.9 - 2018-05-11
8634 This is another release with no functionality changes as part of
8635 changes to Hypothesis's new release tagging scheme.
8636 3.56.8 - 2018-05-10
8638 This is a release with no functionality changes that moves Hypothesis
8639 over to a new release tagging scheme.
8640 3.56.7 - 2018-05-10
8642 This release provides a performance improvement for most tests, but in
8643 particular users of sampled_from() who don't have numpy installed
8644 should see a significant performance improvement.
8645 3.56.6 - 2018-05-09
8647 This patch contains further internal work to support Mypy. There are
8648 no user-visible changes... yet.
8649 3.56.5 - 2018-04-22
8651 This patch contains some internal refactoring to run mypy in CI. There
8652 are no user-visible changes.
8653 3.56.4 - 2018-04-21
8655 This release involves some very minor internal clean up and should have
8656 no user visible effect at all.
8657 3.56.3 - 2018-04-20
8659 This release fixes a problem introduced in 3.56.0 where setting the
8660 hypothesis home directory (through currently undocumented means) would
8661 no longer result in the default database location living in the new
8662 home directory.
8663 3.56.2 - 2018-04-20
8665 This release fixes a problem introduced in 3.56.0 where setting
8666 max_examples to 1 would result in tests failing with Unsatisfiable.
8667 This problem could also occur in other harder to trigger circumstances
8668 (e.g. by setting it to a low value, having a hard to satisfy assump‐
8669 tion, and disabling health checks).
8670 3.56.1 - 2018-04-20
8672 This release fixes a problem that was introduced in 3.56.0: Use of the
8673 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
8674 cated, actually broken due to being read before various setup the dep‐
8675 recation path needed was done. It now works correctly (and emits a dep‐
8676 recation warning).
8677 3.56.0 - 2018-04-17
8679 This release deprecates several redundant or internally oriented set‐
8680 tings, working towards an orthogonal set of configuration options that
8681 are widely useful without requiring any knowledge of our internals (‐
8682 issue #535).
8683 · Deprecated settings that no longer have any effect are no longer
8685 shown in the __repr__ unless set to a non-default value.
8686 · hypothesis.settings.perform_health_check is deprecated, as it dupli‐
8688 cates suppress_health_check.
8689 · hypothesis.settings.max_iterations is deprecated and disabled,
8691 because we can usually get better behaviour from an internal heuris‐
8692 tic than a user-controlled setting.
8693 · hypothesis.settings.min_satisfying_examples is deprecated and dis‐
8695 abled, due to overlap with the filter_too_much healthcheck and poor
8696 interaction with max_examples.
8697 · HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
8699 the profile system instead.
8700 · Examples tried by find() are now reported at debug verbosity level
8702 (as well as verbose level).
8703 3.55.6 - 2018-04-14
8705 This release fixes a somewhat obscure condition (issue #1230) under
8706 which you could occasionally see a failing test trigger an assertion
8707 error inside Hypothesis instead of failing normally.
8708 3.55.5 - 2018-04-14
8710 This patch fixes one possible cause of issue #966. When running Python
8711 2 with hash randomisation, passing a python:bytes object to python:ran‐
8712 dom.seed() would use version=1, which broke derandomize (because the
8713 seed depended on a randomised hash). If derandomize is still nondeter‐
8714 ministic for you, please open an issue.
8715 3.55.4 - 2018-04-13
8717 This patch makes a variety of minor improvements to the documentation,
8718 and improves a few validation messages for invalid inputs.
8719 3.55.3 - 2018-04-12
8721 This release updates the URL metadata associated with the PyPI package
8722 (again). It has no other user visible effects.
8723 3.55.2 - 2018-04-11
8725 This release updates the URL metadata associated with the PyPI package.
8726 It has no other user visible effects.
8727 3.55.1 - 2018-04-06
8729 This patch relaxes constraints in our tests on the expected values
8730 returned by the standard library function hypot() and the internal
8731 helper function cathetus, to fix near-exact test failures on some
8732 32-bit systems used by downstream packagers.
8733 3.55.0 - 2018-04-05
8735 This release includes several improvements to the handling of the data‐
8736 base setting.
8737 · The database_file setting was a historical artefact, and you should
8739 just use database directly.
8740 · The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
8742 favor of load_profile() and the database setting.
8743 · If you have not configured the example database at all and the
8745 default location is not usable (due to e.g. permissions issues),
8746 Hypothesis will fall back to an in-memory database. This is not per‐
8747 sisted between sessions, but means that the defaults work on
8748 read-only filesystems.
8749 3.54.0 - 2018-04-04
8751 This release improves the complex_numbers() strategy, which now sup‐
8752 ports min_magnitude and max_magnitude arguments, along with allow_nan
8753 and allow_infinity like for floats().
8754 Thanks to J.J. Green for this feature.
8756 3.53.0 - 2018-04-01
8758 This release removes support for Django 1.8, which reached end of life
8759 on 2018-04-01. You can see Django's release and support schedule on
8760 the Django Project website.
8761 3.52.3 - 2018-04-01
8763 This patch fixes the min_satisfying_examples settings documentation, by
8764 explaining that example shrinking is tracked at the level of the under‐
8765 lying bytestream rather than the output value.
8766 The output from find() in verbose mode has also been adjusted - see the
8768 example session - to avoid duplicating lines when the example repr is
8769 constant, even if the underlying representation has been shrunken.
8770 3.52.2 - 2018-03-30
8772 This release improves the output of failures with rule based stateful
8773 testing in two ways:
8774 · The output from it is now usually valid Python code.
8776 · When the same value has two different names because it belongs to two
8778 different bundles, it will now display with the name associated with
8779 the correct bundle for a rule argument where it is used.
8780 3.52.1 - 2018-03-29
8782 This release improves the behaviour of stateful testing in two ways:
8783 · Previously some runs would run no steps (issue #376). This should no
8785 longer happen.
8786 · RuleBasedStateMachine tests which used bundles extensively would
8788 often shrink terribly. This should now be significantly improved,
8789 though there is likely a lot more room for improvement.
8790 This release also involves a low level change to how ranges of integers
8792 are handles which may result in other improvements to shrink quality in
8793 some cases.
8794 3.52.0 - 2018-03-24
8796 This release deprecates use of @settings(...) as a decorator, on func‐
8797 tions or methods that are not also decorated with @given. You can
8798 still apply these decorators in any order, though you should only do so
8799 once each.
8800 Applying @given twice was already deprecated, and applying @set‐
8802 tings(...) twice is deprecated in this release and will become an error
8803 in a future version. Neither could ever be used twice to good effect.
8804 Using @settings(...) as the sole decorator on a test is completely
8806 pointless, so this common usage error will become an error in a future
8807 version of Hypothesis.
8808 3.51.0 - 2018-03-24
8810 This release deprecates the average_size argument to lists() and other
8811 collection strategies. You should simply delete it wherever it was
8812 used in your tests, as it no longer has any effect.
8813 In early versions of Hypothesis, the average_size argument was treated
8815 as a hint about the distribution of examples from a strategy. Subse‐
8816 quent improvements to the conceptual model and the engine for generat‐
8817 ing and shrinking examples mean it is more effective to simply describe
8818 what constitutes a valid example, and let our internals handle the dis‐
8819 tribution.
8820 3.50.3 - 2018-03-24
8822 This patch contains some internal refactoring so that we can run with
8823 warnings as errors in CI.
8824 3.50.2 - 2018-03-20
8826 This has no user-visible changes except one slight formatting change to
8827 one docstring, to avoid a deprecation warning.
8828 3.50.1 - 2018-03-20
8830 This patch fixes an internal error introduced in 3.48.0, where a check
8831 for the Django test runner would expose import-time errors in Django
8832 configuration (issue #1167).
8833 3.50.0 - 2018-03-19
8835 This release improves validation of numeric bounds for some strategies.
8836 · integers() and floats() now raise InvalidArgument if passed a
8838 min_value or max_value which is not an instance of Real, instead of
8839 various internal errors.
8840 · floats() now converts its bounding values to the nearest float above
8842 or below the min or max bound respectively, instead of just casting
8843 to float. The old behaviour was incorrect in that you could generate
8844 float(min_value), even when this was less than min_value itself (pos‐
8845 sible with eg. fractions).
8846 · When both bounds are provided to floats() but there are no floats in
8848 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
8849 raised.
8850 · decimals() gives a more useful error message if passed a string that
8852 cannot be converted to Decimal in a context where this error is not
8853 trapped.
8854 Code that previously seemed to work may be explicitly broken if there
8856 were no floats between min_value and max_value (only possible with
8857 non-float bounds), or if a bound was not a Real number but still
8858 allowed in python:math.isnan (some custom classes with a __float__
8859 method).
8860 3.49.1 - 2018-03-15
8862 This patch fixes our tests for Numpy dtype strategies on big-endian
8863 platforms, where the strategy behaved correctly but the test assumed
8864 that the native byte order was little-endian.
8865 There is no user impact unless you are running our test suite on
8867 big-endian platforms. Thanks to Graham Inggs for reporting issue
8868 #1164.
8869 3.49.0 - 2018-03-12
8871 This release deprecates passing elements=None to collection strategies,
8872 such as lists().
8873 Requiring lists(nothing()) or builds(list) instead of lists() means
8875 slightly more typing, but also improves the consistency and discover‐
8876 ability of our API - as well as showing how to compose or construct
8877 strategies in ways that still work in more complex situations.
8878 Passing a nonzero max_size to a collection strategy where the elements
8880 strategy contains no values is now deprecated, and will be an error in
8881 a future version. The equivalent with elements=None is already an
8882 error.
8883 3.48.1 - 2018-03-05
8885 This patch will minimize examples that would come out non-minimal in
8886 previous versions. Thanks to Kyle Reeve for this patch.
8887 3.48.0 - 2018-03-05
8889 This release improves some "unhappy paths" when using Hypothesis with
8890 the standard library python:unittest module:
8891 · Applying @given to a non-test method which is overridden from
8893 python:unittest.TestCase, such as setUp, raises a new health check.
8894 (issue #991)
8895 · Using subTest() within a test decorated with @given would leak inter‐
8897 mediate results when tests were run under the python:unittest test
8898 runner. Individual reporting of failing subtests is now disabled
8899 during a test using @given. (issue #1071)
8900 · @given is still not a class decorator, but the error message if you
8902 try using it on a class has been improved.
8903 As a related improvement, using django:django.test.TestCase with @given
8905 instead of hypothesis.extra.django.TestCase raises an explicit error
8906 instead of running all examples in a single database transaction.
8907 3.47.0 - 2018-03-02
8909 register_profile now accepts keyword arguments for specific settings,
8910 and the parent settings object is now optional. Using a name for a
8911 registered profile which is not a string was never suggested, but it is
8912 now also deprecated and will eventually be an error.
8913 3.46.2 - 2018-03-01
8915 This release removes an unnecessary branch from the code, and has no
8916 user-visible impact.
8917 3.46.1 - 2018-03-01
8919 This changes only the formatting of our docstrings and should have no
8920 user-visible effects.
8921 3.46.0 - 2018-02-26
8923 characters() has improved docs about what arguments are valid, and
8924 additional validation logic to raise a clear error early (instead of
8925 e.g. silently ignoring a bad argument). Categories may be specified as
8926 the Unicode 'general category' (eg u'Nd'), or as the 'major category'
8927 (eg [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
8928 In previous versions, general categories were supported and all other
8930 input was silently ignored. Now, major categories are supported in
8931 addition to general categories (which may change the behaviour of some
8932 existing code), and all other input is deprecated.
8933 3.45.5 - 2018-02-26
8935 This patch improves strategy inference in hypothesis.extra.django to
8936 account for some validators in addition to field type - see issue #1116
8937 for ongoing work in this space.
8938 Specifically, if a CharField or TextField has an attached RegexValida‐
8940 tor, we now use from_regex() instead of text() as the underlying strat‐
8941 egy. This allows us to generate examples of the default User model,
8942 closing issue #1112.
8943 3.45.4 - 2018-02-25
8945 This patch improves some internal debugging information, fixes a typo
8946 in a validation error message, and expands the documentation for new
8947 contributors.
8948 3.45.3 - 2018-02-23
8950 This patch may improve example shrinking slightly for some strategies.
8951 3.45.2 - 2018-02-18
8953 This release makes our docstring style more consistent, thanks to
8954 flake8-docstrings. There are no user-visible changes.
8955 3.45.1 - 2018-02-17
8957 This fixes an indentation issue in docstrings for datetimes(), dates(),
8958 times(), and timedeltas().
8959 3.45.0 - 2018-02-13
8961 This release fixes builds() so that target can be used as a keyword
8962 argument for passing values to the target. The target itself can still
8963 be specified as a keyword argument, but that behavior is now depre‐
8964 cated. The target should be provided as the first positional argument.
8965 3.44.26 - 2018-02-06
8967 This release fixes some formatting issues in the Hypothesis source
8968 code. It should have no externally visible effects.
8969 3.44.25 - 2018-02-05
8971 This release changes the way in which Hypothesis tries to shrink the
8972 size of examples. It probably won't have much impact, but might make
8973 shrinking faster in some cases. It is unlikely but not impossible that
8974 it will change the resulting examples.
8975 3.44.24 - 2018-01-27
8977 This release fixes dependency information when installing Hypothesis
8978 from a binary "wheel" distribution.
8979 · The install_requires for enum34 is resolved at install time, rather
8981 than at build time (with potentially different results).
8982 · Django has fixed their python_requires for versions 2.0.0 onward,
8984 simplifying Python2-compatible constraints for downstream projects.
8985 3.44.23 - 2018-01-24
8987 This release improves shrinking in a class of pathological examples
8988 that you are probably never hitting in practice. If you are hitting
8989 them in practice this should be a significant speed up in shrinking. If
8990 you are not, you are very unlikely to notice any difference. You might
8991 see a slight slow down and/or slightly better falsifying examples.
8992 3.44.22 - 2018-01-23
8994 This release fixes a dependency problem. It was possible to install
8995 Hypothesis with an old version of attrs, which would throw a TypeError
8996 as soon as you tried to import hypothesis. Specifically, you need
8997 attrs 16.0.0 or newer.
8998 Hypothesis will now require the correct version of attrs when
9000 installing.
9001 3.44.21 - 2018-01-22
9003 This change adds some additional structural information that Hypothesis
9004 will use to guide its search.
9005 You mostly shouldn't see much difference from this. The two most likely
9007 effects you would notice are:
9008 1. Hypothesis stores slightly more examples in its database for passing
9010 tests.
9011 2. Hypothesis may find new bugs that it was previously missing, but it
9013 probably won't (this is a basic implementation of the feature that
9014 is intended to support future work. Although it is useful on its
9015 own, it's not very useful on its own).
9016 3.44.20 - 2018-01-21
9018 This is a small refactoring release that changes how Hypothesis tracks
9019 some information about the boundary of examples in its internal repre‐
9020 sentation.
9021 You are unlikely to see much difference in behaviour, but memory usage
9023 and run time may both go down slightly during normal test execution,
9024 and when failing Hypothesis might print its failing example slightly
9025 sooner.
9026 3.44.19 - 2018-01-21
9028 This changes how we compute the default average_size for all collection
9029 strategies. Previously setting a max_size without setting an aver‐
9030 age_size would have the seemingly paradoxical effect of making data
9031 generation slower, because it would raise the average size from its
9032 default. Now setting max_size will either leave the default unchanged
9033 or lower it from its default.
9034 If you are currently experiencing this problem, this may make your
9036 tests substantially faster. If you are not, this will likely have no
9037 effect on you.
9038 3.44.18 - 2018-01-20
9040 This is a small refactoring release that changes how Hypothesis detects
9041 when the structure of data generation depends on earlier values gener‐
9042 ated (e.g. when using flatmap or composite()). It should not have any
9043 observable effect on behaviour.
9044 3.44.17 - 2018-01-15
9046 This release fixes a typo in internal documentation, and has no
9047 user-visible impact.
9048 3.44.16 - 2018-01-13
9050 This release improves test case reduction for recursive data struc‐
9051 tures. Hypothesis now guarantees that whenever a strategy calls itself
9052 recursively (usually this will happen because you are using
9053 deferred()), any recursive call may replace the top level value. e.g.
9054 given a tree structure, Hypothesis will always try replacing it with a
9055 subtree.
9056 Additionally this introduces a new heuristic that may in some circum‐
9058 stances significantly speed up test case reduction - Hypothesis should
9059 be better at immediately replacing elements drawn inside another strat‐
9060 egy with their minimal possible value.
9061 3.44.15 - 2018-01-13
9063 from_type() can now resolve recursive types such as binary trees (issue
9064 #1004). Detection of non-type arguments has also improved, leading to
9065 better error messages in many cases involving forward references.
9066 3.44.14 - 2018-01-08
9068 This release fixes a bug in the shrinker that prevented the optimisa‐
9069 tions in 3.44.6 from working in some cases. It would not have worked
9070 correctly when filtered examples were nested (e.g. with a set of inte‐
9071 gers in some range).
9072 This would not have resulted in any correctness problems, but shrinking
9074 may have been slower than it otherwise could be.
9075 3.44.13 - 2018-01-08
9077 This release changes the average bit length of values drawn from inte‐
9078 gers() to be much smaller. Additionally it changes the shrinking order
9079 so that now size is considered before sign - e.g. -1 will be preferred
9080 to +10.
9081 The new internal format for integers required some changes to the mini‐
9083 mizer to make work well, so you may also see some improvements to exam‐
9084 ple quality in unrelated areas.
9085 3.44.12 - 2018-01-07
9087 This changes Hypothesis's internal implementation of weighted sampling.
9088 This will affect example distribution and quality, but you shouldn't
9089 see any other effects.
9090 3.44.11 - 2018-01-06
9092 This is a change to some internals around how Hypothesis handles avoid‐
9093 ing generating duplicate examples and seeking out novel regions of the
9094 search space.
9095 You are unlikely to see much difference as a result of it, but it fixes
9097 a bug where an internal assertion could theoretically be triggered and
9098 has some minor effects on the distribution of examples so could poten‐
9099 tially find bugs that have previously been missed.
9100 3.44.10 - 2018-01-06
9102 This patch avoids creating debug statements when debugging is disabled.
9103 Profiling suggests this is a 5-10% performance improvement (pull
9104 request #1040).
9105 3.44.9 - 2018-01-06
9107 This patch blacklists null characters ('\x00') in automatically created
9108 strategies for Django CharField and TextField, due to a database issue
9109 which was recently fixed upstream (Hypothesis issue #1045).
9110 3.44.8 - 2018-01-06
9112 This release makes the Hypothesis shrinker slightly less greedy in
9113 order to avoid local minima - when it gets stuck, it makes a small
9114 attempt to search around the final example it would previously have
9115 returned to find a new starting point to shrink from. This should
9116 improve example quality in some cases, especially ones where the test
9117 data has dependencies among parts of it that make it difficult for
9118 Hypothesis to proceed.
9119 3.44.7 - 2018-01-04
9121 This release adds support for Django 2 in the hypothesis-django extra.
9122 This release drops support for Django 1.10, as it is no longer sup‐
9124 ported by the Django team.
9125 3.44.6 - 2018-01-02
9127 This release speeds up test case reduction in many examples by being
9128 better at detecting large shrinks it can use to discard redundant parts
9129 of its input. This will be particularly noticeable in examples that
9130 make use of filtering and for some integer ranges.
9131 3.44.5 - 2018-01-02
9133 Happy new year!
9134 This is a no-op release that updates the year range on all of the copy‐
9136 right headers in our source to include 2018.
9137 3.44.4 - 2017-12-23
9139 This release fixes issue #1044, which slowed tests by up to 6% due to
9140 broken caching.
9141 3.44.3 - 2017-12-21
9143 This release improves the shrinker in cases where examples drawn ear‐
9144 lier can affect how much data is drawn later (e.g. when you draw a
9145 length parameter in a composite and then draw that many elements).
9146 Examples found in cases like this should now be much closer to minimal.
9147 3.44.2 - 2017-12-20
9149 This is a pure refactoring release which changes how Hypothesis manages
9150 its set of examples internally. It should have no externally visible
9151 effects.
9152 3.44.1 - 2017-12-18
9154 This release fixes issue #997, in which under some circumstances the
9155 body of tests run under Hypothesis would not show up when run under
9156 coverage even though the tests were run and the code they called out‐
9157 side of the test file would show up normally.
9158 3.44.0 - 2017-12-17
9160 This release adds a new feature: The @reproduce_failure decorator,
9161 designed to make it easy to use Hypothesis's binary format for examples
9162 to reproduce a problem locally without having to share your example
9163 database between machines.
9164 This also changes when seeds are printed:
9166 · They will no longer be printed for normal falsifying examples, as
9168 there are now adequate ways of reproducing those for all cases, so it
9169 just contributes noise.
9170 · They will once again be printed when reusing examples from the data‐
9172 base, as health check failures should now be more reliable in this
9173 scenario so it will almost always work in this case.
9174 This work was funded by Smarkets.
9176 3.43.1 - 2017-12-17
9178 This release fixes a bug with Hypothesis's database management - exam‐
9179 ples that were found in the course of shrinking were saved in a way
9180 that indicated that they had distinct causes, and so they would all be
9181 retried on the start of the next test. The intended behaviour, which is
9182 now what is implemented, is that only a bounded subset of these exam‐
9183 ples would be retried.
9184 3.43.0 - 2017-12-17
9186 HypothesisDeprecationWarning now inherits from python:FutureWarning
9187 instead of python:DeprecationWarning, as recommended by PEP 565 for
9188 user-facing warnings (issue #618). If you have not changed the default
9189 warnings settings, you will now see each distinct HypothesisDepreca‐
9190 tionWarning instead of only the first.
9191 3.42.2 - 2017-12-12
9193 This patch fixes issue #1017, where instances of a list or tuple sub‐
9194 type used as an argument to a strategy would be coerced to tuple.
9195 3.42.1 - 2017-12-10
9197 This release has some internal cleanup, which makes reading the code
9198 more pleasant and may shrink large examples slightly faster.
9199 3.42.0 - 2017-12-09
9201 This release deprecates faker-extra, which was designed as a transition
9202 strategy but does not support example shrinking or coverage-guided dis‐
9203 covery.
9204 3.41.0 - 2017-12-06
9206 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
9207 pling from multi-dimensional ndarrays still results in a deprecation
9208 warning. Thanks to Charlie Tanksley for this patch.
9209 3.40.1 - 2017-12-04
9211 This release makes two changes:
9212 · It makes the calculation of some of the metadata that Hypothesis uses
9214 for shrinking occur lazily. This should speed up performance of test
9215 case generation a bit because it no longer calculates information it
9216 doesn't need.
9217 · It improves the shrinker for certain classes of nested examples. e.g.
9219 when shrinking lists of lists, the shrinker is now able to concate‐
9220 nate two adjacent lists together into a single list. As a result of
9221 this change, shrinking may get somewhat slower when the minimal exam‐
9222 ple found is large.
9223 3.40.0 - 2017-12-02
9225 This release improves how various ways of seeding Hypothesis interact
9226 with the example database:
9227 · Using the example database with seed() is now deprecated. You should
9229 set database=None if you are doing that. This will only warn if you
9230 actually load examples from the database while using @seed.
9231 · The derandomize will behave the same way as @seed.
9233 · Using --hypothesis-seed will disable use of the database.
9235 · If a test used examples from the database, it will not suggest using
9237 a seed to reproduce it, because that won't work.
9238 This work was funded by Smarkets.
9240 3.39.0 - 2017-12-01
9242 This release adds a new health check that checks if the smallest "natu‐
9243 ral" possible example of your test case is very large - this will tend
9244 to cause Hypothesis to generate bad examples and be quite slow.
9245 This work was funded by Smarkets.
9247 3.38.9 - 2017-11-29
9249 This is a documentation release to improve the documentation of shrink‐
9250 ing behaviour for Hypothesis's strategies.
9251 3.38.8 - 2017-11-29
9253 This release improves the performance of characters() when using black‐
9254 list_characters and from_regex() when using negative character classes.
9255 The problems this fixes were found in the course of work funded by
9257 Smarkets.
9258 3.38.7 - 2017-11-29
9260 This is a patch release for from_regex(), which had a bug in handling
9261 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
9262 rectly when parsing regex.
9263 3.38.6 - 2017-11-28
9265 This patch changes a few byte-string literals from double to single
9266 quotes, thanks to an update in unify. There are no user-visible
9267 changes.
9268 3.38.5 - 2017-11-23
9270 This fixes the repr of strategies using lambda that are defined inside
9271 decorators to include the lambda source.
9272 This would mostly have been visible when using the statistics function‐
9274 ality - lambdas used for e.g. filtering would have shown up with a
9275 <unknown> as their body. This can still happen, but it should happen
9276 less often now.
9277 3.38.4 - 2017-11-22
9279 This release updates the reported statistics so that they show approxi‐
9280 mately what fraction of your test run time is spent in data generation
9281 (as opposed to test execution).
9282 This work was funded by Smarkets.
9284 3.38.3 - 2017-11-21
9286 This is a documentation release, which ensures code examples are up to
9287 date by running them as doctests in CI (issue #711).
9288 3.38.2 - 2017-11-21
9290 This release changes the behaviour of the deadline setting when used
9291 with data(): Time spent inside calls to data.draw will no longer be
9292 counted towards the deadline time.
9293 As a side effect of some refactoring required for this work, the way
9295 flaky tests are handled has changed slightly. You are unlikely to see
9296 much difference from this, but some error messages will have changed.
9297 This work was funded by Smarkets.
9299 3.38.1 - 2017-11-21
9301 This patch has a variety of non-user-visible refactorings, removing
9302 various minor warts ranging from indirect imports to typos in comments.
9303 3.38.0 - 2017-11-18
9305 This release overhauls the health check system in a variety of small
9306 ways. It adds no new features, but is nevertheless a minor release
9307 because it changes which tests are likely to fail health checks.
9308 The most noticeable effect is that some tests that used to fail health
9310 checks will now pass, and some that used to pass will fail. These
9311 should all be improvements in accuracy. In particular:
9312 · New failures will usually be because they are now taking into account
9314 things like use of data() and assume() inside the test body.
9315 · New failures may also be because for some classes of example the way
9317 data generation performance was measured was artificially faster than
9318 real data generation (for most examples that are hitting performance
9319 health checks the opposite should be the case).
9320 · Tests that used to fail health checks and now pass do so because the
9322 health check system used to run in a way that was subtly different
9323 than the main Hypothesis data generation and lacked some of its sup‐
9324 port for e.g. large examples.
9325 If your data generation is especially slow, you may also see your tests
9327 get somewhat faster, as there is no longer a separate health check
9328 phase. This will be particularly noticeable when rerunning test fail‐
9329 ures.
9330 This work was funded by Smarkets.
9332 3.37.0 - 2017-11-12
9334 This is a deprecation release for some health check related features.
9335 The following are now deprecated:
9337 · Passing HealthCheck.exception_in_generation to suppress_health_check.
9339 This no longer does anything even when passed - All errors that
9340 occur during data generation will now be immediately reraised rather
9341 than going through the health check mechanism.
9342 · Passing HealthCheck.random_module to suppress_health_check. This
9344 hasn't done anything for a long time, but was never explicitly depre‐
9345 cated. Hypothesis always seeds the random module when running @given
9346 tests, so this is no longer an error and suppressing it doesn't do
9347 anything.
9348 · Passing non-HealthCheck values in suppress_health_check. This was
9350 previously allowed but never did anything useful.
9351 In addition, passing a non-iterable value as suppress_health_check will
9353 now raise an error immediately (it would never have worked correctly,
9354 but it would previously have failed later). Some validation error mes‐
9355 sages have also been updated.
9356 This work was funded by Smarkets.
9358 3.36.1 - 2017-11-10
9360 This is a yak shaving release, mostly concerned with our own tests.
9361 While getfullargspec() was documented as deprecated in Python 3.5, it
9363 never actually emitted a warning. Our code to silence this (nonexis‐
9364 tent) warning has therefore been removed.
9365 We now run our tests with DeprecationWarning as an error, and made some
9367 minor changes to our own tests as a result. This required similar
9368 upstream updates to coverage and execnet (a test-time dependency via
9369 pytest-xdist).
9370 There is no user-visible change in Hypothesis itself, but we encourage
9372 you to consider enabling deprecations as errors in your own tests.
9373 3.36.0 - 2017-11-06
9375 This release adds a setting to the public API, and does some internal
9376 cleanup:
9377 · The derandomize setting is now documented (issue #890)
9379 · Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
9381 #953)
9382 · Documented the strict setting as deprecated, and updated the build so
9384 our docs always match deprecations in the code.
9385 3.35.0 - 2017-11-06
9387 This minor release supports constraining uuids() to generate a particu‐
9388 lar version of UUID (issue #721).
9389 Thanks to Dion Misic for this feature.
9391 3.34.1 - 2017-11-02
9393 This patch updates the documentation to suggest builds(callable)
9394 instead of just(callable()).
9395 3.34.0 - 2017-11-02
9397 Hypothesis now emits deprecation warnings if you apply @given more than
9398 once to a target.
9399 Applying @given repeatedly wraps the target multiple times. Each wrap‐
9401 per will search the space of of possible parameters separately. This
9402 is equivalent but will be much more inefficient than doing it with a
9403 single call to @given.
9404 For example, instead of @given(booleans()) @given(integers()), you
9406 could write @given(booleans(), integers())
9407 3.33.1 - 2017-11-02
9409 This is a bugfix release:
9410 · builds() would try to infer a strategy for required positional argu‐
9412 ments of the target from type hints, even if they had been given to
9413 builds() as positional arguments (issue #946). Now it only infers
9414 missing required arguments.
9415 · An internal introspection function wrongly reported self as a
9417 required argument for bound methods, which might also have affected
9418 builds(). Now it knows better.
9419 3.33.0 - 2017-10-16
9421 This release supports strategy inference for more Django field types -
9422 you can now omit an argument for Date, Time, Duration, Slug, IP
9423 Address, and UUID fields. (issue #642)
9424 Strategy generation for fields with grouped choices now selects choices
9426 from each group, instead of selecting from the group names.
9427 3.32.2 - 2017-10-15
9429 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
9430 an experimental basis. It has never actually worked, and the new
9431 Hypothesis example database is designed to make such a tool unneces‐
9432 sary.
9433 3.32.1 - 2017-10-13
9435 This patch has two improvements for strategies based on enumerations.
9436 · from_type() now handles enumerations correctly, delegating to sam‐
9438 pled_from(). Previously it noted that Enum.__init__ has no required
9439 arguments and therefore delegated to builds(), which would subse‐
9440 quently fail.
9441 · When sampling from an python:enum.Flag, we also generate combinations
9443 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
9444 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
9445 3.32.0 - 2017-10-09
9447 This changes the default value of the use_coverage setting to True when
9448 running on pypy (it was already True on CPython).
9449 It was previously set to False because we expected it to be too slow,
9451 but recent benchmarking shows that actually performance of the feature
9452 on pypy is fairly acceptable - sometimes it's slower than on CPython,
9453 sometimes it's faster, but it's generally within a factor of two either
9454 way.
9455 3.31.6 - 2017-10-08
9457 This patch improves the quality of strategies inferred from Numpy
9458 dtypes:
9459 · Integer dtypes generated examples with the upper half of their
9461 (non-sign) bits set to zero. The inferred strategies can now produce
9462 any representable integer.
9463 · Fixed-width unicode- and byte-string dtypes now cap the internal
9465 example length, which should improve example and shrink quality.
9466 · Numpy arrays can only store fixed-size strings internally, and allow
9468 shorter strings by right-padding them with null bytes. Inferred
9469 string strategies no longer generate such values, as they can never
9470 be retrieved from an array. This improves shrinking performance by
9471 skipping useless values.
9472 This has already been useful in Hypothesis - we found an overflow bug
9474 in our Pandas support, and as a result indexes() and range_indexes()
9475 now check that min_size and max_size are at least zero.
9476 3.31.5 - 2017-10-08
9478 This release fixes a performance problem in tests where the use_cover‐
9479 age setting is True.
9480 Tests experience a slow-down proportionate to the amount of code they
9482 cover. This is still the case, but the factor is now low enough that
9483 it should be unnoticeable. Previously it was large and became much
9484 larger in 3.30.4.
9485 3.31.4 - 2017-10-08
9487 from_type() failed with a very confusing error if passed a NewType() (‐
9488 issue #901). These psudeo-types are now unwrapped correctly, and
9489 strategy inference works as expected.
9490 3.31.3 - 2017-10-06
9492 This release makes some small optimisations to our use of coverage that
9493 should reduce constant per-example overhead. This is probably only
9494 noticeable on examples where the test itself is quite fast. On no-op
9495 tests that don't test anything you may see up to a fourfold speed
9496 increase (which is still significantly slower than without coverage).
9497 On more realistic tests the speed up is likely to be less than that.
9498 3.31.2 - 2017-09-30
9500 This release fixes some formatting and small typos/grammar issues in
9501 the documentation, specifically the page docs/settings.rst, and the
9502 inline docs for the various settings.
9503 3.31.1 - 2017-09-30
9505 This release improves the handling of deadlines so that they act better
9506 with the shrinking process. This fixes issue #892.
9507 This involves two changes:
9509 1. The deadline is raised during the initial generation and shrinking,
9511 and then lowered to the set value for final replay. This restricts
9512 our attention to examples which exceed the deadline by a more sig‐
9513 nificant margin, which increases their reliability.
9514 2. When despite the above a test still becomes flaky because it is sig‐
9516 nificantly faster on rerun than it was on its first run, the error
9517 message is now more explicit about the nature of this problem, and
9518 includes both the initial test run time and the new test run time.
9519 In addition, this release also clarifies the documentation of the dead‐
9521 line setting slightly to be more explicit about where it applies.
9522 This work was funded by Smarkets.
9524 3.31.0 - 2017-09-29
9526 This release blocks installation of Hypothesis on Python 3.3, which
9527 reached its end of life date on 2017-09-29.
9528 This should not be of interest to anyone but downstream maintainers -
9530 if you are affected, migrate to a secure version of Python as soon as
9531 possible or at least seek commercial support.
9532 3.30.4 - 2017-09-27
9534 This release makes several changes:
9535 1. It significantly improves Hypothesis's ability to use coverage
9537 information to find interesting examples.
9538 2. It reduces the default max_examples setting from 200 to 100. This
9540 takes advantage of the improved algorithm meaning fewer examples are
9541 typically needed to get the same testing and is sufficiently better
9542 at covering interesting behaviour, and offsets some of the perfor‐
9543 mance problems of running under coverage.
9544 3. Hypothesis will always try to start its testing with an example that
9546 is near minimized.
9547 The new algorithm for 1 also makes some changes to Hypothesis's low
9549 level data generation which apply even with coverage turned off. They
9550 generally reduce the total amount of data generated, which should
9551 improve test performance somewhat. Between this and 3 you should see a
9552 noticeable reduction in test runtime (how much so depends on your tests
9553 and how much example size affects their performance. On our benchmarks,
9554 where data generation dominates, we saw up to a factor of two perfor‐
9555 mance improvement, but it's unlikely to be that large.
9556 3.30.3 - 2017-09-25
9558 This release fixes some formatting and small typos/grammar issues in
9559 the documentation, specifically the page docs/details.rst, and some
9560 inline docs linked from there.
9561 3.30.2 - 2017-09-24
9563 This release changes Hypothesis's caching approach for functions in
9564 hypothesis.strategies. Previously it would have cached extremely
9565 aggressively and cache entries would never be evicted. Now it adopts a
9566 least-frequently used, least recently used key invalidation policy, and
9567 is somewhat more conservative about which strategies it caches.
9568 Workloads which create strategies based on dynamic values, e.g. by
9570 using flatmap or composite(), will use significantly less memory.
9571 3.30.1 - 2017-09-22
9573 This release fixes a bug where when running with the use_coverage=True
9574 setting inside an existing running instance of coverage, Hypothesis
9575 would frequently put files that the coveragerc excluded in the report
9576 for the enclosing coverage.
9577 3.30.0 - 2017-09-20
9579 This release introduces two new features:
9580 · When a test fails, either with a health check failure or a falsifying
9582 example, Hypothesis will print out a seed that led to that failure,
9583 if the test is not already running with a fixed seed. You can then
9584 recreate that failure using either the @seed decorator or (if you are
9585 running pytest) with --hypothesis-seed.
9586 · pytest users can specify a seed to use for @given based tests by
9588 passing the --hypothesis-seed command line argument.
9589 This work was funded by Smarkets.
9591 3.29.0 - 2017-09-19
9593 This release makes Hypothesis coverage aware. Hypothesis now runs all
9594 test bodies under coverage, and uses this information to guide its
9595 testing.
9596 The use_coverage setting can be used to disable this behaviour if you
9598 want to test code that is sensitive to coverage being enabled (either
9599 because of performance or interaction with the trace function).
9600 The main benefits of this feature are:
9602 · Hypothesis now observes when examples it discovers cover particular
9604 lines or branches and stores them in the database for later.
9605 · Hypothesis will make some use of this information to guide its explo‐
9607 ration of the search space and improve the examples it finds (this is
9608 currently used only very lightly and will likely improve signifi‐
9609 cantly in future releases).
9610 This also has the following side-effects:
9612 · Hypothesis now has an install time dependency on the coverage pack‐
9614 age.
9615 · Tests that are already running Hypothesis under coverage will likely
9617 get faster.
9618 · Tests that are not running under coverage now run their test bodies
9620 under coverage by default.
9621 This feature is only partially supported under pypy. It is signifi‐
9623 cantly slower than on CPython and is turned off by default as a result,
9624 but it should still work correctly if you want to use it.
9625 3.28.3 - 2017-09-18
9627 This release is an internal change that affects how Hypothesis handles
9628 calculating certain properties of strategies.
9629 The primary effect of this is that it fixes a bug where use of
9631 deferred() could sometimes trigger an internal assertion error. However
9632 the fix for this bug involved some moderately deep changes to how
9633 Hypothesis handles certain constructs so you may notice some additional
9634 knock-on effects.
9635 In particular the way Hypothesis handles drawing data from strategies
9637 that cannot generate any values has changed to bail out sooner than it
9638 previously did. This may speed up certain tests, but it is unlikely to
9639 make much of a difference in practice for tests that were not already
9640 failing with Unsatisfiable.
9641 3.28.2 - 2017-09-18
9643 This is a patch release that fixes a bug in the hypothesis.extra.pandas
9644 documentation where it incorrectly referred to column() instead of col‐
9645 umns().
9646 3.28.1 - 2017-09-16
9648 This is a refactoring release. It moves a number of internal uses of
9649 namedtuple() over to using attrs based classes, and removes a couple of
9650 internal namedtuple classes that were no longer in use.
9651 It should have no user visible impact.
9653 3.28.0 - 2017-09-15
9655 This release adds support for testing pandas via the hypothe‐
9656 sis.extra.pandas module.
9657 It also adds a dependency on attrs.
9659 This work was funded by Stripe.
9661 3.27.1 - 2017-09-14
9663 This release fixes some formatting and broken cross-references in the
9664 documentation, which includes editing docstrings - and thus a patch
9665 release.
9666 3.27.0 - 2017-09-13
9668 This release introduces a deadline setting to Hypothesis.
9669 When set this turns slow tests into errors. By default it is unset but
9671 will warn if you exceed 200ms, which will become the default value in a
9672 future release.
9673 This work was funded by Smarkets.
9675 3.26.0 - 2017-09-12
9677 Hypothesis now emits deprecation warnings if you are using the legacy
9678 SQLite example database format, or the tool for merging them. These
9679 were already documented as deprecated, so this doesn't change their
9680 deprecation status, only that we warn about it.
9681 3.25.1 - 2017-09-12
9683 This release fixes a bug with generating numpy datetime and timedelta
9684 types: When inferring the strategy from the dtype, datetime and
9685 timedelta dtypes with sub-second precision would always produce exam‐
9686 ples with one second resolution. Inferring a strategy from a time
9687 dtype will now always produce example with the same precision.
9688 3.25.0 - 2017-09-12
9690 This release changes how Hypothesis shrinks and replays examples to
9691 take into account that it can encounter new bugs while shrinking the
9692 bug it originally found. Previously it would end up replacing the orig‐
9693 inally found bug with the new bug and show you only that one. Now it is
9694 (often) able to recognise when two bugs are distinct and when it finds
9695 more than one will show both.
9696 3.24.2 - 2017-09-11
9698 This release removes the (purely internal and no longer useful) strat‐
9699 egy_test_suite function and the corresponding strategytests module.
9700 3.24.1 - 2017-09-06
9702 This release improves the reduction of examples involving floating
9703 point numbers to produce more human readable examples.
9704 It also has some general purpose changes to the way the minimizer works
9706 internally, which may see some improvement in quality and slow down of
9707 test case reduction in cases that have nothing to do with floating
9708 point numbers.
9709 3.24.0 - 2017-09-05
9711 Hypothesis now emits deprecation warnings if you use some_strat‐
9712 egy.example() inside a test function or strategy definition (this was
9713 never intended to be supported, but is sufficiently widespread that it
9714 warrants a deprecation path).
9715 3.23.3 - 2017-09-05
9717 This is a bugfix release for decimals() with the places argument.
9718 · No longer fails health checks (issue #725, due to internal filtering)
9720 · Specifying a min_value and max_value without any decimals with places
9722 places between them gives a more useful error message.
9723 · Works for any valid arguments, regardless of the decimal precision
9725 context.
9726 3.23.2 - 2017-09-01
9728 This is a small refactoring release that removes a now-unused parameter
9729 to an internal API. It shouldn't have any user visible effect.
9730 3.23.1 - 2017-09-01
9732 Hypothesis no longer propagates the dynamic scope of settings into
9733 strategy definitions.
9734 This release is a small change to something that was never part of the
9736 public API and you will almost certainly not notice any effect unless
9737 you're doing something surprising, but for example the following code
9738 will now give a different answer in some circumstances:
9739 import hypothesis.strategies as st
9741 from hypothesis import settings
9742 CURRENT_SETTINGS = st.builds(lambda: settings.default)
9744 (We don't actually encourage you writing code like this)
9746 Previously this would have generated the settings that were in effect
9748 at the point of definition of CURRENT_SETTINGS. Now it will generate
9749 the settings that are used for the current test.
9750 It is very unlikely to be significant enough to be visible, but you may
9752 also notice a small performance improvement.
9753 3.23.0 - 2017-08-31
9755 This release adds a unique argument to arrays() which behaves the same
9756 ways as the corresponding one for lists(), requiring all of the ele‐
9757 ments in the generated array to be distinct.
9758 3.22.2 - 2017-08-29
9760 This release fixes an issue where Hypothesis would raise a TypeError
9761 when using the datetime-related strategies if running with PYTHONOPTI‐
9762 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
9763 3.22.1 - 2017-08-28
9765 Hypothesis now transparently handles problems with an internal unicode
9766 cache, including file truncation or read-only filesystems (issue #767).
9767 Thanks to Sam Hames for the patch.
9768 3.22.0 - 2017-08-26
9770 This release provides what should be a substantial performance improve‐
9771 ment to numpy arrays generated using provided numpy support, and adds a
9772 new fill_value argument to arrays() to control this behaviour.
9773 This work was funded by Stripe.
9775 3.21.3 - 2017-08-26
9777 This release fixes some extremely specific circumstances that probably
9778 have never occurred in the wild where users of deferred() might have
9779 seen a python:RuntimeError from too much recursion, usually in cases
9780 where no valid example could have been generated anyway.
9781 3.21.2 - 2017-08-25
9783 This release fixes some minor bugs in argument validation:
9784 · hypothesis.extra.numpy dtype strategies would raise an internal
9786 error instead of an InvalidArgument exception when passed an
9787 invalid endianness specification.
9788 · fractions() would raise an internal error instead of an InvalidAr‐
9790 gument if passed float("nan") as one of its bounds.
9791 · The error message for passing float("nan") as a bound to various
9793 strategies has been improved.
9794 · Various bound arguments will now raise InvalidArgument in cases
9796 where they would previously have raised an internal TypeError or
9797 ValueError from the relevant conversion function.
9798 · streaming() would not have emitted a deprecation warning when
9800 called with an invalid argument.
9801 3.21.1 - 2017-08-24
9803 This release fixes a bug where test failures that were the result of an
9804 @example would print an extra stack trace before re-raising the excep‐
9805 tion.
9806 3.21.0 - 2017-08-23
9808 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
9809 sis's deprecation warnings into errors. Similar functionality can be
9810 achieved by using simplefilter('error', HypothesisDeprecationWarning).
9811 3.20.0 - 2017-08-22
9813 This release renames the relevant arguments on the datetimes(),
9814 dates(), times(), and timedeltas() strategies to min_value and
9815 max_value, to make them consistent with the other strategies in the
9816 module.
9817 The old argument names are still supported but will emit a deprecation
9819 warning when used explicitly as keyword arguments. Arguments passed
9820 positionally will go to the new argument names and are not deprecated.
9821 3.19.3 - 2017-08-22
9823 This release provides a major overhaul to the internals of how Hypothe‐
9824 sis handles shrinking.
9825 This should mostly be visible in terms of getting better examples for
9827 tests which make heavy use of composite(), data() or flatmap where the
9828 data drawn depends a lot on previous choices, especially where size
9829 parameters are affected. Previously Hypothesis would have struggled to
9830 reliably produce good examples here. Now it should do much better. Per‐
9831 formance should also be better for examples with a non-zero min_size.
9832 You may see slight changes to example generation (e.g. improved example
9834 diversity) as a result of related changes to internals, but they are
9835 unlikely to be significant enough to notice.
9836 3.19.2 - 2017-08-21
9838 This release fixes two bugs in hypothesis.extra.numpy:
9839 · unicode_string_dtypes() didn't work at all due to an incorrect dtype
9841 specifier. Now it does.
9842 · Various impossible conditions would have been accepted but would
9844 error when they fail to produced any example. Now they raise an
9845 explicit InvalidArgument error.
9846 3.19.1 - 2017-08-21
9848 This is a bugfix release for issue #739, where bounds for fractions()
9849 or floating-point decimals() were not properly converted to integers
9850 before passing them to the integers strategy. This excluded some val‐
9851 ues that should have been possible, and could trigger internal errors
9852 if the bounds lay between adjacent integers.
9853 You can now bound fractions() with two arbitrarily close fractions.
9855 It is now an explicit error to supply a min_value, max_value, and
9857 max_denominator to fractions() where the value bounds do not include a
9858 fraction with denominator at most max_denominator.
9859 3.19.0 - 2017-08-20
9861 This release adds the from_regex() strategy, which generates strings
9862 that contain a match of a regular expression.
9863 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
9865 then helping to upstream it! (issue #662)
9866 3.18.5 - 2017-08-18
9868 This is a bugfix release for integers(). Previously the strategy would
9869 hit an internal assertion if passed non-integer bounds for min_value
9870 and max_value that had no integers between them. The strategy now
9871 raises InvalidArgument instead.
9872 3.18.4 - 2017-08-18
9874 Release to fix a bug where mocks can be used as test runners under cer‐
9875 tain conditions. Specifically, if a mock is injected into a test via
9876 pytest fixtures or patch decorators, and that mock is the first argu‐
9877 ment in the list, hypothesis will think it represents self and turns
9878 the mock into a test runner. If this happens, the affected test always
9879 passes because the mock is executed instead of the test body. Some‐
9880 times, it will also fail health checks.
9881 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
9883 for this bug fix.
9884 3.18.3 - 2017-08-17
9886 This release should improve the performance of some tests which experi‐
9887 enced a slow down as a result of the 3.13.0 release.
9888 Tests most likely to benefit from this are ones that make extensive use
9890 of min_size parameters, but others may see some improvement as well.
9891 3.18.2 - 2017-08-16
9893 This release fixes a bug introduced in 3.18.0. If the arguments
9894 whitelist_characters and blacklist_characters to characters() contained
9895 overlapping elements, then an InvalidArgument exception would be
9896 raised.
9897 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
9899 3.18.1 - 2017-08-14
9901 This is a bug fix release to fix issue #780, where sets() and similar
9902 would trigger health check errors if their element strategy could only
9903 produce one element (e.g. if it was just()).
9904 3.18.0 - 2017-08-13
9906 This is a feature release:
9907 · characters() now accepts whitelist_characters, particular characters
9909 which will be added to those it produces. (issue #668)
9910 · A bug fix for the internal function _union_interval_lists(), and a
9912 rename to _union_intervals(). It now correctly handles all cases
9913 where intervals overlap, and it always returns the result as a tuple
9914 for tuples.
9915 Thanks to Alex Willmer for these.
9917 3.17.0 - 2017-08-07
9919 This release documents the previously undocumented phases feature, mak‐
9920 ing it part of the public API. It also updates how the example database
9921 is used. Principally:
9922 · A Phases.reuse argument will now correctly control whether examples
9924 from the database are run (it previously did exactly the wrong thing
9925 and controlled whether examples would be saved).
9926 · Hypothesis will no longer try to rerun all previously failing exam‐
9928 ples. Instead it will replay the smallest previously failing example
9929 and a selection of other examples that are likely to trigger any
9930 other bugs that will found. This prevents a previous failure from
9931 dominating your tests unnecessarily.
9932 · As a result of the previous change, Hypothesis will be slower about
9934 clearing out old examples from the database that are no longer fail‐
9935 ing (because it can only clear out ones that it actually runs).
9936 3.16.1 - 2017-08-07
9938 This release makes an implementation change to how Hypothesis handles
9939 certain internal constructs.
9940 The main effect you should see is improvement to the behaviour and per‐
9942 formance of collection types, especially ones with a min_size parame‐
9943 ter. Many cases that would previously fail due to being unable to gen‐
9944 erate enough valid examples will now succeed, and other cases should
9945 run slightly faster.
9946 3.16.0 - 2017-08-04
9948 This release introduces a deprecation of the timeout feature. This
9949 results in the following changes:
9950 · Creating a settings object with an explicit timeout will emit a dep‐
9952 recation warning.
9953 · If your test stops because it hits the timeout (and has not found a
9955 bug) then it will emit a deprecation warning.
9956 · There is a new value unlimited which you can import from hypothesis.
9958 settings(timeout=unlimited) will not cause a deprecation warning.
9959 · There is a new health check, hung_test, which will trigger after a
9961 test has been running for five minutes if it is not suppressed.
9962 3.15.0 - 2017-08-04
9964 This release deprecates two strategies, choices() and streaming().
9965 Both of these are somewhat confusing to use and are entirely redundant
9967 since the introduction of the data() strategy for interactive drawing
9968 in tests, and their use should be replaced with direct use of data()
9969 instead.
9970 3.14.2 - 2017-08-03
9972 This fixes a bug where Hypothesis would not work correctly on Python
9973 2.7 if you had the python:typing module backport installed.
9974 3.14.1 - 2017-08-02
9976 This raises the maximum depth at which Hypothesis starts cutting off
9977 data generation to a more reasonable value which it is harder to hit by
9978 accident.
9979 This resolves (issue #751), in which some examples which previously
9981 worked would start timing out, but it will also likely improve the data
9982 generation quality for complex data types.
9983 3.14.0 - 2017-07-23
9985 Hypothesis now understands inline type annotations (issue #293):
9986 · If the target of builds() has type annotations, a default strategy
9988 for missing required arguments is selected based on the type.
9989 Type-based strategy selection will only override a default if you
9990 pass hypothesis.infer as a keyword argument.
9991 · If @given wraps a function with type annotations, you can pass infer
9993 as a keyword argument and the appropriate strategy will be substi‐
9994 tuted.
9995 · You can check what strategy will be inferred for a type with the new
9997 from_type() function.
9998 · register_type_strategy() teaches Hypothesis which strategy to infer
10000 for custom or unknown types. You can provide a strategy, or for more
10001 complex cases a function which takes the type and returns a strategy.
10002 3.13.1 - 2017-07-20
10004 This is a bug fix release for issue #514 - Hypothesis would continue
10005 running examples after a SkipTest exception was raised, including
10006 printing a falsifying example. Skip exceptions from the standard
10007 python:unittest module, and pytest, nose, or unittest2 modules now
10008 abort the test immediately without printing output.
10009 3.13.0 - 2017-07-16
10011 This release has two major aspects to it: The first is the introduction
10012 of deferred(), which allows more natural definition of recursive
10013 (including mutually recursive) strategies.
10014 The second is a number of engine changes designed to support this sort
10016 of strategy better. These should have a knock-on effect of also improv‐
10017 ing the performance of any existing strategies that currently generate
10018 a lot of data or involve heavy nesting by reducing their typical exam‐
10019 ple size.
10020 3.12.0 - 2017-07-07
10022 This release makes some major internal changes to how Hypothesis repre‐
10023 sents data internally, as a prelude to some major engine changes that
10024 should improve data quality. There are no API changes, but it's a sig‐
10025 nificant enough internal change that a minor version bump seemed war‐
10026 ranted.
10027 User facing impact should be fairly mild, but includes:
10029 · All existing examples in the database will probably be invalidated.
10031 Hypothesis handles this automatically, so you don't need to do any‐
10032 thing, but if you see all your examples disappear that's why.
10033 · Almost all data distributions have changed significantly. Possibly
10035 for the better, possibly for the worse. This may result in new bugs
10036 being found, but it may also result in Hypothesis being unable to
10037 find bugs it previously did.
10038 · Data generation may be somewhat faster if your existing bottleneck
10040 was in draw_bytes (which is often the case for large examples).
10041 · Shrinking will probably be slower, possibly significantly.
10043 If you notice any effects you consider to be a significant regression,
10045 please open an issue about them.
10046 3.11.6 - 2017-06-19
10048 This release involves no functionality changes, but is the first to
10049 ship wheels as well as an sdist.
10050 3.11.5 - 2017-06-18
10052 This release provides a performance improvement to shrinking. For cases
10053 where there is some non-trivial "boundary" value (e.g. the bug happens
10054 for all values greater than some other value), shrinking should now be
10055 substantially faster. Other types of bug will likely see improvements
10056 too.
10057 This may also result in some changes to the quality of the final exam‐
10059 ples - it may sometimes be better, but is more likely to get slightly
10060 worse in some edge cases. If you see any examples where this happens in
10061 practice, please report them.
10062 3.11.4 - 2017-06-17
10064 This is a bugfix release: Hypothesis now prints explicit examples when
10065 running in verbose mode. (issue #313)
10066 3.11.3 - 2017-06-11
10068 This is a bugfix release: Hypothesis no longer emits a warning if you
10069 try to use sampled_from() with python:collections.OrderedDict. (issue
10070 #688)
10071 3.11.2 - 2017-06-10
10073 This is a documentation release. Several outdated snippets have been
10074 updated or removed, and many cross-references are now hyperlinks.
10075 3.11.1 - 2017-05-28
10077 This is a minor ergonomics release. Tracebacks shown by pytest no
10078 longer include Hypothesis internals for test functions decorated with
10079 @given.
10080 3.11.0 - 2017-05-23
10082 This is a feature release, adding datetime-related strategies to the
10083 core strategies.
10084 timezones() allows you to sample pytz timezones from the Olsen data‐
10086 base. Use directly in a recipe for tz-aware datetimes, or compose with
10087 none() to allow a mix of aware and naive output.
10088 The new dates(), times(), datetimes(), and timedeltas() strategies are
10090 all constrained by objects of their type. This means that you can gen‐
10091 erate dates bounded by a single day (i.e. a single date), or datetimes
10092 constrained to the microsecond.
10093 times() and datetimes() take an optional timezones= argument, which
10095 defaults to none() for naive times. You can use our extra strategy
10096 based on pytz, or roll your own timezones strategy with dateutil or
10097 even the standard library.
10098 The old dates, times, and datetimes strategies in hypothe‐
10100 sis.extra.datetimes are deprecated in favor of the new core strategies,
10101 which are more flexible and have no dependencies.
10102 3.10.0 - 2017-05-22
10104 Hypothesis now uses python:inspect.getfullargspec() internally. On
10105 Python 2, there are no visible changes.
10106 On Python 3 @given and @composite now preserve PEP 3107 annotations on
10108 the decorated function. Keyword-only arguments are now either handled
10109 correctly (e.g. @composite), or caught in validation instead of
10110 silently discarded or raising an unrelated error later (e.g. @given).
10111 3.9.1 - 2017-05-22
10113 This is a bugfix release: the default field mapping for a DateTimeField
10114 in the Django extra now respects the USE_TZ setting when choosing a
10115 strategy.
10116 3.9.0 - 2017-05-19
10118 This is feature release, expanding the capabilities of the decimals()
10119 strategy.
10120 · The new (optional) places argument allows you to generate decimals
10122 with a certain number of places (e.g. cents, thousandths, satoshis).
10123 · If allow_infinity is None, setting min_bound no longer excludes posi‐
10125 tive infinity and setting max_value no longer excludes negative
10126 infinity.
10127 · All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
10129 True, or if allow_nan is None and min_value or max_value is None.
10130 · min_value and max_value may be given as decimal strings, e.g.
10132 "1.234".
10133 3.8.5 - 2017-05-16
10135 Hypothesis now imports python:sqlite3 when a SQLite database is used,
10136 rather than at module load, improving compatibility with Python imple‐
10137 mentations compiled without SQLite support (such as BSD or Jython).
10138 3.8.4 - 2017-05-16
10140 This is a compatibility bugfix release. sampled_from() no longer
10141 raises a deprecation warning when sampling from an python:enum.Enum, as
10142 all enums have a reliable iteration order.
10143 3.8.3 - 2017-05-09
10145 This release removes a version check for older versions of pytest when
10146 using the Hypothesis pytest plugin. The pytest plugin will now run
10147 unconditionally on all versions of pytest. This breaks compatibility
10148 with any version of pytest prior to 2.7.0 (which is more than two years
10149 old).
10150 The primary reason for this change is that the version check was a fre‐
10152 quent source of breakage when pytest change their versioning scheme. If
10153 you are not working on pytest itself and are not running a very old
10154 version of it, this release probably doesn't affect you.
10155 3.8.2 - 2017-04-26
10157 This is a code reorganisation release that moves some internal test
10158 helpers out of the main source tree so as to not have changes to them
10159 trigger releases in future.
10160 3.8.1 - 2017-04-26
10162 This is a documentation release. Almost all code examples are now
10163 doctests checked in CI, eliminating stale examples.
10164 3.8.0 - 2017-04-23
10166 This is a feature release, adding the iterables() strategy, equivalent
10167 to lists(...).map(iter) but with a much more useful repr. You can use
10168 this strategy to check that code doesn't accidentally depend on
10169 sequence properties such as indexing support or repeated iteration.
10170 3.7.4 - 2017-04-22
10172 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
10173 ture in the same test could cause the test to fail to fill the argu‐
10174 ments, and throw a TypeError.
10175 3.7.3 - 2017-04-21
10177 This release should include no user visible changes and is purely a
10178 refactoring release. This modularises the behaviour of the core given()
10179 function, breaking it up into smaller and more accessible parts, but
10180 its actual behaviour should remain unchanged.
10181 3.7.2 - 2017-04-21
10183 This reverts an undocumented change in 3.7.1 which broke installation
10184 on debian stable: The specifier for the hypothesis[django]
10185 extra_requires had introduced a wild card, which was not supported on
10186 the default version of pip.
10187 3.7.1 - 2017-04-21
10189 This is a bug fix and internal improvements release.
10190 · In particular Hypothesis now tracks a tree of where it has already
10192 explored. This allows it to avoid some classes of duplicate exam‐
10193 ples, and significantly improves the performance of shrinking failing
10194 examples by allowing it to skip some shrinks that it can determine
10195 can't possibly work.
10196 · Hypothesis will no longer seed the global random arbitrarily unless
10198 you have asked it to using random_module()
10199 · Shrinking would previously have not worked correctly in some special
10201 cases on Python 2, and would have resulted in suboptimal examples.
10202 3.7.0 - 2017-03-20
10204 This is a feature release.
10205 New features:
10207 · Rule based stateful testing now has an @invariant decorator that
10209 specifies methods that are run after init and after every step,
10210 allowing you to encode properties that should be true at all times.
10211 Thanks to Tom Prince for this feature.
10212 · The decimals() strategy now supports allow_nan and allow_infinity
10214 flags.
10215 · There are significantly more strategies available for numpy, includ‐
10217 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
10218 for this feature.
10219 · When using the data() strategy you can now add a label as an argument
10221 to draw(), which will be printed along with the value when an example
10222 fails. Thanks to Peter Inglesby for this feature.
10223 Bug fixes:
10225 · Bug fix: composite() now preserves functions' docstrings.
10227 · The build is now reproducible and doesn't depend on the path you
10229 build it from. Thanks to Chris Lamb for this feature.
10230 · numpy strategies for the void data type did not work correctly.
10232 Thanks to Zac Hatfield Dodds for this fix.
10233 There have also been a number of performance optimizations:
10235 · The permutations() strategy is now significantly faster to use for
10237 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
10238 · Shrinking of failing test cases should have got significantly faster
10240 in some circumstances where it was previously struggling for a long
10241 time.
10242 · Example generation now involves less indirection, which results in a
10244 small speedup in some cases (small enough that you won't really
10245 notice it except in pathological cases).
10246 3.6.1 - 2016-12-20
10248 This release fixes a dependency problem and makes some small behind the
10249 scenes improvements.
10250 · The fake-factory dependency was renamed to faker. If you were depend‐
10252 ing on it through hypothesis[django] or hypothesis[fake-factory]
10253 without pinning it yourself then it would have failed to install
10254 properly. This release changes it so that hypothesis[fakefactory]
10255 (which can now also be installed as hypothesis[faker]) will install
10256 the renamed faker package instead.
10257 · This release also removed the dependency of hypothesis[django] on
10259 hypothesis[fakefactory] - it was only being used for emails. These
10260 now use a custom strategy that isn't from fakefactory. As a result
10261 you should also see performance improvements of tests which generated
10262 User objects or other things with email fields, as well as better
10263 shrinking of email addresses.
10264 · The distribution of code using nested calls to one_of() or the |
10266 operator for combining strategies has been improved, as branches are
10267 now flattened to give a more uniform distribution.
10268 · Examples using composite() or .flatmap should now shrink better. In
10270 particular this will affect things which work by first generating a
10271 length and then generating that many items, which have historically
10272 not shrunk very well.
10273 3.6.0 - 2016-10-31
10275 This release reverts Hypothesis to its old pretty printing of lambda
10276 functions based on attempting to extract the source code rather than
10277 decompile the bytecode. This is unfortunately slightly inferior in
10278 some cases and may result in you occasionally seeing things like lambda
10279 x: <unknown> in statistics reports and strategy reprs.
10280 This removes the dependencies on uncompyle6, xdis and spark-parser.
10282 The reason for this is that the new functionality was based on
10284 uncompyle6, which turns out to introduce a hidden GPLed dependency - it
10285 in turn depended on xdis, and although the library was licensed under
10286 the MIT license, it contained some GPL licensed source code and thus
10287 should have been released under the GPL.
10288 My interpretation is that Hypothesis itself was never in violation of
10290 the GPL (because the license it is under, the Mozilla Public License
10291 v2, is fully compatible with being included in a GPL licensed work),
10292 but I have not consulted a lawyer on the subject. Regardless of the
10293 answer to this question, adding a GPLed dependency will likely cause a
10294 lot of users of Hypothesis to inadvertently be in violation of the GPL.
10295 As a result, if you are running Hypothesis 3.5.x you really should
10297 upgrade to this release immediately.
10298 3.5.3 - 2016-10-05
10300 This is a bug fix release.
10301 Bugs fixed:
10303 · If the same test was running concurrently in two processes and there
10305 were examples already in the test database which no longer failed,
10306 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
10307 Python 2) because an example it was trying to read was deleted before
10308 it was read. (issue #372).
10309 · Drawing from an integers() strategy with both a min_value and a
10311 max_value would reject too many examples needlessly. Now it repeat‐
10312 edly redraws until satisfied. (pull request #366. Thanks to Calen
10313 Pennington for the contribution).
10314 3.5.2 - 2016-09-24
10316 This is a bug fix release.
10317 · The Hypothesis pytest plugin broke pytest support for doctests. Now
10319 it doesn't.
10320 3.5.1 - 2016-09-23
10322 This is a bug fix release.
10323 · Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
10325 bytes and unicode.
10326 · python:unittest.TestCase tests would not have shown up in the new
10328 statistics mode. Now they do.
10329 · Similarly, stateful tests would not have shown up in statistics and
10331 now they do.
10332 · Statistics now print with pytest node IDs (the names you'd get in
10334 pytest verbose mode).
10335 3.5.0 - 2016-09-22
10337 This is a feature release.
10338 · fractions() and decimals() strategies now support min_value and
10340 max_value parameters. Thanks go to Anne Mulhern for the development
10341 of this feature.
10342 · The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
10344 tics parameter that gives detailed statistics about the tests that
10345 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
10346 funding the development of this feature.
10347 · There is a new event() function that can be used to add custom sta‐
10349 tistics.
10350 Additionally there have been some minor bug fixes:
10352 · In some cases Hypothesis should produce fewer duplicate examples
10354 (this will mostly only affect cases with a single parameter).
10355 · pytest command line parameters are now under an option group for
10357 Hypothesis (thanks to David Keijser for fixing this)
10358 · Hypothesis would previously error if you used PEP 3107 function anno‐
10360 tations on your tests under Python 3.4.
10361 · The repr of many strategies using lambdas has been improved to
10363 include the lambda body (this was previously supported in many but
10364 not all cases).
10365 3.4.2 - 2016-07-13
10367 This is a bug fix release, fixing a number of problems with the set‐
10368 tings system:
10369 · Test functions defined using @given can now be called from other
10371 threads (issue #337)
10372 · Attempting to delete a settings property would previously have
10374 silently done the wrong thing. Now it raises an AttributeError.
10375 · Creating a settings object with a custom database_file parameter was
10377 silently getting ignored and the default was being used instead. Now
10378 it's not.
10379 3.4.1 - 2016-07-07
10381 This is a bug fix release for a single bug:
10382 · On Windows when running two Hypothesis processes in parallel (e.g.
10384 using pytest-xdist) they could race with each other and one would
10385 raise an exception due to the non-atomic nature of file renaming on
10386 Windows and the fact that you can't rename over an existing file.
10387 This is now fixed.
10388 3.4.0 - 2016-05-27
10390 This release is entirely provided by Lucas Wiman:
10391 Strategies constructed by the Django extra will now respect much more
10393 of Django's validations out of the box. Wherever possible,
10394 full_clean() should succeed.
10395 In particular:
10397 · The max_length, blank and choices kwargs are now respected.
10399 · Add support for DecimalField.
10401 · If a field includes validators, the list of validators are used to
10403 filter the field strategy.
10404 3.3.0 - 2016-05-27
10406 This release went wrong and is functionally equivalent to 3.2.0. Ignore
10407 it.
10408 3.2.0 - 2016-05-19
10410 This is a small single-feature release:
10411 · All tests using @given now fix the global random seed. This removes
10413 the health check for that. If a non-zero seed is required for the
10414 final falsifying example, it will be reported. Otherwise Hypothesis
10415 will assume randomization was not a significant factor for the test
10416 and be silent on the subject. If you use random_module() this will
10417 continue to work and will always display the seed.
10418 3.1.3 - 2016-05-01
10420 Single bug fix release
10421 · Another charmap problem. In 3.1.2 text() and characters() would break
10423 on systems which had /tmp mounted on a different partition than the
10424 Hypothesis storage directory (usually in home). This fixes that.
10425 3.1.2 - 2016-04-30
10427 Single bug fix release:
10428 · Anything which used a text() or characters() strategy was broken on
10430 Windows and I hadn't updated appveyor to use the new repository loca‐
10431 tion so I didn't notice. This is now fixed and windows support should
10432 work correctly.
10433 3.1.1 - 2016-04-29
10435 Minor bug fix release.
10436 · Fix concurrency issue when running tests that use text() from multi‐
10438 ple processes at once (issue #302, thanks to Alex Chan).
10439 · Improve performance of code using lists() with max_size (thanks to
10441 Cristi Cobzarenco).
10442 · Fix install on Python 2 with ancient versions of pip so that it
10444 installs the enum34 backport (thanks to Donald Stufft for telling me
10445 how to do this).
10446 · Remove duplicated __all__ exports from hypothesis.strategies (thanks
10448 to Piët Delport).
10449 · Update headers to point to new repository location.
10451 · Allow use of strategies that can't be used in find() (e.g. choices())
10453 in stateful testing.
10454 3.1.0 - 2016-03-06
10456 · Add a nothing() strategy that never successfully generates values.
10457 · sampled_from() and one_of() can both now be called with an empty
10459 argument list, in which case they also never generate any values.
10460 · one_of() may now be called with a single argument that is a collec‐
10462 tion of strategies as well as as varargs.
10463 · Add a runner() strategy which returns the instance of the current
10465 test object if there is one.
10466 · 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
10468 can be used as such.
10469 · Tests using RuleBasedStateMachine should now shrink significantly
10471 better.
10472 · Hypothesis now uses a pretty-printing library internally, compatible
10474 with IPython's pretty printing protocol (actually using the same
10475 code). This may improve the quality of output in some cases.
10476 · Add a 'phases' setting that allows more fine grained control over
10478 which parts of the process Hypothesis runs
10479 · Add a suppress_health_check setting which allows you to turn off spe‐
10481 cific health checks in a fine grained manner.
10482 · Fix a bug where lists of non fixed size would always draw one more
10484 element than they included. This mostly didn't matter, but if would
10485 cause problems with empty strategies or ones with side effects.
10486 · Add a mechanism to the Django model generator to allow you to explic‐
10488 itly request the default value (thanks to Jeremy Thurgood for this
10489 one).
10490 3.0.5 - 2016-02-25
10492 · Fix a bug where Hypothesis would now error on pytest development ver‐
10493 sions.
10494 3.0.4 - 2016-02-24
10496 · Fix a bug where Hypothesis would error when running on Python 2.7.3
10497 or earlier because it was trying to pass a python:bytearray object to
10498 python:struct.unpack() (which is only supported since 2.7.4).
10499 3.0.3 - 2016-02-23
10501 · Fix version parsing of pytest to work with pytest release candidates
10502 · More general handling of the health check problem where things could
10504 fail because of a cache miss - now one "free" example is generated
10505 before the start of the health check run.
10506 3.0.2 - 2016-02-18
10508 · Under certain circumstances, strategies involving text() buried
10509 inside some other strategy (e.g. text().filter(...) or recur‐
10510 sive(text(), ...)) would cause a test to fail its health checks the
10511 first time it ran. This was caused by having to compute some related
10512 data and cache it to disk. On travis or anywhere else where the
10513 .hypothesis directory was recreated this would have caused the tests
10514 to fail their health check on every run. This is now fixed for all
10515 the known cases, although there could be others lurking.
10516 3.0.1 - 2016-02-18
10518 · Fix a case where it was possible to trigger an "Unreachable" asser‐
10519 tion when running certain flaky stateful tests.
10520 · Improve shrinking of large stateful tests by eliminating a case where
10522 it was hard to delete early steps.
10523 · Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
10525 cantly by provide a custom implementation for fixed size blocks that
10526 can bypass a lot of machinery.
10527 · Set default home directory based on the current working directory at
10529 the point Hypothesis is imported, not whenever the function first
10530 happens to be called.
10531 3.0.0 - 2016-02-17
10533 Codename: This really should have been 2.1.
10534 Externally this looks like a very small release. It has one small
10536 breaking change that probably doesn't affect anyone at all (some behav‐
10537 iour that never really worked correctly is now outright forbidden) but
10538 necessitated a major version bump and one visible new feature.
10539 Internally this is a complete rewrite. Almost nothing other than the
10541 public API is the same.
10542 New features:
10544 · Addition of data() strategy which allows you to draw arbitrary data
10546 interactively within the test.
10547 · New "exploded" database format which allows you to more easily check
10549 the example database into a source repository while supporting merg‐
10550 ing.
10551 · Better management of how examples are saved in the database.
10553 · Health checks will now raise as errors when they fail. It was too
10555 easy to have the warnings be swallowed entirely.
10556 New limitations:
10558 · choices() and streaming() strategies may no longer be used with
10560 find(). Neither may data() (this is the change that necessitated a
10561 major version bump).
10562 Feature removal:
10564 · The ForkingTestCase executor has gone away. It may return in some
10566 more working form at a later date.
10567 Performance improvements:
10569 · A new model which allows flatmap, composite strategies and stateful
10571 testing to perform much better. They should also be more reliable.
10572 · Filtering may in some circumstances have improved significantly. This
10574 will help especially in cases where you have lots of values with
10575 individual filters on them, such as lists(x.filter(...)).
10576 · Modest performance improvements to the general test runner by avoid‐
10578 ing expensive operations
10579 In general your tests should have got faster. If they've instead got
10581 significantly slower, I'm interested in hearing about it.
10582 Data distribution:
10584 The data distribution should have changed significantly. This may
10586 uncover bugs the previous version missed. It may also miss bugs the
10587 previous version could have uncovered. Hypothesis is now producing less
10588 strongly correlated data than it used to, but the correlations are
10589 extended over more of the structure.
10590 Shrinking:
10592 Shrinking quality should have improved. In particular Hypothesis can
10594 now perform simultaneous shrinking of separate examples within a single
10595 test (previously it was only able to do this for elements of a single
10596 collection). In some cases performance will have improved, in some
10597 cases it will have got worse but generally shouldn't have by much.
10598 Older versions
10600 2.0.0 - 2016-01-10
10601 Codename: A new beginning
10602 This release cleans up all of the legacy that accrued in the course of
10604 Hypothesis 1.0. These are mostly things that were emitting deprecation
10605 warnings in 1.19.0, but there were a few additional changes.
10606 In particular:
10608 · non-strategy values will no longer be converted to strategies when
10610 used in given or find.
10611 · FailedHealthCheck is now an error and not a warning.
10613 · Handling of non-ascii reprs in user types have been simplified by
10615 using raw strings in more places in Python 2.
10616 · given no longer allows mixing positional and keyword arguments.
10618 · given no longer works with functions with defaults.
10620 · given no longer turns provided arguments into defaults - they will
10622 not appear in the argspec at all.
10623 · the basic() strategy no longer exists.
10625 · the n_ary_tree strategy no longer exists.
10627 · the average_list_length setting no longer exists. Note: If you're
10629 using using recursive() this will cause you a significant slow down.
10630 You should pass explicit average_size parameters to collections in
10631 recursive calls.
10632 · @rule can no longer be applied to the same method twice.
10634 · Python 2.6 and 3.3 are no longer officially supported, although in
10636 practice they still work fine.
10637 This also includes two non-deprecation changes:
10639 · given's keyword arguments no longer have to be the rightmost argu‐
10641 ments and can appear anywhere in the method signature.
10642 · The max_shrinks setting would sometimes not have been respected.
10644 1.19.0 - 2016-01-09
10646 Codename: IT COMES
10647 This release heralds the beginning of a new and terrible age of Hypoth‐
10649 esis 2.0.
10650 It's primary purpose is some final deprecations prior to said release.
10652 The goal is that if your code emits no warnings under this release then
10653 it will probably run unchanged under Hypothesis 2.0 (there are some
10654 caveats to this: 2.0 will drop support for some Python versions, and if
10655 you're using internal APIs then as usual that may break without warn‐
10656 ing).
10657 It does have two new features:
10659 · New @seed() decorator which allows you to manually seed a test. This
10661 may be harmlessly combined with and overrides the derandomize set‐
10662 ting.
10663 · settings objects may now be used as a decorator to fix those settings
10665 to a particular @given test.
10666 API changes (old usage still works but is deprecated):
10668 · Settings has been renamed to settings (lower casing) in order to make
10670 the decorator usage more natural.
10671 · Functions for the storage directory that were in hypothesis.settings
10673 are now in a new hypothesis.configuration module.
10674 Additional deprecations:
10676 · the average_list_length setting has been deprecated in favour of
10678 being explicit.
10679 · the basic() strategy has been deprecated as it is impossible to sup‐
10681 port it under a Conjecture based model, which will hopefully be
10682 implemented at some point in the 2.x series.
10683 · the n_ary_tree strategy (which was never actually part of the public
10685 API) has been deprecated.
10686 · Passing settings or random as keyword arguments to given is depre‐
10688 cated (use the new functionality instead)
10689 Bug fixes:
10691 · No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
10693 tion in streaming() values.
10694 · When running in health check mode with non strict, don't print quite
10696 so many errors for an exception in reify.
10697 · When an assumption made in a test or a filter is flaky, tests will
10699 now raise Flaky instead of UnsatisfiedAssumption.
10700 1.18.1 - 2015-12-22
10702 Two behind the scenes changes:
10703 · Hypothesis will no longer write generated code to the file system.
10705 This will improve performance on some systems (e.g. if you're using
10706 PythonAnywhere which is running your code from NFS) and prevent some
10707 annoying interactions with auto-restarting systems.
10708 · Hypothesis will cache the creation of some strategies. This can sig‐
10710 nificantly improve performance for code that uses flatmap or compos‐
10711 ite and thus has to instantiate strategies a lot.
10712 1.18.0 - 2015-12-21
10714 Features:
10715 · Tests and find are now explicitly seeded off the global random mod‐
10717 ule. This means that if you nest one inside the other you will now
10718 get a health check error. It also means that you can control global
10719 randomization by seeding random.
10720 · There is a new random_module() strategy which seeds the global random
10722 module for you and handles things so that you don't get a health
10723 check warning if you use it inside your tests.
10724 · floats() now accepts two new arguments: allow_nan and allow_infinity.
10726 These default to the old behaviour, but when set to False will do
10727 what the names suggest.
10728 Bug fixes:
10730 · Fix a bug where tests that used text() on Python 3.4+ would not actu‐
10732 ally be deterministic even when explicitly seeded or using the deran‐
10733 domize mode, because generation depended on dictionary iteration
10734 order which was affected by hash randomization.
10735 · Fix a bug where with complicated strategies the timing of the initial
10737 health check could affect the seeding of the subsequent test, which
10738 would also render supposedly deterministic tests non-deterministic in
10739 some scenarios.
10740 · In some circumstances flatmap() could get confused by two struc‐
10742 turally similar things it could generate and would produce a flaky
10743 test where the first time it produced an error but the second time it
10744 produced the other value, which was not an error. The same bug was
10745 presumably also possible in composite().
10746 · flatmap() and composite() initial generation should now be moderately
10748 faster. This will be particularly noticeable when you have many val‐
10749 ues drawn from the same strategy in a single run, e.g. constructs
10750 like lists(s.flatmap(f)). Shrinking performance may have suffered,
10751 but this didn't actually produce an interestingly worse result in any
10752 of the standard scenarios tested.
10753 1.17.1 - 2015-12-16
10755 A small bug fix release, which fixes the fact that the 'note' function
10756 could not be used on tests which used the @example decorator to provide
10757 explicit examples.
10758 1.17.0 - 2015-12-15
10760 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
10761 because it contains the following additional change that was not
10762 intended to be in a patch release (it's perfectly stable, but is a
10763 larger change that should have required a minor version bump):
10764 · Hypothesis will now perform a series of "health checks" as part of
10766 running your tests. These detect and warn about some common error
10767 conditions that people often run into which wouldn't necessarily have
10768 caused the test to fail but would cause e.g. degraded performance or
10769 confusing results.
10770 1.16.1 - 2015-12-14
10772 Note: This release has been removed.
10773 A small bugfix release that allows bdists for Hypothesis to be built
10775 under 2.7 - the compat3.py file which had Python 3 syntax wasn't
10776 intended to be loaded under Python 2, but when building a bdist it was.
10777 In particular this would break running setup.py test.
10778 1.16.0 - 2015-12-08
10780 There are no public API changes in this release but it includes a be‐
10781 haviour change that I wasn't comfortable putting in a patch release.
10782 · Functions from hypothesis.strategies will no longer raise InvalidAr‐
10784 gument on bad arguments. Instead the same errors will be raised when
10785 a test using such a strategy is run. This may improve startup time in
10786 some cases, but the main reason for it is so that errors in strate‐
10787 gies won't cause errors in loading, and it can interact correctly
10788 with things like pytest.mark.skipif.
10789 · Errors caused by accidentally invoking the legacy API are now much
10791 less confusing, although still throw NotImplementedError.
10792 · hypothesis.extra.django is 1.9 compatible.
10794 · When tests are run with max_shrinks=0 this will now still rerun the
10796 test on failure and will no longer print "Trying example:" before
10797 each run. Additionally note() will now work correctly when used with
10798 max_shrinks=0.
10799 1.15.0 - 2015-11-24
10801 A release with two new features.
10802 · A 'characters' strategy for more flexible generation of text with
10804 particular character ranges and types, kindly contributed by
10805 Alexander Shorin.
10806 · Add support for preconditions to the rule based stateful testing.
10808 Kindly contributed by Christopher Armstrong
10809 1.14.0 - 2015-11-01
10811 New features:
10812 · Add 'note' function which lets you include additional information in
10814 the final test run's output.
10815 · Add 'choices' strategy which gives you a choice function that emu‐
10817 lates random.choice.
10818 · Add 'uuid' strategy that generates UUIDs'
10820 · Add 'shared' strategy that lets you create a strategy that just gen‐
10822 erates a single shared value for each test run
10823 Bugs:
10825 · Using strategies of the form streaming(x.flatmap(f)) with find or in
10827 stateful testing would have caused InvalidArgument errors when the
10828 resulting values were used (because code that expected to only be
10829 called within a test context would be invoked).
10830 1.13.0 - 2015-10-29
10832 This is quite a small release, but deprecates some public API functions
10833 and removes some internal API functionality so gets a minor version
10834 bump.
10835 · All calls to the 'strategy' function are now deprecated, even ones
10837 which pass just a SearchStrategy instance (which is still a no-op).
10838 · Never documented hypothesis.extra entry_points mechanism has now been
10840 removed ( it was previously how hypothesis.extra packages were loaded
10841 and has been deprecated and unused for some time)
10842 · Some corner cases that could previously have produced an OverflowEr‐
10844 ror when simplifying failing cases using hypothesis.extra.datetimes
10845 (or dates or times) have now been fixed.
10846 · Hypothesis load time for first import has been significantly reduced
10848 - it used to be around 250ms (on my SSD laptop) and now is around
10849 100-150ms. This almost never matters but was slightly annoying when
10850 using it in the console.
10851 · hypothesis.strategies.randoms was previously missing from __all__.
10853 1.12.0 - 2015-10-18
10855 · Significantly improved performance of creating strategies using the
10856 functions from the hypothesis.strategies module by deferring the cal‐
10857 culation of their repr until it was needed. This is unlikely to have
10858 been an performance issue for you unless you were using flatmap, com‐
10859 posite or stateful testing, but for some cases it could be quite a
10860 significant impact.
10861 · A number of cases where the repr of strategies build from lambdas is
10863 improved
10864 · Add dates() and times() strategies to hypothesis.extra.datetimes
10866 · Add new 'profiles' mechanism to the settings system
10868 · Deprecates mutability of Settings, both the Settings.default top
10870 level property and individual settings.
10871 · A Settings object may now be directly initialized from a parent Set‐
10873 tings.
10874 · @given should now give a better error message if you attempt to use
10876 it with a function that uses destructuring arguments (it still won't
10877 work, but it will error more clearly),
10878 · A number of spelling corrections in error messages
10880 · pytest should no longer display the intermediate modules Hypothesis
10882 generates when running in verbose mode
10883 · Hypothesis should now correctly handle printing objects with
10885 non-ascii reprs on python 3 when running in a locale that cannot han‐
10886 dle ascii printing to stdout.
10887 · Add a unique=True argument to lists(). This is equivalent to
10889 unique_by=lambda x: x, but offers a more convenient syntax.
10890 1.11.4 - 2015-09-27
10892 · Hide modifications Hypothesis needs to make to sys.path by undoing
10893 them after we've imported the relevant modules. This is a workaround
10894 for issues cryptography experienced on windows.
10895 · Slightly improved performance of drawing from sampled_from on large
10897 lists of alternatives.
10898 · Significantly improved performance of drawing from one_of or strate‐
10900 gies using | (note this includes a lot of strategies internally -
10901 floats() and integers() both fall into this category). There turned
10902 out to be a massive performance regression introduced in 1.10.0
10903 affecting these which probably would have made tests using Hypothesis
10904 significantly slower than they should have been.
10905 1.11.3 - 2015-09-23
10907 · Better argument validation for datetimes() strategy - previously set‐
10908 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
10909 would not have raised an InvalidArgument error and instead would have
10910 behaved confusingly.
10911 · Compatibility with being run on pytest < 2.7 (achieved by disabling
10913 the plugin).
10914 1.11.2 - 2015-09-23
10916 Bug fixes:
10917 · Settings(database=my_db) would not be correctly inherited when used
10919 as a default setting, so that newly created settings would use the
10920 database_file setting and create an SQLite example database.
10921 · Settings.default.database = my_db would previously have raised an
10923 error and now works.
10924 · Timeout could sometimes be significantly exceeded if during simplifi‐
10926 cation there were a lot of examples tried that didn't trigger the
10927 bug.
10928 · When loading a heavily simplified example using a basic() strategy
10930 from the database this could cause Python to trigger a recursion
10931 error.
10932 · Remove use of deprecated API in pytest plugin so as to not emit warn‐
10934 ing
10935 Misc:
10937 · hypothesis-pytest is now part of hypothesis core. This should have no
10939 externally visible consequences, but you should update your dependen‐
10940 cies to remove hypothesis-pytest and depend on only Hypothesis.
10941 · Better repr for hypothesis.extra.datetimes() strategies.
10943 · Add .close() method to abstract base class for Backend (it was
10945 already present in the main implementation).
10946 1.11.1 - 2015-09-16
10948 Bug fixes:
10949 · When running Hypothesis tests in parallel (e.g. using pytest-xdist)
10951 there was a race condition caused by code generation.
10952 · Example databases are now cached per thread so as to not use sqlite
10954 connections from multiple threads. This should make Hypothesis now
10955 entirely thread safe.
10956 · floats() with only min_value or max_value set would have had a very
10958 bad distribution.
10959 · Running on 3.5, Hypothesis would have emitted deprecation warnings
10961 because of use of inspect.getargspec
10962 1.11.0 - 2015-08-31
10964 · text() with a non-string alphabet would have used the repr() of the
10965 the alphabet instead of its contexts. This is obviously silly. It now
10966 works with any sequence of things convertible to unicode strings.
10967 · @given will now work on methods whose definitions contains no
10969 explicit positional arguments, only varargs (issue #118). This may
10970 have some knock on effects because it means that @given no longer
10971 changes the argspec of functions other than by adding defaults.
10972 · Introduction of new @composite feature for more natural definition of
10974 strategies you'd previously have used flatmap for.
10975 1.10.6 - 2015-08-26
10977 Fix support for fixtures on Django 1.7.
10978 1.10.4 - 2015-08-21
10980 Tiny bug fix release:
10981 · If the database_file setting is set to None, this would have resulted
10983 in an error when running tests. Now it does the same as setting data‐
10984 base to None.
10985 1.10.3 - 2015-08-19
10987 Another small bug fix release.
10988 · lists(elements, unique_by=some_function, min_size=n) would have
10990 raised a ValidationError if n > Settings.default.average_list_length
10991 because it would have wanted to use an average list length shorter
10992 than the minimum size of the list, which is impossible. Now it
10993 instead defaults to twice the minimum size in these circumstances.
10994 · basic() strategy would have only ever produced at most ten distinct
10996 values per run of the test (which is bad if you e.g. have it inside a
10997 list). This was obviously silly. It will now produce a much better
10998 distribution of data, both duplicated and non duplicated.
10999 1.10.2 - 2015-08-19
11001 This is a small bug fix release:
11002 · star imports from hypothesis should now work correctly.
11004 · example quality for examples using flatmap will be better, as the way
11006 it had previously been implemented was causing problems where Hypoth‐
11007 esis was erroneously labelling some examples as being duplicates.
11008 1.10.0 - 2015-08-04
11010 This is just a bugfix and performance release, but it changes some
11011 semi-public APIs, hence the minor version bump.
11012 · Significant performance improvements for strategies which are
11014 one_of() many branches. In particular this included recursive()
11015 strategies. This should take the case where you use one recursive()
11016 strategy as the base strategy of another from unusably slow (tens of
11017 seconds per generated example) to reasonably fast.
11018 · Better handling of just() and sampled_from() for values which have an
11020 incorrect __repr__ implementation that returns non-ASCII unicode on
11021 Python 2.
11022 · Better performance for flatmap from changing the internal morpher API
11024 to be significantly less general purpose.
11025 · Introduce a new semi-public BuildContext/cleanup API. This allows
11027 strategies to register cleanup activities that should run once the
11028 example is complete. Note that this will interact somewhat weirdly
11029 with find.
11030 · Better simplification behaviour for streaming strategies.
11032 · Don't error on lambdas which use destructuring arguments in Python 2.
11034 · Add some better reprs for a few strategies that were missing good
11036 ones.
11037 · The Random instances provided by randoms() are now copyable.
11039 · Slightly more debugging information about simplify when using a debug
11041 verbosity level.
11042 · Support using given for functions with varargs, but not passing argu‐
11044 ments to it as positional.
11045 1.9.0 - 2015-07-27
11047 Codename: The great bundling.
11048 This release contains two fairly major changes.
11050 The first is the deprecation of the hypothesis-extra mechanism. From
11052 now on all the packages that were previously bundled under it other
11053 than hypothesis-pytest (which is a different beast and will remain sep‐
11054 arate). The functionality remains unchanged and you can still import
11055 them from exactly the same location, they just are no longer separate
11056 packages.
11057 The second is that this introduces a new way of building strategies
11059 which lets you build up strategies recursively from other strategies.
11060 It also contains the minor change that calling .example() on a strategy
11062 object will give you examples that are more representative of the
11063 actual data you'll get. There used to be some logic in there to make
11064 the examples artificially simple but this proved to be a bad idea.
11065 1.8.5 - 2015-07-24
11067 This contains no functionality changes but fixes a mistake made with
11068 building the previous package that would have broken installation on
11069 Windows.
11070 1.8.4 - 2015-07-20
11072 Bugs fixed:
11073 · When a call to floats() had endpoints which were not floats but
11075 merely convertible to one (e.g. integers), these would be included in
11076 the generated data which would cause it to generate non-floats.
11077 · Splitting lambdas used in the definition of flatmap, map or filter
11079 over multiple lines would break the repr, which would in turn break
11080 their usage.
11081 1.8.3 - 2015-07-20
11083 "Falsifying example" would not have been printed when the failure came
11084 from an explicit example.
11085 1.8.2 - 2015-07-18
11087 Another small bugfix release:
11088 · When using ForkingTestCase you would usually not get the falsifying
11090 example printed if the process exited abnormally (e.g. due to
11091 os._exit).
11092 · Improvements to the distribution of characters when using text() with
11094 a default alphabet. In particular produces a better distribution of
11095 ascii and whitespace in the alphabet.
11096 1.8.1 - 2015-07-17
11098 This is a small release that contains a workaround for people who have
11099 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
11100 for Hypothesis per se because that's not a thing that is actually sup‐
11101 posed to work, but Hypothesis leans more heavily on repr than is typi‐
11102 cal so it's worth having a workaround for.
11103 1.8.0 - 2015-07-16
11105 New features:
11106 · Much more sensible reprs for strategies, especially ones that come
11108 from hypothesis.strategies. These should now have as reprs python
11109 code that would produce the same strategy.
11110 · lists() accepts a unique_by argument which forces the generated lists
11112 to be only contain elements unique according to some function key
11113 (which must return a hashable value).
11114 · Better error messages from flaky tests to help you debug things.
11116 Mostly invisible implementation details that may result in finding new
11118 bugs in your code:
11119 · Sets and dictionary generation should now produce a better range of
11121 results.
11122 · floats with bounds now focus more on 'critical values', trying to
11124 produce values at edge cases.
11125 · flatmap should now have better simplification for complicated cases,
11127 as well as generally being (I hope) more reliable.
11128 Bug fixes:
11130 · You could not previously use assume() if you were using the forking
11132 executor.
11133 1.7.2 - 2015-07-10
11135 This is purely a bug fix release:
11136 · When using floats() with stale data in the database you could some‐
11138 times get values in your tests that did not respect min_value or
11139 max_value.
11140 · When getting a Flaky error from an unreliable test it would have
11142 incorrectly displayed the example that caused it.
11143 · 2.6 dependency on backports was incorrectly specified. This would
11145 only have caused you problems if you were building a universal wheel
11146 from Hypothesis, which is not how Hypothesis ships, so unless you're
11147 explicitly building wheels for your dependencies and support Python
11148 2.6 plus a later version of Python this probably would never have
11149 affected you.
11150 · If you use flatmap in a way that the strategy on the right hand side
11152 depends sensitively on the left hand side you may have occasionally
11153 seen Flaky errors caused by producing unreliable examples when mini‐
11154 mizing a bug. This use case may still be somewhat fraught to be hon‐
11155 est. This code is due a major rearchitecture for 1.8, but in the
11156 meantime this release fixes the only source of this error that I'm
11157 aware of.
11158 1.7.1 - 2015-06-29
11160 Codename: There is no 1.7.0.
11161 A slight technical hitch with a premature upload means there's was a
11163 yanked 1.7.0 release. Oops.
11164 The major feature of this release is Python 2.6 support. Thanks to Jeff
11166 Meadows for doing most of the work there.
11167 Other minor features
11169 · strategies now has a permutations() function which returns a strategy
11171 yielding permutations of values from a given collection.
11172 · if you have a flaky test it will print the exception that it last saw
11174 before failing with Flaky, even if you do not have verbose reporting
11175 on.
11176 · Slightly experimental git merge script available as "python -m
11178 hypothesis.tools.mergedbs". Instructions on how to use it in the doc‐
11179 string of that file.
11180 Bug fixes:
11182 · Better performance from use of filter. In particular tests which
11184 involve large numbers of heavily filtered strategies should perform a
11185 lot better.
11186 · floats() with a negative min_value would not have worked correctly
11188 (worryingly, it would have just silently failed to run any examples).
11189 This is now fixed.
11190 · tests using sampled_from would error if the number of sampled ele‐
11192 ments was smaller than min_satisfying_examples.
11193 1.6.2 - 2015-06-08
11195 This is just a few small bug fixes:
11196 · Size bounds were not validated for values for a binary() strategy
11198 when reading examples from the database.
11199 · sampled_from is now in __all__ in hypothesis.strategies
11201 · floats no longer consider negative integers to be simpler than posi‐
11203 tive non-integers
11204 · Small floating point intervals now correctly count members, so if you
11206 have a floating point interval so narrow there are only a handful of
11207 values in it, this will no longer cause an error when Hypothesis runs
11208 out of values.
11209 1.6.1 - 2015-05-21
11211 This is a small patch release that fixes a bug where 1.6.0 broke the
11212 use of flatmap with the deprecated API and assumed the passed in func‐
11213 tion returned a SearchStrategy instance rather than converting it to a
11214 strategy.
11215 1.6.0 - 2015-05-21
11217 This is a smallish release designed to fix a number of bugs and smooth
11218 out some weird behaviours.
11219 · Fix a critical bug in flatmap where it would reuse old strategies. If
11221 all your flatmap code was pure you're fine. If it's not, I'm sur‐
11222 prised it's working at all. In particular if you want to use flatmap
11223 with django models, you desperately need to upgrade to this version.
11224 · flatmap simplification performance should now be better in some cases
11226 where it previously had to redo work.
11227 · Fix for a bug where invalid unicode data with surrogates could be
11229 generated during simplification (it was already filtered out during
11230 actual generation).
11231 · The Hypothesis database is now keyed off the name of the test instead
11233 of the type of data. This makes much more sense now with the new
11234 strategies API and is generally more robust. This means you will lose
11235 old examples on upgrade.
11236 · The database will now not delete values which fail to deserialize
11238 correctly, just skip them. This is to handle cases where multiple
11239 incompatible strategies share the same key.
11240 · find now also saves and loads values from the database, keyed off a
11242 hash of the function you're finding from.
11243 · Stateful tests now serialize and load values from the database. They
11245 should have before, really. This was a bug.
11246 · Passing a different verbosity level into a test would not have worked
11248 entirely correctly, leaving off some messages. This is now fixed.
11249 · Fix a bug where derandomized tests with unicode characters in the
11251 function body would error on Python 2.7.
11252 1.5.0 - 2015-05-14
11254 Codename: Strategic withdrawal.
11255 The purpose of this release is a radical simplification of the API for
11257 building strategies. Instead of the old approach of @strategy.extend
11258 and things that get converted to strategies, you just build strategies
11259 directly.
11260 The old method of defining strategies will still work until Hypothesis
11262 2.0, because it's a major breaking change, but will now emit depreca‐
11263 tion warnings.
11264 The new API is also a lot more powerful as the functions for defining
11266 strategies give you a lot of dials to turn. See the updated data sec‐
11267 tion for details.
11268 Other changes:
11270 · Mixing keyword and positional arguments in a call to @given is
11272 deprecated as well.
11273 · There is a new setting called 'strict'. When set to True, Hypothe‐
11275 sis will raise warnings instead of merely printing them. Turning
11276 it on by default is inadvisable because it means that Hypothesis
11277 minor releases can break your code, but it may be useful for mak‐
11278 ing sure you catch all uses of deprecated APIs.
11279 · max_examples in settings is now interpreted as meaning the maximum
11281 number of unique (ish) examples satisfying assumptions. A new set‐
11282 ting max_iterations which defaults to a larger value has the old
11283 interpretation.
11284 · Example generation should be significantly faster due to a new
11286 faster parameter selection algorithm. This will mostly show up for
11287 simple data types - for complex ones the parameter selection is
11288 almost certainly dominated.
11289 · Simplification has some new heuristics that will tend to cut down
11291 on cases where it could previously take a very long time.
11292 · timeout would previously not have been respected in cases where
11294 there were a lot of duplicate examples. You probably wouldn't have
11295 previously noticed this because max_examples counted duplicates,
11296 so this was very hard to hit in a way that mattered.
11297 · A number of internal simplifications to the SearchStrategy API.
11299 · You can now access the current Hypothesis version as hypothe‐
11301 sis.__version__.
11302 · A top level function is provided for running the stateful tests
11304 without the TestCase infrastructure.
11305 1.4.0 - 2015-05-04
11307 Codename: What a state.
11308 The big feature of this release is the new and slightly experimental
11310 stateful testing API. You can read more about that in the appropriate
11311 section.
11312 Two minor features the were driven out in the course of developing
11314 this:
11315 · You can now set settings.max_shrinks to limit the number of times
11317 Hypothesis will try to shrink arguments to your test. If this is set
11318 to <= 0 then Hypothesis will not rerun your test and will just raise
11319 the failure directly. Note that due to technical limitations if
11320 max_shrinks is <= 0 then Hypothesis will print every example it calls
11321 your test with rather than just the failing one. Note also that I
11322 don't consider settings max_shrinks to zero a sensible way to run
11323 your tests and it should really be considered a debug feature.
11324 · There is a new debug level of verbosity which is even more verbose
11326 than verbose. You probably don't want this.
11327 Breakage of semi-public SearchStrategy API:
11329 · It is now a required invariant of SearchStrategy that if u simplifies
11331 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
11332 plifying should not increase the complexity even though it is not
11333 required to decrease it. Enforcing this invariant lead to finding
11334 some bugs where simplifying of integers, floats and sets was subopti‐
11335 mal.
11336 · Integers in basic data are now required to fit into 64 bits. As a
11338 result python integer types are now serialized as strings, and some
11339 types have stopped using quite so needlessly large random seeds.
11340 Hypothesis Stateful testing was then turned upon Hypothesis itself,
11342 which lead to an amazing number of minor bugs being found in Hypothesis
11343 itself.
11344 Bugs fixed (most but not all from the result of stateful testing)
11346 include:
11347 · Serialization of streaming examples was flaky in a way that you would
11349 probably never notice: If you generate a template, simplify it, seri‐
11350 alize it, deserialize it, serialize it again and then deserialize it
11351 you would get the original stream instead of the simplified one.
11352 · If you reduced max_examples below the number of examples already
11354 saved in the database, you would have got a ValueError. Additionally,
11355 if you had more than max_examples in the database all of them would
11356 have been considered.
11357 · @given will no longer count duplicate examples (which it never called
11359 your function with) towards max_examples. This may result in your
11360 tests running slower, but that's probably just because they're trying
11361 more examples.
11362 · General improvements to example search which should result in better
11364 performance and higher quality examples. In particular parameters
11365 which have a history of producing useless results will be more
11366 aggressively culled. This is useful both because it decreases the
11367 chance of useless examples and also because it's much faster to not
11368 check parameters which we were unlikely to ever pick!
11369 · integers_from and lists of types with only one value (e.g. [None])
11371 would previously have had a very high duplication rate so you were
11372 probably only getting a handful of examples. They now have a much
11373 lower duplication rate, as well as the improvements to search making
11374 this less of a problem in the first place.
11375 · You would sometimes see simplification taking significantly longer
11377 than your defined timeout. This would happen because timeout was only
11378 being checked after each successful simplification, so if Hypothesis
11379 was spending a lot of time unsuccessfully simplifying things it
11380 wouldn't stop in time. The timeout is now applied for unsuccessful
11381 simplifications too.
11382 · In Python 2.7, integers_from strategies would have failed during sim‐
11384 plification with an OverflowError if their starting point was at or
11385 near to the maximum size of a 64-bit integer.
11386 · flatmap and map would have failed if called with a function without a
11388 __name__ attribute.
11389 · If max_examples was less than min_satisfying_examples this would
11391 always error. Now min_satisfying_examples is capped to max_examples.
11392 Note that if you have assumptions to satisfy here this will still
11393 cause an error.
11394 Some minor quality improvements:
11396 · Lists of streams, flatmapped strategies and basic strategies should
11398 now now have slightly better simplification.
11399 1.3.0 - 2015-05-22
11401 New features:
11402 · New verbosity level API for printing intermediate results and excep‐
11404 tions.
11405 · New specifier for strings generated from a specified alphabet.
11407 · Better error messages for tests that are failing because of a lack of
11409 enough examples.
11410 Bug fixes:
11412 · Fix error where use of ForkingTestCase would sometimes result in too
11414 many open files.
11415 · Fix error where saving a failing example that used flatmap could
11417 error.
11418 · Implement simplification for sampled_from, which apparently never
11420 supported it previously. Oops.
11421 General improvements:
11423 · Better range of examples when using one_of or sampled_from.
11425 · Fix some pathological performance issues when simplifying lists of
11427 complex values.
11428 · Fix some pathological performance issues when simplifying examples
11430 that require unicode strings with high codepoints.
11431 · Random will now simplify to more readable examples.
11433 1.2.1 - 2015-04-16
11435 A small patch release for a bug in the new executors feature. Tests
11436 which require doing something to their result in order to fail would
11437 have instead reported as flaky.
11438 1.2.0 - 2015-04-15
11440 Codename: Finders keepers.
11441 A bunch of new features and improvements.
11443 · Provide a mechanism for customizing how your tests are executed.
11445 · Provide a test runner that forks before running each example. This
11447 allows better support for testing native code which might trigger a
11448 segfault or a C level assertion failure.
11449 · Support for using Hypothesis to find examples directly rather than as
11451 just as a test runner.
11452 · New streaming type which lets you generate infinite lazily loaded
11454 streams of data - perfect for if you need a number of examples but
11455 don't know how many.
11456 · Better support for large integer ranges. You can now use inte‐
11458 gers_in_range with ranges of basically any size. Previously large
11459 ranges would have eaten up all your memory and taken forever.
11460 · Integers produce a wider range of data than before - previously they
11462 would only rarely produce integers which didn't fit into a machine
11463 word. Now it's much more common. This percolates to other numeric
11464 types which build on integers.
11465 · Better validation of arguments to @given. Some situations that would
11467 previously have caused silently wrong behaviour will now raise an
11468 error.
11469 · Include +/- sys.float_info.max in the set of floating point edge
11471 cases that Hypothesis specifically tries.
11472 · Fix some bugs in floating point ranges which happen when given +/-
11474 sys.float_info.max as one of the endpoints... (really any two floats
11475 that are sufficiently far apart so that x, y are finite but y - x is
11476 infinite). This would have resulted in generating infinite values
11477 instead of ones inside the range.
11478 1.1.1 - 2015-04-07
11480 Codename: Nothing to see here
11481 This is just a patch release put out because it fixed some internal
11483 bugs that would block the Django integration release but did not actu‐
11484 ally affect anything anyone could previously have been using. It also
11485 contained a minor quality fix for floats that I'd happened to have fin‐
11486 ished in time.
11487 · Fix some internal bugs with object lifecycle management that were
11489 impossible to hit with the previously released versions but broke
11490 hypothesis-django.
11491 · Bias floating point numbers somewhat less aggressively towards very
11493 small numbers
11494 1.1.0 - 2015-04-06
11496 Codename: No-one mention the M word.
11497 · Unicode strings are more strongly biased towards ascii characters.
11499 Previously they would generate all over the space. This is mostly so
11500 that people who try to shape their unicode strings with assume() have
11501 less of a bad time.
11502 · A number of fixes to data deserialization code that could theoreti‐
11504 cally have caused mysterious bugs when using an old version of a
11505 Hypothesis example database with a newer version. To the best of my
11506 knowledge a change that could have triggered this bug has never actu‐
11507 ally been seen in the wild. Certainly no-one ever reported a bug of
11508 this nature.
11509 · Out of the box support for Decimal and Fraction.
11511 · new dictionary specifier for dictionaries with variable keys.
11513 · Significantly faster and higher quality simplification, especially
11515 for collections of data.
11516 · New filter() and flatmap() methods on Strategy for better ways of
11518 building strategies out of other strategies.
11519 · New BasicStrategy class which allows you to define your own strate‐
11521 gies from scratch without needing an existing matching strategy or
11522 being exposed to the full horror or non-public nature of the Search‐
11523 Strategy interface.
11524 1.0.0 - 2015-03-27
11526 Codename: Blast-off!
11527 There are no code changes in this release. This is precisely the 0.9.2
11529 release with some updated documentation.
11530 0.9.2 - 2015-03-26
11532 Codename: T-1 days.
11533 · floats_in_range would not actually have produced floats_in_range
11535 unless that range happened to be (0, 1). Fix this.
11536 0.9.1 - 2015-03-25
11538 Codename: T-2 days.
11539 · Fix a bug where if you defined a strategy using map on a lambda then
11541 the results would not be saved in the database.
11542 · Significant performance improvements when simplifying examples using
11544 lists, strings or bounded integer ranges.
11545 0.9.0 - 2015-03-23
11547 Codename: The final countdown
11548 This release could also be called 1.0-RC1.
11550 It contains a teeny tiny bugfix, but the real point of this release is
11552 to declare feature freeze. There will be zero functionality changes
11553 between 0.9.0 and 1.0 unless something goes really really wrong. No new
11554 features will be added, no breaking API changes will occur, etc. This
11555 is the final shakedown before I declare Hypothesis stable and ready to
11556 use and throw a party to celebrate.
11557 Bug bounty for any bugs found between now and 1.0: I will buy you a
11559 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
11560 we ever find ourselves in the same city at the same time.
11561 The one tiny bugfix:
11563 · Under pypy, databases would fail to close correctly when garbage col‐
11565 lected, leading to a memory leak and a confusing error message if you
11566 were repeatedly creating databases and not closing them. It is very
11567 unlikely you were doing this and the chances of you ever having
11568 noticed this bug are very low.
11569 0.7.2 - 2015-03-22
11571 Codename: Hygienic macros or bust
11572 · You can now name an argument to @given 'f' and it won't break (issue
11574 #38)
11575 · strategy_test_suite is now named strategy_test_suite as the documen‐
11577 tation claims and not in fact strategy_test_suitee
11578 · Settings objects can now be used as a context manager to temporarily
11580 override the default values inside their context.
11581 0.7.1 - 2015-03-21
11583 Codename: Point releases go faster
11584 · Better string generation by parametrizing by a limited alphabet
11586 · Faster string simplification - previously if simplifying a string
11588 with high range unicode characters it would try every unicode charac‐
11589 ter smaller than that. This was pretty pointless. Now it stops after
11590 it's a short range (it can still reach smaller ones through recursive
11591 calls because of other simplifying operations).
11592 · Faster list simplification by first trying a binary chop down the
11594 middle
11595 · Simultaneous simplification of identical elements in a list. So if a
11597 bug only triggers when you have duplicates but you drew e.g. [-17,
11598 -17], this will now simplify to [0, 0].
11599 0.7.0, - 2015-03-20
11601 Codename: Starting to look suspiciously real
11602 This is probably the last minor release prior to 1.0. It consists of
11604 stability improvements, a few usability things designed to make Hypoth‐
11605 esis easier to try out, and filing off some final rough edges from the
11606 API.
11607 · Significant speed and memory usage improvements
11609 · Add an example() method to strategy objects to give an example of the
11611 sort of data that the strategy generates.
11612 · Remove .descriptor attribute of strategies
11614 · Rename descriptor_test_suite to strategy_test_suite
11616 · Rename the few remaining uses of descriptor to specifier (descriptor
11618 already has a defined meaning in Python)
11619 0.6.0 - 2015-03-13
11621 Codename: I'm sorry, were you using that API?
11622 This is primarily a "simplify all the weird bits of the API" release.
11624 As a result there are a lot of breaking changes. If you just use @given
11625 with core types then you're probably fine.
11626 In particular:
11628 · Stateful testing has been removed from the API
11630 · The way the database is used has been rendered less useful (sorry).
11632 The feature for reassembling values saved from other tests doesn't
11633 currently work. This will probably be brought back in post 1.0.
11634 · SpecificationMapper is no longer a thing. Instead there is an
11636 ExtMethod called strategy which you extend to specify how to convert
11637 other types to strategies.
11638 · Settings are now extensible so you can add your own for configuring a
11640 strategy
11641 · MappedSearchStrategy no longer needs an unpack method
11643 · Basically all the SearchStrategy internals have changed massively. If
11645 you implemented SearchStrategy directly rather than using Mapped‐
11646 SearchStrategy talk to me about fixing it.
11647 · Change to the way extra packages work. You now specify the package.
11649 This must have a load() method. Additionally any modules in the pack‐
11650 age will be loaded in under hypothesis.extra
11651 Bug fixes:
11653 · Fix for a bug where calling falsify on a lambda with a non-ascii
11655 character in its body would error.
11656 Hypothesis Extra:
11658 hypothesis-fakefactory: An extension for using faker data in hypothe‐
11660 sis. Depends
11661 on fake-factory.
11662 0.5.0 - 2015-02-10
11664 Codename: Read all about it.
11665 Core hypothesis:
11667 · Add support back in for pypy and python 3.2
11669 · @given functions can now be invoked with some arguments explicitly
11671 provided. If all arguments that hypothesis would have provided are
11672 passed in then no falsification is run.
11673 · Related to the above, this means that you can now use pytest fixtures
11675 and mark.parametrize with Hypothesis without either interfering with
11676 the other.
11677 · Breaking change: @given no longer works for functions with varargs
11679 (varkwargs are fine). This might be added back in at a later date.
11680 · Windows is now fully supported. A limited version (just the tests
11682 with none of the extras) of the test suite is run on windows with
11683 each commit so it is now a first class citizen of the Hypothesis
11684 world.
11685 · Fix a bug for fuzzy equality of equal complex numbers with different
11687 reprs (this can happen when one coordinate is zero). This shouldn't
11688 affect users - that feature isn't used anywhere public facing.
11689 · Fix generation of floats on windows and 32-bit builds of python. I
11691 was using some struct.pack logic that only worked on certain word
11692 sizes.
11693 · When a test times out and hasn't produced enough examples this now
11695 raises a Timeout subclass of Unfalsifiable.
11696 · Small search spaces are better supported. Previously something like a
11698 @given(bool, bool) would have failed because it couldn't find enough
11699 examples. Hypothesis is now aware of the fact that these are small
11700 search spaces and will not error in this case.
11701 · Improvements to parameter search in the case of hard to satisfy
11703 assume. Hypothesis will now spend less time exploring parameters that
11704 are unlikely to provide anything useful.
11705 · Increase chance of generating "nasty" floats
11707 · Fix a bug that would have caused unicode warnings if you had a sam‐
11709 pled_from that was mixing unicode and byte strings.
11710 · Added a standard test suite that you can use to validate a custom
11712 strategy you've defined is working correctly.
11713 Hypothesis extra:
11715 First off, introducing Hypothesis extra packages!
11717 These are packages that are separated out from core Hypothesis because
11719 they have one or more dependencies. Every hypothesis-extra package is
11720 pinned to a specific point release of Hypothesis and will have some
11721 version requirements on its dependency. They use entry_points so you
11722 will usually not need to explicitly import them, just have them
11723 installed on the path.
11724 This release introduces two of them:
11726 hypothesis-datetime:
11728 Does what it says on the tin: Generates datetimes for Hypothesis. Just
11730 install the package and datetime support will start working.
11731 Depends on pytz for timezone support
11733 hypothesis-pytest:
11735 A very rudimentary pytest plugin. All it does right now is hook the
11737 display of falsifying examples into pytest reporting.
11738 Depends on pytest.
11740 0.4.3 - 2015-02-05
11742 Codename: TIL narrow Python builds are a thing
11743 This just fixes the one bug.
11745 · Apparently there is such a thing as a "narrow python build" and OS X
11747 ships with these by default for python 2.7. These are builds where
11748 you only have two bytes worth of unicode. As a result, generating
11749 unicode was completely broken on OS X. Fix this by only generating
11750 unicode codepoints in the range supported by the system.
11751 0.4.2 - 2015-02-04
11753 Codename: O(dear)
11754 This is purely a bugfix release:
11756 · Provide sensible external hashing for all core types. This will sig‐
11758 nificantly improve performance of tracking seen examples which hap‐
11759 pens in literally every falsification run. For Hypothesis fixing this
11760 cut 40% off the runtime of the test suite. The behaviour is quadratic
11761 in the number of examples so if you're running the default configura‐
11762 tion this will be less extreme (Hypothesis's test suite runs at a
11763 higher number of examples than default), but you should still see a
11764 significant improvement.
11765 · Fix a bug in formatting of complex numbers where the string could get
11767 incorrectly truncated.
11768 0.4.1 - 2015-02-03
11770 Codename: Cruel and unusual edge cases
11771 This release is mostly about better test case generation.
11773 Enhancements:
11775 · Has a cool release name
11777 · text_type (str in python 3, unicode in python 2) example generation
11779 now actually produces interesting unicode instead of boring ascii
11780 strings.
11781 · floating point numbers are generated over a much wider range, with
11783 particular attention paid to generating nasty numbers - nan, infin‐
11784 ity, large and small values, etc.
11785 · examples can be generated using pieces of examples previously saved
11787 in the database. This allows interesting behaviour that has previ‐
11788 ously been discovered to be propagated to other examples.
11789 · improved parameter exploration algorithm which should allow it to
11791 more reliably hit interesting edge cases.
11792 · Timeout can now be disabled entirely by setting it to any value <= 0.
11794 Bug fixes:
11796 · The descriptor on a OneOfStrategy could be wrong if you had descrip‐
11798 tors which were equal but should not be coalesced. e.g. a strategy
11799 for one_of((frozenset({int}), {int})) would have reported its
11800 descriptor as {int}. This is unlikely to have caused you any problems
11801 · If you had strategies that could produce NaN (which float previously
11803 couldn't but e.g. a Just(float('nan')) could) then this would have
11804 sent hypothesis into an infinite loop that would have only been ter‐
11805 minated when it hit the timeout.
11806 · Given elements that can take a long time to minimize, minimization of
11808 floats or tuples could be quadratic or worse in the that value. You
11809 should now see much better performance for simplification, albeit at
11810 some cost in quality.
11811 Other:
11813 · A lot of internals have been been rewritten. This shouldn't affect
11815 you at all, but it opens the way for certain of hypothesis's oddities
11816 to be a lot more extensible by users. Whether this is a good thing
11817 may be up for debate...
11818 0.4.0 - 2015-01-21
11820 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
11821 stores data in a local SQLite database and will reuse it for all tests
11822 of the same type.
11823 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
11825 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
11826 ther notice. The previous license remains applicable to all code prior
11827 to 0.4.0.
11828 Enhancements:
11830 · Printing of failing examples. I was finding that the pytest runner
11832 was not doing a good job of displaying these, and that Hypothesis
11833 itself could do much better.
11834 · Drop dependency on six for cross-version compatibility. It was easy
11836 enough to write the shim for the small set of features that we care
11837 about and this lets us avoid a moderately complex dependency.
11838 · Some improvements to statistical distribution of selecting from small
11840 (<= 3 elements)
11841 · Improvements to parameter selection for finding examples.
11843 Bugs fixed:
11845 · could_have_produced for lists, dicts and other collections would not
11847 have examined the elements and thus when using a union of different
11848 types of list this could result in Hypothesis getting confused and
11849 passing a value to the wrong strategy. This could potentially result
11850 in exceptions being thrown from within simplification.
11851 · sampled_from would not work correctly on a single element list.
11853 · Hypothesis could get very confused by values which are equal despite
11855 having different types being used in descriptors. Hypothesis now has
11856 its own more specific version of equality it uses for descriptors and
11857 tracking. It is always more fine grained than Python equality: Things
11858 considered != are not considered equal by hypothesis, but some things
11859 that are considered == are distinguished. If your test suite uses
11860 both frozenset and set tests this bug is probably affecting you.
11861 0.3.2 - 2015-01-16
11863 · Fix a bug where if you specified floats_in_range with integer argu‐
11864 ments Hypothesis would error in example simplification.
11865 · Improve the statistical distribution of the floats you get for the
11867 floats_in_range strategy. I'm not sure whether this will affect users
11868 in practice but it took my tests for various conditions from flaky to
11869 rock solid so it at the very least improves discovery of the artifi‐
11870 cial cases I'm looking for.
11871 · Improved repr() for strategies and RandomWithSeed instances.
11873 · Add detection for flaky test cases where hypothesis managed to find
11875 an example which breaks it but on the final invocation of the test it
11876 does not raise an error. This will typically happen with too much
11877 recursion errors but could conceivably happen in other circumstances
11878 too.
11879 · Provide a "derandomized" mode. This allows you to run hypothesis with
11881 zero real randomization, making your build nice and deterministic.
11882 The tests run with a seed calculated from the function they're test‐
11883 ing so you should still get a good distribution of test cases.
11884 · Add a mechanism for more conveniently defining tests which just sam‐
11886 ple from some collection.
11887 · Fix for a really subtle bug deep in the internals of the strategy ta‐
11889 ble. In some circumstances if you were to define instance strategies
11890 for both a parent class and one or more of its subclasses you would
11891 under some circumstances get the strategy for the wrong superclass of
11892 an instance. It is very unlikely anyone has ever encountered this in
11893 the wild, but it is conceivably possible given that a mix of namedtu‐
11894 ple and tuple are used fairly extensively inside hypothesis which do
11895 exhibit this pattern of strategy.
11896 0.3.1 - 2015-01-13
11898 · Support for generation of frozenset and Random values
11899 · Correct handling of the case where a called function mutates it argu‐
11901 ment. This involved introducing a notion of a strategies knowing how
11902 to copy their argument. The default method should be entirely accept‐
11903 able and the worst case is that it will continue to have the old be‐
11904 haviour if you don't mark your strategy as mutable, so this shouldn't
11905 break anything.
11906 · Fix for a bug where some strategies did not correctly implement
11908 could_have_produced. It is very unlikely that any of these would have
11909 been seen in the wild, and the consequences if they had been would
11910 have been minor.
11911 · Re-export the @given decorator from the main hypothesis namespace.
11913 It's still available at the old location too.
11914 · Minor performance optimisation for simplifying long lists.
11916 0.3.0 - 2015-01-12
11918 · Complete redesign of the data generation system. Extreme breaking
11919 change for anyone who was previously writing their own SearchStrategy
11920 implementations. These will not work any more and you'll need to mod‐
11921 ify them.
11922 · New settings system allowing more global and modular control of Veri‐
11924 fier behaviour.
11925 · Decouple SearchStrategy from the StrategyTable. This leads to much
11927 more composable code which is a lot easier to understand.
11928 · A significant amount of internal API renaming and moving. This may
11930 also break your code.
11931 · Expanded available descriptors, allowing for generating integers or
11933 floats in a specific range.
11934 · Significantly more robust. A very large number of small bug fixes,
11936 none of which anyone is likely to have ever noticed.
11937 · Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
11939 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
11940 these to remain officially supported for a very long time. I would
11941 not be surprised to add pypy support back in later but I'm not going
11942 to do so until I know someone cares about it. In the meantime it will
11943 probably still work.
11944 0.2.2 - 2015-01-08
11946 · Fix an embarrassing complete failure of the installer caused by my
11947 being bad at version control
11948 0.2.1 - 2015-01-07
11950 · Fix a bug in the new stateful testing feature where you could make
11951 __init__ a @requires method. Simplification would not always work if
11952 the prune method was able to successfully shrink the test.
11953 0.2.0 - 2015-01-07
11955 · It's aliiive.
11956 · Improve python 3 support using six.
11958 · Distinguish between byte and unicode types.
11960 · Fix issues where FloatStrategy could raise.
11962 · Allow stateful testing to request constructor args.
11964 · Fix for issue where test annotations would timeout based on when the
11966 module was loaded instead of when the test started
11967 0.1.4 - 2013-12-14
11969 · Make verification runs time bounded with a configurable timeout
11970 0.1.3 - 2013-05-03
11972 · Bugfix: Stateful testing behaved incorrectly with subclassing.
11973 · Complex number support
11975 · support for recursive strategies
11977 · different error for hypotheses with unsatisfiable assumptions
11979 0.1.2 - 2013-03-24
11981 · Bugfix: Stateful testing was not minimizing correctly and could throw
11982 exceptions.
11983 · Better support for recursive strategies.
11985 · Support for named tuples.
11987 · Much faster integer generation.
11989 0.1.1 - 2013-03-24
11991 · Python 3.x support via 2to3.
11992 · Use new style classes (oops).
11994 0.1.0 - 2013-03-23
11996 · Introduce stateful testing.
11997 · Massive rewrite of internals to add flags and strategies.
11999 0.0.5 - 2013-03-13
12001 · No changes except trying to fix packaging
12002 0.0.4 - 2013-03-13
12004 · No changes except that I checked in a failing test case for 0.0.3 so
12005 had to replace the release. Doh
12006 0.0.3 - 2013-03-13
12008 · Improved a few internals.
12009 · Opened up creating generators from instances as a general API.
12011 · Test integration.
12013 0.0.2 - 2013-03-12
12015 · Starting to tighten up on the internals.
12016 · Change API to allow more flexibility in configuration.
12018 · More testing.
12020 0.0.1 - 2013-03-10
12022 · Initial release.
12023 · Basic working prototype. Demonstrates idea, probably shouldn't be
12025 used.
12026
12028 Hypothesis development is managed by me, David R. MacIver. I am the
12029 primary author of Hypothesis.
12030 However, I no longer do unpaid feature development on Hypothesis. My
12032 roles as leader of the project are:
12033 1. Helping other people do feature development on Hypothesis
12035 2. Fixing bugs and other code health issues
12037 3. Improving documentation
12039 4. General release management work
12041 5. Planning the general roadmap of the project
12043 6. Doing sponsored development on tasks that are too large or in depth
12045 for other people to take on
12046 So all new features must either be sponsored or implemented by someone
12048 else. That being said, the maintenance team takes an active role in
12049 shepherding pull requests and helping people write a new feature (see
12050 CONTRIBUTING.rst for details and pull request #154 for an example of
12051 how the process goes). This isn't "patches welcome", it's "we will help
12052 you write a patch".
12053 Release policy
12055 Hypothesis releases follow semantic versioning.
12056 We maintain backwards-compatibility wherever possible, and use depreca‐
12058 tion warnings to mark features that have been superseded by a newer
12059 alternative. If you want to detect this, you can upgrade warnings to
12060 errors in the usual ways.
12061 We use continuous deployment to ensure that you can always use our new‐
12063 est and shiniest features - every change to the source tree is automat‐
12064 ically built and published on PyPI as soon as it's merged onto master,
12065 after code review and passing our extensive test suite.
12066 Project roadmap
12068 Hypothesis does not have a long-term release plan. We respond to bug
12069 reports as they are made; new features are released as and when someone
12070 volunteers to write and maintain them.
12071
12073 For questions you are happy to ask in public, the Hypothesis community
12074 is a friendly place where I or others will be more than happy to help
12075 you out. You're also welcome to ask questions on Stack Overflow. If you
12076 do, please tag them with 'python-hypothesis' so someone sees them.
12077 For bugs and enhancements, please file an issue on the GitHub issue
12079 tracker. Note that as per the development policy, enhancements will
12080 probably not get implemented unless you're willing to pay for develop‐
12081 ment or implement them yourself (with assistance from me). Bugs will
12082 tend to get fixed reasonably promptly, though it is of course on a best
12083 effort basis.
12084 To see the versions of Python, optional dependencies, test runners, and
12086 operating systems Hypothesis supports (meaning incompatibility is
12087 treated as a bug), see supported.
12088 If you need to ask questions privately or want more of a guarantee of
12090 bugs being fixed promptly, please contact me on
12091 hypothesis-support@drmaciver.com to talk about availability of support
12092 contracts.
12093
12095 Downstream packagers often want to package Hypothesis. Here are some
12096 guidelines.
12097 The primary guideline is this: If you are not prepared to keep up with
12099 the Hypothesis release schedule, don't. You will annoy me and are doing
12100 your users a disservice.
12101 Hypothesis has a very frequent release schedule. It's rare that it goes
12103 a week without a release, and there are often multiple releases in a
12104 given week.
12105 If you are prepared to keep up with this schedule, you might find the
12107 rest of this document useful.
12108 Release tarballs
12110 These are available from the GitHub releases page. The tarballs on PyPI
12111 are intended for installation from a Python tool such as pip and should
12112 not be considered complete releases. Requests to include additional
12113 files in them will not be granted. Their absence is not a bug.
12114 Dependencies
12116 Python versions
12117 Hypothesis is designed to work with a range of Python versions - we
12118 support all versions of CPython with upstream support. We also support
12119 the latest versions of PyPy for Python 3.
12120 Other Python libraries
12122 Hypothesis has mandatory dependencies on the following libraries:
12123 · attrs
12125 · sortedcontainers
12127 Hypothesis has optional dependencies on the following libraries:
12129 extras_require = {
12131 "cli": ["click>=7.0", "black>=19.10b0"],
12132 "ghostwriter": ["black>=19.10b0"],
12133 "pytz": ["pytz>=2014.1"],
12134 "dateutil": ["python-dateutil>=1.4"],
12135 "lark": ["lark-parser>=0.6.5"],
12136 "numpy": ["numpy>=1.9.0"],
12137 "pandas": ["pandas>=0.25"],
12138 "pytest": ["pytest>=4.3"],
12139 "dpcontracts": ["dpcontracts>=0.4"],
12140 "redis": ["redis>=3.0.0"],
12141 # zoneinfo is an odd one: every dependency is conditional, because they're
12142 # only necessary on old versions of Python or Windows systems.
12143 "zoneinfo": [
12144 "tzdata>=2020.4 ; sys_platform == 'win32'",
12145 "backports.zoneinfo>=0.2.1 ; python_version<'3.9'",
12146 "importlib_resources>=3.3.0 ; python_version<'3.7'",
12147 ],
12148 # We only support Django versions with upstream support - see
12149 # https://www.djangoproject.com/download/#supported-versions
12150 "django": ["pytz>=2014.1", "django>=2.2"],
12151 }
12152 The way this works when installing Hypothesis normally is that these
12155 features become available if the relevant library is installed.
12156 Specifically for pytest, our plugin supports versions of pytest which
12158 have been out of upstream support for some time. Hypothesis tests can
12159 still be executed by even older versions of pytest - you just won't
12160 have the plugin to provide automatic marks, helpful usage warnings, and
12161 per-test statistics.
12162 Testing Hypothesis
12164 If you want to test Hypothesis as part of your packaging you will prob‐
12165 ably not want to use the mechanisms Hypothesis itself uses for running
12166 its tests, because it has a lot of logic for installing and testing
12167 against different versions of Python.
12168 The tests must be run with fairly recent tooling; check the
12170 requirements/ directory for details.
12171 The organisation of the tests is described in the
12173 hypothesis-python/tests/README.rst.
12174 Examples
12176 · arch linux
12177 · fedora
12179 · gentoo
12181
12183 One of the things that is often concerning for people using randomized
12184 testing is the question of how to reproduce failing test cases.
12185 NOTE:
12187 It is better to think about the data Hypothesis generates as being
12188 arbitrary, rather than random. We deliberately generate any valid
12189 data that seems likely to cause errors, so you shouldn't rely on any
12190 expected distribution of or relationships between generated data.
12191 You can read about "swarm testing" and "coverage guided fuzzing" if
12192 you're interested, because you don't need to know for Hypothesis!
12193 Fortunately Hypothesis has a number of features to support reproducing
12195 test failures. The one you will use most commonly when developing
12196 locally is the example database, which means that you shouldn't have to
12197 think about the problem at all for local use - test failures will just
12198 automatically reproduce without you having to do anything.
12199 The example database is perfectly suitable for sharing between
12201 machines, but there currently aren't very good work flows for that, so
12202 Hypothesis provides a number of ways to make examples reproducible by
12203 adding them to the source code of your tests. This is particularly use‐
12204 ful when e.g. you are trying to run an example that has failed on your
12205 CI, or otherwise share them between machines.
12206 Providing explicit examples
12208 The simplest way to reproduce a failed test is to ask Hypothesis to run
12209 the failing example it printed. For example, if Falsifying example:
12210 test(n=1) was printed you can decorate test with @example(n=1).
12211 @example can also be used to ensure a specific example is always exe‐
12213 cuted as a regression test or to cover some edge case - basically com‐
12214 bining a Hypothesis test and a traditional parametrized test.
12215 hypothesis.example(*args, **kwargs)
12217 A decorator which ensures a specific example is always tested.
12218 Hypothesis will run all examples you've asked for first. If any of them
12220 fail it will not go on to look for more examples.
12221 It doesn't matter whether you put the example decorator before or after
12223 given. Any permutation of the decorators in the above will do the same
12224 thing.
12225 Note that examples can be positional or keyword based. If they're posi‐
12227 tional then they will be filled in from the right when calling, so
12228 either of the following styles will work as expected:
12229 @given(text())
12231 @example("Hello world")
12232 @example(x="Some very long string")
12233 def test_some_code(x):
12234 assert True
12235 from unittest import TestCase
12238 class TestThings(TestCase):
12241 @given(text())
12242 @example("Hello world")
12243 @example(x="Some very long string")
12244 def test_some_code(self, x):
12245 assert True
12246 As with @given, it is not permitted for a single example to be a mix of
12248 positional and keyword arguments. Either are fine, and you can use one
12249 in one example and the other in another example if for some reason you
12250 really want to, but a single example must be consistent.
12251 Reproducing a test run with @seed
12253 hypothesis.seed(seed)
12254 seed: Start the test execution from a specific seed.
12255 May be any hashable object. No exact meaning for seed is pro‐
12257 vided other than that for a fixed seed value Hypothesis will try
12258 the same actions (insofar as it can given external sources of
12259 non- determinism. e.g. timing and hash randomization).
12260 Overrides the derandomize setting, which is designed to enable
12262 deterministic builds rather than reproducing observed failures.
12263 When a test fails unexpectedly, usually due to a health check failure,
12265 Hypothesis will print out a seed that led to that failure, if the test
12266 is not already running with a fixed seed. You can then recreate that
12267 failure using either the @seed decorator or (if you are running pytest)
12268 with --hypothesis-seed. For example, the following test function and
12269 RuleBasedStateMachine will each check the same examples each time they
12270 are executed, thanks to @seed():
12271 @seed(1234)
12273 @given(x=...)
12274 def test(x):
12275 ...
12276 @seed(6789)
12279 class MyModel(RuleBasedStateMachine):
12280 ...
12281 The seed will not be printed if you could simply use @example instead.
12283 Reproducing an example with @reproduce_failure
12285 Hypothesis has an opaque binary representation that it uses for all
12286 examples it generates. This representation is not intended to be stable
12287 across versions or with respect to changes in the test, but can be used
12288 to to reproduce failures with the @reproduce_failure decorator.
12289 hypothesis.reproduce_failure(version, blob)
12291 Run the example that corresponds to this data blob in order to
12292 reproduce a failure.
12293 A test with this decorator always runs only one example and
12295 always fails. If the provided example does not cause a failure,
12296 or is in some way invalid for this test, then this will fail
12297 with a DidNotReproduce error.
12298 This decorator is not intended to be a permanent addition to
12300 your test suite. It's simply some code you can add to ease
12301 reproduction of a problem in the event that you don't have
12302 access to the test database. Because of this, no compatibility
12303 guarantees are made between different versions of Hypothesis -
12304 its API may change arbitrarily from version to version.
12305 The intent is that you should never write this decorator by hand, but
12307 it is instead provided by Hypothesis. When a test fails with a falsi‐
12308 fying example, Hypothesis may print out a suggestion to use @repro‐
12309 duce_failure on the test to recreate the problem as follows:
12310 >>> from hypothesis import settings, given, PrintSettings
12312 >>> import hypothesis.strategies as st
12313 >>> @given(st.floats())
12314 ... @settings(print_blob=True)
12315 ... def test(f):
12316 ... assert f == f
12317 ...
12318 >>> try:
12319 ... test()
12320 ... except AssertionError:
12321 ... pass
12322 Falsifying example: test(f=nan)
12323 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
12325 Adding the suggested decorator to the test should reproduce the failure
12327 (as long as everything else is the same - changing the versions of
12328 Python or anything else involved, might of course affect the behaviour
12329 of the test! Note that changing the version of Hypothesis will result
12330 in a different error - each @reproduce_failure invocation is specific
12331 to a Hypothesis version).
12332 By default these messages are not printed. If you want to see these
12334 you must set the print_blob setting to True.
12335
12337 David R. MacIver
12338
12340 2013-2021, David R. MacIver
123415.43.9 Feb 01, 2021 HYPOTHESIS(1)
