1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
6 hypothesis - Hypothesis Documentation
7 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15 Think of a normal unit test as being something like the following:
17 1. Set up some data.
19 2. Perform some operations on the data.
21 3. Assert something about the result.
23 Hypothesis lets you write tests which instead look like this:
25 1. For all data matching some specification.
27 2. Perform some operations on the data.
29 3. Assert something about the result.
31 This is often called property-based testing, and was popularised by the
33 Haskell library Quickcheck.
34 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the fu‐
41 ture.
42 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47 • Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51 • If you delete an object, it is no longer visible.
53 • If you serialize and then deserialize a value, then you get the same
55 value back.
56 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77 def encode(input_string):
79 count = 1
80 prev = ""
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94 def decode(lst):
97 q = ""
98 for character, count in lst:
99 q += character * count
100 return q
101 We want to write a test for this that will check some invariant of
103 these functions.
104 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108 Let's see how you'd do that with Hypothesis:
110 from hypothesis import given
112 from hypothesis.strategies import text
113 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127 Anyway, this test immediately finds a bug in the code:
129 Falsifying example: test_decode_inverts_encode(s='')
131 UnboundLocalError: local variable 'character' referenced before assignment
133 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136 If we fix that by just adding the following code to the beginning of
138 the function then Hypothesis tells us the code is correct (by doing
139 nothing as you'd expect a passing test to).
140 if not input_string:
142 return []
143 If we wanted to make sure this example was always checked we could add
145 it in explicitly by using the @example decorator:
146 from hypothesis import given, example
148 from hypothesis.strategies import text
149 @given(text())
152 @example("")
153 def test_decode_inverts_encode(s):
154 assert decode(encode(s)) == s
155 This can be useful to show other developers (or your future self) what
157 kinds of data are valid inputs, or to ensure that particular edge cases
158 such as "" are tested every time. It's also great for regression tests
159 because although Hypothesis will remember failing examples, we don't
160 recommend distributing that database.
161 It's also worth noting that both example and given support keyword ar‐
163 guments as well as positional. The following would have worked just as
164 well:
165 @given(s=text())
167 @example(s="")
168 def test_decode_inverts_encode(s):
169 assert decode(encode(s)) == s
170 Suppose we had a more interesting bug and forgot to reset the count
172 each time. Say we missed a line in our encode method:
173 def encode(input_string):
175 count = 1
176 prev = ""
177 lst = []
178 for character in input_string:
179 if character != prev:
180 if prev:
181 entry = (prev, count)
182 lst.append(entry)
183 # count = 1 # Missing reset operation
184 prev = character
185 else:
186 count += 1
187 entry = (character, count)
188 lst.append(entry)
189 return lst
190 Hypothesis quickly informs us of the following example:
192 Falsifying example: test_decode_inverts_encode(s='001')
194 Note that the example provided is really quite simple. Hypothesis
196 doesn't just find any counter-example to your tests, it knows how to
197 simplify the examples it finds to produce small easy to understand
198 ones. In this case, two identical values are enough to set the count to
199 a number different from one, followed by another distinct value which
200 should have reset the count but in this case didn't.
201 The examples Hypothesis provides are valid Python code you can run. Any
203 arguments that you explicitly provide when calling the function are not
204 generated by Hypothesis, and if you explicitly provide all the argu‐
205 ments Hypothesis will just call the underlying function once rather
206 than running it multiple times.
207 Installing
209 Hypothesis is available on PyPI as "hypothesis". You can install it
210 with:
211 pip install hypothesis
213 You can install the dependencies for optional extensions with e.g. pip
215 install hypothesis[pandas,django].
216 If you want to install directly from the source code (e.g. because you
218 want to make changes and install the changed version), check out the
219 instructions in CONTRIBUTING.rst.
220 Running tests
222 In our example above we just let pytest discover and run our tests, but
223 we could also have run it explicitly ourselves:
224 if __name__ == "__main__":
226 test_decode_inverts_encode()
227 We could also have done this as a python:unittest.TestCase:
229 import unittest
231 class TestEncoding(unittest.TestCase):
234 @given(text())
235 def test_decode_inverts_encode(self, s):
236 self.assertEqual(decode(encode(s)), s)
237 if __name__ == "__main__":
240 unittest.main()
241 A detail: This works because Hypothesis ignores any arguments it hasn't
243 been told to provide (positional arguments start from the right), so
244 the self argument to the test is simply ignored and works as normal.
245 This also means that Hypothesis will play nicely with other ways of pa‐
246 rameterizing tests. e.g it works fine if you use pytest fixtures for
247 some arguments and Hypothesis for others.
248 Writing tests
250 A test in Hypothesis consists of two parts: A function that looks like
251 a normal test in your test framework of choice but with some additional
252 arguments, and a @given decorator that specifies how to provide those
253 arguments.
254 Here are some other examples of how you could use that:
256 from hypothesis import given
258 import hypothesis.strategies as st
259 @given(st.integers(), st.integers())
262 def test_ints_are_commutative(x, y):
263 assert x + y == y + x
264 @given(x=st.integers(), y=st.integers())
267 def test_ints_cancel(x, y):
268 assert (x + y) - y == x
269 @given(st.lists(st.integers()))
272 def test_reversing_twice_gives_same_list(xs):
273 # This will generate lists of arbitrary length (usually between 0 and
274 # 100 elements) whose elements are integers.
275 ys = list(xs)
276 ys.reverse()
277 ys.reverse()
278 assert xs == ys
279 @given(st.tuples(st.booleans(), st.text()))
282 def test_look_tuples_work_too(t):
283 # A tuple is generated as the one you provided, with the corresponding
284 # types in those positions.
285 assert len(t) == 2
286 assert isinstance(t[0], bool)
287 assert isinstance(t[1], str)
288 Note that as we saw in the above example you can pass arguments to
290 @given either as positional or as keywords.
291 Where to start
293 You should now know enough of the basics to write some tests for your
294 code using Hypothesis. The best way to learn is by doing, so go have a
295 try.
296 If you're stuck for ideas for how to use this sort of test for your
298 code, here are some good starting points:
299 1. Try just calling functions with appropriate arbitrary data and see
301 if they crash. You may be surprised how often this works. e.g. note
302 that the first bug we found in the encoding example didn't even get
303 as far as our assertion: It crashed because it couldn't handle the
304 data we gave it, not because it did the wrong thing.
305 2. Look for duplication in your tests. Are there any cases where you're
307 testing the same thing with multiple different examples? Can you
308 generalise that to a single test using Hypothesis?
309 3. This piece is designed for an F# implementation, but is still very
311 good advice which you may find helps give you good ideas for using
312 Hypothesis.
313 If you have any trouble getting started, don't feel shy about asking
315 for help.
316
318 This is an account of slightly less common Hypothesis features that you
319 don't need to get started but will nevertheless make your life easier.
320 Additional test output
322 Normally the output of a failing test will look something like:
323 Falsifying example: test_a_thing(x=1, y="foo")
325 With the repr of each keyword argument being printed.
327 Sometimes this isn't enough, either because you have a value with a
329 __repr__() method that isn't very descriptive or because you need to
330 see the output of some intermediate steps of your test. That's where
331 the note function comes in:
332 hypothesis.note(value)
334 Report this value in the final execution.
335 >>> from hypothesis import given, note, strategies as st
337 >>> @given(st.lists(st.integers()), st.randoms())
338 ... def test_shuffle_is_noop(ls, r):
339 ... ls2 = list(ls)
340 ... r.shuffle(ls2)
341 ... note("Shuffle: %r" % (ls2))
342 ... assert ls == ls2
343 ...
344 >>> try:
345 ... test_shuffle_is_noop()
346 ... except AssertionError:
347 ... print('ls != ls2')
348 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
349 Shuffle: [1, 0]
350 ls != ls2
351 The note is printed in the final run of the test in order to include
353 any additional information you might need in your test.
354 Test statistics
356 If you are using pytest you can see a number of statistics about the
357 executed tests by passing the command line argument --hypothe‐
358 sis-show-statistics. This will include some general statistics about
359 the test:
360 For example if you ran the following with --hypothesis-show-statistics:
362 from hypothesis import given, strategies as st
364 @given(st.integers())
367 def test_integers(i):
368 pass
369 You would see:
371 - during generate phase (0.06 seconds):
373 - Typical runtimes: < 1ms, ~ 47% in data generation
374 - 100 passing examples, 0 failing examples, 0 invalid examples
375 - Stopped because settings.max_examples=100
376 The final "Stopped because" line is particularly important to note: It
378 tells you the setting value that determined when the test should stop
379 trying new examples. This can be useful for understanding the behaviour
380 of your tests. Ideally you'd always want this to be max_examples.
381 In some cases (such as filtered and recursive strategies) you will see
383 events mentioned which describe some aspect of the data generation:
384 from hypothesis import given, strategies as st
386 @given(st.integers().filter(lambda x: x % 2 == 0))
389 def test_even_integers(i):
390 pass
391 You would see something like:
393 test_even_integers:
395 - during generate phase (0.08 seconds):
397 - Typical runtimes: < 1ms, ~ 57% in data generation
398 - 100 passing examples, 0 failing examples, 12 invalid examples
399 - Events:
400 * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
401 * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
402 - Stopped because settings.max_examples=100
403 You can also mark custom events in a test using the event function:
405 hypothesis.event(value)
407 Record an event that occurred this test. Statistics on number of
408 test runs with each event will be reported at the end if you run
409 Hypothesis in statistics reporting mode.
410 Events should be strings or convertible to them.
412 from hypothesis import given, event, strategies as st
414 @given(st.integers().filter(lambda x: x % 2 == 0))
417 def test_even_integers(i):
418 event("i mod 3 = %d" % (i % 3,))
419 You will then see output like:
421 test_even_integers:
423 - during generate phase (0.09 seconds):
425 - Typical runtimes: < 1ms, ~ 59% in data generation
426 - 100 passing examples, 0 failing examples, 32 invalid examples
427 - Events:
428 * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
429 * 31.06%, i mod 3 = 2
430 * 28.79%, i mod 3 = 0
431 * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
432 * 15.91%, i mod 3 = 1
433 - Stopped because settings.max_examples=100
434 Arguments to event can be any hashable type, but two events will be
436 considered the same if they are the same when converted to a string
437 with python:str.
438 Making assumptions
440 Sometimes Hypothesis doesn't give you exactly the right sort of data
441 you want - it's mostly of the right shape, but some examples won't work
442 and you don't want to care about them. You can just ignore these by
443 aborting the test early, but this runs the risk of accidentally testing
444 a lot less than you think you are. Also it would be nice to spend less
445 time on bad examples - if you're running 100 examples per test (the de‐
446 fault) and it turns out 70 of those examples don't match your needs,
447 that's a lot of wasted time.
448 hypothesis.assume(condition)
450 Calling assume is like an assert that marks the example as bad,
451 rather than failing the test.
452 This allows you to specify properties that you assume will be
454 true, and let Hypothesis try to avoid similar examples in fu‐
455 ture.
456 For example suppose you had the following test:
458 @given(floats())
460 def test_negation_is_self_inverse(x):
461 assert x == -(-x)
462 Running this gives us:
464 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
466 AssertionError
467 This is annoying. We know about NaN and don't really care about it, but
469 as soon as Hypothesis finds a NaN example it will get distracted by
470 that and tell us about it. Also the test will fail and we want it to
471 pass.
472 So let's block off this particular example:
474 from math import isnan
476 @given(floats())
479 def test_negation_is_self_inverse_for_non_nan(x):
480 assume(not isnan(x))
481 assert x == -(-x)
482 And this passes without a problem.
484 In order to avoid the easy trap where you assume a lot more than you
486 intended, Hypothesis will fail a test when it can't find enough exam‐
487 ples passing the assumption.
488 If we'd written:
490 @given(floats())
492 def test_negation_is_self_inverse_for_non_nan(x):
493 assume(False)
494 assert x == -(-x)
495 Then on running we'd have got the exception:
497 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
499 How good is assume?
501 Hypothesis has an adaptive exploration strategy to try to avoid things
502 which falsify assumptions, which should generally result in it still
503 being able to find examples in hard to find situations.
504 Suppose we had the following:
506 @given(lists(integers()))
508 def test_sum_is_positive(xs):
509 assert sum(xs) > 0
510 Unsurprisingly this fails and gives the falsifying example [].
512 Adding assume(xs) to this removes the trivial empty example and gives
514 us [0].
515 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
517 of positive integers is positive.
518 The reason that this should be surprising is not that it doesn't find a
520 counter-example, but that it finds enough examples at all.
521 In order to make sure something interesting is happening, suppose we
523 wanted to try this for long lists. e.g. suppose we added an as‐
524 sume(len(xs) > 10) to it. This should basically never find an example:
525 a naive strategy would find fewer than one in a thousand examples, be‐
526 cause if each element of the list is negative with probability
527 one-half, you'd have to have ten of these go the right way by chance.
528 In the default configuration Hypothesis gives up long before it's tried
529 1000 examples (by default it tries 200).
530 Here's what happens if we try to run this:
532 @given(lists(integers()))
534 def test_sum_is_positive(xs):
535 assume(len(xs) > 1)
536 assume(all(x > 0 for x in xs))
537 print(xs)
538 assert sum(xs) > 0
539 In: test_sum_is_positive()
541 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
543 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
544 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2, ...]
545 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
546 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...]
547 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
548 As you can see, Hypothesis doesn't find many examples here, but it
550 finds some - enough to keep it happy.
551 In general if you can shape your strategies better to your tests you
553 should - for example integers(1, 1000) is a lot better than assume(1 <=
554 x <= 1000), but assume will take you a long way if you can't.
555 Defining strategies
557 The type of object that is used to explore the examples given to your
558 test function is called a SearchStrategy. These are created using the
559 functions exposed in the hypothesis.strategies module.
560 Many of these strategies expose a variety of arguments you can use to
562 customize generation. For example for integers you can specify min and
563 max values of integers you want. If you want to see exactly what a
564 strategy produces you can ask for an example:
565 >>> integers(min_value=0, max_value=10).example()
567 1
568 Many strategies are built out of other strategies. For example, if you
570 want to define a tuple you need to say what goes in each element:
571 >>> from hypothesis.strategies import tuples
573 >>> tuples(integers(), integers()).example()
574 (-24597, 12566)
575 Further details are available in a separate document.
577 The gory details of given parameters
579 hypothesis.given(*_given_arguments, **_given_kwargs)
580 A decorator for turning a test function that accepts arguments
581 into a randomized test.
582 This is the main entry point to Hypothesis.
584 The @given decorator may be used to specify which arguments of a func‐
586 tion should be parametrized over. You can use either positional or key‐
587 word arguments, but not a mixture of both.
588 For example all of the following are valid uses:
590 @given(integers(), integers())
592 def a(x, y):
593 pass
594 @given(integers())
597 def b(x, y):
598 pass
599 @given(y=integers())
602 def c(x, y):
603 pass
604 @given(x=integers())
607 def d(x, y):
608 pass
609 @given(x=integers(), y=integers())
612 def e(x, **kwargs):
613 pass
614 @given(x=integers(), y=integers())
617 def f(x, *args, **kwargs):
618 pass
619 class SomeTest(TestCase):
622 @given(integers())
623 def test_a_thing(self, x):
624 pass
625 The following are not:
627 @given(integers(), integers(), integers())
629 def g(x, y):
630 pass
631 @given(integers())
634 def h(x, *args):
635 pass
636 @given(integers(), x=integers())
639 def i(x, y):
640 pass
641 @given()
644 def j(x, y):
645 pass
646 The rules for determining what are valid uses of given are as follows:
648 1. You may pass any keyword argument to given.
650 2. Positional arguments to given are equivalent to the rightmost named
652 arguments for the test function.
653 3. Positional arguments may not be used if the underlying test function
655 has varargs, arbitrary keywords, or keyword-only arguments.
656 4. Functions tested with given may not have any defaults.
658 The reason for the "rightmost named arguments" behaviour is so that us‐
660 ing @given with instance methods works: self will be passed to the
661 function as normal and not be parametrized over.
662 The function returned by given has all the same arguments as the origi‐
664 nal test, minus those that are filled in by @given. Check the notes on
665 framework compatibility to see how this affects other testing libraries
666 you may be using.
667 Targeted example generation
669 Targeted property-based testing combines the advantages of both
670 search-based and property-based testing. Instead of being completely
671 random, T-PBT uses a search-based component to guide the input genera‐
672 tion towards values that have a higher probability of falsifying a
673 property. This explores the input space more effectively and requires
674 fewer tests to find a bug or achieve a high confidence in the system
675 being tested than random PBT. (Löscher and Sagonas)
676 This is not always a good idea - for example calculating the search
678 metric might take time better spent running more uniformly-random test
679 cases - but Hypothesis has experimental support for targeted PBT you
680 may wish to try.
681 hypothesis.target(observation, *, label='')
683 Calling this function with an int or float observation gives it
684 feedback with which to guide our search for inputs that will
685 cause an error, in addition to all the usual heuristics. Obser‐
686 vations must always be finite.
687 Hypothesis will try to maximize the observed value over several
689 examples; almost any metric will work so long as it makes sense
690 to increase it. For example, -abs(error) is a metric that in‐
691 creases as error approaches zero.
692 Example metrics:
694 • Number of elements in a collection, or tasks in a queue
696 • Mean or maximum runtime of a task (or both, if you use label)
698 • Compression ratio for data (perhaps per-algorithm or
700 per-level)
701 • Number of steps taken by a state machine
703 The optional label argument can be used to distinguish between
705 and therefore separately optimise distinct observations, such as
706 the mean and standard deviation of a dataset. It is an error to
707 call target() with any label more than once per test case.
708 NOTE:
710 The more examples you run, the better this technique works.
711 As a rule of thumb, the targeting effect is noticeable above
713 max_examples=1000, and immediately obvious by around ten
714 thousand examples per label used by your test.
715 NOTE:
717 hypothesis.target is considered experimental, and may be rad‐
718 ically changed or even removed in a future version. If you
719 find it useful, please let us know so we can share and build
720 on that success!
721 Test statistics include the best score seen for each label,
723 which can help avoid the threshold problem when the minimal ex‐
724 ample shrinks right down to the threshold of failure (issue
725 #2180).
726 We recommend that users also skim the papers introducing targeted PBT;
728 from ISSTA 2017 and ICST 2018. For the curious, the initial implemen‐
729 tation in Hypothesis uses hill-climbing search via a mutating fuzzer,
730 with some tactics inspired by simulated annealing to avoid getting
731 stuck and endlessly mutating a local maximum.
732 Custom function execution
734 Hypothesis provides you with a hook that lets you control how it runs
735 examples.
736 This lets you do things like set up and tear down around each example,
738 run examples in a subprocess, transform coroutine tests into normal
739 tests, etc. For example, TransactionTestCase in the Django extra runs
740 each example in a separate database transaction.
741 The way this works is by introducing the concept of an executor. An ex‐
743 ecutor is essentially a function that takes a block of code and run it.
744 The default executor is:
745 def default_executor(function):
747 return function()
748 You define executors by defining a method execute_example on a class.
750 Any test methods on that class with @given used on them will use
751 self.execute_example as an executor with which to run tests. For exam‐
752 ple, the following executor runs all its code twice:
753 from unittest import TestCase
755 class TestTryReallyHard(TestCase):
758 @given(integers())
759 def test_something(self, i):
760 perform_some_unreliable_operation(i)
761 def execute_example(self, f):
763 f()
764 return f()
765 Note: The functions you use in map, etc. will run inside the executor.
767 i.e. they will not be called until you invoke the function passed to
768 execute_example.
769 An executor must be able to handle being passed a function which re‐
771 turns None, otherwise it won't be able to run normal test cases. So for
772 example the following executor is invalid:
773 from unittest import TestCase
775 class TestRunTwice(TestCase):
778 def execute_example(self, f):
779 return f()()
780 and should be rewritten as:
782 from unittest import TestCase
784 class TestRunTwice(TestCase):
787 def execute_example(self, f):
788 result = f()
789 if callable(result):
790 result = result()
791 return result
792 An alternative hook is provided for use by test runner extensions such
794 as pytest-trio, which cannot use the execute_example method. This is
795 not recommended for end-users - it is better to write a complete test
796 function directly, perhaps by using a decorator to perform the same
797 transformation before applying @given.
798 @given(x=integers())
800 @pytest.mark.trio
801 async def test(x):
802 ...
803 # Illustrative code, inside the pytest-trio plugin
806 test.hypothesis.inner_test = lambda x: trio.run(test, x)
807 For authors of test runners however, assigning to the inner_test attri‐
809 bute of the hypothesis attribute of the test will replace the interior
810 test.
811 NOTE:
813 The new inner_test must accept and pass through all the *args and
814 **kwargs expected by the original test.
815 If the end user has also specified a custom executor using the exe‐
817 cute_example method, it - and all other execution-time logic - will be
818 applied to the new inner test assigned by the test runner.
819 Making random code deterministic
821 While Hypothesis' example generation can be used for nondeterministic
822 tests, debugging anything nondeterministic is usually a very frustrat‐
823 ing exercise. To make things worse, our example shrinking relies on
824 the same input causing the same failure each time - though we show the
825 un-shrunk failure and a decent error message if it doesn't.
826 By default, Hypothesis will handle the global random and numpy.random
828 random number generators for you, and you can register others:
829 hypothesis.register_random(r)
831 Register the given Random instance for management by Hypothesis.
832 You can pass random.Random instances (or other objects with
834 seed, getstate, and setstate methods) to register_random(r) to
835 have their states seeded and restored in the same way as the
836 global PRNGs from the random and numpy.random modules.
837 All global PRNGs, from e.g. simulation or scheduling frameworks,
839 should be registered to prevent flaky tests. Hypothesis will
840 ensure that the PRNG state is consistent for all test runs, or
841 reproducibly varied if you choose to use the random_module()
842 strategy.
843 Inferred strategies
845 In some cases, Hypothesis can work out what to do when you omit argu‐
846 ments. This is based on introspection, not magic, and therefore has
847 well-defined limits.
848 builds() will check the signature of the target (using get‐
850 fullargspec()). If there are required arguments with type annotations
851 and no strategy was passed to builds(), from_type() is used to fill
852 them in. You can also pass the special value hypothesis.infer as a
853 keyword argument, to force this inference for arguments with a default
854 value.
855 >>> def func(a: int, b: str):
857 ... return [a, b]
858 >>> builds(func).example()
859 [-6993, '']
860 hypothesis.infer
862 @given does not perform any implicit inference for required arguments,
864 as this would break compatibility with pytest fixtures. infer can be
865 used as a keyword argument to explicitly fill in an argument from its
866 type annotation.
867 @given(a=infer)
869 def test(a: int):
870 pass
871 # is equivalent to
874 @given(a=integers())
875 def test(a):
876 pass
877 Limitations
879 Hypothesis does not inspect PEP 484 type comments at runtime. While
880 from_type() will work as usual, inference in builds() and @given will
881 only work if you manually create the __annotations__ attribute (e.g. by
882 using @annotations(...) and @returns(...) decorators).
883 The python:typing module is provisional and has a number of internal
885 changes between Python 3.5.0 and 3.6.1, including at minor versions.
886 These are all supported on a best-effort basis, but you may encounter
887 problems with an old version of the module. Please report them to us,
888 and consider updating to a newer version of Python as a workaround.
889 Type annotations in Hypothesis
891 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
892 patible tool, the type checker should automatically pick up our type
893 hints.
894 NOTE:
896 Hypothesis' type hints may make breaking changes between minor re‐
897 leases.
898 Upstream tools and conventions about type hints remain in flux - for
900 example the python:typing module itself is provisional, and Mypy has
901 not yet reached version 1.0 - and we plan to support the latest ver‐
902 sion of this ecosystem, as well as older versions where practical.
903 We may also find more precise ways to describe the type of various
905 interfaces, or change their type and runtime behaviour together in a
906 way which is otherwise backwards-compatible. We often omit type
907 hints for deprecated features or arguments, as an additional form of
908 warning.
909 There are known issues inferring the type of examples generated by de‐
911 ferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
912 ies(). We will fix these, and require correspondingly newer versions
913 of Mypy for type hinting, as the ecosystem improves.
914 Writing downstream type hints
916 Projects that provide Hypothesis strategies and use type hints may wish
917 to annotate their strategies too. This is a supported use-case, again
918 on a best-effort provisional basis. For example:
919 def foo_strategy() -> SearchStrategy[Foo]:
921 ...
922 class hypothesis.strategies.SearchStrategy
924 SearchStrategy is the type of all strategy objects. It is a generic
926 type, and covariant in the type of the examples it creates. For exam‐
927 ple:
928 • integers() is of type SearchStrategy[int].
930 • lists(integers()) is of type SearchStrategy[List[int]].
932 • SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
934 a subtype of Animal (as seems likely).
935 WARNING:
937 SearchStrategy should only be used in type hints. Please do not in‐
938 herit from, compare to, or otherwise use it in any way outside of
939 type hints. The only supported way to construct objects of this
940 type is to use the functions provided by the hypothesis.strategies
941 module!
942 The Hypothesis pytest plugin
944 Hypothesis includes a tiny plugin to improve integration with pytest,
945 which is activated by default (but does not affect other test runners).
946 It aims to improve the integration between Hypothesis and Pytest by
947 providing extra information and convenient access to config options.
948 • pytest --hypothesis-show-statistics can be used to display test and
950 data generation statistics.
951 • pytest --hypothesis-profile=<profile name> can be used to load a set‐
953 tings profile. pytest --hypothesis-verbosity=<level name> can be
954 used to override the current verbosity level.
955 • pytest --hypothesis-seed=<an int> can be used to reproduce a failure
957 with a particular seed.
958 Finally, all tests that are defined with Hypothesis automatically have
960 @pytest.mark.hypothesis applied to them. See here for information on
961 working with markers.
962 NOTE:
964 Pytest will load the plugin automatically if Hypothesis is in‐
965 stalled. You don't need to do anything at all to use it.
966 Use with external fuzzers
968 TIP:
969 Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
970 Use HypoFuzz to fuzz your whole test suite,
971 and find more bugs without more tests!
972 Sometimes, you might want to point a traditional fuzzer such as
975 python-afl, pythonfuzz, or Google's atheris (for Python and native ex‐
976 tensions) at your code. Wouldn't it be nice if you could use any of
977 your @given tests as fuzz targets, instead of converting bytestrings
978 into your objects by hand?
979 @given(st.text())
981 def test_foo(s):
982 ...
983 # This is a traditional fuzz target - call it with a bytestring,
986 # or a binary IO object, and it runs the test once.
987 fuzz_target = test_foo.hypothesis.fuzz_one_input
988 # For example:
990 fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
991 fuzz_target(io.BytesIO(...))
992 Depending on the input to fuzz_one_input, one of three things will hap‐
994 pen:
995 • If the bytestring was invalid, for example because it was too short
997 or failed a filter or assume() too many times, fuzz_one_input returns
998 None.
999 • If the bytestring was valid and the test passed, fuzz_one_input re‐
1001 turns a canonicalised and pruned buffer which will replay that test
1002 case. This is provided as an option to improve the performance of
1003 mutating fuzzers, but can safely be ignored.
1004 • If the test failed, i.e. raised an exception, fuzz_one_input will add
1006 the pruned buffer to the Hypothesis example database and then
1007 re-raise that exception. All you need to do to reproduce, minimize,
1008 and de-duplicate all the failures found via fuzzing is run your test
1009 suite!
1010 Note that the interpretation of both input and output bytestrings is
1012 specific to the exact version of Hypothesis you are using and the
1013 strategies given to the test, just like the example database and @re‐
1014 produce_failure decorator.
1015 Interaction with settings
1017 fuzz_one_input uses just enough of Hypothesis' internals to drive your
1018 test function with a fuzzer-provided bytestring, and most settings
1019 therefore have no effect in this mode. We recommend running your tests
1020 the usual way before fuzzing to get the benefits of healthchecks, as
1021 well as afterwards to replay, shrink, deduplicate, and report whatever
1022 errors were discovered.
1023 • The database setting is used by fuzzing mode - adding failures to the
1025 database to be replayed when you next run your tests is our preferred
1026 reporting mechanism and reponse to the 'fuzzer taming' problem.
1027 • The verbosity and stateful_step_count settings work as usual.
1029 The deadline, derandomize, max_examples, phases, print_blob, re‐
1031 port_multiple_bugs, and suppress_health_check settings do not affect
1032 fuzzing mode.
1033
1035 Hypothesis tries to have good defaults for its behaviour, but sometimes
1036 that's not enough and you need to tweak it.
1037 The mechanism for doing this is the settings object. You can set up a
1039 @given based test to use this using a settings decorator:
1040 @given invocation is as follows:
1042 from hypothesis import given, settings
1044 @given(integers())
1047 @settings(max_examples=500)
1048 def test_this_thoroughly(x):
1049 pass
1050 This uses a settings object which causes the test to receive a much
1052 larger set of examples than normal.
1053 This may be applied either before or after the given and the results
1055 are the same. The following is exactly equivalent:
1056 from hypothesis import given, settings
1058 @settings(max_examples=500)
1061 @given(integers())
1062 def test_this_thoroughly(x):
1063 pass
1064 Available settings
1066 class hypothesis.settings(parent=None, *, max_examples=not_set, deran‐
1067 domize=not_set, database=not_set, verbosity=not_set, phases=not_set,
1068 stateful_step_count=not_set, report_multiple_bugs=not_set, sup‐
1069 press_health_check=not_set, deadline=not_set, print_blob=not_set)
1070 A settings object controls a variety of parameters that are used
1071 in falsification. These may control both the falsification
1072 strategy and the details of the data that is generated.
1073 Default values are picked up from the settings.default object
1075 and changes made there will be picked up in newly created set‐
1076 tings.
1077 database
1079 An instance of ExampleDatabase that will be used to save
1080 examples to and load previous examples from. May be None
1081 in which case no storage will be used.
1082 See the example database documentation for a list of
1084 built-in example database implementations, and how to de‐
1085 fine custom implementations.
1086 default value: (dynamically calculated)
1088 deadline
1090 If set, a duration (as timedelta, or integer or float
1091 number of milliseconds) that each individual example
1092 (i.e. each time your test function is called, not the
1093 whole decorated test) within a test is not allowed to ex‐
1094 ceed. Tests which take longer than that may be converted
1095 into errors (but will not necessarily be if close to the
1096 deadline, to allow some variability in test run time).
1097 Set this to None to disable this behaviour entirely.
1099 default value: timedelta(milliseconds=200)
1101 derandomize
1103 If True, seed Hypothesis' random number generator using a
1104 hash of the test function, so that every run will test
1105 the same set of examples until you update Hypothesis,
1106 Python, or the test function.
1107 This allows you to check for regressions and look for
1109 bugs using separate settings profiles - for example run‐
1110 ning quick deterministic tests on every commit, and a
1111 longer non-deterministic nightly testing run.
1112 default value: False
1114 max_examples
1116 Once this many satisfying examples have been considered
1117 without finding any counter-example, falsification will
1118 terminate.
1119 The default value is chosen to suit a workflow where the
1121 test will be part of a suite that is regularly executed
1122 locally or on a CI server, balancing total running time
1123 against the chance of missing a bug.
1124 If you are writing one-off tests, running tens of thou‐
1126 sands of examples is quite reasonable as Hypothesis may
1127 miss uncommon bugs with default settings. For very com‐
1128 plex code, we have observed Hypothesis finding novel bugs
1129 after several million examples while testing SymPy. If
1130 you are running more than 100k examples for a test, con‐
1131 sider using our integration for coverage-guided fuzzing -
1132 it really shines when given minutes or hours to run.
1133 default value: 100
1135 phases Control which phases should be run. See the full documen‐
1137 tation for more details
1138 default value: (Phase.explicit, Phase.reuse, Phase.gener‐
1140 ate, Phase.target, Phase.shrink)
1141 print_blob
1143 If set to True, Hypothesis will print code for failing
1144 examples that can be used with @reproduce_failure to re‐
1145 produce the failing example. The default is True if the
1146 CI or TF_BUILD env vars are set, False otherwise.
1147 default value: (dynamically calculated)
1149 report_multiple_bugs
1151 Because Hypothesis runs the test many times, it can some‐
1152 times find multiple bugs in a single run. Reporting all
1153 of them at once is usually very useful, but replacing the
1154 exceptions can occasionally clash with debuggers. If
1155 disabled, only the exception with the smallest minimal
1156 example is raised.
1157 default value: True
1159 stateful_step_count
1161 Number of steps to run a stateful program for before giv‐
1162 ing up on it breaking.
1163 default value: 50
1165 suppress_health_check
1167 A list of HealthCheck items to disable.
1168 default value: ()
1170 verbosity
1172 Control the verbosity level of Hypothesis messages
1173 default value: Verbosity.normal
1175 Controlling what runs
1177 Hypothesis divides tests into five logically distinct phases:
1178 1. Running explicit examples provided with the @example decorator.
1180 2. Rerunning a selection of previously failing examples to reproduce a
1182 previously seen error
1183 3. Generating new examples.
1185 4. Mutating examples for targeted property-based testing.
1187 5. Attempting to shrink an example found in previous phases (other than
1189 phase 1 - explicit examples cannot be shrunk). This turns poten‐
1190 tially large and complicated examples which may be hard to read into
1191 smaller and simpler ones.
1192 The phases setting provides you with fine grained control over which of
1194 these run, with each phase corresponding to a value on the Phase enum:
1195 class hypothesis.Phase
1197 1. Phase.explicit controls whether explicit examples are run.
1199 2. Phase.reuse controls whether previous examples will be reused.
1201 3. Phase.generate controls whether new examples will be generated.
1203 4. Phase.target controls whether examples will be mutated for target‐
1205 ing.
1206 5. Phase.shrink controls whether examples will be shrunk.
1208 The phases argument accepts a collection with any subset of these. e.g.
1210 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1211 ples and shrink them, but will not run explicit examples or reuse pre‐
1212 vious failures, while settings(phases=[Phase.explicit]) will only run
1213 the explicit examples.
1214 Seeing intermediate result
1216 To see what's going on while Hypothesis runs your tests, you can turn
1217 up the verbosity setting.
1218 >>> from hypothesis import find, settings, Verbosity
1220 >>> from hypothesis.strategies import lists, integers
1221 >>> @given(lists(integers()))
1222 ... @settings(verbosity=Verbosity.verbose)
1223 ... def f(x): assert not any(x)
1224 ... f()
1225 Trying example: []
1226 Falsifying example: [-1198601713, -67, 116, -29578]
1227 Shrunk example to [-1198601713]
1228 Shrunk example to [-1198601600]
1229 Shrunk example to [-1191228800]
1230 Shrunk example to [-8421504]
1231 Shrunk example to [-32896]
1232 Shrunk example to [-128]
1233 Shrunk example to [64]
1234 Shrunk example to [32]
1235 Shrunk example to [16]
1236 Shrunk example to [8]
1237 Shrunk example to [4]
1238 Shrunk example to [3]
1239 Shrunk example to [2]
1240 Shrunk example to [1]
1241 [1]
1242 The four levels are quiet, normal, verbose and debug. normal is the de‐
1244 fault, while in quiet mode Hypothesis will not print anything out, not
1245 even the final falsifying example. debug is basically verbose but a bit
1246 more so. You probably don't want it.
1247 If you are using pytest, you may also need to disable output capturing
1249 for passing tests.
1250 Building settings objects
1252 Settings can be created by calling settings with any of the available
1253 settings values. Any absent ones will be set to defaults:
1254 >>> from hypothesis import settings
1256 >>> settings().max_examples
1257 100
1258 >>> settings(max_examples=10).max_examples
1259 10
1260 You can also pass a 'parent' settings object as the first argument, and
1262 any settings you do not specify as keyword arguments will be copied
1263 from the parent settings:
1264 >>> parent = settings(max_examples=10)
1266 >>> child = settings(parent, deadline=None)
1267 >>> parent.max_examples == child.max_examples == 10
1268 True
1269 >>> parent.deadline
1270 200
1271 >>> child.deadline is None
1272 True
1273 Default settings
1275 At any given point in your program there is a current default settings,
1276 available as settings.default. As well as being a settings object in
1277 its own right, all newly created settings objects which are not explic‐
1278 itly based off another settings are based off the default, so will in‐
1279 herit any values that are not explicitly set from it.
1280 You can change the defaults by using profiles.
1282 Settings profiles
1284 Depending on your environment you may want different default settings.
1285 For example: during development you may want to lower the number of ex‐
1286 amples to speed up the tests. However, in a CI environment you may want
1287 more examples so you are more likely to find bugs.
1288 Hypothesis allows you to define different settings profiles. These pro‐
1290 files can be loaded at any time.
1291 static settings.register_profile(name, parent=None, **kwargs)
1293 Registers a collection of values to be used as a settings pro‐
1294 file.
1295 Settings profiles can be loaded by name - for example, you might
1297 create a 'fast' profile which runs fewer examples, keep the 'de‐
1298 fault' profile, and create a 'ci' profile that increases the
1299 number of examples and uses a different database to store fail‐
1300 ures.
1301 The arguments to this method are exactly as for settings: op‐
1303 tional parent settings, and keyword arguments for each setting
1304 that will be set differently to parent (or settings.default, if
1305 parent is None).
1306 static settings.get_profile(name)
1308 Return the profile with the given name.
1309 static settings.load_profile(name)
1311 Loads in the settings defined in the profile provided.
1312 If the profile does not exist, InvalidArgument will be raised.
1314 Any setting not defined in the profile will be the library de‐
1315 fined default for that setting.
1316 Loading a profile changes the default settings but will not change the
1318 behaviour of tests that explicitly change the settings.
1319 >>> from hypothesis import settings
1321 >>> settings.register_profile("ci", max_examples=1000)
1322 >>> settings().max_examples
1323 100
1324 >>> settings.load_profile("ci")
1325 >>> settings().max_examples
1326 1000
1327 Instead of loading the profile and overriding the defaults you can re‐
1329 trieve profiles for specific tests.
1330 >>> settings.get_profile("ci").max_examples
1332 1000
1333 Optionally, you may define the environment variable to load a profile
1335 for you. This is the suggested pattern for running your tests on CI.
1336 The code below should run in a conftest.py or any setup/initialization
1337 section of your test suite. If this variable is not defined the Hy‐
1338 pothesis defined defaults will be loaded.
1339 >>> import os
1341 >>> from hypothesis import settings, Verbosity
1342 >>> settings.register_profile("ci", max_examples=1000)
1343 >>> settings.register_profile("dev", max_examples=10)
1344 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1345 >>> settings.load_profile(os.getenv(u'HYPOTHESIS_PROFILE', 'default'))
1346 If you are using the hypothesis pytest plugin and your profiles are
1348 registered by your conftest you can load one with the command line op‐
1349 tion --hypothesis-profile.
1350 $ pytest tests --hypothesis-profile <profile-name>
1352
1354 Most things should be easy to generate and everything should be possi‐
1355 ble.
1356 To support this principle Hypothesis provides strategies for most
1358 built-in types with arguments to constrain or adjust the output, as
1359 well as higher-order strategies that can be composed to generate more
1360 complex types.
1361 This document is a guide to what strategies are available for generat‐
1363 ing data and how to build them. Strategies have a variety of other im‐
1364 portant internal features, such as how they simplify, but the data they
1365 can generate is the only public part of their API.
1366 Core strategies
1368 Functions for building strategies are all available in the hypothe‐
1369 sis.strategies module. The salient functions from it are as follows:
1370 hypothesis.strategies.binary(*, min_size=0, max_size=None)
1372 Generates python:bytes.
1373 The generated python:bytes will have a length of at least
1375 min_size and at most max_size. If max_size is None there is no
1376 upper limit.
1377 Examples from this strategy shrink towards smaller strings and
1379 lower byte values.
1380 hypothesis.strategies.booleans()
1382 Returns a strategy which generates instances of python:bool.
1383 Examples from this strategy will shrink towards False (i.e.
1385 shrinking will replace True with False where possible).
1386 hypothesis.strategies.builds(target, /, *args, **kwargs)
1388 Generates values by drawing from args and kwargs and passing
1389 them to the callable (provided as the first positional argument)
1390 in the appropriate argument position.
1391 e.g. builds(target, integers(), flag=booleans()) would draw an
1393 integer i and a boolean b and call target(i, flag=b).
1394 If the callable has type annotations, they will be used to infer
1396 a strategy for required arguments that were not passed to
1397 builds. You can also tell builds to infer a strategy for an op‐
1398 tional argument by passing the special value hypothesis.infer as
1399 a keyword argument to builds, instead of a strategy for that ar‐
1400 gument to the callable.
1401 If the callable is a class defined with attrs, missing required
1403 arguments will be inferred from the attribute on a best-effort
1404 basis, e.g. by checking attrs standard validators. Dataclasses
1405 are handled natively by the inference from type hints.
1406 Examples from this strategy shrink by shrinking the argument
1408 values to the callable.
1409 hypothesis.strategies.characters(*, whitelist_categories=None, black‐
1411 list_categories=None, blacklist_characters=None, min_codepoint=None,
1412 max_codepoint=None, whitelist_characters=None)
1413 Generates characters, length-one python:strings, following spec‐
1414 ified filtering rules.
1415 • When no filtering rules are specified, any character can be
1417 produced.
1418 • If min_codepoint or max_codepoint is specified, then only
1420 characters having a codepoint in that range will be produced.
1421 • If whitelist_categories is specified, then only characters
1423 from those Unicode categories will be produced. This is a fur‐
1424 ther restriction, characters must also satisfy min_codepoint
1425 and max_codepoint.
1426 • If blacklist_categories is specified, then any character from
1428 those categories will not be produced. Any overlap between
1429 whitelist_categories and blacklist_categories will raise an
1430 exception, as each character can only belong to a single
1431 class.
1432 • If whitelist_characters is specified, then any additional
1434 characters in that list will also be produced.
1435 • If blacklist_characters is specified, then any characters in
1437 that list will be not be produced. Any overlap between
1438 whitelist_characters and blacklist_characters will raise an
1439 exception.
1440 The _codepoint arguments must be integers between zero and
1442 python:sys.maxunicode. The _characters arguments must be col‐
1443 lections of length-one unicode strings, such as a unicode
1444 string.
1445 The _categories arguments must be used to specify either the
1447 one-letter Unicode major category or the two-letter Unicode
1448 general category. For example, ('Nd', 'Lu') signifies "Number,
1449 decimal digit" and "Letter, uppercase". A single letter ('major
1450 category') can be given to match all corresponding categories,
1451 for example 'P' for characters in any punctuation category.
1452 Examples from this strategy shrink towards the codepoint for
1454 '0', or the first allowable codepoint after it if '0' is ex‐
1455 cluded.
1456 hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magni‐
1458 tude=None, allow_infinity=None, allow_nan=None)
1459 Returns a strategy that generates complex numbers.
1460 This strategy draws complex numbers with constrained magnitudes.
1462 The min_magnitude and max_magnitude parameters should be
1463 non-negative Real numbers; a value of None corresponds an infi‐
1464 nite upper bound.
1465 If min_magnitude is nonzero or max_magnitude is finite, it is an
1467 error to enable allow_nan. If max_magnitude is finite, it is an
1468 error to enable allow_infinity.
1469 The magnitude contraints are respected up to a relative error of
1471 (around) floating-point epsilon, due to implementation via the
1472 system sqrt function.
1473 Examples from this strategy shrink by shrinking their real and
1475 imaginary parts, as floats().
1476 If you need to generate complex numbers with particular real and
1478 imaginary parts or relationships between parts, consider using
1479 builds(complex, ...) or @composite respectively.
1480 hypothesis.strategies.composite(f)
1482 Defines a strategy that is built out of potentially arbitrarily
1483 many other strategies.
1484 This is intended to be used as a decorator. See the full docu‐
1486 mentation for more details about how to use this function.
1487 Examples from this strategy shrink by shrinking the output of
1489 each draw call.
1490 hypothesis.strategies.data()
1492 This isn't really a normal strategy, but instead gives you an
1493 object which can be used to draw data interactively from other
1494 strategies.
1495 See the rest of the documentation for more complete information.
1497 Examples from this strategy do not shrink (because there is only
1499 one), but the result of calls to each draw() call shrink as they
1500 normally would.
1501 class hypothesis.strategies.DataObject(data)
1503 This type only exists so that you can write type hints for tests
1504 using the data() strategy. Do not use it directly!
1505 hypothesis.strategies.dates(min_value=datetime.date.min,
1507 max_value=datetime.date.max)
1508 A strategy for dates between min_value and max_value.
1509 Examples from this strategy shrink towards January 1st 2000.
1511 hypothesis.strategies.datetimes(min_value=datetime.datetime.min,
1513 max_value=datetime.datetime.max, *, timezones=none(), allow_imagi‐
1514 nary=True)
1515 A strategy for generating datetimes, which may be time‐
1516 zone-aware.
1517 This strategy works by drawing a naive datetime between
1519 min_value and max_value, which must both be naive (have no time‐
1520 zone).
1521 timezones must be a strategy that generates either None, for
1523 naive datetimes, or tzinfo objects for 'aware' datetimes. You
1524 can construct your own, though we recommend using the dateutil
1525 package and hypothesis.extra.dateutil.timezones() strategy, and
1526 also provide hypothesis.extra.pytz.timezones().
1527 You may pass allow_imaginary=False to filter out "imaginary"
1529 datetimes which did not (or will not) occur due to daylight sav‐
1530 ings, leap seconds, timezone and calendar adjustments, etc.
1531 Imaginary datetimes are allowed by default, because malformed
1532 timestamps are a common source of bugs. Note that because pytz
1533 predates PEP 495, this does not work correctly with timezones
1534 that use a negative DST offset (such as "Europe/Dublin").
1535 Examples from this strategy shrink towards midnight on January
1537 1st 2000, local time.
1538 hypothesis.strategies.decimals(min_value=None, max_value=None, *, al‐
1540 low_nan=None, allow_infinity=None, places=None)
1541 Generates instances of python:decimal.Decimal, which may be:
1542 • A finite rational number, between min_value and max_value.
1544 • Not a Number, if allow_nan is True. None means "allow NaN,
1546 unless min_value and max_value are not None".
1547 • Positive or negative infinity, if max_value and min_value re‐
1549 spectively are None, and allow_infinity is not False. None
1550 means "allow infinity, unless excluded by the min and max val‐
1551 ues".
1552 Note that where floats have one NaN value, Decimals have four:
1554 signed, and either quiet or signalling. See the decimal module
1555 docs for more information on special values.
1556 If places is not None, all finite values drawn from the strategy
1558 will have that number of digits after the decimal place.
1559 Examples from this strategy do not have a well defined shrink
1561 order but try to maximize human readability when shrinking.
1562 hypothesis.strategies.deferred(definition)
1564 A deferred strategy allows you to write a strategy that refer‐
1565 ences other strategies that have not yet been defined. This al‐
1566 lows for the easy definition of recursive and mutually recursive
1567 strategies.
1568 The definition argument should be a zero-argument function that
1570 returns a strategy. It will be evaluated the first time the
1571 strategy is used to produce an example.
1572 Example usage:
1574 >>> import hypothesis.strategies as st
1576 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1577 >>> x.example()
1578 (((False, (True, True)), (False, True)), (True, True))
1579 >>> x.example()
1580 True
1581 Mutual recursion also works fine:
1583 >>> a = st.deferred(lambda: st.booleans() | b)
1585 >>> b = st.deferred(lambda: st.tuples(a, a))
1586 >>> a.example()
1587 True
1588 >>> b.example()
1589 (False, (False, ((False, True), False)))
1590 Examples from this strategy shrink as they normally would from
1592 the strategy returned by the definition.
1593 hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class
1595 'dict'>, min_size=0, max_size=None)
1596 Generates dictionaries of type dict_class with keys drawn from
1597 the keys argument and values drawn from the values argument.
1598 The size parameters have the same interpretation as for lists().
1600 Examples from this strategy shrink by trying to remove keys from
1602 the generated dictionary, and by shrinking each generated key
1603 and value.
1604 hypothesis.strategies.emails()
1606 A strategy for generating email addresses as unicode strings.
1607 The address format is specified in RFC 5322#section-3.4.1. Val‐
1608 ues shrink towards shorter local-parts and host domains.
1609 This strategy is useful for generating "user data" for tests, as
1611 mishandling of email addresses is a common source of bugs.
1612 hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)
1614 Generates a dictionary of the same type as mapping with a fixed
1615 set of keys mapping to strategies. mapping must be a dict sub‐
1616 class.
1617 Generated values have all keys present in mapping, with the cor‐
1619 responding values drawn from mapping[key]. If mapping is an in‐
1620 stance of OrderedDict the keys will also be in the same order,
1621 otherwise the order is arbitrary.
1622 If optional is passed, the generated value may or may not con‐
1624 tain each key from optional and a value drawn from the corre‐
1625 sponding strategy.
1626 Examples from this strategy shrink by shrinking each individual
1628 value in the generated dictionary, and omitting optional
1629 key-value pairs.
1630 hypothesis.strategies.floats(min_value=None, max_value=None, *, al‐
1632 low_nan=None, allow_infinity=None, width=64, exclude_min=False, ex‐
1633 clude_max=False)
1634 Returns a strategy which generates floats.
1635 • If min_value is not None, all values will be >= min_value (or
1637 > min_value if exclude_min).
1638 • If max_value is not None, all values will be <= max_value (or
1640 < max_value if exclude_max).
1641 • If min_value or max_value is not None, it is an error to en‐
1643 able allow_nan.
1644 • If both min_value and max_value are not None, it is an error
1646 to enable allow_infinity.
1647 Where not explicitly ruled out by the bounds, all of infinity,
1649 -infinity and NaN are possible values generated by this strat‐
1650 egy.
1651 The width argument specifies the maximum number of bits of pre‐
1653 cision required to represent the generated float. Valid values
1654 are 16, 32, or 64. Passing width=32 will still use the builtin
1655 64-bit float class, but always for values which can be exactly
1656 represented as a 32-bit float.
1657 The exclude_min and exclude_max argument can be used to generate
1659 numbers from open or half-open intervals, by excluding the re‐
1660 spective endpoints. Excluding either signed zero will also ex‐
1661 clude the other. Attempting to exclude an endpoint which is
1662 None will raise an error; use allow_infinity=False to generate
1663 finite floats. You can however use e.g. min_value=-math.inf,
1664 exclude_min=True to exclude only one infinite endpoint.
1665 Examples from this strategy have a complicated and hard to ex‐
1667 plain shrinking behaviour, but it tries to improve "human read‐
1668 ability". Finite numbers will be preferred to infinity and in‐
1669 finity will be preferred to NaN.
1670 hypothesis.strategies.fractions(min_value=None, max_value=None, *,
1672 max_denominator=None)
1673 Returns a strategy which generates Fractions.
1674 If min_value is not None then all generated values are no less
1676 than min_value. If max_value is not None then all generated
1677 values are no greater than max_value. min_value and max_value
1678 may be anything accepted by the Fraction constructor.
1679 If max_denominator is not None then the denominator of any gen‐
1681 erated values is no greater than max_denominator. Note that
1682 max_denominator must be None or a positive integer.
1683 Examples from this strategy shrink towards smaller denominators,
1685 then closer to zero.
1686 hypothesis.strategies.from_regex(regex, *, fullmatch=False)
1688 Generates strings that contain a match for the given regex (i.e.
1689 ones for which python:re.search() will return a non-None re‐
1690 sult).
1691 regex may be a pattern or compiled regex. Both byte-strings and
1693 unicode strings are supported, and will generate examples of the
1694 same type.
1695 You can use regex flags such as python:re.IGNORECASE or
1697 python:re.DOTALL to control generation. Flags can be passed ei‐
1698 ther in compiled regex or inside the pattern with a (?iLmsux)
1699 group.
1700 Some regular expressions are only partly supported - the under‐
1702 lying strategy checks local matching and relies on filtering to
1703 resolve context-dependent expressions. Using too many of these
1704 constructs may cause health-check errors as too many examples
1705 are filtered out. This mainly includes (positive or negative)
1706 lookahead and lookbehind groups.
1707 If you want the generated string to match the whole regex you
1709 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1710 gle character string, while "." will return any string, and
1711 r"\A.$" will return a single character optionally followed by a
1712 "\n". Alternatively, passing fullmatch=True will ensure that
1713 the whole string is a match, as if you had used the \A and \Z
1714 markers.
1715 Examples from this strategy shrink towards shorter strings and
1717 lower character values, with exact behaviour that may depend on
1718 the pattern.
1719 hypothesis.strategies.from_type(thing)
1721 Looks up the appropriate search strategy for the given type.
1722 from_type is used internally to fill in missing arguments to
1724 builds() and can be used interactively to explore what strate‐
1725 gies are available or to debug type resolution.
1726 You can use register_type_strategy() to handle your custom
1728 types, or to globally redefine certain strategies - for example
1729 excluding NaN from floats, or use timezone-aware instead of
1730 naive time and datetime strategies.
1731 The resolution logic may be changed in a future version, but
1733 currently tries these five options:
1734 1. If thing is in the default lookup mapping or user-registered
1736 lookup, return the corresponding strategy. The default
1737 lookup covers all types with Hypothesis strategies, including
1738 extras where possible.
1739 2. If thing is from the python:typing module, return the corre‐
1741 sponding strategy (special logic).
1742 3. If thing has one or more subtypes in the merged lookup, re‐
1744 turn the union of the strategies for those types that are not
1745 subtypes of other elements in the lookup.
1746 4. Finally, if thing has type annotations for all required argu‐
1748 ments, and is not an abstract class, it is resolved via
1749 builds().
1750 5. Because abstract types cannot be instantiated, we treat ab‐
1752 stract types as the union of their concrete subclasses. Note
1753 that this lookup works via inheritance but not via register,
1754 so you may still need to use register_type_strategy().
1755 There is a valuable recipe for leveraging from_type() to gener‐
1757 ate "everything except" values from a specified type. I.e.
1758 def everything_except(excluded_types):
1760 return (
1761 from_type(type).flatmap(from_type)
1762 .filter(lambda x: not isinstance(x, excluded_types))
1763 )
1764 For example, everything_except(int) returns a strategy that can
1766 generate anything that from_type() can ever generate, except for
1767 instances of python:int, and excluding instances of types added
1768 via register_type_strategy().
1769 This is useful when writing tests which check that invalid input
1771 is rejected in a certain way.
1772 hypothesis.strategies.frozensets(elements, *, min_size=0,
1774 max_size=None)
1775 This is identical to the sets function but instead returns
1776 frozensets.
1777 hypothesis.strategies.functions(*, like=lambda : ..., returns=none(),
1779 pure=False)
1780 A strategy for functions, which can be used in callbacks.
1781 The generated functions will mimic the interface of like, which
1783 must be a callable (including a class, method, or function).
1784 The return value for the function is drawn from the returns ar‐
1785 gument, which must be a strategy.
1786 If pure=True, all arguments passed to the generated function
1788 must be hashable, and if passed identical arguments the original
1789 return value will be returned again - not regenerated, so beware
1790 mutable values.
1791 If pure=False, generated functions do not validate their argu‐
1793 ments, and may return a different value if called again with the
1794 same arguments.
1795 Generated functions can only be called within the scope of the
1797 @given which created them. This strategy does not support .ex‐
1798 ample().
1799 hypothesis.strategies.integers(min_value=None, max_value=None)
1801 Returns a strategy which generates integers.
1802 If min_value is not None then all values will be >= min_value.
1804 If max_value is not None then all values will be <= max_value
1805 Examples from this strategy will shrink towards zero, and nega‐
1807 tive values will also shrink towards positive (i.e. -n may be
1808 replaced by +n).
1809 hypothesis.strategies.ip_addresses(*, v=None, network=None)
1811 Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Ad‐
1812 dresses, or leave unspecified to allow both versions.
1813 network may be an IPv4Network or IPv6Network, or a string repre‐
1815 senting a network such as "127.0.0.0/24" or "2001:db8::/32". As
1816 well as generating addresses within a particular routable net‐
1817 work, this can be used to generate addresses from a reserved
1818 range listed in the IANA registries.
1819 If you pass both v and network, they must be for the same ver‐
1821 sion.
1822 hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None,
1824 unique_by=None, unique=False)
1825 This has the same behaviour as lists, but returns iterables in‐
1826 stead.
1827 Some iterables cannot be indexed (e.g. sets) and some do not
1829 have a fixed length (e.g. generators). This strategy produces
1830 iterators, which cannot be indexed and do not have a fixed
1831 length. This ensures that you do not accidentally depend on se‐
1832 quence behaviour.
1833 hypothesis.strategies.just(value)
1835 Return a strategy which only generates value.
1836 Note: value is not copied. Be wary of using mutable values.
1838 If value is the result of a callable, you can use
1840 builds(callable) instead of just(callable()) to get a fresh
1841 value each time.
1842 Examples from this strategy do not shrink (because there is only
1844 one).
1845 hypothesis.strategies.lists(elements, *, min_size=0, max_size=None,
1847 unique_by=None, unique=False)
1848 Returns a list containing values drawn from elements with length
1849 in the interval [min_size, max_size] (no bounds in that direc‐
1850 tion if these are None). If max_size is 0, only the empty list
1851 will be drawn.
1852 If unique is True (or something that evaluates to True), we com‐
1854 pare direct object equality, as if unique_by was lambda x: x.
1855 This comparison only works for hashable types.
1856 If unique_by is not None it must be a callable or tuple of
1858 callables returning a hashable type when given a value drawn
1859 from elements. The resulting list will satisfy the condition
1860 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1861 If unique_by is a tuple of callables the uniqueness will be re‐
1863 spective to each callable.
1864 For example, the following will produce two columns of integers
1866 with both columns being unique respectively.
1867 >>> twoints = st.tuples(st.integers(), st.integers())
1869 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1870 Examples from this strategy shrink by trying to remove elements
1872 from the list, and by shrinking each individual element of the
1873 list.
1874 hypothesis.strategies.none()
1876 Return a strategy which only generates None.
1877 Examples from this strategy do not shrink (because there is only
1879 one).
1880 hypothesis.strategies.nothing()
1882 This strategy never successfully draws a value and will always
1883 reject on an attempt to draw.
1884 Examples from this strategy do not shrink (because there are
1886 none).
1887 hypothesis.strategies.one_of(*args)
1889 Return a strategy which generates values from any of the argu‐
1890 ment strategies.
1891 This may be called with one iterable argument instead of multi‐
1893 ple strategy arguments, in which case one_of(x) and one_of(*x)
1894 are equivalent.
1895 Examples from this strategy will generally shrink to ones that
1897 come from strategies earlier in the list, then shrink according
1898 to behaviour of the strategy that produced them. In order to get
1899 good shrinking behaviour, try to put simpler strategies first.
1900 e.g. one_of(none(), text()) is better than one_of(text(),
1901 none()).
1902 This is especially important when using recursive strategies.
1904 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1905 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1906 st.none()) will shrink very badly indeed.
1907 hypothesis.strategies.permutations(values)
1909 Return a strategy which returns permutations of the ordered col‐
1910 lection values.
1911 Examples from this strategy shrink by trying to become closer to
1913 the original order of values.
1914 hypothesis.strategies.random_module()
1916 The Hypothesis engine handles PRNG state for the stdlib and
1917 Numpy random modules internally, always seeding them to zero and
1918 restoring the previous state after the test.
1919 If having a fixed seed would unacceptably weaken your tests, and
1921 you cannot use a random.Random instance provided by randoms(),
1922 this strategy calls python:random.seed() with an arbitrary inte‐
1923 ger and passes you an opaque object whose repr displays the seed
1924 value for debugging. If numpy.random is available, that state
1925 is also managed.
1926 Examples from these strategy shrink to seeds closer to zero.
1928 hypothesis.strategies.randoms(*, note_method_calls=False, use_true_ran‐
1930 dom=None)
1931 Generates instances of random.Random. The generated Random in‐
1932 stances are of a special HypothesisRandom subclass.
1933 • If note_method_calls is set to True, Hypothesis will print the
1935 randomly drawn values in any falsifying test case. This can be
1936 helpful for debugging the behaviour of randomized algorithms.
1937 • If use_true_random is set to True then values will be drawn
1939 from their usual distribution, otherwise they will actually be
1940 Hypothesis generated values (and will be shrunk accordingly
1941 for any failing test case). Setting use_true_random=False will
1942 tend to expose bugs that would occur with very low probability
1943 when it is set to True, and this flag should only be set to
1944 True when your code relies on the distribution of values for
1945 correctness.
1946 hypothesis.strategies.recursive(base, extend, *, max_leaves=100)
1948 base: A strategy to start from.
1949 extend: A function which takes a strategy and returns a new
1951 strategy.
1952 max_leaves: The maximum number of elements to be drawn from base
1954 on a given run.
1955 This returns a strategy S such that S = extend(base | S). That
1957 is, values may be drawn from base, or from any strategy reach‐
1958 able by mixing applications of | and extend.
1959 An example may clarify: recursive(booleans(), lists) would re‐
1961 turn a strategy that may return arbitrarily nested and mixed
1962 lists of booleans. So e.g. False, [True], [False, []], and
1963 [[[[True]]]] are all valid values to be drawn from that strat‐
1964 egy.
1965 Examples from this strategy shrink by trying to reduce the
1967 amount of recursion and by shrinking according to the shrinking
1968 behaviour of base and the result of extend.
1969 hypothesis.strategies.register_type_strategy(custom_type, strategy)
1971 Add an entry to the global type-to-strategy lookup.
1972 This lookup is used in builds() and @given.
1974 builds() will be used automatically for classes with type anno‐
1976 tations on __init__ , so you only need to register a strategy if
1977 one or more arguments need to be more tightly defined than their
1978 type-based default, or if you want to supply a strategy for an
1979 argument with a default value.
1980 strategy may be a search strategy, or a function that takes a
1982 type and returns a strategy (useful for generic types).
1983 Note that you may not register a parametrised generic type (such
1985 as MyCollection[int]) directly, because the resolution logic
1986 does not handle this case correctly. Instead, you may register
1987 a function for MyCollection and inspect the type parameters
1988 within that function.
1989 hypothesis.strategies.runner(*, default=not_set)
1991 A strategy for getting "the current test runner", whatever that
1992 may be. The exact meaning depends on the entry point, but it
1993 will usually be the associated 'self' value for it.
1994 If there is no current test runner and a default is provided,
1996 return that default. If no default is provided, raises Invali‐
1997 dArgument.
1998 Examples from this strategy do not shrink (because there is only
2000 one).
2001 hypothesis.strategies.sampled_from(elements)
2003 Returns a strategy which generates any value present in ele‐
2004 ments.
2005 Note that as with just(), values will not be copied and thus you
2007 should be careful of using mutable data.
2008 sampled_from supports ordered collections, as well as Enum ob‐
2010 jects. Flag objects may also generate any combination of their
2011 members.
2012 Examples from this strategy shrink by replacing them with values
2014 earlier in the list. So e.g. sampled_from([10, 1]) will shrink
2015 by trying to replace 1 values with 10, and sampled_from([1, 10])
2016 will shrink by trying to replace 10 values with 1.
2017 hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)
2019 This has the same behaviour as lists, but returns sets instead.
2020 Note that Hypothesis cannot tell if values are drawn from ele‐
2022 ments are hashable until running the test, so you can define a
2023 strategy for sets of an unhashable type but it will fail at test
2024 time.
2025 Examples from this strategy shrink by trying to remove elements
2027 from the set, and by shrinking each individual element of the
2028 set.
2029 hypothesis.strategies.shared(base, *, key=None)
2031 Returns a strategy that draws a single shared value per run,
2032 drawn from base. Any two shared instances with the same key will
2033 share the same value, otherwise the identity of this strategy
2034 will be used. That is:
2035 >>> s = integers() # or any other strategy
2037 >>> x = shared(s)
2038 >>> y = shared(s)
2039 In the above x and y may draw different (or potentially the
2041 same) values. In the following they will always draw the same:
2042 >>> x = shared(s, key="hi")
2044 >>> y = shared(s, key="hi")
2045 Examples from this strategy shrink as per their base strategy.
2047 hypothesis.strategies.slices(size)
2049 Generates slices that will select indices up to the supplied
2050 size
2051 Generated slices will have start and stop indices that range
2053 from -size to size - 1 and will step in the appropriate direc‐
2054 tion. Slices should only produce an empty selection if the start
2055 and end are the same.
2056 Examples from this strategy shrink toward 0 and smaller values
2058 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
2060 gories=('Cs')), *, min_size=0, max_size=None)
2061 Generates strings with characters drawn from alphabet, which
2062 should be a collection of length one strings or a strategy gen‐
2063 erating such strings.
2064 The default alphabet strategy can generate the full unicode
2066 range but excludes surrogate characters because they are invalid
2067 in the UTF-8 encoding. You can use characters() without argu‐
2068 ments to find surrogate-related bugs such as bpo-34454.
2069 min_size and max_size have the usual interpretations. Note that
2071 Python measures string length by counting codepoints: U+00C5 Å
2072 is a single character, while U+0041 U+030A Å is two - the A,
2073 and a combining ring above.
2074 Examples from this strategy shrink towards shorter strings, and
2076 with the characters in the text shrinking as per the alphabet
2077 strategy. This strategy does not normalize() examples, so gen‐
2078 erated strings may be in any or none of the 'normal forms'.
2079 hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min,
2081 max_value=datetime.timedelta.max)
2082 A strategy for timedeltas between min_value and max_value.
2083 Examples from this strategy shrink towards zero.
2085 hypothesis.strategies.times(min_value=datetime.time.min,
2087 max_value=datetime.time.max, *, timezones=none())
2088 A strategy for times between min_value and max_value.
2089 The timezones argument is handled as for datetimes().
2091 Examples from this strategy shrink towards midnight, with the
2093 timezone component shrinking as for the strategy that provided
2094 it.
2095 hypothesis.strategies.timezone_keys(*, allow_prefix=True)
2097 A strategy for IANA timezone names.
2098 As well as timezone names like "UTC", "Australia/Sydney", or
2100 "America/New_York", this strategy can generate:
2101 • Aliases such as "Antarctica/McMurdo", which links to "Pa‐
2103 cific/Auckland".
2104 • Deprecated names such as "Antarctica/South_Pole", which also
2106 links to "Pacific/Auckland". Note that most but not all dep‐
2107 recated timezone names are also aliases.
2108 • Timezone names with the "posix/" or "right/" prefixes, unless
2110 allow_prefix=False.
2111 These strings are provided separately from Tzinfo objects - such
2113 as ZoneInfo instances from the timezones() strategy - to facili‐
2114 tate testing of timezone logic without needing workarounds to
2115 access non-canonical names.
2116 NOTE:
2118 The python:zoneinfo module is new in Python 3.9, so you will
2119 need to install the backports.zoneinfo module on earlier ver‐
2120 sions, and the importlib_resources backport on Python 3.6.
2121 pip install hypothesis[zoneinfo] will install these condi‐
2123 tional dependencies if and only if they are needed.
2124 On Windows, you may need to access IANA timezone data via the
2126 tzdata package. For non-IANA timezones, such as Windows-native
2127 names or GNU TZ strings, we recommend using sampled_from() with
2128 the dateutil package, e.g. dateutil:dateutil.tz.tzwin.list().
2129 hypothesis.strategies.timezones(*, no_cache=False)
2131 A strategy for python:zoneinfo.ZoneInfo objects.
2132 If no_cache=True, the generated instances are constructed using
2134 ZoneInfo.no_cache instead of the usual constructor. This may
2135 change the semantics of your datetimes in surprising ways, so
2136 only use it if you know that you need to!
2137 NOTE:
2139 The python:zoneinfo module is new in Python 3.9, so you will
2140 need to install the backports.zoneinfo module on earlier ver‐
2141 sions, and the importlib_resources backport on Python 3.6.
2142 pip install hypothesis[zoneinfo] will install these condi‐
2144 tional dependencies if and only if they are needed.
2145 hypothesis.strategies.tuples(*args)
2147 Return a strategy which generates a tuple of the same length as
2148 args by generating the value at index i from args[i].
2149 e.g. tuples(integers(), integers()) would generate a tuple of
2151 length two with both values an integer.
2152 Examples from this strategy shrink by shrinking their component
2154 parts.
2155 hypothesis.strategies.uuids(*, version=None)
2157 Returns a strategy that generates UUIDs.
2158 If the optional version argument is given, value is passed
2160 through to UUID and only UUIDs of that version will be gener‐
2161 ated.
2162 All returned values from this will be unique, so e.g. if you do
2164 lists(uuids()) the resulting list will never contain duplicates.
2165 Examples from this strategy don't have any meaningful shrink or‐
2167 der.
2168 Provisional strategies
2170 This module contains various provisional APIs and strategies.
2171 It is intended for internal use, to ease code reuse, and is not stable.
2173 Point releases may move or break the contents at any time!
2174 Internet strategies should conform to RFC 3986 or the authoritative
2176 definitions it links to. If not, report the bug!
2177 hypothesis.provisional.domains(*, max_length=255, max_ele‐
2179 ment_length=63)
2180 Generate RFC 1035 compliant fully qualified domain names.
2181 hypothesis.provisional.urls()
2183 A strategy for RFC 3986, generating http/https URLs.
2184 Shrinking
2186 When using strategies it is worth thinking about how the data shrinks.
2187 Shrinking is the process by which Hypothesis tries to produce human
2188 readable examples when it finds a failure - it takes a complex example
2189 and turns it into a simpler one.
2190 Each strategy defines an order in which it shrinks - you won't usually
2192 need to care about this much, but it can be worth being aware of as it
2193 can affect what the best way to write your own strategies is.
2194 The exact shrinking behaviour is not a guaranteed part of the API, but
2196 it doesn't change that often and when it does it's usually because we
2197 think the new way produces nicer examples.
2198 Possibly the most important one to be aware of is one_of(), which has a
2200 preference for values produced by strategies earlier in its argument
2201 list. Most of the others should largely "do the right thing" without
2202 you having to think about it.
2203 Adapting strategies
2205 Often it is the case that a strategy doesn't produce exactly what you
2206 want it to and you need to adapt it. Sometimes you can do this in the
2207 test, but this hurts reuse because you then have to repeat the adaption
2208 in every test.
2209 Hypothesis gives you ways to build strategies from other strategies
2211 given functions for transforming the data.
2212 Mapping
2214 map is probably the easiest and most useful of these to use. If you
2215 have a strategy s and a function f, then an example s.map(f).example()
2216 is f(s.example()), i.e. we draw an example from s and then apply f to
2217 it.
2218 e.g.:
2220 >>> lists(integers()).map(sorted).example()
2222 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2223 Note that many things that you might use mapping for can also be done
2225 with builds().
2226 Filtering
2228 filter lets you reject some examples. s.filter(f).example() is some ex‐
2229 ample of s such that f(example) is truthy.
2230 >>> integers().filter(lambda x: x > 11).example()
2232 26126
2233 >>> integers().filter(lambda x: x > 11).example()
2234 23324
2235 It's important to note that filter isn't magic and if your condition is
2237 too hard to satisfy then this can fail:
2238 >>> integers().filter(lambda x: False).example()
2240 Traceback (most recent call last):
2241 ...
2242 hypothesis.errors.Unsatisfiable: Could not find any valid examples in 20 tries
2243 In general you should try to use filter only to avoid corner cases that
2245 you don't want rather than attempting to cut out a large chunk of the
2246 search space.
2247 A technique that often works well here is to use map to first transform
2249 the data and then use filter to remove things that didn't work out. So
2250 for example if you wanted pairs of integers (x,y) such that x < y you
2251 could do the following:
2252 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2254 [-8543729478746591815, 3760495307320535691]
2255 Chaining strategies together
2257 Finally there is flatmap. flatmap draws an example, then turns that ex‐
2258 ample into a strategy, then draws an example from that strategy.
2259 It may not be obvious why you want this at first, but it turns out to
2261 be quite useful because it lets you generate different types of data
2262 with relationships to each other.
2263 For example suppose we wanted to generate a list of lists of the same
2265 length:
2266 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2268 ... lambda n: lists(lists(integers(), min_size=n, max_size=n)))
2269 >>> rectangle_lists.example()
2270 []
2271 >>> rectangle_lists.filter(lambda x: len(x) >= 10).example()
2272 [[], [], [], [], [], [], [], [], [], []]
2273 >>> rectangle_lists.filter(lambda t: len(t) >= 3 and len(t[0]) >= 3).example()
2274 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2275 >>> rectangle_lists.filter(lambda t: sum(len(s) for s in t) >= 10).example()
2276 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2277 In this example we first choose a length for our tuples, then we build
2279 a strategy which generates lists containing lists precisely of that
2280 length. The finds show what simple examples for this look like.
2281 Most of the time you probably don't want flatmap, but unlike filter and
2283 map which are just conveniences for things you could just do in your
2284 tests, flatmap allows genuinely new data generation that you wouldn't
2285 otherwise be able to easily do.
2286 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2288 don't know Haskell, ignore everything in these parentheses. You do not
2289 need to understand anything about monads to use this, or anything else
2290 in Hypothesis).
2291 Recursive data
2293 Sometimes the data you want to generate has a recursive definition.
2294 e.g. if you wanted to generate JSON data, valid JSON is:
2295 1. Any float, any boolean, any unicode string.
2297 2. Any list of valid JSON data
2299 3. Any dictionary mapping unicode strings to valid JSON data.
2301 The problem is that you cannot call a strategy recursively and expect
2303 it to not just blow up and eat all your memory. The other problem here
2304 is that not all unicode strings display consistently on different ma‐
2305 chines, so we'll restrict them in our doctest.
2306 The way Hypothesis handles this is with the recursive() strategy which
2308 you pass in a base case and a function that, given a strategy for your
2309 data type, returns a new strategy for it. So for example:
2310 >>> from string import printable; from pprint import pprint
2312 >>> json = recursive(none() | booleans() | floats() | text(printable),
2313 ... lambda children: lists(children, 1) | dictionaries(text(printable), children, min_size=1))
2314 >>> pprint(json.example())
2315 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2316 >>> pprint(json.example())
2317 {'de(l': None,
2318 'nK': {'(Rt)': None,
2319 '+hoZh1YU]gy8': True,
2320 '8z]EIFA06^li^': 'LFE{Q',
2321 '9,': 'l{cA=/'}}
2322 That is, we start with our leaf data and then we augment it by allowing
2324 lists and dictionaries of anything we can generate as JSON data.
2325 The size control of this works by limiting the maximum number of values
2327 that can be drawn from the base strategy. So for example if we wanted
2328 to only generate really small JSON we could do this as:
2329 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2331 >>> small_lists.example()
2332 True
2333 >>> small_lists.example()
2334 [False]
2335 Composite strategies
2337 The @composite decorator lets you combine other strategies in more or
2338 less arbitrary ways. It's probably the main thing you'll want to use
2339 for complicated custom strategies.
2340 The composite decorator works by converting a function that returns one
2342 example into a function that returns a strategy that produces such ex‐
2343 amples - which you can pass to @given, modify with .map or .filter, and
2344 generally use like any other strategy.
2345 It does this by giving you a special function draw as the first argu‐
2347 ment, which can be used just like the corresponding method of the
2348 data() strategy within a test. In fact, the implementation is almost
2349 the same - but defining a strategy with @composite makes code reuse
2350 easier, and usually improves the display of failing examples.
2351 For example, the following gives you a list and an index into it:
2353 >>> @composite
2355 ... def list_and_index(draw, elements=integers()):
2356 ... xs = draw(lists(elements, min_size=1))
2357 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2358 ... return (xs, i)
2359 draw(s) is a function that should be thought of as returning s.exam‐
2361 ple(), except that the result is reproducible and will minimize cor‐
2362 rectly. The decorated function has the initial argument removed from
2363 the list, but will accept all the others in the expected order. De‐
2364 faults are preserved.
2365 >>> list_and_index()
2367 list_and_index()
2368 >>> list_and_index().example()
2369 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2370 >>> list_and_index(booleans())
2372 list_and_index(elements=booleans())
2373 >>> list_and_index(booleans()).example()
2374 ([True, False], 0)
2375 Note that the repr will work exactly like it does for all the built-in
2377 strategies: it will be a function that you can call to get the strategy
2378 in question, with values provided only if they do not match the de‐
2379 faults.
2380 You can use assume inside composite functions:
2382 @composite
2384 def distinct_strings_with_common_characters(draw):
2385 x = draw(text(min_size=1))
2386 y = draw(text(alphabet=x))
2387 assume(x != y)
2388 return (x, y)
2389 This works as assume normally would, filtering out any examples for
2391 which the passed in argument is falsey.
2392 Take care that your function can cope with adversarial draws, or ex‐
2394 plicitly rejects them using the .filter() method or assume() - our mu‐
2395 tation and shrinking logic can do some strange things, and a naive im‐
2396 plementation might lead to serious performance problems. For example:
2397 @composite
2399 def reimplementing_sets_strategy(draw, elements=st.integers(), size=5):
2400 # The bad way: if Hypothesis keeps generating e.g. zero,
2401 # we'll keep looping for a very long time.
2402 result = set()
2403 while len(result) < size:
2404 result.add(draw(elements))
2405 # The good way: use a filter, so Hypothesis can tell what's valid!
2406 for _ in range(size):
2407 result.add(draw(elements.filter(lambda x: x not in result)))
2408 return result
2409 If @composite is used to decorate a method or classmethod, the draw ar‐
2411 gument must come before self or cls. While we therefore recommend
2412 writing strategies as standalone functions and using the
2413 register_type_strategy() function to associate them with a class, meth‐
2414 ods are supported and the @composite decorator may be applied either
2415 before or after @classmethod or @staticmethod. See issue #2578 and
2416 pull request #2634 for more details.
2417 Drawing interactively in tests
2419 There is also the data() strategy, which gives you a means of using
2420 strategies interactively. Rather than having to specify everything up
2421 front in @given you can draw from strategies in the body of your test.
2422 This is similar to @composite, but even more powerful as it allows you
2424 to mix test code with example generation. The downside of this power
2425 is that data() is incompatible with explicit @example(...)s - and the
2426 mixed code is often harder to debug when something goes wrong.
2427 If you need values that are affected by previous draws but which don't
2429 depend on the execution of your test, stick to the simpler @composite.
2430 @given(data())
2432 def test_draw_sequentially(data):
2433 x = data.draw(integers())
2434 y = data.draw(integers(min_value=x))
2435 assert x < y
2436 If the test fails, each draw will be printed with the falsifying exam‐
2438 ple. e.g. the above is wrong (it has a boundary condition error), so
2439 will print:
2440 Falsifying example: test_draw_sequentially(data=data(...))
2442 Draw 1: 0
2443 Draw 2: 0
2444 As you can see, data drawn this way is simplified as usual.
2446 Optionally, you can provide a label to identify values generated by
2448 each call to data.draw(). These labels can be used to identify values
2449 in the output of a falsifying example.
2450 For instance:
2452 @given(data())
2454 def test_draw_sequentially(data):
2455 x = data.draw(integers(), label="First number")
2456 y = data.draw(integers(min_value=x), label="Second number")
2457 assert x < y
2458 will produce the output:
2460 Falsifying example: test_draw_sequentially(data=data(...))
2462 Draw 1 (First number): 0
2463 Draw 2 (Second number): 0
2464
2466 Hypothesis has minimal dependencies, to maximise compatibility and make
2467 installing Hypothesis as easy as possible.
2468 Our integrations with specific packages are therefore provided by extra
2470 modules that need their individual dependencies installed in order to
2471 work. You can install these dependencies using the setuptools extra
2472 feature as e.g. pip install hypothesis[django]. This will check in‐
2473 stallation of compatible versions.
2474 You can also just install hypothesis into a project using them, ignore
2476 the version constraints, and hope for the best.
2477 In general "Which version is Hypothesis compatible with?" is a hard
2479 question to answer and even harder to regularly test. Hypothesis is al‐
2480 ways tested against the latest compatible version and each package will
2481 note the expected compatibility range. If you run into a bug with any
2482 of these please specify the dependency version.
2483 There are separate pages for django and numpy.
2485 hypothesis[cli]
2487 $ hypothesis --help
2488 Usage: hypothesis [OPTIONS] COMMAND [ARGS]...
2489 Options:
2491 --version Show the version and exit.
2492 -h, --help Show this message and exit.
2493 Commands:
2495 fuzz [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
2496 write `hypothesis write` writes property-based tests for you!
2497 This module requires the click package, and provides Hypothesis' com‐
2499 mand-line interface, for e.g. 'ghostwriting' tests via the terminal.
2500 It's also where HypoFuzz adds the hypothesis fuzz command (learn more
2501 about that here).
2502 hypothesis[lark]
2504 This extra can be used to generate strings matching any context-free
2505 grammar, using the Lark parser library.
2506 It currently only supports Lark's native EBNF syntax, but we plan to
2508 extend this to support other common syntaxes such as ANTLR and RFC 5234
2509 ABNF. Lark already supports loading grammars from nearley.js, so you
2510 may not have to write your own at all.
2511 Note that as Lark is at version 0.x, this module may break API compati‐
2513 bility in minor releases if supporting the latest version of Lark would
2514 otherwise be infeasible. We may also be quite aggressive in bumping
2515 the minimum version of Lark, unless someone volunteers to either fund
2516 or do the maintainence.
2517 hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None)
2519 A strategy for strings accepted by the given context-free gram‐
2520 mar.
2521 grammar must be a Lark object, which wraps an EBNF specifica‐
2523 tion. The Lark EBNF grammar reference can be found here.
2524 from_lark will automatically generate strings matching the non‐
2526 terminal start symbol in the grammar, which was supplied as an
2527 argument to the Lark class. To generate strings matching a dif‐
2528 ferent symbol, including terminals, you can override this by
2529 passing the start argument to from_lark. Note that Lark may re‐
2530 move unreachable productions when the grammar is compiled, so
2531 you should probably pass the same value for start to both.
2532 Currently from_lark does not support grammars that need custom
2534 lexing. Any lexers will be ignored, and any undefined terminals
2535 from the use of %declare will result in generation errors. To
2536 define strategies for such terminals, pass a dictionary mapping
2537 their name to a corresponding strategy as the explicit argument.
2538 The hypothesmith project includes a strategy for Python source,
2540 based on a grammar and careful post-processing.
2541 Example grammars, which may provide a useful starting point for your
2543 tests, can be found in the Lark repository and in this third-party col‐
2544 lection.
2545 hypothesis[pytz]
2547 This module provides pytz timezones.
2548 You can use this strategy to make hypothesis.strategies.datetimes() and
2550 hypothesis.strategies.times() produce timezone-aware values.
2551 hypothesis.extra.pytz.timezones()
2553 Any timezone in the Olsen database, as a pytz tzinfo object.
2554 This strategy minimises to UTC, or the smallest possible fixed
2556 offset, and is designed for use with hypothesis.strategies.date‐
2557 times().
2558 hypothesis[dateutil]
2560 This module provides dateutil timezones.
2561 You can use this strategy to make datetimes() and times() produce time‐
2563 zone-aware values.
2564 hypothesis.extra.dateutil.timezones()
2566 Any timezone from dateutil.
2567 This strategy minimises to UTC, or the timezone with the small‐
2569 est offset from UTC as of 2000-01-01, and is designed for use
2570 with datetimes().
2571 Note that the timezones generated by the strategy may vary de‐
2573 pending on the configuration of your machine. See the dateutil
2574 documentation for more information.
2575
2577 Writing tests with Hypothesis frees you from the tedium of deciding on
2578 and writing out specific inputs to test. Now, the hypothesis.ex‐
2579 tra.ghostwriter module can write your test functions for you too!
2580 The idea is to provide an easy way to start property-based testing, and
2582 a seamless transition to more complex test code - because ghostwritten
2583 tests are source code that you could have written for yourself.
2584 So just pick a function you'd like tested, and feed it to one of the
2586 functions below. They follow imports, use but do not require type an‐
2587 notations, and generally do their best to write you a useful test. You
2588 can also use our command-line interface:
2589 $ hypothesis write --help
2591 Usage: hypothesis write [OPTIONS] FUNC...
2592 `hypothesis write` writes property-based tests for you!
2594 Type annotations are helpful but not required for our advanced
2596 introspection and templating logic. Try running the examples below to see
2597 how it works:
2598 hypothesis write gzip
2600 hypothesis write numpy.matmul
2601 hypothesis write re.compile --except re.error
2602 hypothesis write --equivalent ast.literal_eval eval
2603 hypothesis write --roundtrip json.dumps json.loads
2604 hypothesis write --style=unittest --idempotent sorted
2605 hypothesis write --binary-op operator.add
2606 Options:
2608 --roundtrip start by testing write/read or encode/decode!
2609 --equivalent very useful when optimising or refactoring code
2610 --idempotent
2611 --binary-op
2612 --style [pytest|unittest] pytest-style function, or unittest-style method?
2613 -e, --except OBJ_NAME dotted name of exception(s) to ignore
2614 -h, --help Show this message and exit.
2615 NOTE:
2617 The ghostwriter requires black, but the generated code only requires
2618 Hypothesis itself.
2619 NOTE:
2621 Legal questions? While the ghostwriter fragments and logic is under
2622 the MPL-2.0 license like the rest of Hypothesis, the output from the
2623 ghostwriter is made available under the Creative Commons Zero (CC0)
2624 public domain dedication, so you can use it without any restric‐
2625 tions.
2626 hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(),
2628 style='pytest')
2629 Guess which ghostwriters to use, for a module or collection of
2630 functions.
2631 As for all ghostwriters, the except_ argument should be an
2633 python:Exception or tuple of exceptions, and style may be either
2634 "pytest" to write test functions or "unittest" to write test
2635 methods and TestCase.
2636 After finding the public functions attached to any modules, the
2638 magic ghostwriter looks for pairs of functions to pass to
2639 roundtrip(), then checks for binary_operation() and ufunc()
2640 functions, and any others are passed to fuzz().
2641 For example, try hypothesis write gzip on the command line!
2643 hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest')
2645 Write source code for a property-based test of func.
2646 The resulting test checks that valid input only leads to ex‐
2648 pected exceptions. For example:
2649 from re import compile, error
2651 from hypothesis.extra import ghostwriter
2652 ghostwriter.fuzz(compile, except_=error)
2654 Gives:
2656 # This test code was written by the `hypothesis.extra.ghostwriter` module
2658 # and is provided under the Creative Commons Zero public domain dedication.
2659 import re
2660 from hypothesis import given, reject, strategies as st
2661 # TODO: replace st.nothing() with an appropriate strategy
2663 @given(pattern=st.nothing(), flags=st.just(0))
2665 def test_fuzz_compile(pattern, flags):
2666 try:
2667 re.compile(pattern=pattern, flags=flags)
2668 except re.error:
2669 reject()
2670 Note that it includes all the required imports. Because the
2672 pattern parameter doesn't have annotations or a default argu‐
2673 ment, you'll need to specify a strategy - for example text() or
2674 binary(). After that, you have a test!
2675 hypothesis.extra.ghostwriter.idempotent(func, *, except_=(),
2677 style='pytest')
2678 Write source code for a property-based test of func.
2679 The resulting test checks that if you call func on it's own out‐
2681 put, the result does not change. For example:
2682 from typing import Sequence
2684 from hypothesis.extra import ghostwriter
2685 def timsort(seq: Sequence[int]) -> Sequence[int]:
2687 return sorted(seq)
2688 ghostwriter.idempotent(timsort)
2690 Gives:
2692 # This test code was written by the `hypothesis.extra.ghostwriter` module
2694 # and is provided under the Creative Commons Zero public domain dedication.
2695 from hypothesis import given, strategies as st
2697 @given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
2699 def test_idempotent_timsort(seq):
2700 result = timsort(seq=seq)
2701 repeat = timsort(seq=result)
2702 assert result == repeat, (result, repeat)
2703 hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(),
2705 style='pytest')
2706 Write source code for a property-based test of funcs.
2707 The resulting test checks that if you call the first function,
2709 pass the result to the second (and so on), the final result is
2710 equal to the first input argument.
2711 This is a very powerful property to test, especially when the
2713 config options are varied along with the object to round-trip.
2714 For example, try ghostwriting a test for python:json.dumps() -
2715 would you have thought of all that?
2716 hypothesis write --roundtrip json.dumps json.loads
2718 hypothesis.extra.ghostwriter.equivalent(*funcs, except_=(),
2720 style='pytest')
2721 Write source code for a property-based test of funcs.
2722 The resulting test checks that calling each of the functions
2724 gives the same result. This can be used as a classic 'oracle',
2725 such as testing a fast sorting algorithm against the
2726 python:sorted() builtin, or for differential testing where none
2727 of the compared functions are fully trusted but any difference
2728 indicates a bug (e.g. running a function on different numbers of
2729 threads, or simply multiple times).
2730 The functions should have reasonably similar signatures, as only
2732 the common parameters will be passed the same arguments - any
2733 other parameters will be allowed to vary.
2734 hypothesis.extra.ghostwriter.binary_operation(func, *, associa‐
2736 tive=True, commutative=True, identity=infer, distributes_over=None, ex‐
2737 cept_=(), style='pytest')
2738 Write property tests for the binary operation func.
2739 While binary operations are not particularly common, they have
2741 such nice properties to test that it seems a shame not to demon‐
2742 strate them with a ghostwriter. For an operator f, test that:
2743 • if associative, f(a, f(b, c)) == f(f(a, b), c)
2745 • if commutative, f(a, b) == f(b, a)
2747 • if identity is not None, f(a, identity) == a
2749 • if distributes_over is +, f(a, b) + f(a, c) == f(a, b+c)
2751 For example:
2753 ghostwriter.binary_operation(
2755 operator.mul,
2756 identity=1,
2757 distributes_over=operator.add,
2758 style="unittest",
2759 )
2760 hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest')
2762 Write a property-based test for the array unfunc func.
2763 The resulting test checks that your ufunc or gufunc has the ex‐
2765 pected broadcasting and dtype casting behaviour. You will prob‐
2766 ably want to add extra assertions, but as with the other ghost‐
2767 writers this gives you a great place to start.
2768 hypothesis write numpy.matmul
2770
2772 Hypothesis offers a number of features specific for Django testing,
2773 available in the hypothesis[django] extra. This is tested against each
2774 supported series with mainstream or extended support - if you're still
2775 getting security patches, you can test with Hypothesis.
2776 class hypothesis.extra.django.TestCase
2778 Using it is quite straightforward: All you need to do is subclass
2780 hypothesis.extra.django.TestCase or
2781 hypothesis.extra.django.TransactionTestCase and you can use @given as
2782 normal, and the transactions will be per example rather than per test
2783 function as they would be if you used @given with a normal django test
2784 suite (this is important because your test function will be called mul‐
2785 tiple times and you don't want them to interfere with each other). Test
2786 cases on these classes that do not use @given will be run as normal.
2787 class hypothesis.extra.django.TransactionTestCase
2789 We recommend avoiding TransactionTestCase unless you really have to run
2791 each test case in a database transaction. Because Hypothesis runs this
2792 in a loop, the performance problems it normally has are significantly
2793 exacerbated and your tests will be really slow. If you are using
2794 TransactionTestCase, you may need to use @settings(sup‐
2795 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2796 example generation.
2797 Having set up a test class, you can now pass @given a strategy for
2799 Django models:
2800 For example, using the trivial django project we have for testing:
2802 >>> from hypothesis.extra.django import from_model
2804 >>> from toystore.models import Customer
2805 >>> c = from_model(Customer).example()
2806 >>> c
2807 <Customer: Customer object>
2808 >>> c.email
2809 'jaime.urbina@gmail.com'
2810 >>> c.name
2811 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2812 >>> c.age
2813 -873375803
2814 Hypothesis has just created this with whatever the relevant type of
2816 data is.
2817 Obviously the customer's age is implausible, which is only possible be‐
2819 cause we have not used (eg) MinValueValidator to set the valid range
2820 for this field (or used a PositiveSmallIntegerField, which would only
2821 need a maximum value validator).
2822 If you do have validators attached, Hypothesis will only generate exam‐
2824 ples that pass validation. Sometimes that will mean that we fail a
2825 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2826 egy to skip validation at the strategy level:
2827 NOTE:
2829 Inference from validators will be much more powerful when issue
2830 #1116 is implemented, but there will always be some edge cases that
2831 require you to pass an explicit strategy.
2832 >>> from hypothesis.strategies import integers
2834 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2835 >>> c
2836 <Customer: Customer object>
2837 >>> c.age
2838 5
2839 Tips and tricks
2841 Custom field types
2842 If you have a custom Django field type you can register it with Hypoth‐
2843 esis's model deriving functionality by registering a default strategy
2844 for it:
2845 >>> from toystore.models import CustomishField, Customish
2847 >>> from_model(Customish).example()
2848 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2849 customish for model Customish
2850 >>> from hypothesis.extra.django import register_field_strategy
2851 >>> from hypothesis.strategies import just
2852 >>> register_field_strategy(CustomishField, just("hi"))
2853 >>> x = from_model(Customish).example()
2854 >>> x.customish
2855 'hi'
2856 Note that this mapping is on exact type. Subtypes will not inherit it.
2858 Generating child models
2860 For the moment there's no explicit support in hypothesis-django for
2861 generating dependent models. i.e. a Company model will generate no
2862 Shops. However if you want to generate some dependent models as well,
2863 you can emulate this by using the flatmap function as follows:
2864 from hypothesis.strategies import lists, just
2866 def generate_with_shops(company):
2869 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
2870 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
2873 Let's unpack what this is doing:
2875 The way flatmap works is that we draw a value from the original strat‐
2877 egy, then apply a function to it which gives us a new strategy. We then
2878 draw a value from that strategy. So in this case we're first drawing a
2879 company, and then we're drawing a list of shops belonging to that com‐
2880 pany: The just strategy is a strategy such that drawing it always pro‐
2881 duces the individual value, so from_model(Shop, company=just(company))
2882 is a strategy that generates a Shop belonging to the original company.
2883 So the following code would give us a list of shops all belonging to
2885 the same company:
2886 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
2888 The only difference from this and the above is that we want the com‐
2890 pany, not the shops. This is where the inner map comes in. We build the
2891 list of shops and then throw it away, instead returning the company we
2892 started for. This works because the models that Hypothesis generates
2893 are saved in the database, so we're essentially running the inner
2894 strategy purely for the side effect of creating those children in the
2895 database.
2896 Generating primary key values
2898 If your model includes a custom primary key that you want to generate
2899 using a strategy (rather than a default auto-increment primary key)
2900 then Hypothesis has to deal with the possibility of a duplicate primary
2901 key.
2902 If a model strategy generates a value for the primary key field, Hy‐
2904 pothesis will create the model instance with update_or_create(), over‐
2905 writing any existing instance in the database for this test case with
2906 the same primary key.
2907 On the subject of MultiValueField
2909 Django forms feature the MultiValueField which allows for several
2910 fields to be combined under a single named field, the default example
2911 of this is the SplitDateTimeField.
2912 class CustomerForm(forms.Form):
2914 name = forms.CharField()
2915 birth_date_time = forms.SplitDateTimeField()
2916 from_form supports MultiValueField subclasses directly, however if you
2918 want to define your own strategy be forewarned that Django binds data
2919 for a MultiValueField in a peculiar way. Specifically each sub-field is
2920 expected to have its own entry in data addressed by the field name
2921 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
2922 ValueField, so form data for the example above might look like this:
2923 {
2925 "name": "Samuel John",
2926 "birth_date_time_0": "2018-05-19", # the date, as the first sub-field
2927 "birth_date_time_1": "15:18:00", # the time, as the second sub-field
2928 }
2929 Thus, if you want to define your own strategies for such a field you
2931 must address your sub-fields appropriately:
2932 from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))
2934
2936 numpy
2937 Hypothesis offers a number of strategies for NumPy testing, available
2938 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
2939 package.
2940 The centerpiece is the arrays() strategy, which generates arrays with
2942 any dtype, shape, and contents you can specify or give a strategy for.
2943 To make this as useful as possible, strategies are provided to generate
2944 array shapes and generate all kinds of fixed-size or compound dtypes.
2945 hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0,
2947 max_size=None, min_value=None, max_value=None, allow_nan=None, al‐
2948 low_infinity=None, exclude_min=None, exclude_max=None)
2949 Creates a strategy which can generate any value of the given
2950 dtype.
2951 Compatible **kwargs are passed to the inferred strategy function
2953 for integers, floats, and strings. This allows you to customise
2954 the min and max values, control the length or contents of
2955 strings, or exclude non-finite numbers. This is particularly
2956 useful when kwargs are passed through from arrays() which allow
2957 a variety of numeric dtypes, as it seamlessly handles the width
2958 or representable bounds for you. See issue #2552 for more de‐
2959 tail.
2960 hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None,
2962 fill=None, unique=False)
2963 Returns a strategy for generating numpy:numpy.ndarrays.
2964 • dtype may be any valid input to dtype (this includes dtype ob‐
2966 jects), or a strategy that generates such values.
2967 • shape may be an integer >= 0, a tuple of such integers, or a
2969 strategy that generates such values.
2970 • elements is a strategy for generating values to put in the ar‐
2972 ray. If it is None a suitable value will be inferred based on
2973 the dtype, which may give any legal value (including eg NaN
2974 for floats). If a mapping, it will be passed as **kwargs to
2975 from_dtype()
2976 • fill is a strategy that may be used to generate a single back‐
2978 ground value for the array. If None, a suitable default will
2979 be inferred based on the other arguments. If set to nothing()
2980 then filling behaviour will be disabled entirely and every el‐
2981 ement will be generated independently.
2982 • unique specifies if the elements of the array should all be
2984 distinct from one another. Note that in this case multiple NaN
2985 values may still be allowed. If fill is also set, the only
2986 valid values for it to return are NaN values (anything for
2987 which numpy:numpy.isnan returns True. So e.g. for complex num‐
2988 bers (nan+1j) is also a valid fill). Note that if unique is
2989 set to True the generated values must be hashable.
2990 Arrays of specified dtype and shape are generated for example
2992 like this:
2993 >>> import numpy as np
2995 >>> arrays(np.int8, (2, 3)).example()
2996 array([[-8, 6, 3],
2997 [-6, 4, 6]], dtype=int8)
2998 • See What you can generate and how.
3000 >>> import numpy as np
3002 >>> from hypothesis.strategies import floats
3003 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
3004 array([ 0.88974794, 0.77387938, 0.1977879 ])
3005 Array values are generated in two parts:
3007 1. Some subset of the coordinates of the array are populated
3009 with a value drawn from the elements strategy (or its in‐
3010 ferred form).
3011 2. If any coordinates were not assigned in the previous step, a
3013 single value is drawn from the fill strategy and is assigned
3014 to all remaining places.
3015 You can set fill to nothing() if you want to disable this behav‐
3017 iour and draw a value for every element.
3018 If fill is set to None then it will attempt to infer the correct
3020 behaviour automatically: If unique is True, no filling will oc‐
3021 cur by default. Otherwise, if it looks safe to reuse the values
3022 of elements across multiple coordinates (this will be the case
3023 for any inferred strategy, and for most of the builtins, but is
3024 not the case for mutable values or strategies built with
3025 flatmap, map, composite, etc) then it will use the elements
3026 strategy as the fill, else it will default to having no fill.
3027 Having a fill helps Hypothesis craft high quality examples, but
3029 its main importance is when the array generated is large: Hy‐
3030 pothesis is primarily designed around testing small examples. If
3031 you have arrays with hundreds or more elements, having a fill
3032 value is essential if you want your tests to run in reasonable
3033 time.
3034 hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None,
3036 min_side=1, max_side=None)
3037 Return a strategy for array shapes (tuples of int >= 1).
3038 hypothesis.extra.numpy.scalar_dtypes()
3040 Return a strategy that can return any non-flexible scalar dtype.
3041 hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?',
3043 sizes=(8, 16, 32, 64))
3044 Return a strategy for unsigned integer dtypes.
3045 endianness may be < for little-endian, > for big-endian, = for
3047 native byte order, or ? to allow either byte order. This argu‐
3048 ment only applies to dtypes of more than one byte.
3049 sizes must be a collection of integer sizes in bits. The de‐
3051 fault (8, 16, 32, 64) covers the full range of sizes.
3052 hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16,
3054 32, 64))
3055 Return a strategy for signed integer dtypes.
3056 endianness and sizes are treated as for
3058 unsigned_integer_dtypes().
3059 hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16,
3061 32, 64))
3062 Return a strategy for floating-point dtypes.
3063 sizes is the size in bits of floating-point number. Some ma‐
3065 chines support 96- or 128-bit floats, but these are not gener‐
3066 ated by default.
3067 Larger floats (96 and 128 bit real parts) are not supported on
3069 all platforms and therefore disabled by default. To generate
3070 these dtypes, include these values in the sizes argument.
3071 hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?',
3073 sizes=(64, 128))
3074 Return a strategy for complex-number dtypes.
3075 sizes is the total size in bits of a complex number, which con‐
3077 sists of two floats. Complex halfs (a 16-bit real part) are not
3078 supported by numpy and will not be generated by this strategy.
3079 hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_pe‐
3081 riod='ns', endianness='?')
3082 Return a strategy for datetime64 dtypes, with various precisions
3083 from year to attosecond.
3084 hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_pe‐
3086 riod='ns', endianness='?')
3087 Return a strategy for timedelta64 dtypes, with various preci‐
3088 sions from year to attosecond.
3089 hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1,
3091 max_len=16)
3092 Return a strategy for generating bytestring dtypes, of various
3093 lengths and byteorder.
3094 While Hypothesis' string strategies can generate empty strings,
3096 string dtypes with length 0 indicate that size is still to be
3097 determined, so the minimum length for string dtypes is 1.
3098 hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?',
3100 min_len=1, max_len=16)
3101 Return a strategy for generating unicode string dtypes, of vari‐
3102 ous lengths and byteorder.
3103 While Hypothesis' string strategies can generate empty strings,
3105 string dtypes with length 0 indicate that size is still to be
3106 determined, so the minimum length for string dtypes is 1.
3107 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
3109 *, min_size=1, max_size=5, allow_subarrays=False)
3110 Return a strategy for generating array (compound) dtypes, with
3111 members drawn from the given subtype strategy.
3112 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
3114 *, max_leaves=10, max_itemsize=None)
3115 Return the most-general dtype strategy.
3116 Elements drawn from this strategy may be simple (from the sub‐
3118 type_strategy), or several such values drawn from array_dtypes()
3119 with allow_subarrays=True. Subdtypes in an array dtype may be
3120 nested to any depth, subject to the max_leaves argument.
3121 hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0,
3123 max_size=None)
3124 Return a strategy for generating permissible tuple-values for
3125 the axis argument for a numpy sequential function (e.g.
3126 numpy:numpy.sum()), given an array of the specified dimensional‐
3127 ity.
3128 All tuples will have an length >= min_size and <= max_size. The
3130 default value for max_size is ndim.
3131 Examples from this strategy shrink towards an empty tuple, which
3133 render most sequential functions as no-ops.
3134 The following are some examples drawn from this strategy.
3136 >>> [valid_tuple_axes(3).example() for i in range(4)]
3138 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
3139 valid_tuple_axes can be joined with other strategies to generate
3141 any type of valid axis object, i.e. integers, tuples, and None:
3142 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
3144 hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0,
3146 max_dims=None, min_side=1, max_side=None)
3147 Return a strategy for generating shapes that are broadcast-com‐
3148 patible with the provided shape.
3149 Examples from this strategy shrink towards a shape with length
3151 min_dims. The size of an aligned dimension shrinks towards size
3152 1. The size of an unaligned dimension shrink towards min_side.
3153 • shape a tuple of integers
3155 • min_dims The smallest length that the generated shape can pos‐
3157 sess.
3158 • max_dims The largest length that the generated shape can pos‐
3160 sess. The default-value for max_dims is min(32,
3161 max(len(shape), min_dims) + 2).
3162 • min_side The smallest size that an unaligned dimension can
3164 possess.
3165 • max_side The largest size that an unaligned dimension can pos‐
3167 sess. The default value is 2 + 'size-of-largest-aligned-di‐
3168 mension'.
3169 The following are some examples drawn from this strategy.
3171 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
3173 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
3174 hypothesis.extra.numpy.mutually_broadcastable_shapes(*,
3176 num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0,
3177 max_dims=None, min_side=1, max_side=None)
3178 Return a strategy for generating a specified number of shapes,
3179 N, that are mutually-broadcastable with one another and with the
3180 provided "base-shape".
3181 The strategy will generate a named-tuple of:
3183 • input_shapes: the N generated shapes
3185 • result_shape: the resulting shape, produced by broadcasting
3187 the N shapes with the base-shape
3188 Each shape produced from this strategy shrinks towards a shape
3190 with length min_dims. The size of an aligned dimension shrinks
3191 towards being having a size of 1. The size of an unaligned di‐
3192 mension shrink towards min_side.
3193 • num_shapes The number of mutually broadcast-compatible shapes
3195 to generate.
3196 • base-shape The shape against which all generated shapes can
3198 broadcast. The default shape is empty, which corresponds to a
3199 scalar and thus does not constrain broadcasting at all.
3200 • min_dims The smallest length that any generated shape can pos‐
3202 sess.
3203 • max_dims The largest length that any generated shape can pos‐
3205 sess. It cannot exceed 32, which is the greatest supported
3206 dimensionality for a numpy array. The default-value for
3207 max_dims is 2 + max(len(shape), min_dims), capped at 32.
3208 • min_side The smallest size that an unaligned dimension can
3210 possess.
3211 • max_side The largest size that an unaligned dimension can pos‐
3213 sess. The default value is 2 + 'size-of-largest-aligned-di‐
3214 mension'.
3215 The following are some examples drawn from this strategy.
3217 >>> # Draw three shapes, and each shape is broadcast-compatible with `(2, 3)`
3219 >>> for _ in range(5):
3220 ... mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3)).example()
3221 BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
3222 BroadcastableShapes(input_shapes=((3,), (1,), (2, 1)), result_shape=(2, 3))
3223 BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
3224 BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
3225 BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))
3226 Use with Generalised Universal Function signatures
3228 A universal function (or ufunc for short) is a function that op‐
3230 erates on ndarrays in an element-by-element fashion, supporting
3231 array broadcasting, type casting, and several other standard
3232 features. A generalised ufunc operates on sub-arrays rather
3233 than elements, based on the "signature" of the function. Com‐
3234 pare e.g. numpy:numpy.add (ufunc) to numpy:numpy.matmul (gu‐
3235 func).
3236 To generate shapes for a gufunc, you can pass the signature ar‐
3238 gument instead of num_shapes. This must be a gufunc signature
3239 string; which you can write by hand or access as e.g. np.mat‐
3240 mul.signature on generalised ufuncs.
3241 In this case, the side arguments are applied to the 'core dimen‐
3243 sions' as well, ignoring any frozen dimensions. base_shape and
3244 the dims arguments are applied to the 'loop dimensions', and if
3245 necessary, the dimensionality of each shape is silently capped
3246 to respect the 32-dimension limit.
3247 The generated result_shape is the real result shape of applying
3249 the gufunc to arrays of the generated input_shapes, even where
3250 this is different to broadcasting the loop dimensions.
3251 gufunc-compatible shapes shrink their loop dimensions as above,
3253 towards omitting optional core dimensions, and smaller-size core
3254 dimensions.
3255 >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
3257 >>> for _ in range(3):
3258 ... mutually_broadcastable_shapes(signature=np.matmul.signature).example()
3259 BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
3260 BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
3261 BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))
3262 hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0,
3264 max_dims=None, allow_newaxis=False, allow_ellipsis=True)
3265 The basic_indices strategy generates basic indexes for arrays
3266 of the specified shape, which may include dimensions of size
3267 zero.
3268 It generates tuples containing some mix of integers,
3270 python:slice objects, ... (Ellipsis), and numpy:numpy.newaxis;
3271 which when used to index a shape-shaped array will produce ei‐
3272 ther a scalar or a shared-memory view. When a length-one tuple
3273 would be generated, this strategy may instead return the element
3274 which will index the first axis, e.g. 5 instead of (5,).
3275 • shape: the array shape that will be indexed, as a tuple of in‐
3277 tegers >= 0. This must be at least two-dimensional for a tu‐
3278 ple to be a valid basic index; for one-dimensional arrays use
3279 slices() instead.
3280 • min_dims: the minimum dimensionality of the resulting view
3282 from use of the generated index. When min_dims == 0, scalars
3283 and zero-dimensional arrays are both allowed.
3284 • max_dims: the maximum dimensionality of the resulting view.
3286 If not specified, it defaults to max(len(shape), min_dims) +
3287 2.
3288 • allow_ellipsis: whether ...` is allowed in the index.
3290 • allow_newaxis: whether numpy:numpy.newaxis is allowed in the
3292 index.
3293 Note that the length of the generated tuple may be anywhere be‐
3295 tween zero and min_dims. It may not match the length of shape,
3296 or even the dimensionality of the array view resulting from its
3297 use!
3298 hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=ar‐
3300 ray_shapes(), dtype='int')
3301 Return a search strategy for tuples of integer-arrays that, when
3302 used to index into an array of shape shape, given an array whose
3303 shape was drawn from result_shape.
3304 Examples from this strategy shrink towards the tuple of in‐
3306 dex-arrays:
3307 len(shape) * (np.zeros(drawn_result_shape, dtype), )
3309 • shape a tuple of integers that indicates the shape of the ar‐
3311 ray, whose indices are being generated.
3312 • result_shape a strategy for generating tuples of integers,
3314 which describe the shape of the resulting index arrays. The
3315 default is array_shapes(). The shape drawn from this strategy
3316 determines the shape of the array that will be produced when
3317 the corresponding example from integer_array_indices is used
3318 as an index.
3319 • dtype the integer data type of the generated index-arrays.
3321 Negative integer indices can be generated if a signed integer
3322 type is specified.
3323 Recall that an array can be indexed using a tuple of integer-ar‐
3325 rays to access its members in an arbitrary order, producing an
3326 array with an arbitrary shape. For example:
3327 >>> from numpy import array
3329 >>> x = array([-0, -1, -2, -3, -4])
3330 >>> ind = (array([[4, 0], [0, 1]]),) # a tuple containing a 2D integer-array
3331 >>> x[ind] # the resulting array is commensurate with the indexing array(s)
3332 array([[-4, 0],
3333 [0, -1]])
3334 Note that this strategy does not accommodate all variations of
3336 so-called 'advanced indexing', as prescribed by NumPy's nomen‐
3337 clature. Combinations of basic and advanced indexes are too
3338 complex to usefully define in a standard strategy; we leave ap‐
3339 plication-specific strategies to the user. Advanced-boolean in‐
3340 dexing can be defined as arrays(shape=..., dtype=bool), and is
3341 similarly left to the user.
3342 pandas
3344 Hypothesis provides strategies for several of the core pandas data
3345 types: pandas.Index, pandas.Series and pandas.DataFrame.
3346 The general approach taken by the pandas module is that there are mul‐
3348 tiple strategies for generating indexes, and all of the other strate‐
3349 gies take the number of entries they contain from their index strategy
3350 (with sensible defaults). So e.g. a Series is specified by specifying
3351 its numpy.dtype (and/or a strategy for generating elements for it).
3352 hypothesis.extra.pandas.indexes(*, elements=None, dtype=None,
3354 min_size=0, max_size=None, unique=True)
3355 Provides a strategy for producing a pandas.Index.
3356 Arguments:
3358 • elements is a strategy which will be used to generate the in‐
3360 dividual values of the index. If None, it will be inferred
3361 from the dtype. Note: even if the elements strategy produces
3362 tuples, the generated value will not be a MultiIndex, but in‐
3363 stead be a normal index whose elements are tuples.
3364 • dtype is the dtype of the resulting index. If None, it will be
3366 inferred from the elements strategy. At least one of dtype or
3367 elements must be provided.
3368 • min_size is the minimum number of elements in the index.
3370 • max_size is the maximum number of elements in the index. If
3372 None then it will default to a suitable small size. If you
3373 want larger indexes you should pass a max_size explicitly.
3374 • unique specifies whether all of the elements in the resulting
3376 index should be distinct.
3377 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
3379 Provides a strategy which generates an Index whose values are 0,
3380 1, ..., n for some n.
3381 Arguments:
3383 • min_size is the smallest number of elements the index can
3385 have.
3386 • max_size is the largest number of elements the index can have.
3388 If None it will default to some suitable value based on
3389 min_size.
3390 hypothesis.extra.pandas.series(*, elements=None, dtype=None, in‐
3392 dex=None, fill=None, unique=False)
3393 Provides a strategy for producing a pandas.Series.
3394 Arguments:
3396 • elements: a strategy that will be used to generate the indi‐
3398 vidual values in the series. If None, we will attempt to infer
3399 a suitable default from the dtype.
3400 • dtype: the dtype of the resulting series and may be any value
3402 that can be passed to numpy.dtype. If None, will use pandas's
3403 standard behaviour to infer it from the type of the elements
3404 values. Note that if the type of values that comes out of your
3405 elements strategy varies, then so will the resulting dtype of
3406 the series.
3407 • index: If not None, a strategy for generating indexes for the
3409 resulting Series. This can generate either pandas.Index ob‐
3410 jects or any sequence of values (which will be passed to the
3411 Index constructor).
3412 You will probably find it most convenient to use the indexes()
3414 or range_indexes() function to produce values for this argu‐
3415 ment.
3416 Usage:
3418 >>> series(dtype=int).example()
3420 0 -2001747478
3421 1 1153062837
3422 class hypothesis.extra.pandas.column(name=None, elements=None,
3424 dtype=None, fill=None, unique=False)
3425 Data object for describing a column in a DataFrame.
3426 Arguments:
3428 • name: the column name, or None to default to the column posi‐
3430 tion. Must be hashable, but can otherwise be any value sup‐
3431 ported as a pandas column name.
3432 • elements: the strategy for generating values in this column,
3434 or None to infer it from the dtype.
3435 • dtype: the dtype of the column, or None to infer it from the
3437 element strategy. At least one of dtype or elements must be
3438 provided.
3439 • fill: A default value for elements of the column. See arrays()
3441 for a full explanation.
3442 • unique: If all values in this column should be distinct.
3444 hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, ele‐
3446 ments=None, fill=None, unique=False)
3447 A convenience function for producing a list of column objects of
3448 the same general shape.
3449 The names_or_number argument is either a sequence of values, the
3451 elements of which will be used as the name for individual column
3452 objects, or a number, in which case that many unnamed columns
3453 will be created. All other arguments are passed through verbatim
3454 to create the columns.
3455 hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, in‐
3457 dex=None)
3458 Provides a strategy for producing a pandas.DataFrame.
3459 Arguments:
3461 • columns: An iterable of column objects describing the shape of
3463 the generated DataFrame.
3464 • rows: A strategy for generating a row object. Should generate
3466 either dicts mapping column names to values or a sequence map‐
3467 ping column position to the value in that position (note that
3468 unlike the pandas.DataFrame constructor, single values are not
3469 allowed here. Passing e.g. an integer is an error, even if
3470 there is only one column).
3471 At least one of rows and columns must be provided. If both are
3473 provided then the generated rows will be validated against the
3474 columns and an error will be raised if they don't match.
3475 Caveats on using rows:
3477 • In general you should prefer using columns to rows, and only
3479 use rows if the columns interface is insufficiently flexible
3480 to describe what you need - you will get better performance
3481 and example quality that way.
3482 • If you provide rows and not columns, then the shape and
3484 dtype of the resulting DataFrame may vary. e.g. if you have
3485 a mix of int and float in the values for one column in your
3486 row entries, the column will sometimes have an integral
3487 dtype and sometimes a float.
3488 • index: If not None, a strategy for generating indexes for the
3490 resulting DataFrame. This can generate either pandas.Index ob‐
3491 jects or any sequence of values (which will be passed to the
3492 Index constructor).
3493 You will probably find it most convenient to use the indexes()
3495 or range_indexes() function to produce values for this argu‐
3496 ment.
3497 Usage:
3499 The expected usage pattern is that you use column and columns()
3501 to specify a fixed shape of the DataFrame you want as follows.
3502 For example the following gives a two column data frame:
3503 >>> from hypothesis.extra.pandas import column, data_frames
3505 >>> data_frames([
3506 ... column('A', dtype=int), column('B', dtype=float)]).example()
3507 A B
3508 0 2021915903 1.793898e+232
3509 1 1146643993 inf
3510 2 -2096165693 1.000000e+07
3511 If you want the values in different columns to interact in some
3513 way you can use the rows argument. For example the following
3514 gives a two column DataFrame where the value in the first column
3515 is always at most the value in the second:
3516 >>> from hypothesis.extra.pandas import column, data_frames
3518 >>> import hypothesis.strategies as st
3519 >>> data_frames(
3520 ... rows=st.tuples(st.floats(allow_nan=False),
3521 ... st.floats(allow_nan=False)).map(sorted)
3522 ... ).example()
3523 0 1
3524 0 -3.402823e+38 9.007199e+15
3525 1 -1.562796e-298 5.000000e-01
3526 You can also combine the two:
3528 >>> from hypothesis.extra.pandas import columns, data_frames
3530 >>> import hypothesis.strategies as st
3531 >>> data_frames(
3532 ... columns=columns(["lo", "hi"], dtype=float),
3533 ... rows=st.tuples(st.floats(allow_nan=False),
3534 ... st.floats(allow_nan=False)).map(sorted)
3535 ... ).example()
3536 lo hi
3537 0 9.314723e-49 4.353037e+45
3538 1 -9.999900e-01 1.000000e+07
3539 2 -2.152861e+134 -1.069317e-73
3540 (Note that the column dtype must still be specified and will not
3542 be inferred from the rows. This restriction may be lifted in fu‐
3543 ture).
3544 Combining rows and columns has the following behaviour:
3546 • The column names and dtypes will be used.
3548 • If the column is required to be unique, this will be enforced.
3550 • Any values missing from the generated rows will be provided
3552 using the column's fill.
3553 • Any values in the row not present in the column specification
3555 (if dicts are passed, if there are keys with no corresponding
3556 column name, if sequences are passed if there are too many
3557 items) will result in InvalidArgument being raised.
3558 Supported versions
3560 There is quite a lot of variation between pandas versions. We only com‐
3561 mit to supporting the latest version of pandas, but older minor ver‐
3562 sions are supported on a "best effort" basis. Hypothesis is currently
3563 tested against and confirmed working with every Pandas minor version
3564 from 0.25 through to 1.1.
3565 Releases that are not the latest patch release of their minor version
3567 are not tested or officially supported, but will probably also work un‐
3568 less you hit a pandas bug.
3569
3571 Hypothesis tries to detect common mistakes and things that will cause
3572 difficulty at run time in the form of a number of 'health checks'.
3573 These include detecting and warning about:
3575 • Strategies with very slow data generation
3577 • Strategies which filter out too much
3579 • Recursive strategies which branch too much
3581 • Tests that are unlikely to complete in a reasonable amount of time.
3583 If any of these scenarios are detected, Hypothesis will emit a warning
3585 about them.
3586 The general goal of these health checks is to warn you about things
3588 that you are doing that might appear to work but will either cause Hy‐
3589 pothesis to not work correctly or to perform badly.
3590 To selectively disable health checks, use the suppress_health_check
3592 setting. The argument for this parameter is a list with elements drawn
3593 from any of the class-level attributes of the HealthCheck class. Using
3594 a value of HealthCheck.all() will disable all health checks.
3595 class hypothesis.HealthCheck(value)
3597 Arguments for suppress_health_check.
3598 Each member of this enum is a type of health check to suppress.
3600 data_too_large = 1
3602 Check for when the typical size of the examples you are
3603 generating exceeds the maximum allowed size too often.
3604 filter_too_much = 2
3606 Check for when the test is filtering out too many exam‐
3607 ples, either through use of assume() or filter(), or oc‐
3608 casionally for Hypothesis internal reasons.
3609 too_slow = 3
3611 Check for when your data generation is extremely slow and
3612 likely to hurt testing.
3613 return_value = 5
3615 Checks if your tests return a non-None value (which will
3616 be ignored and is unlikely to do what you want).
3617 large_base_example = 7
3619 Checks if the natural example to shrink towards is very
3620 large.
3621 not_a_test_method = 8
3623 Checks if @given has been applied to a method defined by
3624 python:unittest.TestCase (i.e. not a test).
3625 Deprecations
3627 We also use a range of custom exception and warning types, so you can
3628 see exactly where an error came from - or turn only our warnings into
3629 errors.
3630 class hypothesis.errors.HypothesisDeprecationWarning
3632 A deprecation warning issued by Hypothesis.
3633 Actually inherits from FutureWarning, because DeprecationWarning
3635 is hidden by the default warnings filter.
3636 You can configure the Python python:warnings to handle these
3638 warnings differently to others, either turning them into errors
3639 or suppressing them entirely. Obviously we would prefer the
3640 former!
3641 Deprecated features will be continue to emit warnings for at least six
3643 months, and then be removed in the following major release. Note how‐
3644 ever that not all warnings are subject to this grace period; sometimes
3645 we strengthen validation by adding a warning and these may become er‐
3646 rors immediately at a major release.
3647
3649 When Hypothesis finds a bug it stores enough information in its data‐
3650 base to reproduce it. This enables you to have a classic testing work‐
3651 flow of find a bug, fix a bug, and be confident that this is actually
3652 doing the right thing because Hypothesis will start by retrying the ex‐
3653 amples that broke things last time.
3654 Limitations
3656 The database is best thought of as a cache that you never need to in‐
3657 validate: Information may be lost when you upgrade a Hypothesis version
3658 or change your test, so you shouldn't rely on it for correctness - if
3659 there's an example you want to ensure occurs each time then there's a
3660 feature for including them in your source code - but it helps the de‐
3661 velopment workflow considerably by making sure that the examples you've
3662 just found are reproduced.
3663 The database also records examples that exercise less-used parts of
3665 your code, so the database may update even when no failing examples
3666 were found.
3667 Upgrading Hypothesis and changing your tests
3669 The design of the Hypothesis database is such that you can put arbi‐
3670 trary data in the database and not get wrong behaviour. When you up‐
3671 grade Hypothesis, old data might be invalidated, but this should happen
3672 transparently. It can never be the case that e.g. changing the strategy
3673 that generates an argument gives you data from the old strategy.
3674 ExampleDatabase implementations
3676 Hypothesis' default database setting creates a
3677 DirectoryBasedExampleDatabase in your current working directory, under
3678 .hypothesis/examples. If this location is unusable, e.g. because you
3679 do not have read or write permissions, Hypothesis will emit a warning
3680 and fall back to an InMemoryExampleDatabase.
3681 Hypothesis provides the following ExampleDatabase implementations:
3683 class hypothesis.database.InMemoryExampleDatabase
3685 A non-persistent example database, implemented in terms of a
3686 dict of sets.
3687 This can be useful if you call a test function several times in
3689 a single session, or for testing other database implementations,
3690 but because it does not persist between runs we do not recommend
3691 it for general use.
3692 class hypothesis.database.DirectoryBasedExampleDatabase(path)
3694 Use a directory to store Hypothesis examples as files.
3695 Each test corresponds to a directory, and each example to a file
3697 within that directory. While the contents are fairly opaque, a
3698 DirectoryBasedExampleDatabase can be shared by checking the di‐
3699 rectory into version control, for example with the following
3700 .gitignore:
3701 # Ignore files cached by Hypothesis...
3703 .hypothesis/*
3704 # except for the examples directory
3705 !.hypothesis/examples/
3706 Note however that this only makes sense if you also pin to an
3708 exact version of Hypothesis, and we would usually recommend im‐
3709 plementing a shared database with a network datastore - see
3710 ExampleDatabase, and the MultiplexedDatabase helper.
3711 class hypothesis.database.ReadOnlyDatabase(db)
3713 A wrapper to make the given database read-only.
3714 The implementation passes through fetch, and turns save, delete,
3716 and move into silent no-ops.
3717 Note that this disables Hypothesis' automatic discarding of
3719 stale examples. It is designed to allow local machines to ac‐
3720 cess a shared database (e.g. from CI servers), without propagat‐
3721 ing changes back from a local or in-development branch.
3722 class hypothesis.database.MultiplexedDatabase(*dbs)
3724 A wrapper around multiple databases.
3725 Each save, fetch, move, or delete operation will be run against
3727 all of the wrapped databases. fetch does not yield duplicate
3728 values, even if the same value is present in two or more of the
3729 wrapped databases.
3730 This combines well with a ReadOnlyDatabase, as follows:
3732 local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
3734 shared = CustomNetworkDatabase()
3735 settings.register_profile("ci", database=shared)
3737 settings.register_profile(
3738 "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
3739 )
3740 settings.load_profile("ci" if os.environ.get("CI") else "dev")
3741 So your CI system or fuzzing runs can populate a central shared
3743 database; while local runs on development machines can reproduce
3744 any failures from CI but will only cache their own failures lo‐
3745 cally and cannot remove examples from the shared database.
3746 class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_af‐
3748 ter=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:')
3749 Store Hypothesis examples as sets in the given redis.Redis data‐
3750 store.
3751 This is particularly useful for shared databases, as per the
3753 recipe for a MultiplexedDatabase.
3754 NOTE:
3756 If a test has not been run for expire_after, those examples
3757 will be allowed to expire. The default time-to-live persists
3758 examples between weekly runs.
3759 Defining your own ExampleDatabase
3761 You can define your ExampleDatabase, for example to use a shared datas‐
3762 tore, with just a few methods:
3763 class hypothesis.database.ExampleDatabase(*args, **kwargs)
3765 An abstract base class for storing examples in Hypothesis' in‐
3766 ternal format.
3767 An ExampleDatabase maps each bytes key to many distinct bytes
3769 values, like a Mapping[bytes, AbstractSet[bytes]].
3770 abstract save(key, value)
3772 Save value under key.
3773 If this value is already present for this key, silently
3775 do nothing.
3776 abstract fetch(key)
3778 Return an iterable over all values matching this key.
3779 abstract delete(key, value)
3781 Remove this value from this key.
3782 If this value is not present, silently do nothing.
3784 move(src, dest, value)
3786 Move value from key src to key dest. Equivalent to
3787 delete(src, value) followed by save(src, value), but may
3788 have a more efficient implementation.
3789 Note that value will be inserted at dest regardless of
3791 whether it is currently present at src.
3792
3794 With @given, your tests are still something that you mostly write your‐
3795 self, with Hypothesis providing some data. With Hypothesis's stateful
3796 testing, Hypothesis instead tries to generate not just data but entire
3797 tests. You specify a number of primitive actions that can be combined
3798 together, and then Hypothesis will try to find sequences of those ac‐
3799 tions that result in a failure.
3800 NOTE:
3802 This style of testing is often called model-based testing, but in
3803 Hypothesis is called stateful testing (mostly for historical reasons
3804 - the original implementation of this idea in Hypothesis was more
3805 closely based on ScalaCheck's stateful testing where the name is
3806 more apt). Both of these names are somewhat misleading: You don't
3807 really need any sort of formal model of your code to use this, and
3808 it can be just as useful for pure APIs that don't involve any state
3809 as it is for stateful ones.
3810 It's perhaps best to not take the name of this sort of testing too
3812 seriously. Regardless of what you call it, it is a powerful form of
3813 testing which is useful for most non-trivial APIs.
3814 You may not need state machines
3816 The basic idea of stateful testing is to make Hypothesis choose actions
3817 as well as values for your test, and state machines are a great declar‐
3818 ative way to do just that.
3819 For simpler cases though, you might not need them at all - a standard
3821 test with @given might be enough, since you can use data() in branches
3822 or loops. In fact, that's how the state machine explorer works inter‐
3823 nally. For more complex workloads though, where a higher level API
3824 comes into it's own, keep reading!
3825 Rule-based state machines
3827 class hypothesis.stateful.RuleBasedStateMachine
3828 A RuleBasedStateMachine gives you a structured way to define
3829 state machines.
3830 The idea is that a state machine carries a bunch of types of
3832 data divided into Bundles, and has a set of rules which may read
3833 data from bundles (or just from normal strategies) and push data
3834 onto bundles. At any given point a random applicable rule will
3835 be executed.
3836 A rule is very similar to a normal @given based test in that it takes
3838 values drawn from strategies and passes them to a user defined test
3839 function. The key difference is that where @given based tests must be
3840 independent, rules can be chained together - a single test run may in‐
3841 volve multiple rule invocations, which may interact in various ways.
3842 Rules can take normal strategies as arguments, or a specific kind of
3844 strategy called a Bundle. A Bundle is a named collection of generated
3845 values that can be reused by other operations in the test. They are
3846 populated with the results of rules, and may be used as arguments to
3847 rules, allowing data to flow from one rule to another, and rules to
3848 work on the results of previous computations or actions.
3849 You can think of each value that gets added to any Bundle as being as‐
3851 signed to a new variable. Drawing a value from the bundle strategy
3852 means choosing one of the corresponding variables and using that value,
3853 and consumes() as a del statement for that variable. If you can re‐
3854 place use of Bundles with instance attributes of the class that is of‐
3855 ten simpler, but often Bundles are strictly more powerful.
3856 The following rule based state machine example is a simplified version
3858 of a test for Hypothesis's example database implementation. An example
3859 database maps keys to sets of values, and in this test we compare one
3860 implementation of it to a simplified in memory model of its behaviour,
3861 which just stores the same values in a Python dict. The test then runs
3862 operations against both the real database and the in-memory representa‐
3863 tion of it and looks for discrepancies in their behaviour.
3864 import shutil
3866 import tempfile
3867 from collections import defaultdict
3869 import hypothesis.strategies as st
3870 from hypothesis.database import DirectoryBasedExampleDatabase
3871 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
3872 class DatabaseComparison(RuleBasedStateMachine):
3875 def __init__(self):
3876 super(DatabaseComparison, self).__init__()
3877 self.tempd = tempfile.mkdtemp()
3878 self.database = DirectoryBasedExampleDatabase(self.tempd)
3879 self.model = defaultdict(set)
3880 keys = Bundle("keys")
3882 values = Bundle("values")
3883 @rule(target=keys, k=st.binary())
3885 def add_key(self, k):
3886 return k
3887 @rule(target=values, v=st.binary())
3889 def add_value(self, v):
3890 return v
3891 @rule(k=keys, v=values)
3893 def save(self, k, v):
3894 self.model[k].add(v)
3895 self.database.save(k, v)
3896 @rule(k=keys, v=values)
3898 def delete(self, k, v):
3899 self.model[k].discard(v)
3900 self.database.delete(k, v)
3901 @rule(k=keys)
3903 def values_agree(self, k):
3904 assert set(self.database.fetch(k)) == self.model[k]
3905 def teardown(self):
3907 shutil.rmtree(self.tempd)
3908 TestDBComparison = DatabaseComparison.TestCase
3911 In this we declare two bundles - one for keys, and one for values. We
3913 have two trivial rules which just populate them with data (k and v),
3914 and three non-trivial rules: save saves a value under a key and delete
3915 removes a value from a key, in both cases also updating the model of
3916 what should be in the database. values_agree then checks that the con‐
3917 tents of the database agrees with the model for a particular key.
3918 We can then integrate this into our test suite by getting a unittest
3920 TestCase from it:
3921 TestTrees = DatabaseComparison.TestCase
3923 # Or just run with pytest's unittest support
3925 if __name__ == "__main__":
3926 unittest.main()
3927 This test currently passes, but if we comment out the line where we
3929 call self.model[k].discard(v), we would see the following output when
3930 run under pytest:
3931 AssertionError: assert set() == {b''}
3933 ------------ Hypothesis ------------
3935 state = DatabaseComparison()
3937 var1 = state.add_key(k=b'')
3938 var2 = state.add_value(v=var1)
3939 state.save(k=var1, v=var2)
3940 state.delete(k=var1, v=var2)
3941 state.values_agree(k=var1)
3942 state.teardown()
3943 Note how it's printed out a very short program that will demonstrate
3945 the problem. The output from a rule based state machine should gener‐
3946 ally be pretty close to Python code - if you have custom repr implemen‐
3947 tations that don't return valid Python then it might not be, but most
3948 of the time you should just be able to copy and paste the code into a
3949 test to reproduce it.
3950 You can control the detailed behaviour with a settings object on the
3952 TestCase (this is a normal hypothesis settings object using the de‐
3953 faults at the time the TestCase class was first referenced). For exam‐
3954 ple if you wanted to run fewer examples with larger programs you could
3955 change the settings to:
3956 DatabaseComparison.TestCase.settings = settings(
3958 max_examples=50, stateful_step_count=100
3959 )
3960 Which doubles the number of steps each program runs and halves the num‐
3962 ber of test cases that will be run.
3963 Rules
3965 As said earlier, rules are the most common feature used in RuleBased‐
3966 StateMachine. They are defined by applying the rule() decorator on a
3967 function. Note that RuleBasedStateMachine must have at least one rule
3968 defined and that a single function cannot be used to define multiple
3969 rules (this to avoid having multiple rules doing the same things). Due
3970 to the stateful execution method, rules generally cannot take arguments
3971 from other sources such as fixtures or pytest.mark.parametrize - con‐
3972 sider providing them via a strategy such as sampled_from() instead.
3973 hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)
3975 Decorator for RuleBasedStateMachine. Any name present in target
3976 or targets will define where the end result of this function
3977 should go. If both are empty then the end result will be dis‐
3978 carded.
3979 target must be a Bundle, or if the result should go to multiple
3981 bundles you can pass a tuple of them as the targets argument.
3982 It is invalid to use both arguments for a single rule. If the
3983 result should go to exactly one of several bundles, define a
3984 separate rule for each case.
3985 kwargs then define the arguments that will be passed to the
3987 function invocation. If their value is a Bundle, or if it is
3988 consumes(b) where b is a Bundle, then values that have previ‐
3989 ously been produced for that bundle will be provided. If con‐
3990 sumes is used, the value will also be removed from the bundle.
3991 Any other kwargs should be strategies and values from them will
3993 be provided.
3994 hypothesis.stateful.consumes(bundle)
3996 When introducing a rule in a RuleBasedStateMachine, this func‐
3997 tion can be used to mark bundles from which each value used in a
3998 step with the given rule should be removed. This function re‐
3999 turns a strategy object that can be manipulated and combined
4000 like any other.
4001 For example, a rule declared with
4003 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
4005 sumes(b3)))
4006 will consume a value from Bundle b2 and several values from Bun‐
4008 dle b3 to populate value2 and value3 each time it is executed.
4009 hypothesis.stateful.multiple(*args)
4011 This function can be used to pass multiple results to the tar‐
4012 get(s) of a rule. Just use return multiple(result1, result2,
4013 ...) in your rule.
4014 It is also possible to use return multiple() with no arguments
4016 in order to end a rule without passing any result.
4017 Initializes
4019 Initializes are a special case of rules that are guaranteed to be run
4020 at most once at the beginning of a run (i.e. before any normal rule is
4021 called). Note if multiple initialize rules are defined, they may be
4022 called in any order, and that order will vary from run to run.
4023 Initializes are typically useful to populate bundles:
4025 hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)
4027 Decorator for RuleBasedStateMachine.
4028 An initialize decorator behaves like a rule, but the decorated
4030 method is called at most once in a run. All initialize decorated
4031 methods will be called before any rule decorated methods, in an
4032 arbitrary order.
4033 import hypothesis.strategies as st
4035 from hypothesis.stateful import RuleBasedStateMachine, Bundle, rule, initialize
4036 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
4038 class NumberModifier(RuleBasedStateMachine):
4041 folders = Bundle("folders")
4043 files = Bundle("files")
4044 @initialize(target=folders)
4046 def init_folders(self):
4047 return "/"
4048 @rule(target=folders, name=name_strategy)
4050 def create_folder(self, parent, name):
4051 return "%s/%s" % (parent, name)
4052 @rule(target=files, name=name_strategy)
4054 def create_file(self, parent, name):
4055 return "%s/%s" % (parent, name)
4056 Preconditions
4058 While it's possible to use assume() in RuleBasedStateMachine rules, if
4059 you use it in only a few rules you can quickly run into a situation
4060 where few or none of your rules pass their assumptions. Thus, Hypothe‐
4061 sis provides a precondition() decorator to avoid this problem. The
4062 precondition() decorator is used on rule-decorated functions, and must
4063 be given a function that returns True or False based on the RuleBased‐
4064 StateMachine instance.
4065 hypothesis.stateful.precondition(precond)
4067 Decorator to apply a precondition for rules in a RuleBased‐
4068 StateMachine. Specifies a precondition for a rule to be consid‐
4069 ered as a valid step in the state machine. The given function
4070 will be called with the instance of RuleBasedStateMachine and
4071 should return True or False. Usually it will need to look at at‐
4072 tributes on that instance.
4073 For example:
4075 class MyTestMachine(RuleBasedStateMachine):
4077 state = 1
4078 @precondition(lambda self: self.state != 0)
4080 @rule(numerator=integers())
4081 def divide_with(self, numerator):
4082 self.state = numerator / self.state
4083 This is better than using assume in your rule since more valid
4085 rules should be able to be run.
4086 from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
4088 class NumberModifier(RuleBasedStateMachine):
4091 num = 0
4093 @rule()
4095 def add_one(self):
4096 self.num += 1
4097 @precondition(lambda self: self.num != 0)
4099 @rule()
4100 def divide_with_one(self):
4101 self.num = 1 / self.num
4102 By using precondition() here instead of assume(), Hypothesis can filter
4104 the inapplicable rules before running them. This makes it much more
4105 likely that a useful sequence of steps will be generated.
4106 Note that currently preconditions can't access bundles; if you need to
4108 use preconditions, you should store relevant data on the instance in‐
4109 stead.
4110 Invariants
4112 Often there are invariants that you want to ensure are met after every
4113 step in a process. It would be possible to add these as rules that are
4114 run, but they would be run zero or multiple times between other rules.
4115 Hypothesis provides a decorator that marks a function to be run after
4116 every step.
4117 hypothesis.stateful.invariant()
4119 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
4120 chine. The decorated function will be run after every rule and
4121 can raise an exception to indicate failed invariants.
4122 For example:
4124 class MyTestMachine(RuleBasedStateMachine):
4126 state = 1
4127 @invariant()
4129 def is_nonzero(self):
4130 assert self.state != 0
4131 from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
4133 class NumberModifier(RuleBasedStateMachine):
4136 num = 0
4138 @rule()
4140 def add_two(self):
4141 self.num += 2
4142 if self.num > 50:
4143 self.num += 1
4144 @invariant()
4146 def divide_with_one(self):
4147 assert self.num % 2 == 0
4148 NumberTest = NumberModifier.TestCase
4151 Invariants can also have precondition()s applied to them, in which case
4153 they will only be run if the precondition function returns true.
4154 Note that currently invariants can't access bundles; if you need to use
4156 invariants, you should store relevant data on the instance instead.
4157 More fine grained control
4159 If you want to bypass the TestCase infrastructure you can invoke these
4160 manually. The stateful module exposes the function run_state_ma‐
4161 chine_as_test, which takes an arbitrary function returning a RuleBased‐
4162 StateMachine and an optional settings parameter and does the same as
4163 the class based runTest provided.
4164 This is not recommended as it bypasses some important internal func‐
4166 tions, including reporting of statistics such as runtimes and event()
4167 calls. It was originally added to support custom __init__ methods, but
4168 you can now use initialize() rules instead.
4169
4171 Hypothesis does its level best to be compatible with everything you
4172 could possibly need it to be compatible with. Generally you should just
4173 try it and expect it to work. If it doesn't, you can be surprised and
4174 check this document for the details.
4175 Hypothesis versions
4177 Backwards compatibility is better than backporting fixes, so we use se‐
4178 mantic versioning and only support the most recent version of Hypothe‐
4179 sis. See support for more information.
4180 Documented APIs will not break except between major version bumps. All
4182 APIs mentioned in this documentation are public unless explicitly noted
4183 as provisional, in which case they may be changed in minor releases.
4184 Undocumented attributes, modules, and behaviour may include breaking
4185 changes in patch releases.
4186 Python versions
4188 Hypothesis is supported and tested on CPython 3.6+, i.e. all versions
4189 of CPython with upstream support,
4190 Hypothesis also supports the latest PyPy for Python 3.6. 32-bit builds
4192 of CPython also work, though they are currently only tested on Windows.
4193 In general Hypothesis does not officially support anything except the
4195 latest patch release of any version of Python it supports. Earlier re‐
4196 leases should work and bugs in them will get fixed if reported, but
4197 they're not tested in CI and no guarantees are made.
4198 Operating systems
4200 In theory Hypothesis should work anywhere that Python does. In practice
4201 it is only known to work and regularly tested on OS X, Windows and
4202 Linux, and you may experience issues running it elsewhere.
4203 If you're using something else and it doesn't work, do get in touch and
4205 I'll try to help, but unless you can come up with a way for me to run a
4206 CI server on that operating system it probably won't stay fixed due to
4207 the inevitable march of time.
4208 Testing frameworks
4210 In general Hypothesis goes to quite a lot of effort to generate things
4211 that look like normal Python test functions that behave as closely to
4212 the originals as possible, so it should work sensibly out of the box
4213 with every test framework.
4214 If your testing relies on doing something other than calling a function
4216 and seeing if it raises an exception then it probably won't work out of
4217 the box. In particular things like tests which return generators and
4218 expect you to do something with them (e.g. nose's yield based tests)
4219 will not work. Use a decorator or similar to wrap the test to take this
4220 form, or ask the framework maintainer to support our hooks for insert‐
4221 ing such a wrapper later.
4222 In terms of what's actually known to work:
4224 • Hypothesis integrates as smoothly with pytest and unittest as we
4226 can make it, and this is verified as part of the CI.
4227 • pytest fixtures work in the usual way for tests that have been
4229 decorated with @given - just avoid passing a strategy for each ar‐
4230 gument that will be supplied by a fixture. However, each fixture
4231 will run once for the whole function, not once per example. Deco‐
4232 rating a fixture function with @given is meaningless.
4233 • The python:unittest.mock.patch() decorator works with @given, but
4235 we recommend using it as a context manager within the decorated
4236 test to ensure that the mock is per-test-case and avoid poor in‐
4237 teractions with Pytest fixtures.
4238 • Nose works fine with Hypothesis, and this is tested as part of the
4240 CI. yield based tests simply won't work.
4241 • Integration with Django's testing requires use of the hypothe‐
4243 sis-django package. The issue is that in Django's tests' normal
4244 mode of execution it will reset the database once per test rather
4245 than once per example, which is not what you want.
4246 • Coverage works out of the box with Hypothesis; our own test suite
4248 has 100% branch coverage.
4249 Optional packages
4251 The supported versions of optional packages, for strategies in hypothe‐
4252 sis.extra, are listed in the documentation for that extra. Our general
4253 goal is to support all versions that are supported upstream.
4254 Regularly verifying this
4256 Everything mentioned above as explicitly supported is checked on every
4257 commit with GitHub Actions. Our continuous delivery pipeline runs all
4258 of these checks before publishing each release, so when we say they're
4259 supported we really mean it.
4260
4262 This is a collection of examples of how to use Hypothesis in interest‐
4263 ing ways. It's small for now but will grow over time.
4264 All of these examples are designed to be run under pytest, and nose
4266 should work too.
4267 How not to sort by a partial order
4269 The following is an example that's been extracted and simplified from a
4270 real bug that occurred in an earlier version of Hypothesis. The real
4271 bug was a lot harder to find.
4272 Suppose we've got the following type:
4274 class Node(object):
4276 def __init__(self, label, value):
4277 self.label = label
4278 self.value = tuple(value)
4279 def __repr__(self):
4281 return "Node(%r, %r)" % (self.label, self.value)
4282 def sorts_before(self, other):
4284 if len(self.value) >= len(other.value):
4285 return False
4286 return other.value[: len(self.value)] == self.value
4287 Each node is a label and a sequence of some data, and we have the rela‐
4289 tionship sorts_before meaning the data of the left is an initial seg‐
4290 ment of the right. So e.g. a node with value [1, 2] will sort before a
4291 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
4292 before the other.
4293 We have a list of nodes, and we want to topologically sort them with
4295 respect to this ordering. That is, we want to arrange the list so that
4296 if x.sorts_before(y) then x appears earlier in the list than y. We
4297 naively think that the easiest way to do this is to extend the partial
4298 order defined here to a total order by breaking ties arbitrarily and
4299 then using a normal sorting algorithm. So we define the following code:
4300 from functools import total_ordering
4302 @total_ordering
4305 class TopoKey(object):
4306 def __init__(self, node):
4307 self.value = node
4308 def __lt__(self, other):
4310 if self.value.sorts_before(other.value):
4311 return True
4312 if other.value.sorts_before(self.value):
4313 return False
4314 return self.value.label < other.value.label
4316 def sort_nodes(xs):
4319 xs.sort(key=TopoKey)
4320 This takes the order defined by sorts_before and extends it by breaking
4322 ties by comparing the node labels.
4323 But now we want to test that it works.
4325 First we write a function to verify that our desired outcome holds:
4327 def is_prefix_sorted(xs):
4329 for i in range(len(xs)):
4330 for j in range(i + 1, len(xs)):
4331 if xs[j].sorts_before(xs[i]):
4332 return False
4333 return True
4334 This will return false if it ever finds a pair in the wrong order and
4336 return true otherwise.
4337 Given this function, what we want to do with Hypothesis is assert that
4339 for all sequences of nodes, the result of calling sort_nodes on it is
4340 sorted.
4341 First we need to define a strategy for Node:
4343 from hypothesis import settings, strategies
4345 import hypothesis.strategies as s
4346 NodeStrategy = s.builds(Node, s.integers(), s.lists(s.booleans(), max_size=10))
4348 We want to generate short lists of values so that there's a decent
4350 chance of one being a prefix of the other (this is also why the choice
4351 of bool as the elements). We then define a strategy which builds a node
4352 out of an integer and one of those short lists of booleans.
4353 We can now write a test:
4355 from hypothesis import given
4357 @given(s.lists(NodeStrategy))
4360 def test_sorting_nodes_is_prefix_sorted(xs):
4361 sort_nodes(xs)
4362 assert is_prefix_sorted(xs)
4363 this immediately fails with the following example:
4365 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
4367 The reason for this is that because False is not a prefix of (True,
4369 True) nor vice versa, sorting things the first two nodes are equal be‐
4370 cause they have equal labels. This makes the whole order non-transi‐
4371 tive and produces basically nonsense results.
4372 But this is pretty unsatisfying. It only works because they have the
4374 same label. Perhaps we actually wanted our labels to be unique. Let's
4375 change the test to do that.
4376 def deduplicate_nodes_by_label(nodes):
4378 table = {node.label: node for node in nodes}
4379 return list(table.values())
4380 We define a function to deduplicate nodes by labels, and can now map
4382 that over a strategy for lists of nodes to give us a strategy for lists
4383 of nodes with unique labels:
4384 @given(s.lists(NodeStrategy).map(deduplicate_nodes_by_label))
4386 def test_sorting_nodes_is_prefix_sorted(xs):
4387 sort_nodes(xs)
4388 assert is_prefix_sorted(xs)
4389 Hypothesis quickly gives us an example of this still being wrong:
4391 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))]
4393 Now this is a more interesting example. None of the nodes will sort
4395 equal. What is happening here is that the first node is strictly less
4396 than the last node because (False,) is a prefix of (False, False). This
4397 is in turn strictly less than the middle node because neither is a pre‐
4398 fix of the other and -2 < -1. The middle node is then less than the
4399 first node because -1 < 0.
4400 So, convinced that our implementation is broken, we write a better one:
4402 def sort_nodes(xs):
4404 for i in range(1, len(xs)):
4405 j = i - 1
4406 while j >= 0:
4407 if xs[j].sorts_before(xs[j + 1]):
4408 break
4409 xs[j], xs[j + 1] = xs[j + 1], xs[j]
4410 j -= 1
4411 This is just insertion sort slightly modified - we swap a node back‐
4413 wards until swapping it further would violate the order constraints.
4414 The reason this works is because our order is a partial order already
4415 (this wouldn't produce a valid result for a general topological sorting
4416 - you need the transitivity).
4417 We now run our test again and it passes, telling us that this time
4419 we've successfully managed to sort some nodes without getting it com‐
4420 pletely wrong. Go us.
4421 Time zone arithmetic
4423 This is an example of some tests for pytz which check that various
4424 timezone conversions behave as you would expect them to. These tests
4425 should all pass, and are mostly a demonstration of some useful sorts of
4426 thing to test with Hypothesis, and how the datetimes() strategy works.
4427 >>> from datetime import timedelta
4429 >>> from hypothesis.extra.pytz import timezones
4430 >>> from hypothesis.strategies import datetimes
4431 >>> # The datetimes strategy is naive by default, so tell it to use timezones
4433 >>> aware_datetimes = datetimes(timezones=timezones())
4434 >>> @given(aware_datetimes, timezones(), timezones())
4436 ... def test_convert_via_intermediary(dt, tz1, tz2):
4437 ... """Test that converting between timezones is not affected
4438 ... by a detour via another timezone.
4439 ... """
4440 ... assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
4441 >>> @given(aware_datetimes, timezones())
4443 ... def test_convert_to_and_fro(dt, tz2):
4444 ... """If we convert to a new timezone and back to the old one
4445 ... this should leave the result unchanged.
4446 ... """
4447 ... tz1 = dt.tzinfo
4448 ... assert dt == dt.astimezone(tz2).astimezone(tz1)
4449 >>> @given(aware_datetimes, timezones())
4451 ... def test_adding_an_hour_commutes(dt, tz):
4452 ... """When converting between timezones it shouldn't matter
4453 ... if we add an hour here or add an hour there.
4454 ... """
4455 ... an_hour = timedelta(hours=1)
4456 ... assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
4457 >>> @given(aware_datetimes, timezones())
4459 ... def test_adding_a_day_commutes(dt, tz):
4460 ... """When converting between timezones it shouldn't matter
4461 ... if we add a day here or add a day there.
4462 ... """
4463 ... a_day = timedelta(days=1)
4464 ... assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
4465 >>> # And we can check that our tests pass
4467 >>> test_convert_via_intermediary()
4468 >>> test_convert_to_and_fro()
4469 >>> test_adding_an_hour_commutes()
4470 >>> test_adding_a_day_commutes()
4471 Condorcet's paradox
4473 A classic paradox in voting theory, called Condorcet's paradox, is that
4474 majority preferences are not transitive. That is, there is a population
4475 and a set of three candidates A, B and C such that the majority of the
4476 population prefer A to B, B to C and C to A.
4477 Wouldn't it be neat if we could use Hypothesis to provide an example of
4479 this?
4480 Well as you can probably guess from the presence of this section, we
4482 can! The main trick is to decide how we want to represent the result
4483 of an election - for this example, we'll use a list of "votes", where
4484 each vote is a list of candidates in the voters preferred order. With‐
4485 out further ado, here is the code:
4486 from hypothesis import given, assume
4488 from hypothesis.strategies import lists, permutations
4489 from collections import Counter
4490 # We need at least three candidates and at least three voters to have a
4492 # paradox; anything less can only lead to victories or at worst ties.
4493 @given(lists(permutations(["A", "B", "C"]), min_size=3))
4494 def test_elections_are_transitive(election):
4495 all_candidates = {"A", "B", "C"}
4496 # First calculate the pairwise counts of how many prefer each candidate
4498 # to the other
4499 counts = Counter()
4500 for vote in election:
4501 for i in range(len(vote)):
4502 for j in range(i + 1, len(vote)):
4503 counts[(vote[i], vote[j])] += 1
4504 # Now look at which pairs of candidates one has a majority over the
4506 # other and store that.
4507 graph = {}
4508 for i in all_candidates:
4509 for j in all_candidates:
4510 if counts[(i, j)] > counts[(j, i)]:
4511 graph.setdefault(i, set()).add(j)
4512 # Now for each triple assert that it is transitive.
4514 for x in all_candidates:
4515 for y in graph.get(x, ()):
4516 for z in graph.get(y, ()):
4517 assert x not in graph.get(z, ())
4518 The example Hypothesis gives me on my first run (your mileage may of
4520 course vary) is:
4521 [["A", "B", "C"], ["B", "C", "A"], ["C", "A", "B"]]
4523 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
4525 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
4526 and 2) prefer C to A. This is in fact basically the canonical example
4527 of the voting paradox.
4528 Fuzzing an HTTP API
4530 Hypothesis's support for testing HTTP services is somewhat nascent.
4531 There are plans for some fully featured things around this, but right
4532 now they're probably quite far down the line.
4533 But you can do a lot yourself without any explicit support! Here's a
4535 script I wrote to throw arbitrary data against the API for an entirely
4536 fictitious service called Waspfinder (this is only lightly obfuscated
4537 and you can easily figure out who I'm actually talking about, but I
4538 don't want you to run this code and hammer their API without their per‐
4539 mission).
4540 All this does is use Hypothesis to generate arbitrary JSON data match‐
4542 ing the format their API asks for and check for 500 errors. More ad‐
4543 vanced tests which then use the result and go on to do other things are
4544 definitely also possible. The swagger-conformance package provides an
4545 excellent example of this!
4546 import unittest
4548 from hypothesis import given, assume, settings, strategies as st
4549 from collections import namedtuple
4550 import requests
4551 import os
4552 import random
4553 import time
4554 import math
4555 Goal = namedtuple("Goal", ("slug",))
4558 # We just pass in our API credentials via environment variables.
4561 waspfinder_token = os.getenv("WASPFINDER_TOKEN")
4562 waspfinder_user = os.getenv("WASPFINDER_USER")
4563 assert waspfinder_token is not None
4564 assert waspfinder_user is not None
4565 GoalData = st.fixed_dictionaries(
4567 {
4568 "title": st.text(),
4569 "goal_type": st.sampled_from(
4570 ["hustler", "biker", "gainer", "fatloser", "inboxer", "drinker", "custom"]
4571 ),
4572 "goaldate": st.one_of(st.none(), st.floats()),
4573 "goalval": st.one_of(st.none(), st.floats()),
4574 "rate": st.one_of(st.none(), st.floats()),
4575 "initval": st.floats(),
4576 "panic": st.floats(),
4577 "secret": st.booleans(),
4578 "datapublic": st.booleans(),
4579 }
4580 )
4581 needs2 = ["goaldate", "goalval", "rate"]
4584 class WaspfinderTest(unittest.TestCase):
4587 @given(GoalData)
4588 def test_create_goal_dry_run(self, data):
4589 # We want slug to be unique for each run so that multiple test runs
4590 # don't interfere with each other. If for some reason some slugs trigger
4591 # an error and others don't we'll get a Flaky error, but that's OK.
4592 slug = hex(random.getrandbits(32))[2:]
4593 # Use assume to guide us through validation we know about, otherwise
4595 # we'll spend a lot of time generating boring examples.
4596 # Title must not be empty
4598 assume(data["title"])
4599 # Exactly two of these values should be not None. The other will be
4601 # inferred by the API.
4602 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
4604 for v in data.values():
4605 if isinstance(v, float):
4606 assume(not math.isnan(v))
4607 data["slug"] = slug
4608 # The API nicely supports a dry run option, which means we don't have
4610 # to worry about the user account being spammed with lots of fake goals
4611 # Otherwise we would have to make sure we cleaned up after ourselves
4612 # in this test.
4613 data["dryrun"] = True
4614 data["auth_token"] = waspfinder_token
4615 for d, v in data.items():
4616 if v is None:
4617 data[d] = "null"
4618 else:
4619 data[d] = str(v)
4620 result = requests.post(
4621 "https://waspfinder.example.com/api/v1/users/"
4622 "%s/goals.json" % (waspfinder_user,),
4623 data=data,
4624 )
4625 # Let's not hammer the API too badly. This will of course make the
4627 # tests even slower than they otherwise would have been, but that's
4628 # life.
4629 time.sleep(1.0)
4630 # For the moment all we're testing is that this doesn't generate an
4632 # internal error. If we didn't use the dry run option we could have
4633 # then tried doing more with the result, but this is a good start.
4634 self.assertNotEqual(result.status_code, 500)
4635 if __name__ == "__main__":
4638 unittest.main()
4639
4641 The Hypothesis community is small for the moment but is full of excel‐
4642 lent people who can answer your questions and help you out. Please do
4643 join us. The two major places for community discussion are:
4644 • The mailing list.
4646 • An IRC channel, #hypothesis on freenode, which is more active than
4648 the mailing list.
4649 Feel free to use these to ask for help, provide feedback, or discuss
4651 anything remotely Hypothesis related at all. If you post a question on
4652 Stack Overflow, please use the python-hypothesis tag!
4653 Please note that the Hypothesis code of conduct applies in all Hypothe‐
4655 sis community spaces.
4656 If you would like to cite Hypothesis, please consider our suggested ci‐
4658 tation.
4659 If you like repo badges, we suggest the following badge, which you can
4661 add with reStructuredText or Markdown, respectively: [image]
4662 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
4664 :alt: Tested with Hypothesis
4665 :target: https://hypothesis.readthedocs.io
4666 [](https://hypothesis.readthedocs.io/)
4668 Finally, we have a beautiful logo which appears online, and often on
4670 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
4671 [image]
4672 As well as being beautiful, dragonflies actively hunt down bugs for a
4674 living! You can find the images and a usage guide in the brand direc‐
4675 tory on GitHub, or find us at conferences where we often have stickers
4676 and sometimes other swag.
4677
4679 What is Hypothesis for?
4680 From the perspective of a user, the purpose of Hypothesis is to make it
4682 easier for you to write better tests.
4683 From my perspective as the author, that is of course also a purpose of
4685 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
4686 mania for a moment), the larger purpose of Hypothesis is to drag the
4687 world kicking and screaming into a new and terrifying age of high qual‐
4688 ity software.
4689 Software is, as they say, eating the world. Software is also terrible.
4691 It's buggy, insecure and generally poorly thought out. This combination
4692 is clearly a recipe for disaster.
4693 And the state of software testing is even worse. Although it's fairly
4695 uncontroversial at this point that you should be testing your code, can
4696 you really say with a straight face that most projects you've worked on
4697 are adequately tested?
4698 A lot of the problem here is that it's too hard to write good tests.
4700 Your tests encode exactly the same assumptions and fallacies that you
4701 had when you wrote the code, so they miss exactly the same bugs that
4702 you missed when you wrote the code.
4703 Meanwhile, there are all sorts of tools for making testing better that
4705 are basically unused. The original Quickcheck is from 1999 and the ma‐
4706 jority of developers have not even heard of it, let alone used it.
4707 There are a bunch of half-baked implementations for most languages, but
4708 very few of them are worth using.
4709 The goal of Hypothesis is to bring advanced testing techniques to the
4711 masses, and to provide an implementation that is so high quality that
4712 it is easier to use them than it is not to use them. Where I can, I
4713 will beg, borrow and steal every good idea I can find that someone has
4714 had to make software testing better. Where I can't, I will invent new
4715 ones.
4716 Quickcheck is the start, but I also plan to integrate ideas from fuzz
4718 testing (a planned future feature is to use coverage information to
4719 drive example selection, and the example saving database is already in‐
4720 spired by the workflows people use for fuzz testing), and am open to
4721 and actively seeking out other suggestions and ideas.
4722 The plan is to treat the social problem of people not using these ideas
4724 as a bug to which there is a technical solution: Does property-based
4725 testing not match your workflow? That's a bug, let's fix it by figur‐
4726 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
4727 tom data for your application? That's a bug. Let's fix it by figuring
4728 out how to make it easier, or how to take something you're already us‐
4729 ing to specify your data and derive a generator from that automati‐
4730 cally. Find the explanations of these advanced ideas hopelessly obtuse
4731 and hard to follow? That's a bug. Let's provide you with an easy API
4732 that lets you test your code better without a PhD in software verifica‐
4733 tion.
4734 Grand ambitions, I know, and I expect ultimately the reality will be
4736 somewhat less grand, but so far in about three months of development,
4737 Hypothesis has become the most solid implementation of Quickcheck ever
4738 seen in a mainstream language (as long as we don't count Scala as main‐
4739 stream yet), and at the same time managed to significantly push forward
4740 the state of the art, so I think there's reason to be optimistic.
4741
4743 This is a page for listing people who are using Hypothesis and how ex‐
4744 cited they are about that. If that's you and your name is not on the
4745 list, this file is in Git and I'd love it if you sent me a pull request
4746 to fix that.
4747 Stripe
4749 At Stripe we use Hypothesis to test every piece of our machine learning
4750 model training pipeline (powered by scikit). Before we migrated, our
4751 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
4752 resentative at all of our actual very complex data. Because we needed
4753 to craft examples for each test, we took the easy way out and lived
4754 with extremely low test coverage.
4755 Hypothesis changed all that. Once we had our strategies for generating
4757 Dataframes of features it became trivial to slightly customize each
4758 strategy for new tests. Our coverage is now close to 90%.
4759 Full-stop, property-based testing is profoundly more powerful - and has
4761 caught or prevented far more bugs - than our old style of example-based
4762 testing.
4763 Kristian Glass - Director of Technology at LaterPay GmbH
4765 Hypothesis has been brilliant for expanding the coverage of our test
4766 cases, and also for making them much easier to read and understand, so
4767 we're sure we're testing the things we want in the way we want.
4768 Seth Morton
4770 When I first heard about Hypothesis, I knew I had to include it in my
4771 two open-source Python libraries, natsort and fastnumbers . Quite
4772 frankly, I was a little appalled at the number of bugs and "holes" I
4773 found in the code. I can now say with confidence that my libraries are
4774 more robust to "the wild." In addition, Hypothesis gave me the confi‐
4775 dence to expand these libraries to fully support Unicode input, which I
4776 never would have had the stomach for without such thorough testing ca‐
4777 pabilities. Thanks!
4778 Sixty North
4780 At Sixty North we use Hypothesis for testing Segpy an open source
4781 Python library for shifting data between Python data structures and SEG
4782 Y files which contain geophysical data from the seismic reflection sur‐
4783 veys used in oil and gas exploration.
4784 This is our first experience of property-based testing – as opposed to
4786 example-based testing. Not only are our tests more powerful, they are
4787 also much better explanations of what we expect of the production code.
4788 In fact, the tests are much closer to being specifications. Hypothesis
4789 has located real defects in our code which went undetected by tradi‐
4790 tional test cases, simply because Hypothesis is more relentlessly devi‐
4791 ous about test case generation than us mere humans! We found Hypothe‐
4792 sis particularly beneficial for Segpy because SEG Y is an antiquated
4793 format that uses legacy text encodings (EBCDIC) and even a legacy
4794 floating point format we implemented from scratch in Python.
4795 Hypothesis is sure to find a place in most of our future Python code‐
4797 bases and many existing ones too.
4798 mulkieran
4800 Just found out about this excellent QuickCheck for Python implementa‐
4801 tion and ran up a few tests for my bytesize package last night. Refuted
4802 a few hypotheses in the process.
4803 Looking forward to using it with a bunch of other projects as well.
4805 Adam Johnson
4807 I have written a small library to serialize dicts to MariaDB's dynamic
4808 columns binary format, mariadb-dyncol. When I first developed it, I
4809 thought I had tested it really well - there were hundreds of test
4810 cases, some of them even taken from MariaDB's test suite itself. I was
4811 ready to release.
4812 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
4814 Wow! It found bug after bug after bug. Even after a first release, I
4815 thought of a way to make the tests do more validation, which revealed a
4816 further round of bugs! Most impressively, Hypothesis found a compli‐
4817 cated off-by-one error in a condition with 4095 versus 4096 bytes of
4818 data - something that I would never have found.
4819 Long live Hypothesis! (Or at least, property-based testing).
4821 Josh Bronson
4823 Adopting Hypothesis improved bidict's test coverage and significantly
4824 increased our ability to make changes to the code with confidence that
4825 correct behavior would be preserved. Thank you, David, for the great
4826 testing tool.
4827 Cory Benfield
4829 Hypothesis is the single most powerful tool in my toolbox for working
4830 with algorithmic code, or any software that produces predictable output
4831 from a wide range of sources. When using it with Priority, Hypothesis
4832 consistently found errors in my assumptions and extremely subtle bugs
4833 that would have taken months of real-world use to locate. In some
4834 cases, Hypothesis found subtle deviations from the correct output of
4835 the algorithm that may never have been noticed at all.
4836 When it comes to validating the correctness of your tools, nothing
4838 comes close to the thoroughness and power of Hypothesis.
4839 Jon Moore
4841 One extremely satisfied user here. Hypothesis is a really solid imple‐
4842 mentation of property-based testing, adapted well to Python, and with
4843 good features such as failure-case shrinkers. I first used it on a
4844 project where we needed to verify that a vendor's Python and non-Python
4845 implementations of an algorithm matched, and it found about a dozen
4846 cases that previous example-based testing and code inspections had not.
4847 Since then I've been evangelizing for it at our firm.
4848 Russel Winder
4850 I am using Hypothesis as an integral part of my Python workshops. Test‐
4851 ing is an integral part of Python programming and whilst unittest and,
4852 better, pytest can handle example-based testing, property-based testing
4853 is increasingly far more important than example-base testing, and Hy‐
4854 pothesis fits the bill.
4855 Wellfire Interactive
4857 We've been using Hypothesis in a variety of client projects, from test‐
4858 ing Django-related functionality to domain-specific calculations. It
4859 both speeds up and simplifies the testing process since there's so much
4860 less tedious and error-prone work to do in identifying edge cases. Test
4861 coverage is nice but test depth is even nicer, and it's much easier to
4862 get meaningful test depth using Hypothesis.
4863 Cody Kochmann
4865 Hypothesis is being used as the engine for random object generation
4866 with my open source function fuzzer battle_tested which maps all behav‐
4867 iors of a function allowing you to minimize the chance of unexpected
4868 crashes when running code in production.
4869 With how efficient Hypothesis is at generating the edge cases that
4871 cause unexpected behavior occur, battle_tested is able to map out the
4872 entire behavior of most functions in less than a few seconds.
4873 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
4875 ing it.
4876 Merchise Autrement
4878 Just minutes after our first use of hypothesis we uncovered a subtle
4879 bug in one of our most used library. Since then, we have increasingly
4880 used hypothesis to improve the quality of our testing in libraries and
4881 applications as well.
4882 Florian Kromer
4884 At Roboception GmbH I use Hypothesis to implement fully automated
4885 stateless and stateful reliability tests for the 3D sensor rc_visard
4886 and robotic software components .
4887 Thank you very much for creating the (probably) most powerful prop‐
4889 erty-based testing framework.
4890 Reposit Power
4892 With a micro-service architecture, testing between services is made
4893 easy using Hypothesis in integration testing. Ensuring everything is
4894 running smoothly is vital to help maintain a secure network of Virtual
4895 Power Plants.
4896 It allows us to find potential bugs and edge cases with relative ease
4898 and minimal overhead. As our architecture relies on services communi‐
4899 cating effectively, Hypothesis allows us to strictly test for the kind
4900 of data which moves around our services, particularly our backend
4901 Python applications.
4902 Your name goes here
4904 I know there are many more, because I keep finding out about new people
4905 I'd never even heard of using Hypothesis. If you're looking to way to
4906 give back to a tool you love, adding your name here only takes a moment
4907 and would really help a lot. As per instructions at the top, just send
4908 me a pull request and I'll add you to the list.
4909
4911 The following is a non-exhaustive list of open source projects I know
4912 are using Hypothesis. If you're aware of any others please add them to
4913 the list! The only inclusion criterion right now is that if it's a
4914 Python library then it should be available on PyPI.
4915 You can find hundreds more from the Hypothesis page at libraries.io,
4917 and thousands on GitHub. Hypothesis has over 100,000 downloads per
4918 week, and was used by more than 4% of Python users surveyed by the PSF
4919 in 2018.
4920 • aur
4922 • argon2_cffi
4924 • attrs
4926 • axelrod
4928 • bidict
4930 • binaryornot
4932 • brotlipy
4934 • chardet
4936 • cmph-cffi
4938 • cryptography
4940 • dbus-signature-pyparsing
4942 • dry-python/returns
4944 • fastnumbers
4946 • flocker
4948 • flownetpy
4950 • funsize
4952 • fusion-index
4954 • hyper-h2
4956 • into-dbus-python
4958 • justbases
4960 • justbytes
4962 • loris
4964 • mariadb-dyncol
4966 • mercurial
4968 • natsort
4970 • poliastro
4972 • pretext
4974 • priority
4976 • PyCEbox
4978 • PyPy
4980 • pyrsistent
4982 • python-humble-utils
4984 • pyudev
4986 • qutebrowser
4988 • RubyMarshal
4990 • Segpy
4992 • simoa
4994 • srt
4996 • tchannel
4998 • vdirsyncer
5000 • wcag-contrast-ratio
5002 • yacluster
5004 • yturl
5006
5008 Hypothesis has been eagerly used and extended by the open source commu‐
5009 nity. This page lists extensions and applications; you can find more
5010 or newer packages by searching PyPI by keyword or filter by classifier,
5011 or search libraries.io.
5012 If there's something missing which you think should be here, let us
5014 know!
5015 NOTE:
5017 Being listed on this page does not imply that the Hypothesis main‐
5018 tainers endorse a package.
5019 External strategies
5021 Some packages provide strategies directly:
5022 • hypothesis-fspaths - strategy to generate filesystem paths.
5024 • hypothesis-geojson - strategy to generate GeoJson.
5026 • hypothesis-geometry - strategies to generate geometric objects.
5028 • hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
5030 • hypothesis-sqlalchemy - strategies to generate SQLAlchemy objects.
5032 • hypothesis-ros - strategies to generate messages and parameters for
5034 the Robot Operating System.
5035 • hypothesis-csv - strategy to generate CSV files.
5037 • hypothesis-networkx - strategy to generate networkx graphs.
5039 • hypothesis-bio - strategies for bioinformatics data, such as DNA,
5041 codons, FASTA, and FASTQ formats.
5042 • hypothesmith - strategy to generate syntatically-valid Python code.
5044 Others provide a function to infer a strategy from some other schema:
5046 • hypothesis-jsonschema - infer strategies from JSON schemas.
5048 • lollipop-hypothesis - infer strategies from lollipop schemas.
5050 • hypothesis-drf - infer strategies from a djangorestframework seri‐
5052 aliser.
5053 • hypothesis-graphql - infer strategies from GraphQL schemas.
5055 • hypothesis-mongoengine - infer strategies from a mongoengine model.
5057 • hypothesis-pb - infer strategies from Protocol Buffer schemas.
5059 Other cool things
5061 schemathesis is a tool for testing web applications built with Open API
5062 / Swagger specifications. It reads the schema and generates test cases
5063 which will ensure that the application is compliant with its schema.
5064 The application under test could be written in any language, the only
5065 thing you need is a valid API schema in a supported format. Includes
5066 CLI and convenient pytest integration. Powered by Hypothesis and
5067 hypothesis-jsonschema, inspired by the earlier swagger-conformance li‐
5068 brary.
5069 Trio is an async framework with "an obsessive focus on usability and
5071 correctness", so naturally it works with Hypothesis! pytest-trio in‐
5072 cludes a custom hook that allows @given(...) to work with Trio-style
5073 async test functions, and hypothesis-trio includes stateful testing ex‐
5074 tensions to support concurrent programs.
5075 pymtl3 is "an open-source Python-based hardware generation, simulation,
5077 and verification framework with multi-level hardware modeling support",
5078 which ships with Hypothesis integrations to check that all of those
5079 levels are eqivalent, from function-level to register-transfer level
5080 and even to hardware.
5081 libarchimedes makes it easy to use Hypothesis in the Hy language, a
5083 Lisp embedded in Python.
5084 battle_tested is a fuzzing tool that will show you how your code can
5086 fail - by trying all kinds of inputs and reporting whatever happens.
5087 pytest-subtesthack functions as a workaround for issue #377.
5089 returns uses Hypothesis to verify that Higher Kinded Types correctly
5091 implement functor, applicative, monad, and other laws; allowing a
5092 declarative approach to be combined with traditional pythonic code.
5093 Writing an extension
5095 See CONTRIBUTING.rst for more information.
5096 New strategies can be added to Hypothesis, or published as an external
5098 package on PyPI - either is fine for most strategies. If in doubt, ask!
5099 It's generally much easier to get things working outside, because
5101 there's more freedom to experiment and fewer requirements in stability
5102 and API style. We're happy to review and help with external packages as
5103 well as pull requests!
5104 If you're thinking about writing an extension, please name it hypothe‐
5106 sis-{something} - a standard prefix makes the community more visible
5107 and searching for extensions easier. And make sure you use the Frame‐
5108 work :: Hypothesis trove classifier!
5109 On the other hand, being inside gets you access to some deeper imple‐
5111 mentation features (if you need them) and better long-term guarantees
5112 about maintenance. We particularly encourage pull requests for new
5113 composable primitives that make implementing other strategies easier,
5114 or for widely used types in the standard library. Strategies for other
5115 things are also welcome; anything with external dependencies just goes
5116 in hypothesis.extra.
5117 Registering strategies via setuptools entry points
5119 If you would like to ship Hypothesis strategies for a custom type - ei‐
5120 ther as part of the upstream library, or as a third-party extension,
5121 there's a catch: from_type() only works after the corresponding call to
5122 register_type_strategy(). This means that either
5123 • you have to try importing Hypothesis to register the strategy when
5125 your library is imported, though that's only useful at test time, or
5126 • the user has to call a 'register the strategies' helper that you pro‐
5128 vide before running their tests
5129 Entry points are Python's standard way of automating the latter: when
5131 you register a "hypothesis" entry point in your setup.py, we'll import
5132 and run it automatically when hypothesis is imported. Nothing happens
5133 unless Hypothesis is already in use, and it's totally seamless for
5134 downstream users!
5135 Let's look at an example. You start by adding a function somewhere in
5137 your package that does all the Hypothesis-related setup work:
5138 # mymodule.py
5140 class MyCustomType:
5143 def __init__(self, x: int):
5144 assert x >= 0, f"got {x}, but only positive numbers are allowed"
5145 self.x = x
5146 def _hypothesis_setup_hook():
5149 import hypothesis.strategies as st
5150 st.register_type_strategy(MyCustomType, st.integers(min_value=0))
5152 and then tell setuptools that this is your "hypothesis" entry point:
5154 # setup.py
5156 # You can list a module to import by dotted name
5158 entry_points = {"hypothesis": ["_ = mymodule.a_submodule"]}
5159 # Or name a specific function too, and Hypothesis will call it for you
5161 entry_points = {"hypothesis": ["_ = mymodule:_hypothesis_setup_hook"]}
5162 And that's all it takes!
5164 NOTE:
5166 On Python 3.6 and 3.7, where the importlib.metadata module is not in
5167 the standard library, loading entry points requires either the
5168 importlib_metadata (preferred) or setuptools (fallback) package to
5169 be installed.
5170 Interaction with pytest-cov
5172 Because pytest does not load plugins from entrypoints in any particular
5173 order, using the Hypothesis entrypoint may import your module before
5174 pytest-cov starts. This is a known issue, but there are workarounds.
5175 You can use coverage run pytest ... instead of pytest --cov ..., opting
5177 out of the pytest plugin entirely. Alternatively, you can ensure that
5178 Hypothesis is loaded after coverage measurement is started by disabling
5179 the entrypoint, and loading our pytest plugin from your conftest.py in‐
5180 stead:
5181 echo "pytest_plugins = ['hypothesis.extra.pytestplugin']\n" > tests/conftest.py
5183 pytest -p "no:hypothesispytest" ...
5184
5186 This is a record of all past Hypothesis releases and what went into
5187 them, in reverse chronological order. All previous releases should
5188 still be available on PyPI.
5189 Hypothesis 5.x
5191 5.43.9 - 2021-01-02
5192 This patch fixes issue #2722, where certain orderings of regis‐
5193 ter_type_strategy(), ForwardRef, and from_type() could trigger an in‐
5194 ternal error.
5195 5.43.8 - 2021-01-02
5197 This patch makes some strategies for collections with a uniqueness con‐
5198 straint much more efficient, including dictionaries(keys=sam‐
5199 pled_from(...), values=..) and lists(tuples(sampled_from(...), ...),
5200 unique_by=lambda x: x[0]). (related to issue #2036)
5201 5.43.7 - 2021-01-02
5203 This patch extends our faster special case for sampled_from() elements
5204 in unique lists() to account for chains of .map(...) and .filter(...)
5205 calls (issue #2036).
5206 5.43.6 - 2021-01-02
5208 This patch improves the type annotations on assume() and @repro‐
5209 duce_failure().
5210 5.43.5 - 2021-01-01
5212 This patch updates our copyright headers to include 2021. Happy new
5213 year!
5214 5.43.4 - 2020-12-24
5216 This change fixes a documentation error in the database setting.
5217 The previous documentation suggested that callers could specify a data‐
5219 base path string, or the special string ":memory:", but this setting
5220 has never actually allowed string arguments.
5221 Permitted values are None, and instances of ExampleDatabase.
5223 5.43.3 - 2020-12-11
5225 This patch fixes issue #2696, an internal error triggered when the @ex‐
5226 ample decorator was used and the verbosity setting was quiet.
5227 5.43.2 - 2020-12-10
5229 This patch improves the error message from the data_frames() strategy
5230 when both the rows and columns arguments are given, but there is a
5231 missing entry in rows and the corresponding column has no fill value (‐
5232 issue #2678).
5233 5.43.1 - 2020-12-10
5235 This patch improves the error message if builds() is passed an Enum
5236 which cannot be called without arguments, to suggest using sam‐
5237 pled_from() (issue #2693).
5238 5.43.0 - 2020-12-09
5240 This release adds new timezones() and timezone_keys() strategies (issue
5241 #2630) based on the new python:zoneinfo module in Python 3.9.
5242 pip install hypothesis[zoneinfo] will ensure that you have the appro‐
5244 priate backports installed if you need them.
5245 5.42.3 - 2020-12-09
5247 This patch fixes an internal error in datetimes() with allow_imagi‐
5248 nary=False where the timezones argument can generate tzinfo=None (issue
5249 #2662).
5250 5.42.2 - 2020-12-09
5252 This patch teaches hypothesis.extra.django.from_field() to infer more
5253 efficient strategies by inspecting (not just filtering by) field val‐
5254 idators for numeric and string fields (issue #1116).
5255 5.42.1 - 2020-12-09
5257 This patch refactors hypothesis.settings to use type-annotated keyword
5258 arguments instead of **kwargs, which makes tab-completion much more
5259 useful - as well as type-checkers like mypy.
5260 5.42.0 - 2020-12-09
5262 This patch teaches the magic() ghostwriter to recognise "en/de" func‐
5263 tion roundtrips other than the common encode/decode pattern, such as
5264 encrypt/decrypt or, encipher/decipher.
5265 5.41.5 - 2020-12-05
5267 This patch adds a performance optimisation to avoid saving redundant
5268 seeds when using the .fuzz_one_input hook.
5269 5.41.4 - 2020-11-28
5271 This patch fixes issue #2657, where passing unicode patterns compiled
5272 with python:re.IGNORECASE to from_regex() could trigger an internal er‐
5273 ror when casefolding a character creates a longer string (e.g.
5274 "\u0130".lower() -> "i\u0370").
5275 5.41.3 - 2020-11-18
5277 This patch adds a final fallback clause to our plugin logic to fail
5278 with a warning rather than error on Python < 3.8 when neither the
5279 importlib_metadata (preferred) or setuptools (fallback) packages are
5280 available.
5281 5.41.2 - 2020-11-08
5283 This patch fixes urls() strategy ensuring that ~ (tilde) is treated as
5284 one of the url-safe characters (issue #2658).
5285 5.41.1 - 2020-11-03
5287 This patch improves our CLI help and documentation.
5288 5.41.0 - 2020-10-30
5290 Hypothesis now shrinks examples where the error is raised while drawing
5291 from a strategy. This makes complicated custom strategies much easier
5292 to debug, at the cost of a slowdown for use-cases where you catch and
5293 ignore such errors.
5294 5.40.0 - 2020-10-30
5296 This release teaches from_type() how to handle ChainMap, Counter,
5297 Deque, Generator, Match, OrderedDict, Pattern, and Set (issue #2654).
5298 5.39.0 - 2020-10-30
5300 from_type() now knows how to resolve PEP 585 parameterized standard
5301 collection types, which are new in Python 3.9 (issue #2629).
5302 5.38.1 - 2020-10-26
5304 This patch fixes builds(), so that when passed infer for an argument
5305 with a non-Optional type annotation and a default value of None to
5306 build a class which defines an explicit __signature__ attribute, either
5307 None or that type may be generated.
5308 This is unlikely to happen unless you are using pydantic (issue #2648).
5310 5.38.0 - 2020-10-24
5312 This release improves our support for @st.composite on a python:class‐
5313 method or python:staticmethod (issue #2578).
5314 5.37.5 - 2020-10-24
5316 This patch fixes from_type() with Iterable[T] (issue #2645).
5317 5.37.4 - 2020-10-20
5319 This patch teaches the magic() ghostwriter to recognise that pairs of
5320 functions like rgb_to_hsv() and hsv_to_rgb() should roundtrip().
5321 5.37.3 - 2020-10-15
5323 This patch improves builds() and from_type() support for explicitly de‐
5324 fined __signature__ attributes, from version 5.8.3, to support generic
5325 types from the python:typing module.
5326 Thanks to Rónán Carrigan for identifying and fixing this problem!
5328 5.37.2 - 2020-10-14
5330 This patch fixes from_lark() with version 0.10.1+ of the lark-parser
5331 package.
5332 5.37.1 - 2020-10-07
5334 This patch fixes some broken links in the lark extra documentation.
5335 5.37.0 - 2020-10-03
5337 This release adds a new RedisExampleDatabase, along with the ReadOnly‐
5338 Database and MultiplexedDatabase helpers, to support team workflows
5339 where failing examples can be seamlessly shared between everyone on the
5340 team - and your CI servers or buildbots.
5341 5.36.2 - 2020-10-02
5343 This patch ensures that if the "hypothesis" entry point is callable, we
5344 call it after importing it. You can still use non-callable entry
5345 points (like modules), which are only imported.
5346 We also prefer importlib.metadata or the backport over pkg_resources,
5348 which makes import hypothesis around 200 milliseconds faster (issue
5349 #2571).
5350 5.36.1 - 2020-09-25
5352 This patch adds some helpful suggestions to error messages you might
5353 see while learning to use the @example() decorator (issue #2611) or the
5354 one_of() strategy.
5355 5.36.0 - 2020-09-24
5357 This release upgrades the from_dtype() strategy to pass optional
5358 **kwargs to the inferred strategy, and upgrades the arrays() strategy
5359 to accept an elements=kwargs dict to pass through to from_dtype().
5360 arrays(floating_dtypes(), shape, elements={"min_value": -10,
5362 "max_value": 10}) is a particularly useful pattern, as it allows for
5363 any floating dtype without triggering the roundoff warning for smaller
5364 types or sacrificing variety for larger types (issue #2552).
5365 5.35.4 - 2020-09-21
5367 This patch reformats our code with the latest black to take advantage
5368 of the support for magic trailing commas.
5369 5.35.3 - 2020-09-15
5371 This release significantly improves the performance of Hypothesis's in‐
5372 ternal implementation of automaton learning. However this code does not
5373 run as part of the user-accessible API so this has no user-visible im‐
5374 pact.
5375 5.35.2 - 2020-09-14
5377 This patch ensures that, when the generate phases is disabled, we can
5378 replay up to max_examples examples from the database - which is very
5379 useful when using Hypothesis with a fuzzer.
5380 Thanks to Afrida Tabassum for fixing issue #2585!
5382 5.35.1 - 2020-09-14
5384 This patch changes some internal python:struct.Struct.format strings
5385 from bytes to str, to avoid python:BytesWarning when running python
5386 -bb.
5387 Thanks to everyone involved in pytest-xdist issue 596, bpo-16349,
5389 bpo-21071, and bpo-41777 for their work on this - it was a remarkably
5390 subtle issue!
5391 5.35.0 - 2020-09-11
5393 The target() function now accepts integers as well as floats.
5394 5.34.1 - 2020-09-11
5396 This patch adds explicit Optional annotations to our public API, to
5397 better support users who run mypy with --strict or no_implicit_op‐
5398 tional=True.
5399 Thanks to Krzysztof Przybyła for bringing this to our attention and
5401 writing the patch!
5402 5.34.0 - 2020-09-11
5404 This release drops support for Python 3.5, which reached end of life
5405 upstream on 2020-09-13.
5406 5.33.2 - 2020-09-09
5408 This patch fixes a problem with builds() that was not able to generate
5409 valid data for annotated classes with constructors.
5410 Thanks to Nikita Sobolev for fixing issue #2603!
5412 5.33.1 - 2020-09-07
5414 This patch improves the error message from the hypothesis write command
5415 if black (required for the ghostwriter) is not installed.
5416 Thanks to Nikita Sobolev for fixing issue #2604!
5418 5.33.0 - 2020-09-06
5420 When reporting failing examples, or tried examples in verbose mode, Hy‐
5421 pothesis now identifies which were from @example(...) explicit exam‐
5422 ples.
5423 5.32.1 - 2020-09-06
5425 This patch contains some internal refactoring. Thanks to Felix Sheldon
5426 for fixing issue #2516!
5427 5.32.0 - 2020-09-04
5429 An array drawn from arrays() will own its own memory; previously most
5430 arrays returned by this strategy were views.
5431 5.31.0 - 2020-09-04
5433 builds() will use the __signature__ attribute of the target, if it ex‐
5434 ists, to retrieve type hints. Previously python:typ‐
5435 ing.get_type_hints(), was used by default. If argument names varied
5436 between the __annotations__ and __signature__, they would not be sup‐
5437 plied to the target.
5438 This was particularily an issue for pydantic models which use an alias
5440 generator.
5441 5.30.1 - 2020-09-04
5443 This patch makes the ghostwriter much more robust when passed unusual
5444 modules.
5445 • improved support for non-resolvable type annotations
5447 • magic() can now write equivalent() tests
5449 • running magic() on modules where some names in __all__ are undefined
5451 skips such names, instead of raising an error
5452 • magic() now knows to skip mocks
5454 • improved handling of import-time errors found by the ghostwriter CLI
5456 5.30.0 - 2020-08-30
5458 register_type_strategy() now supports python:typing.TypeVar, which was
5459 previously hard-coded, and allows a variety of types to be generated
5460 for an unconstrained TypeVar instead of just text().
5461 Thanks again to Nikita Sobolev for all your work on advanced types!
5463 5.29.4 - 2020-08-28
5465 This release fixes some hard to trigger bugs in Hypothesis's automata
5466 learning code. This code is only run as part of the Hypothesis build
5467 process, and not for user code, so this release has no user visible im‐
5468 pact.
5469 5.29.3 - 2020-08-27
5471 This patch adds type annotations to the hypothesis.database module.
5472 There is no runtime change, but your typechecker might notice.
5473 5.29.2 - 2020-08-27
5475 This patch tracks some additional information in Hypothesis internals,
5476 and has no user-visible impact.
5477 5.29.1 - 2020-08-27
5479 This release fixes a bug in some Hypothesis internal support code for
5480 learning automata. This mostly doesn't have any user visible impact,
5481 although it slightly affects the learned shrink passes so shrinking may
5482 be subtly different.
5483 5.29.0 - 2020-08-24
5485 This release adds support for entry-points, which allows for smoother
5486 integration of third-party Hypothesis extensions and external li‐
5487 braries. Unless you're publishing a library with Hypothesis integra‐
5488 tion, you'll probably only ever use this indirectly!
5489 5.28.0 - 2020-08-24
5491 from_type() can now resolve TypeVar instances when the bound is a For‐
5492 wardRef, so long as that name is in fact defined in the same module as
5493 the typevar (no TYPE_CHECKING tricks, sorry). This feature requires
5494 Python 3.7 or later.
5495 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
5497 5.27.0 - 2020-08-20
5499 This patch adds two new ghostwriters to test binary operations, like
5500 python:operator.add(), and Numpy ufuncs and gufuncs like np.matmul().
5501 5.26.1 - 2020-08-19
5503 This release improves the performance of some methods in Hypothesis's
5504 internal automaton library. These are currently only lightly used by
5505 user code, but this may result in slightly faster shrinking.
5506 5.26.0 - 2020-08-17
5508 register_type_strategy() no longer accepts parametrised user-defined
5509 generic types, because the resolution logic was quite badly broken (‐
5510 issue #2537).
5511 Instead of registering a strategy for e.g. MyCollection[int], you
5513 should register a function for MyCollection and inspect the type param‐
5514 eters within that function.
5515 Thanks to Nikita Sobolev for the bug report, design assistance, and
5517 pull request to implement this feature!
5518 5.25.0 - 2020-08-16
5520 Tired of writing tests? Or new to Hypothesis and not sure where to
5521 start?
5522 This release is for you! With our new Ghostwriter functions and hy‐
5524 pothesis write ... command-line interface, you can stop writing tests
5525 entirely... or take the source code Hypothesis writes for you as a
5526 starting point.
5527 This has been in the works for months, from issue #2118 to versions
5529 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who
5530 reviewed pull requests or commented on demos, and to Timothy Crosley's
5531 hypothesis-auto project for inspiration.
5532 5.24.4 - 2020-08-14
5534 This patch adds yet more internal functions to support a new feature
5535 we're working on, like version 5.18.3 and version 5.23.6. We promise
5536 it's worth the wait!
5537 5.24.3 - 2020-08-13
5539 This release fixes a small internal bug in Hypothesis's internal autom‐
5540 aton library. Fortunately this bug was currently impossible to hit in
5541 user facing code, so this has no user visible impact.
5542 5.24.2 - 2020-08-12
5544 This release improves shrink quality by allowing Hypothesis to automat‐
5545 ically learn new shrink passes for difficult to shrink tests.
5546 The automatic learning is not currently accessible in user code (it
5548 still needs significant work on robustness and performance before it is
5549 ready for that), but this release includes learned passes that should
5550 improve shrinking quality for tests which use any of the text(),
5551 floats(), datetimes(), emails(), and complex_numbers() strategies.
5552 5.24.1 - 2020-08-12
5554 This patch updates some docstrings, without changing runtime behaviour.
5555 5.24.0 - 2020-08-10
5557 The functions() strategy has a new argument pure=True, which ensures
5558 that the same return value is generated for identical calls to the gen‐
5559 erated function (issue #2538).
5560 Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!
5562 5.23.12 - 2020-08-10
5564 This release removes a number of Hypothesis's internal "shrink passes"
5565 - transformations it makes to a generated test case during shrinking -
5566 which appeared to be redundant with other transformations.
5567 It is unlikely that you will see much impact from this. If you do, it
5569 will likely show up as a change in shrinking performance (probably
5570 slower, maybe faster), or possibly in worse shrunk results. If you en‐
5571 counter the latter, please let us know.
5572 5.23.11 - 2020-08-04
5574 This release fixes a bug in some internal Hypothesis support code. It
5575 has no user visible impact.
5576 5.23.10 - 2020-08-04
5578 This release improves the quality of shrunk test cases in some special
5579 cases. Specifically, it should get shrinking unstuck in some scenarios
5580 which require simultaneously changing two parts of the generated test
5581 case.
5582 5.23.9 - 2020-08-03
5584 This release improves the performance of some internal support code. It
5585 has no user visible impact, as that code is not currently run during
5586 normal Hypothesis operation.
5587 5.23.8 - 2020-07-31
5589 This release adds a heuristic to detect when shrinking has finished de‐
5590 spite the fact that there are many more possible transformations to
5591 try. This will be particularly useful for tests where the minimum fail‐
5592 ing test case is very large despite there being many smaller test cases
5593 possible, where it is likely to speed up shrinking dramatically.
5594 In some cases it is likely that this will result in worse shrunk test
5596 cases. In those cases rerunning the test will result in further shrink‐
5597 ing.
5598 5.23.7 - 2020-07-29
5600 This release makes some performance improvements to shrinking. They
5601 should only be noticeable for tests that are currently particularly
5602 slow to shrink.
5603 5.23.6 - 2020-07-29
5605 This patch adds some more internal functions to support a new feature
5606 we're working on, like version 5.18.3. There is still no user-visible
5607 change... yet.
5608 5.23.5 - 2020-07-29
5610 This release makes some changes to internal support code that is not
5611 currently used in production Hypothesis. It has no user visible effect
5612 at present.
5613 5.23.4 - 2020-07-29
5615 This release improves shrinking quality in some special cases.
5616 5.23.3 - 2020-07-27
5618 This release fixes issue #2507, where lazy evaluation meant that the
5619 values drawn from a sampled_from() strategy could depend on mutations
5620 of the sampled sequence that happened after the strategy was construc‐
5621 ted.
5622 5.23.2 - 2020-07-27
5624 This patch fixes issue #2462, a bug in our handling of unittest.Test‐
5625 Case.subTest(). Thanks to Israel Fruchter for fixing this at the Eu‐
5626 roPython sprints!
5627 5.23.1 - 2020-07-26
5629 This release improves the behaviour of the characters() strategy when
5630 shrinking, by changing which characters are considered smallest to pre‐
5631 fer more "normal" ascii characters where available.
5632 5.23.0 - 2020-07-26
5634 The default print_blob setting is now smarter. It defaults to True in
5635 CI and False for local development.
5636 Thanks to Hugo van Kemenade for implementing this feature at the Eu‐
5638 roPython sprints!
5639 5.22.0 - 2020-07-25
5641 The slices() strategy can now generate slices for empty sequences,
5642 slices with negative start and stop indices (from the end of the se‐
5643 quence), and step=None in place of step=1.
5644 Thanks to Sangarshanan for implementing this feature at the EuroPython
5646 sprints!
5647 5.21.0 - 2020-07-23
5649 This release ensures that tests which raise RecursionError are not re‐
5650 ported as flaky simply because we run them from different initial stack
5651 depths (issue #2494).
5652 5.20.4 - 2020-07-23
5654 This release improves the performance of the sample method on objects
5655 obtained from randoms() when use_true_random=False. This should mostly
5656 only be noticeable when the sample size is a large fraction of the pop‐
5657 ulation size, but may also help avoid health check failures in some
5658 other cases.
5659 5.20.3 - 2020-07-21
5661 This release makes some internal changes for testing purposes and
5662 should have no user visible effect.
5663 5.20.2 - 2020-07-18
5665 This release fixes a small caching bug in Hypothesis internals that may
5666 under some circumstances have resulted in a less diverse set of test
5667 cases being generated than was intended.
5668 Fixing this problem revealed some performance problems that could occur
5670 during targeted property based testing, so this release also fixes
5671 those. Targeted property-based testing should now be significantly
5672 faster in some cases, but this may be at the cost of reduced effective‐
5673 ness.
5674 5.20.1 - 2020-07-17
5676 This patch updates our formatting to use isort 5. There is no
5677 user-visible change.
5678 5.20.0 - 2020-07-17
5680 The basic_indices() strategy can now generate bare indexers in place of
5681 length-one tuples. Thanks to Andrea for this patch!
5682 5.19.3 - 2020-07-15
5684 This patch removes an internal use of distutils in order to avoid this
5685 setuptools warning for some users.
5686 5.19.2 - 2020-07-13
5688 This patch contains a small internal refactoring with no user-visible
5689 impact.
5690 Thanks to Andrea for writing this at the SciPy 2020 Sprints!
5692 5.19.1 - 2020-07-12
5694 This release slightly improves shrinking behaviour. This should mainly
5695 only impact stateful tests, but may have some minor positive impact on
5696 shrinking collections (lists, sets, etc).
5697 5.19.0 - 2020-06-30
5699 This release improves the randoms() strategy by adding support for Ran‐
5700 dom instances where Hypothesis generates the random values rather than
5701 having them be "truly" random.
5702 5.18.3 - 2020-06-27
5704 This patch adds some internal functions to support a new feature we're
5705 working on. There is no user-visible change... yet.
5706 5.18.2 - 2020-06-26
5708 This patch improves our docs for the derandomize setting.
5709 5.18.1 - 2020-06-25
5711 This release consists of some internal refactoring to the shrinker in
5712 preparation for future work. It has no user visible impact.
5713 5.18.0 - 2020-06-22
5715 This release teaches Hypothesis to shorten tracebacks for explicit ex‐
5716 amples, as we already do for generated examples, so that you can focus
5717 on your code rather than ours.
5718 If you have multiple failing explicit examples, they will now all be
5720 reported. To report only the first failure, you can use the re‐
5721 port_multiple_bugs=False setting as for generated examples.
5722 5.17.0 - 2020-06-22
5724 This patch adds strategy inference for the Literal, NewType, Type, De‐
5725 faultDict, and TypedDict types from the typing_extensions backport on
5726 PyPI.
5727 5.16.3 - 2020-06-21
5729 This patch precomputes some of the setup logic for our experimental ex‐
5730 ternal fuzzer integration and sets deadline=None in fuzzing mode, sav‐
5731 ing around 150us on each iteration.
5732 This is around two-thirds the runtime to fuzz an empty test with
5734 @given(st.none()), and nice to have even as a much smaller fraction of
5735 the runtime for non-trivial tests.
5736 5.16.2 - 2020-06-19
5738 This patch fixes an internal error when warning about the use of func‐
5739 tion-scoped fixtures for parametrised tests where the parametrised
5740 value contained a % character. Thanks to Bryant for reporting and fix‐
5741 ing this bug!
5742 5.16.1 - 2020-06-10
5744 If you pass a python:list or python:tuple where a strategy was ex‐
5745 pected, the error message now mentions sampled_from() as an example
5746 strategy.
5747 Thanks to the enthusiastic participants in the PyCon Mentored Sprints
5749 who suggested adding this hint.
5750 5.16.0 - 2020-05-27
5752 functions() can now infer the appropriate returns strategy if you pass
5753 a like function with a return-type annotation. Before, omitting the
5754 returns argument would generate functions that always returned None.
5755 5.15.1 - 2020-05-21
5757 Fix from_type() with generic types under Python 3.9.
5758 5.15.0 - 2020-05-19
5760 This patch fixes an error that happens when multiple threads create new
5761 strategies.
5762 5.14.0 - 2020-05-13
5764 Passing min_magnitude=None to complex_numbers() is now deprecated - you
5765 can explicitly pass min_magnitude=0, or omit the argument entirely.
5766 5.13.1 - 2020-05-13
5768 This patch fixes an internal error in from_type() for python:typ‐
5769 ing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and
5770 fixing issue #2427!
5771 5.13.0 - 2020-05-12
5773 This release upgrades the test statistics available via the --hypothe‐
5774 sis-show-statistics option to include separate information on each of
5775 the phases (issue #1555).
5776 5.12.2 - 2020-05-12
5778 This patch teaches the from_type() internals to return slightly more
5779 efficient strategies for some generic sets and mappings.
5780 5.12.1 - 2020-05-12
5782 This patch adds a # noqa comment for flake8 3.8.0, which disagrees with
5783 mypy about how to write the type of ....
5784 5.12.0 - 2020-05-10
5786 This release limits the maximum duration of the shrinking phase to five
5787 minutes, so that Hypothesis does not appear to hang when making very
5788 slow progress shrinking a failing example (issue #2340).
5789 If one of your tests triggers this logic, we would really appreciate a
5791 bug report to help us improve the shrinker for difficult but realistic
5792 workloads.
5793 5.11.0 - 2020-05-07
5795 This release improves the interaction between assume() and the @exam‐
5796 ple() decorator, so that the following test no longer fails with Unsat‐
5797 isfiedAssumption (issue #2125):
5798 @given(value=floats(0, 1))
5800 @example(value=0.56789) # used to make the test fail!
5801 @pytest.mark.parametrize("threshold", [0.5, 1])
5802 def test_foo(threshold, value):
5803 assume(value < threshold)
5804 ...
5805 5.10.5 - 2020-05-04
5807 If you have django installed but don't use it, this patch will make im‐
5808 port hypothesis a few hundred milliseconds faster (e.g. 0.704s ->
5809 0.271s).
5810 Thanks to importtime-waterfall for highlighting this problem and Jake
5812 Vanderplas for the solution - it's impossible to misuse code from a
5813 module you haven't imported!
5814 5.10.4 - 2020-04-24
5816 This patch improves the internals of builds() type inference, to handle
5817 recursive forward references in certain dataclasses. This is useful
5818 for e.g. hypothesmith's forthcoming LibCST mode.
5819 5.10.3 - 2020-04-22
5821 This release reverses the order in which some operations are tried dur‐
5822 ing shrinking. This should generally be a slight performance improve‐
5823 ment, but most tests are unlikely to notice much difference.
5824 5.10.2 - 2020-04-22
5826 This patch fixes issue #2406, where use of pandas:pandas.Timestamp ob‐
5827 jects as bounds for the datetimes() strategy caused an internal error.
5828 This bug was introduced in version 5.8.1.
5829 5.10.1 - 2020-04-19
5831 This release is a small internal refactoring to how shrinking interacts
5832 with targeted property-based testing that should have no user user vis‐
5833 ible impact.
5834 5.10.0 - 2020-04-18
5836 This release improves our support for datetimes and times around DST
5837 transitions.
5838 times() and datetimes() are now sometimes generated with fold=1, indi‐
5840 cating that they represent the second occurrence of a given wall-time
5841 when clocks are set backwards. This may be set even when there is no
5842 transition, in which case the fold value should be ignored.
5843 For consistency, timezones provided by the pytz package can now gener‐
5845 ate imaginary times (such as the hour skipped over when clocks 'spring
5846 forward' to daylight saving time, or during some historical timezone
5847 transitions). All other timezones have always supported generation of
5848 imaginary times.
5849 If you prefer the previous behaviour, datetimes() now takes an argument
5851 allow_imaginary which defaults to True but can be set to False for any
5852 timezones strategy.
5853 5.9.1 - 2020-04-16
5855 This patch fixes the rendering of binary() docstring by using the
5856 proper backticks syntax.
5857 5.9.0 - 2020-04-15
5859 Failing tests which use target() now report the highest score observed
5860 for each target alongside the failing example(s), even without explic‐
5861 itly showing test statistics.
5862 This improves the debugging workflow for tests of accuracy, which as‐
5864 sert that the total imprecision is within some error budget - for exam‐
5865 ple, abs(a - b) < 0.5. Previously, shrinking to a minimal failing ex‐
5866 ample could often make errors seem smaller or more subtle than they re‐
5867 ally are (see the threshold problem, and issue #2180).
5868 5.8.6 - 2020-04-15
5870 This patch improves the docstring of binary(), the python:repr() of
5871 sampled_from() on an python:enum.Enum subclass, and a warning in our
5872 pytest plugin. There is no change in runtime behaviour.
5873 5.8.5 - 2020-04-15
5875 This release (potentially very significantly) improves the performance
5876 of failing tests in some rare cases, mostly only relevant when using
5877 targeted property-based testing, by stopping further optimisation of
5878 unrelated test cases once a failing example is found.
5879 5.8.4 - 2020-04-14
5881 This release fixes issue #2395, where under some circumstances targeted
5882 property-based testing could cause Hypothesis to get caught in an infi‐
5883 nite loop.
5884 5.8.3 - 2020-04-12
5886 This patch teaches builds() and from_type() to use the __signature__
5887 attribute of classes where it has been set, improving our support for
5888 Pydantic models (in pydantic >= 1.5).
5889 5.8.2 - 2020-04-12
5891 This release improves the performance of the part of the core engine
5892 that deliberately generates duplicate values.
5893 5.8.1 - 2020-04-12
5895 This patch improves dates() shrinking, to simplify year, month, and day
5896 like datetimes() rather than minimizing the number of days since
5897 2000-01-01.
5898 5.8.0 - 2020-03-24
5900 This release adds a .hypothesis.fuzz_one_input attribute to @given
5901 tests, for easy integration with external fuzzers such as python-afl
5902 (supporting issue #171).
5903 5.7.2 - 2020-03-24
5905 This patch fixes issue #2341, ensuring that the printed output from a
5906 stateful test cannot use variable names before they are defined.
5907 5.7.1 - 2020-03-23
5909 This patch fixes issue #2375, preventing incorrect failure when a func‐
5910 tion scoped fixture is overridden with a higher scoped fixture.
5911 5.7.0 - 2020-03-19
5913 This release allows the array_dtypes() strategy to generate Numpy
5914 dtypes which have field titles in addition to field names. We expect
5915 this to expose latent bugs where code expects that set(dtype.names) ==
5916 set(dtype.fields), though the latter may include titles.
5917 5.6.1 - 2020-03-18
5919 This makes model a positional-only argument to from_model(), to support
5920 models with a field literally named "model" (issue #2369).
5921 5.6.0 - 2020-02-29
5923 This release adds an explicit warning for tests that are both decorated
5924 with @given(...) and request a function-scoped pytest fixture, because
5925 such fixtures are only executed once for all Hypothesis test cases and
5926 that often causes trouble (issue #377).
5927 It's very difficult to fix this on the pytest side, so since 2015 our
5929 advice has been "just don't use function-scoped fixtures with Hypothe‐
5930 sis". Now we detect and warn about the issue at runtime!
5931 5.5.5 - 2020-02-29
5933 This release cleans up the internal machinery for stateful, after we
5934 dropped the legacy APIs in Hypothesis 5.0 (issue #2218). There is no
5935 user-visible change.
5936 5.5.4 - 2020-02-16
5938 This patch fixes issue #2351, arrays() would raise a confusing error if
5939 we inferred a strategy for datetime64 or timedelta64 values with vary‐
5940 ing time units.
5941 We now infer an internally-consistent strategy for such arrays, and
5943 have a more helpful error message if an inconsistent strategy is ex‐
5944 plicitly specified.
5945 5.5.3 - 2020-02-14
5947 This patch improves the signature of builds() by specifying target as a
5948 positional-only argument on Python 3.8 (see PEP 570). The semantics of
5949 builds() have not changed at all - this just clarifies the documenta‐
5950 tion.
5951 5.5.2 - 2020-02-13
5953 This release makes Hypothesis faster at generating test cases that con‐
5954 tain duplicated values in their inputs.
5955 5.5.1 - 2020-02-07
5957 This patch has some tiny internal code clean-ups, with no user-visible
5958 change.
5959 5.5.0 - 2020-02-07
5961 Our style guide suggests that optional parameters should usually be
5962 keyword-only arguments (see PEP 3102) to prevent confusion based on po‐
5963 sitional arguments - for example, hypothesis.strategies.floats() takes
5964 up to four boolean flags and many of the Numpy strategies have both
5965 dims and side bounds.
5966 This release converts most optional parameters in our API to use key‐
5968 word-only arguments - and adds a compatibility shim so you get warnings
5969 rather than errors everywhere (issue #2130).
5970 5.4.2 - 2020-02-06
5972 This patch fixes compatibility with Python 3.5.2 (issue #2334). Note
5973 that we only test the latest patch of each minor version, though as in
5974 this case we usually accept pull requests for older patch versions.
5975 5.4.1 - 2020-02-01
5977 This patch improves the repr of from_type(), so that in most cases it
5978 will display the strategy it resolves to rather than from_type(...).
5979 The latter form will continue to be used where resolution is not imme‐
5980 diately successful, e.g. invalid arguments or recursive type defini‐
5981 tions involving forward references.
5982 5.4.0 - 2020-01-30
5984 This release removes support for Python 3.5.0 and 3.5.1, where the
5985 python:typing module was quite immature (e.g. missing overload() and
5986 Type).
5987 Note that Python 3.5 will reach its end-of-life in September 2020, and
5989 new releases of Hypothesis may drop support somewhat earlier.
5990 NOTE:
5992 pip install hypothesis should continue to give you the latest com‐
5993 patible version. If you have somehow ended up with an incompatible
5994 version, you need to update your packaging stack to pip >= 9.0 and
5995 setuptools >= 24.2 - see here for details. Then pip uninstall hy‐
5996 pothesis && pip install hypothesis will get you back to a compatible
5997 version.
5998 5.3.1 - 2020-01-26
6000 This patch does some minor internal cleanup; there is no user-visible
6001 change.
6002 5.3.0 - 2020-01-21
6004 The standard library ipaddress module is new in Python 3, and this re‐
6005 lease adds the new ip_addresses() strategy to generate IPv4Addresses
6006 and/or IPv6Addresses (depending on the v and network arguments).
6007 If you use them in type annotations, from_type() now has strategies
6009 registered for ipaddress address, network, and interface types.
6010 The provisional strategies for IP address strings are therefore depre‐
6012 cated.
6013 5.2.1 - 2020-01-21
6015 This patch reverts version 5.2, due to a strange issue where indexing
6016 an array of strings can raise an error instead of returning an item
6017 which contains certain surrogate characters.
6018 5.2.0 - 2020-01-19
6020 This release allows from_dtype() to generate Unicode strings which can‐
6021 not be encoded in UTF-8, but are valid in Numpy arrays (which use
6022 UTF-32).
6023 5.1.6 - 2020-01-19
6025 This patch fixes issue #2320, where from_type(Set[Hashable]) could
6026 raise an internal error because Decimal("snan") is of a hashable type,
6027 but raises an error when hashed. We now ensure that set elements and
6028 dict keys in generic types can actually be hashed.
6029 5.1.5 - 2020-01-12
6031 This patch fixes an internal error when running in an IPython repl or
6032 Jupyter notebook on Windows (issue #2319), and an internal error on
6033 Python 3.5.1 (issue #2318).
6034 5.1.4 - 2020-01-11
6036 This patch fixes a bug where errors in third-party extensions such as
6037 hypothesis-trio or hypothesis-jsonschema were incorrectly considered to
6038 be Hypothesis internal errors, which could result in confusing error
6039 messages.
6040 Thanks to Vincent Michel for reporting and fixing the bug!
6042 5.1.3 - 2020-01-11
6044 This release converts the type hint comments on our public API to PEP
6045 484 type annotations.
6046 Thanks to Ivan Levkivskyi for com2ann - with the refactoring tools from
6048 5.0.1 it made this process remarkably easy!
6049 5.1.2 - 2020-01-09
6051 This patch makes multiple() iterable, so that output like a, b =
6052 state.some_rule() is actually executable and can be used to reproduce
6053 failing examples.
6054 Thanks to Vincent Michel for reporting and fixing issue #2311!
6056 5.1.1 - 2020-01-06
6058 This patch contains many small refactorings to replace our Python 2
6059 compatibility functions with their native Python 3 equivalents. Since
6060 Hypothesis is now Python 3 only, there is no user-visible change.
6061 5.1.0 - 2020-01-03
6063 This release teaches from_type() how to generate python:datetime.time‐
6064 zone. As a result, you can now generate python:datetime.tzinfo objects
6065 without having pytz installed.
6066 If your tests specifically require pytz timezones, you should be using
6068 hypothesis.extra.pytz.timezones() instead of st.from_type(tzinfo).
6069 5.0.1 - 2020-01-01
6071 This patch contains mostly-automated refactorings to remove code that
6072 we only needed to support Python 2. Since Hypothesis is now Python 3
6073 only (hurray!), there is no user-visible change.
6074 Our sincere thanks to the authors of autoflake, black, isort, and
6076 pyupgrade, who have each and collectively made this kind of update
6077 enormously easier.
6078 5.0.0 - 2020-01-01
6080 Welcome to the next major version of Hypothesis!
6081 There are no new features here, as we release those in minor versions.
6083 Instead, 5.0 is a chance for us to remove deprecated features (many al‐
6084 ready converted into no-ops), and turn a variety of warnings into er‐
6085 rors.
6086 If you were running on the last version of Hypothesis 4.x without any
6088 Hypothesis deprecation warnings, this will be a very boring upgrade.
6089 In fact, nothing will change for you at all.
6090 NOTE:
6092 This release drops support for Python 2, which has passed its end of
6093 life date. The Python 3 Statement outlines our reasons, and lists
6094 many other packages that have made the same decision.
6095 pip install hypothesis should continue to give you the latest com‐
6097 patible version. If you have somehow ended up with Hypothesis 5.0
6098 on Python 2, you need to update your packaging stack to pip >= 9.0
6099 and setuptools >= 24.2 - see here for details. Then pip uninstall
6100 hypothesis && pip install hypothesis will get you back to a compati‐
6101 ble version.
6102 Strategies
6104 • integers() bounds must be equal to an integer, though they can still
6105 be other types.
6106 • If fractions() is passed a max_denominator, the bounds must have at
6108 most that denominator.
6109 • floats() bounds must be exactly representable as a floating-point
6111 number with the given width. If not, the error message includes the
6112 nearest such number.
6113 • sampled_from([]) is now an error.
6115 • The values from the elements and fill strategies for hypothesis.ex‐
6117 tra.numpy.arrays() must be losslessly representable in an array of
6118 the given dtype.
6119 • The min_size and max_size arguments to all collection strategies must
6121 be of type python:int (or max_size may be None).
6122 Miscellaneous
6124 • The .example() method of strategies (intended for interactive explo‐
6125 ration) no longer takes a random argument.
6126 • It is now an error to apply @example, @seed, or @reproduce_failure
6128 without also applying @given.
6129 • You may pass either the target or targets argument to stateful rules,
6131 but not both.
6132 • deadline must be None (to disable), a timedelta, or an integer or
6134 float number of milliseconds.
6135 • Both of derandomize and print_blob must be either True or False,
6137 where they previously accepted other values.
6138 • stateful_step_count must be at least one.
6140 • max_examples must be at least one. To disable example generation,
6142 use the phases setting.
6143 Removals
6145 • hypothesis.stateful.GenericStateMachine in favor of hypothesis.state‐
6146 ful.RuleBasedStateMachine
6147 • hypothesis.extra.django.models.models in favor of hypothesis.ex‐
6149 tra.django.from_model() and hypothesis.extra.django.models.add_de‐
6150 fault_field_mapping in favor of hypothesis.extra.django.regis‐
6151 ter_field_strategy()
6152 • hypothesis.HealthCheck.hung_test, without replacement
6154 • hypothesis.settings.buffer, without replacement
6156 • hypothesis.PrintSettings, because hypothesis.settings.print_blob
6158 takes True or False
6159 • hypothesis.settings.timeout, in favor of hypothesis.settings.deadline
6161 • hypothesis.unlimited without replacement (only only useful as argu‐
6163 ment to timeout)
6164 Hypothesis 4.x
6166 4.57.1 - 2019-12-29
6167 This patch improves the type hints and documentation for the django ex‐
6168 tra. There is no runtime change.
6169 4.57.0 - 2019-12-28
6171 This release improves support for the SupportsOp protocols from the
6172 python:typing module when using on from_type() as outlined in issue
6173 #2292. The following types now generate much more varied strategies
6174 when called with from_type():
6175 • python:typing.SupportsAbs
6177 • python:typing.SupportsBytes
6179 • python:typing.SupportsComplex
6181 • python:typing.SupportsInt
6183 • python:typing.SupportsFloat
6185 • python:typing.SupportsRound
6187 Note that using from_type() with one of the above strategies will not
6189 ensure that the the specified function will execute successfully (ie :
6190 the strategy returned for from_type(typing.SupportsAbs) may include
6191 NaNs or things which cause the python:abs() function to error. )
6192 Thanks to Lea Provenzano for this patch.
6194 4.56.3 - 2019-12-22
6196 This release fixes a small internal bug in shrinking which could have
6197 caused it to perform slightly more tests than were necessary. Fixing
6198 this shouldn't have much effect but it will make shrinking slightly
6199 faster.
6200 4.56.2 - 2019-12-21
6202 This release removes an internal heuristic that was no longer providing
6203 much benefit. It is unlikely that there will be any user visible ef‐
6204 fect.
6205 4.56.1 - 2019-12-19
6207 This release further improves the optimisation algorithm for targeted
6208 property-based testing.
6209 4.56.0 - 2019-12-18
6211 This release enables deprecation warnings even when the verbosity set‐
6212 ting is quiet, in preparation for Hypothesis 5.0 (issue #2218).
6213 Warnings can still be filtered by the standard mechanisms provided in
6215 the standard-library python:warnings module.
6216 4.55.4 - 2019-12-18
6218 This release improves Hypothesis's management of the set of test cases
6219 it tracks between runs. It will only do anything if you have Phase.tar‐
6220 get enabled and an example database set. In those circumstances it
6221 should result in a more thorough and faster set of examples that are
6222 tried on each run.
6223 4.55.3 - 2019-12-18
6225 This release makes Hypothesis better at generating test cases where
6226 generated values are duplicated in different parts of the test case.
6227 This will be especially noticeable with reasonably complex values, as
6228 it was already able to do this for simpler ones such as integers or
6229 floats.
6230 4.55.2 - 2019-12-17
6232 This release expands the set of test cases that Hypothesis saves in its
6233 database for future runs to include a representative set of "struc‐
6234 turally different" test cases - e.g. it might try to save test cases
6235 where a given list is empty or not.
6236 Currently this is unlikely to have much user visible impact except to
6238 produce slightly more consistent behaviour between consecutive runs of
6239 a test suite. It is mostly groundwork for future improvements which
6240 will exploit this functionality more effectively.
6241 4.55.1 - 2019-12-16
6243 This patch fixes issue #2257, where from_type() could incorrectly gen‐
6244 erate bytestrings when passed a generic python:typing.Sequence such as
6245 Sequence[set].
6246 4.55.0 - 2019-12-16
6248 This release adds database support for targeted property-based testing,
6249 so the best examples based on the targeting will be saved and reused
6250 between runs. This is mostly laying groundwork for future features in
6251 this area, but will also make targeted property-based tests more useful
6252 during development, where the same tests tend to get run over and over
6253 again.
6254 If max_examples is large, this may increase memory usage significantly
6256 under some circumstances, but these should be relatively rare.
6257 This release also adds a dependency on the sortedcontainers package.
6259 4.54.2 - 2019-12-16
6261 This release improves the optimisation algorithm for targeted prop‐
6262 erty-based testing, so that it will find higher quality results more
6263 reliably. Specifically, in cases where it would previously have got
6264 near a local optimum, it will now tend to achieve the locally optimal
6265 value.
6266 4.54.1 - 2019-12-16
6268 This release is mostly internal changes in support of better testing of
6269 the core engine. You are unlikely to see much effect, although some in‐
6270 ternal heuristics have changed slightly.
6271 4.54.0 - 2019-12-15
6273 This release adds a dedicated phase for targeted property-based test‐
6274 ing, and (somewhat) improves the targeting algorithm so that it will
6275 find higher quality results more reliably. This comes at a cost of
6276 making it more likely to get stuck in a local optimum.
6277 4.53.3 - 2019-12-15
6279 This patch fixes from_type() with python:typing.Hashable and
6280 python:typing.Sized, which previously failed with an internal error on
6281 Python 3.7 or later.
6282 Thanks to Lea Provenzano for both reporting issue #2272 and writing the
6284 patch!
6285 4.53.2 - 2019-12-11
6287 This release reorganises a number of the Hypothesis internal modules
6288 into a package structure. If you are only depending on the public API
6289 it should have no effect. If you are depending on the internal API
6290 (which you shouldn't be, and which we don't guarantee compatibility on)
6291 you may have to rename some imports.
6292 4.53.1 - 2019-12-09
6294 This release changes the size distribution of the number of steps run
6295 in stateful testing: It will now almost always run the maximum number
6296 of steps permitted.
6297 4.53.0 - 2019-12-09
6299 statistics now include the best score seen for each label, which can
6300 help avoid the threshold problem when the minimal example shrinks
6301 right down to the threshold of failure (issue #2180).
6302 4.52.0 - 2019-12-09
6304 This release changes the stateful_step_count setting to raise an error
6305 if set to 0. This is a backwards compatible change because a value of 0
6306 would never have worked and attempting to run it would have resulted in
6307 an internal assertion error.
6308 4.51.1 - 2019-12-09
6310 This release makes a small internal change to the distribution of test
6311 cases. It is unlikely to have much user visible impact.
6312 4.51.0 - 2019-12-07
6314 This release deprecates use of @example, @seed, or @reproduce_failure
6315 without @given.
6316 Thanks to Nick Anyos for the patch!
6318 4.50.8 - 2019-12-05
6320 This patch makes certain uses of Bundles more efficient in stateful
6321 testing (issue #2078).
6322 4.50.7 - 2019-12-05
6324 This release refactors some of Hypothesis's internal interfaces for
6325 representing data generation. It should have no user visible effect.
6326 4.50.6 - 2019-12-02
6328 This patch removes some old debugging helpers in our Numpy extra which
6329 have not been needed since issue #1963 and issue #2245.
6330 4.50.5 - 2019-12-01
6332 This patch fixes issue #2229, where Numpy arrays of unsized strings
6333 would only ever have strings of size one due to an interaction between
6334 our generation logic and Numpy's allocation strategy.
6335 4.50.4 - 2019-12-01
6337 This patch fixes a rare internal error in strategies for a list of
6338 unique items sampled from a short non-unique sequence (issue #2247).
6339 The bug was discovered via hypothesis-jsonschema.
6340 4.50.3 - 2019-12-01
6342 This release improves the error message when @settings tries to inherit
6343 settings from a parent argument that isn't a settings instance.
6344 4.50.2 - 2019-11-29
6346 This release improves Hypothesis's "Falsifying example" output, by
6347 breaking output across multiple lines where necessary, and by removing
6348 irrelevant information from the stateful testing output.
6349 4.50.1 - 2019-11-29
6351 This patch adds flake8-comprehensions to our linter suite. There is no
6352 user-visible change - expect perhaps via some strange microbenchmarks -
6353 but certain parts of the code now have a clear and more consistent
6354 style.
6355 4.50.0 - 2019-11-28
6357 This release fixes some cases where we might previously have failed to
6358 run the validation logic for some strategies. As a result tests which
6359 would previously have been silently testing significantly less than
6360 they should may now start to raise InvalidArgument now that these er‐
6361 rors are caught.
6362 4.49.0 - 2019-11-28
6364 This release significantly improves the data distribution in rule based
6365 stateful testing, by using a technique called Swarm Testing (Groce,
6366 Alex, et al. "Swarm testing." Proceedings of the 2012 International
6367 Symposium on Software Testing and Analysis. ACM, 2012.) to select
6368 which rules are run in any given test case. This should allow it to
6369 find many issues that it would previously have missed.
6370 This change is likely to be especially beneficial for stateful tests
6372 with large numbers of rules.
6373 4.48.1 - 2019-11-28
6375 This release adds some heuristics to test case generation that try to
6376 ensure that test cases generated early on will be relatively small.
6377 This fixes a bug introduced in Hypothesis 4.42.0 which would cause oc‐
6379 casional too_slow failures on some tests.
6380 4.48.0 - 2019-11-28
6382 This release revokes the deprecation of find, as we've now rebuilt it
6383 on top of @given, which means it has minimal maintenance burden and
6384 we're happy to support it.
6385 4.47.5 - 2019-11-28
6387 This release rebuilds find() on top of @given in order to have more
6388 code in common. It should have minimal user visible effect.
6389 4.47.4 - 2019-11-27
6391 This patch removes an internal compatibility shim that we no longer
6392 need.
6393 4.47.3 - 2019-11-26
6395 This patch fixes several typos in our docstrings and comments, with no
6396 change in behaviour. Thanks to Dmitry Dygalo for identifying and fix‐
6397 ing them!
6398 4.47.2 - 2019-11-25
6400 This release fixes an internal issue where Hypothesis would sometimes
6401 generate test cases that were above its intended maximum size. This
6402 would only have happened rarely and probably would not have caused ma‐
6403 jor problems when it did.
6404 Users of the new targeted property-based testing might see minor im‐
6406 pact (possibly slightly faster tests and slightly worse target scores),
6407 but only in the unlikely event that they were hitting this problem.
6408 Other users should not see any effect at all.
6409 4.47.1 - 2019-11-24
6411 This release removes some unused code from the core engine. There is
6412 no user-visible change.
6413 4.47.0 - 2019-11-24
6415 This release commonizes some code between running explicit examples and
6416 normal test execution. The main user visible impact of this is that
6417 deadlines are now enforced when running explicit examples.
6418 4.46.1 - 2019-11-23
6420 This patch ensures that a KeyboardInterrupt received during example
6421 generation is not treated as a mystery test failure but instead propa‐
6422 gates to the top level, not recording the interrupted generation in the
6423 conjecture data tree. Thanks to Anne Archibald for identifying and
6424 fixing the problem.
6425 4.46.0 - 2019-11-22
6427 This release changes the behaviour of floats() when excluding signed
6428 zeros - floats(max_value=0.0, exclude_max=True) can no longer generate
6429 -0.0 nor the much rarer floats(min_value=-0.0, exclude_min=True) gener‐
6430 ate +0.0.
6431 The correct interaction between signed zeros and exclusive endpoints
6433 was unclear; we now enforce the invariant that floats() will never gen‐
6434 erate a value equal to an excluded endpoint (issue #2201).
6435 If you prefer the old behaviour, you can pass floats(max_value=-0.0) or
6437 floats(min_value=0.0) which is exactly equivalent and has not changed.
6438 If you had two endpoints equal to zero, we recommend clarifying your
6439 tests by using just() or sampled_from() instead of floats().
6440 4.45.1 - 2019-11-20
6442 This patch improves the error message when invalid arguments are passed
6443 to rule() or invariant() (issue #2149).
6444 Thanks to Benjamin Palmer for this bugfix!
6446 4.45.0 - 2019-11-20
6448 This release supports python:typing.Final and python:typing.TypedDict
6449 in from_type().
6450 4.44.5 - 2019-11-20
6452 This patch disables our pytest plugin when running on versions of
6453 pytest before 4.3, the oldest our plugin supports. Note that at time
6454 of writing the Pytest developers only support 4.6 and later!
6455 Hypothesis tests using @given() work on any test runner, but our inte‐
6457 grations to e.g. avoid example database collisions when combined with
6458 @pytest.mark.parametrize eventually drop support for obsolete versions.
6459 4.44.4 - 2019-11-20
6461 This patch adds some internal comments and clarifications to the Hy‐
6462 pothesis implementation. There is no user-visible change.
6463 4.44.3 - 2019-11-20
6465 This patch avoids importing test runners such as pytest, unittest2, or
6466 nose solely to access their special "skip test" exception types - if
6467 the module is not in sys.modules, the exception can't be raised anyway.
6468 This fixes a problem where importing an otherwise unused module could
6470 cause spurious errors due to import-time side effects (and possibly
6471 -Werror).
6472 4.44.2 - 2019-11-12
6474 This release fixes @given to only complain about missing keyword-only
6475 arguments if the associated test function is actually called.
6476 This matches the behaviour of other InvalidArgument errors produced by
6478 @given.
6479 4.44.1 - 2019-11-11
6481 This patch allows Hypothesis to run in environments that do not specify
6482 a __file__, such as a python:zipapp (issue #2196).
6483 4.44.0 - 2019-11-11
6485 This release adds a signature argument to mutually_broad‐
6486 castable_shapes() (issue #2174), which allows us to generate shapes
6487 which are valid for functions like numpy:numpy.matmul that require
6488 shapes which are not simply broadcastable.
6489 Thanks to everyone who has contributed to this feature over the last
6491 year, and a particular shout-out to Zac Hatfield-Dodds and Ryan Sok‐
6492 laski for mutually_broadcastable_shapes() and to Ryan Turner for the
6493 downstream hypothesis-gufunc project.
6494 4.43.9 - 2019-11-11
6496 This patch fixes issue #2108, where the first test using data() to draw
6497 from characters() or text() would be flaky due to unreliable test tim‐
6498 ings.
6499 Time taken by lazy instantiation of strategies is now counted towards
6501 drawing from the strategy, rather than towards the deadline for the
6502 test function.
6503 4.43.8 - 2019-11-08
6505 This release ensures that the strategies passed to @given are properly
6506 validated when applied to a test method inside a test class.
6507 This should result in clearer error messages when some of those strate‐
6509 gies are invalid.
6510 4.43.7 - 2019-11-08
6512 This release changes how Hypothesis manages its search space in cases
6513 where it generates redundant data. This should cause it to generate
6514 significantly fewer duplicated examples (especially with short integer
6515 ranges), and may cause it to produce more useful examples in some cases
6516 (especially ones where there is a significant amount of filtering).
6517 4.43.6 - 2019-11-07
6519 This patch refactors width handling in floats(); you may notice small
6520 performance improvements but the main purpose is to enable work on
6521 issue #1704 (improving shrinking of bounded floats).
6522 4.43.5 - 2019-11-06
6524 This patch removes an unused internal flag. There is no user-visible
6525 change.
6526 4.43.4 - 2019-11-05
6528 This patch corrects the exception type and error message you get if you
6529 attempt to use data() to draw from something which is not a strategy.
6530 This never worked, but the error is more helpful now.
6531 4.43.3 - 2019-11-05
6533 We've adopted flake8-bugbear to check for a few more style issues, and
6534 this patch implements the minor internal cleanups it suggested. There
6535 is no user-visible change.
6536 4.43.2 - 2019-11-05
6538 This patch fixes the formatting of some documentation, but there is no
6539 change to any executed code.
6540 4.43.1 - 2019-11-04
6542 Python 3.8's new python:typing.Literal type - see PEP 586 for details -
6543 is now supported in from_type().
6544 4.43.0 - 2019-11-04
6546 This release adds the strategy mutually_broadcastable_shapes(), which
6547 generates multiple array shapes that are mutually broadcast-compatible
6548 with an optional user-specified base-shape.
6549 This is a generalisation of broadcastable_shapes(). It relies heavily
6551 on non-public internals for performance when generating and shrinking
6552 examples. We intend to support generating shapes matching a ufunc sig‐
6553 nature in a future version (issue #2174).
6554 Thanks to Ryan Soklaski, Zac Hatfield-Dodds, and @rdturnermtl who con‐
6556 tributed to this new feature.
6557 4.42.10 - 2019-11-03
6559 This release fixes from_type() when used with bounded or constrained
6560 python:typing.TypeVar objects (issue #2094).
6561 Previously, distinct typevars with the same constraints would be
6563 treated as all single typevar, and in cases where a typevar bound was
6564 resolved to a union of subclasses this could result in mixed types be‐
6565 ing generated for that typevar.
6566 4.42.9 - 2019-11-03
6568 This patch ensures that the default value broadcastable_shapes()
6569 chooses for max_dims is always valid (at most 32), even if you pass
6570 min_dims=32.
6571 4.42.8 - 2019-11-02
6573 This patch ensures that we only add profile information to the pytest
6574 header if running either pytest or Hypothesis in verbose mode, matching
6575 the builtin cache plugin (issue #2155).
6576 4.42.7 - 2019-11-02
6578 This patch makes stateful step printing expand the result of a step
6579 into multiple variables when you return multiple() (issue #2139).
6580 Thanks to Joseph Weston for reporting and fixing this bug!
6581 4.42.6 - 2019-11-02
6583 This release fixes a bug (issue #2166) where a Unicode character info
6584 cache file was generated but never used on subsequent test runs, caus‐
6585 ing tests to run more slowly than they should have.
6586 Thanks to Robert Knight for this bugfix!
6588 4.42.5 - 2019-11-01
6590 This patch corrects some internal documentation. There is no user-vis‐
6591 ible change.
6592 4.42.4 - 2019-11-01
6594 This release fixes a bug (issue #2160) where decorators applied after
6595 @settings and before @given were ignored.
6596 Thanks to Tom Milligan for this bugfix!
6598 4.42.3 - 2019-10-30
6600 This release updates Hypothesis's formatting to the new version of
6601 black, and has absolutely no user visible effect.
6602 4.42.2 - 2019-10-30
6604 This release fixes a bug in recursive() which would have meant that in
6605 practice max_leaves was treated as if it was lower than it actually is
6606 - specifically it would be capped at the largest power of two smaller
6607 than it. It is now handled correctly.
6608 4.42.1 - 2019-10-30
6610 Python 3.8's new python:typing.SupportsIndex type - see PEP 357 for de‐
6611 tails - is now supported in from_type().
6612 Thanks to Grigorios Giannakopoulos for the patch!
6614 4.42.0 - 2019-10-27
6616 This release significantly simplifies Hypothesis's internal logic for
6617 data generation, by removing a number of heuristics of questionable or
6618 unproven value.
6619 The results of this change will vary significantly from test to test.
6621 Most test suites will see significantly faster data generation and
6622 lower memory usage. The "quality" of the generated data may go up or
6623 down depending on your particular test suites.
6624 If you see any significant regressions in Hypothesis's ability to find
6626 bugs in your code as a result of this release, please file an issue to
6627 let us know.
6628 Users of the new targeted property-based testing functionality are
6630 reasonably likely to see improvements in data generation, as this re‐
6631 lease changes the search algorithm for targeted property based testing
6632 to one that is more likely to be productive than the existing approach.
6633 4.41.3 - 2019-10-21
6635 This patch is to ensure that our internals remain comprehensible to
6636 mypy 0.740 - there is no user-visible change.
6637 4.41.2 - 2019-10-17
6639 This patch changes some internal hashes to SHA384, to better support
6640 users subject to FIPS-140. There is no user-visible API change.
6641 Thanks to Paul Kehrer for this contribution!
6643 4.41.1 - 2019-10-16
6645 This release makes --hypothesis-show-statistics much more useful for
6646 tests using a RuleBasedStateMachine, by simplifying the reprs so that
6647 events are aggregated correctly.
6648 4.41.0 - 2019-10-16
6650 This release upgrades the fixed_dictionaries() strategy to support op‐
6651 tional keys (issue #1913).
6652 4.40.2 - 2019-10-16
6654 This release makes some minor internal changes in support of improving
6655 the Hypothesis test suite. It should not have any user visible impact.
6656 4.40.1 - 2019-10-14
6658 This release changes how Hypothesis checks if a parameter to a test
6659 function is a mock object. It is unlikely to have any noticeable ef‐
6660 fect, but may result in a small performance improvement, especially for
6661 test functions where a mock object is being passed as the first argu‐
6662 ment.
6663 4.40.0 - 2019-10-09
6665 This release fixes a bug where our example database logic did not dis‐
6666 tinguish between failing examples based on arguments from a
6667 @pytest.mark.parametrize(...). This could in theory cause data loss if
6668 a common failure overwrote a rare one, and in practice caused occa‐
6669 sional file-access collisions in highly concurrent workloads (e.g. dur‐
6670 ing a 300-way parametrize on 16 cores).
6671 For internal reasons this also involves bumping the minimum supported
6673 version of pytest to 4.3
6674 Thanks to Peter C Kroon for the Hacktoberfest patch!
6676 4.39.3 - 2019-10-09
6678 This patch improves our type hints on the emails(), functions(), inte‐
6679 gers(), iterables(), and slices() strategies, as well as the .filter()
6680 method.
6681 There is no runtime change, but if you use mypy or a similar
6683 type-checker on your tests the results will be a bit more precise.
6684 4.39.2 - 2019-10-09
6686 This patch improves the performance of unique collections such as
6687 sets() of just() or booleans() strategies. They were already pretty
6688 good though, so you're unlikely to notice much!
6689 4.39.1 - 2019-10-09
6691 If a value in a dict passed to fixed_dictionaries() is not a strategy,
6692 Hypothesis now tells you which one.
6693 4.39.0 - 2019-10-07
6695 This release adds the basic_indices() strategy, to generate basic in‐
6696 dexes for arrays of the specified shape (issue #1930).
6697 It generates tuples containing some mix of integers, python:slice ob‐
6699 jects, ... (Ellipsis), and numpy:numpy.newaxis; which when used to in‐
6700 dex an array of the specified shape produce either a scalar or a
6701 shared-memory view of the array. Note that the index tuple may be
6702 longer or shorter than the array shape, and may produce a view with an‐
6703 other dimensionality again!
6704 Thanks to Lampros Mountrakis, Ryan Soklaski, and Zac Hatfield-Dodds for
6706 their collaboration on this surprisingly subtle strategy!
6707 4.38.3 - 2019-10-04
6709 This patch defers creation of the .hypothesis directory until we have
6710 something to store in it, meaning that it will appear when Hypothesis
6711 is used rather than simply installed.
6712 Thanks to Peter C Kroon for the Hacktoberfest patch!
6714 4.38.2 - 2019-10-02
6716 This patch bumps our dependency on attrs to >=19.2.0; but there are no
6717 user-visible changes to Hypothesis.
6718 4.38.1 - 2019-10-01
6720 This is a comment-only patch which tells mypy 0.730 to ignore some in‐
6721 ternal compatibility shims we use to support older Pythons.
6722 4.38.0 - 2019-10-01
6724 This release adds the hypothesis.target() function, which implements
6725 experimental support for targeted property-based testing (issue #1779).
6726 By calling target() in your test function, Hypothesis can do a
6728 hill-climbing search for bugs. If you can calculate a suitable metric
6729 such as the load factor or length of a queue, this can help you find
6730 bugs with inputs that are highly improbably from unguided generation -
6731 however good our heuristics, example diversity, and deduplication logic
6732 might be. After all, those features are at work in targeted PBT too!
6733 4.37.0 - 2019-09-28
6735 This release emits a warning if you use the .example() method of a
6736 strategy in a non-interactive context.
6737 given() is a much better choice for writing tests, whether you care
6739 about performance, minimal examples, reproducing failures, or even just
6740 the variety of inputs that will be tested!
6741 4.36.2 - 2019-09-20
6743 This patch disables part of the typing-based inference for the attrs
6744 package under Python 3.5.0, which has some incompatible internal de‐
6745 tails (issue #2095).
6746 4.36.1 - 2019-09-17
6748 This patch fixes a bug in strategy inference for attrs classes where
6749 Hypothesis would fail to infer a strategy for attributes of a generic
6750 type such as Union[int, str] or List[bool] (issue #2091).
6751 Thanks to Jonathan Gayvallet for the bug report and this patch!
6753 4.36.0 - 2019-09-09
6755 This patch deprecates min_len or max_len of 0 in byte_string_dtypes()
6756 and unicode_string_dtypes(). The lower limit is now 1.
6757 Numpy uses a length of 0 in these dtypes to indicate an undetermined
6759 size, chosen from the data at array creation. However, as the arrays()
6760 strategy creates arrays before filling them, strings were truncated to
6761 1 byte.
6762 4.35.1 - 2019-09-09
6764 This patch improves the messaging that comes from invalid size argu‐
6765 ments to collection strategies such as lists().
6766 4.35.0 - 2019-09-04
6768 This release improves the from_lark() strategy, tightening argument
6769 validation and adding the explicit argument to allow use with terminals
6770 that use @declare instead of a string or regular expression.
6771 This feature is required to handle features such as indent and dedent
6773 tokens in Python code, which can be generated with the hypothesmith
6774 package.
6775 4.34.0 - 2019-08-23
6777 The from_type() strategy now knows to look up the subclasses of ab‐
6778 stract types, which cannot be instantiated directly.
6779 This is very useful for hypothesmith to support libCST.
6781 4.33.1 - 2019-08-21
6783 This patch works around a crash when an incompatible version of Numpy
6784 is installed under PyPy 5.10 (Python 2.7).
6785 If you are still using Python 2, please upgrade to Python 3 as soon as
6787 possible - it will be unsupported at the end of this year.
6788 4.33.0 - 2019-08-20
6790 This release improves the domains() strategy, as well as the urls() and
6791 the emails() strategies which use it. These strategies now use the
6792 full IANA list of Top Level Domains and are correct as per RFC 1035.
6793 Passing tests using these strategies may now fail.
6795 Thanks to TechDragon for this improvement.
6797 4.32.3 - 2019-08-05
6799 This patch tidies up the repr of several settings-related objects, at
6800 runtime and in the documentation, and deprecates the undocumented edge
6801 case that phases=None was treated like phases=tuple(Phase).
6802 It also fixes from_lark() with lark 0.7.2 and later.
6804 4.32.2 - 2019-07-30
6806 This patch updates some internal comments for mypy 0.720. There is no
6807 user-visible impact.
6808 4.32.1 - 2019-07-29
6810 This release changes how the shrinker represents its progress inter‐
6811 nally. For large generated test cases this should result in signifi‐
6812 cantly less memory usage and possibly faster shrinking. Small generated
6813 test cases may be slightly slower to shrink but this shouldn't be very
6814 noticeable.
6815 4.32.0 - 2019-07-28
6817 This release makes arrays() more pedantic about elements strategies
6818 that cannot be exactly represented as array elements.
6819 In practice, you will see new warnings if you were using a float16 or
6821 float32 dtype without passing floats() the width=16 or width=32 argu‐
6822 ments respectively.
6823 The previous behaviour could lead to silent truncation, and thus some
6825 elements being equal to an explicitly excluded bound (issue #1899).
6826 4.31.1 - 2019-07-28
6828 This patch changes an internal use of MD5 to SHA hashes, to better sup‐
6829 port users subject to FIPS-140. There is no user-visible or API
6830 change.
6831 Thanks to Alex Gaynor for this patch.
6833 4.31.0 - 2019-07-24
6835 This release simplifies the logic of the print_blob setting by removing
6836 the option to set it to PrintSettings.INFER. As a result the
6837 print_blob setting now takes a single boolean value, and the use of
6838 PrintSettings is deprecated.
6839 4.28.2 - 2019-07-14
6841 This patch improves the docstrings of several Hypothesis strategies, by
6842 clarifying markup and adding cross-references. There is no runtime
6843 change.
6844 Thanks to Elizabeth Williams and Serah Njambi Rono for their contribu‐
6846 tions at the SciPy 2019 sprints!
6847 4.28.1 - 2019-07-12
6849 This patch improves the behaviour of the text() strategy when passed an
6850 alphabet which is not a strategy. The value is now interpreted as
6851 whitelist_characters to characters() instead of a sequence for sam‐
6852 pled_from(), which standardises the distribution of examples and the
6853 shrinking behaviour.
6854 You can get the previous behaviour by using lists(sampled_from(alpha‐
6856 bet)).map("".map) instead.
6857 4.28.0 - 2019-07-11
6859 This release deprecates find(). The .example() method is a better re‐
6860 placement if you want an example, and for the rare occasions where you
6861 want the minimal example you can get it from @given.
6862 @given has steadily outstripped find() in both features and performance
6864 over recent years, and as we do not have the resources to maintain and
6865 test both we think it is better to focus on just one.
6866 4.27.0 - 2019-07-08
6868 This release refactors the implementation of the .example() method, to
6869 more accurately represent the data which will be generated by @given.
6870 As a result, calling s.example() on an empty strategy s (such as noth‐
6872 ing()) now raises Unsatisfiable instead of the old NoExamples excep‐
6873 tion.
6874 4.26.4 - 2019-07-07
6876 This patch ensures that the Pandas extra will keep working when Python
6877 3.8 removes abstract base classes from the top-level python:collections
6878 namespace. This also fixes the relevant warning in Python 3.7, but
6879 there is no other difference in behaviour and you do not need to do
6880 anything.
6881 4.26.3 - 2019-07-05
6883 This release fixes issue #2027, by changing the way Hypothesis tries
6884 to generate distinct examples to be more efficient.
6885 This may result in slightly different data distribution, and should im‐
6887 prove generation performance in general, but should otherwise have min‐
6888 imal user impact.
6889 4.26.2 - 2019-07-04
6891 This release fixes issue #1864, where some simple tests would perform
6892 very slowly, because they would run many times with each subsequent run
6893 being progressively slower. They will now stop after a more reasonable
6894 number of runs without hitting this problem.
6895 Unless you are hitting exactly this issue, it is unlikely that this re‐
6897 lease will have any effect, but certain classes of custom generators
6898 that are currently very slow may become a bit faster, or start to trig‐
6899 ger health check failures.
6900 4.26.1 - 2019-07-04
6902 This release adds the strategy integer_array_indices(), which generates
6903 tuples of Numpy arrays that can be used for advanced indexing to select
6904 an array of a specified shape.
6905 4.26.0 - 2019-07-04
6907 This release significantly improves the performance of drawing unique
6908 collections whose elements are drawn from sampled_from() strategies.
6909 As a side effect, this detects an error condition that would previously
6911 have passed silently: When the min_size argument on a collection with
6912 distinct elements is greater than the number of elements being sampled,
6913 this will now raise an error.
6914 4.25.1 - 2019-07-03
6916 This release removes some defunct internal functionality that was only
6917 being used for testing. It should have no user visible impact.
6918 4.25.0 - 2019-07-03
6920 This release deprecates and disables the buffer_size setting, which
6921 should have been treated as a private implementation detail all along.
6922 We recommend simply deleting this settings argument.
6923 4.24.6 - 2019-06-26
6925 This patch makes datetimes() more efficient, as it now handles short
6926 months correctly by construction instead of filtering.
6927 4.24.5 - 2019-06-23
6929 This patch improves the development experience by simplifying the
6930 tracebacks you will see when e.g. you have used the .map(...) method of
6931 a strategy and the mapped function raises an exception.
6932 No new exceptions can be raised, nor existing exceptions change any‐
6934 thing but their traceback. We're simply using if-statements rather
6935 than exceptions for control flow in a certain part of the internals!
6936 4.24.4 - 2019-06-21
6938 This patch fixes issue #2014, where our compatibility layer broke with
6939 version 3.7.4 of the typing module backport on PyPI.
6940 This issue only affects Python 2. We remind users that Hypothesis,
6942 like many other packages, will drop Python 2 support on 2020-01-01 and
6943 already has several features that are only available on Python 3.
6944 4.24.3 - 2019-06-07
6946 This patch improves the implementation of an internal wrapper on Python
6947 3.8 beta1 (and will break on the alphas; but they're not meant to be
6948 stable). On other versions, there is no change at all.
6949 Thanks to Daniel Hahler for the patch, and Victor Stinner for his work
6951 on bpo-37032 that made it possible.
6952 4.24.2 - 2019-06-06
6954 Deprecation messages for functions in hypothesis.extra.django.models
6955 now explicitly name the deprecated function to make it easier to track
6956 down usages. Thanks to Kristian Glass for this contribution!
6957 4.24.1 - 2019-06-04
6959 This patch fixes issue #1999, a spurious bug raised when a @st.compos‐
6960 ite function was passed a keyword-only argument.
6961 Thanks to Jim Nicholls for his fantastic bug report.
6963 4.24.0 - 2019-05-29
6965 This release deprecates GenericStateMachine, in favor of RuleBased‐
6966 StateMachine. Rule-based stateful testing is significantly faster, es‐
6967 pecially during shrinking.
6968 If your use-case truly does not fit rule-based stateful testing, we
6970 recommend writing a custom test function which drives your specific
6971 control-flow using data().
6972 4.23.9 - 2019-05-28
6974 This patch fixes a very rare example database issue with file permis‐
6975 sions.
6976 When running a test that uses both @given and pytest.mark.parametrize,
6978 using pytest-xdist on Windows, with failing examples in the database,
6979 two attempts to read a file could overlap and we caught FileNotFound
6980 but not other OSErrors.
6981 4.23.8 - 2019-05-26
6983 This patch has a minor cleanup of the internal engine. There is no
6984 user-visible impact.
6985 4.23.7 - 2019-05-26
6987 This patch clarifies some error messages when the test function signa‐
6988 ture is incompatible with the arguments to @given, especially when the
6989 @settings() decorator is also used (issue #1978).
6990 4.23.6 - 2019-05-19
6992 This release adds the pyupgrade fixer to our code style, for consistent
6993 use of dict and set literals and comprehensions.
6994 4.23.5 - 2019-05-16
6996 This release slightly simplifies a small part of the core engine.
6997 There is no user-visible change.
6998 4.23.4 - 2019-05-09
7000 Fixes a minor formatting issue the docstring of from_type()
7001 4.23.3 - 2019-05-09
7003 Adds a recipe to the docstring of from_type() that describes a means
7004 for drawing values for "everything except" a specified type. This
7005 recipe is especially useful for writing tests that perform input-type
7006 validation.
7007 4.23.2 - 2019-05-08
7009 This patch uses autoflake to remove some pointless pass statements,
7010 which improves our workflow but has no user-visible impact.
7011 4.23.1 - 2019-05-08
7013 This patch fixes an OverflowError in from_type(xrange) on Python 2.
7014 It turns out that not only do the start and stop values have to fit in
7016 a C long, but so does stop - start. We now handle this even on 32bit
7017 platforms, but remind users that Python2 will not be supported after
7018 2019 without specific funding.
7019 4.23.0 - 2019-05-08
7021 This release implements the slices() strategy, to generate slices of a
7022 length-size sequence.
7023 Thanks to Daniel J. West for writing this patch at the PyCon 2019
7025 sprints!
7026 4.22.3 - 2019-05-07
7028 This patch exposes DataObject, solely to support more precise type
7029 hints. Objects of this type are provided by data(), and can be used to
7030 draw examples from strategies intermixed with your test code.
7031 4.22.2 - 2019-05-07
7033 This patch fixes the very rare issue #1798 in array_dtypes(), which
7034 caused an internal error in our tests.
7035 4.22.1 - 2019-05-07
7037 This patch fixes a rare bug in from_type(range).
7038 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
7040 Sprints.
7041 4.22.0 - 2019-05-07
7043 The unique_by argument to lists now accepts a tuple of callables such
7044 that every element of the generated list will be unique with respect to
7045 each callable in the tuple (issue #1916).
7046 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
7048 4.21.1 - 2019-05-06
7050 This patch cleans up the internals of one_of(). You may see a slight
7051 change to the distribution of examples from this strategy but there is
7052 no change to the public API.
7053 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
7055 sprints!
7056 4.21.0 - 2019-05-05
7058 The from_type() strategy now supports python:slice objects.
7059 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
7061 Mentored Sprints.
7062 4.20.0 - 2019-05-05
7064 This release improves the array_shapes() strategy, to choose an appro‐
7065 priate default for max_side based on the min_side, and max_dims based
7066 on the min_dims. An explicit error is raised for dimensions greater
7067 than 32, which are not supported by Numpy, as for other invalid combi‐
7068 nations of arguments.
7069 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
7071 tored Sprints.
7072 4.19.0 - 2019-05-05
7074 The from_type() strategy now supports python:range objects (or xrange
7075 on Python 2).
7076 Thanks to Katrina Durance for writing this feature at the PyCon 2019
7078 Mentored Sprints.
7079 4.18.3 - 2019-04-30
7081 This release fixes a very rare edge case in the test-case mutator,
7082 which could cause an internal error with certain unusual tests.
7083 4.18.2 - 2019-04-30
7085 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
7086 changed the representation of code objects to support positional-only
7087 arguments. Note however that Hypothesis does not (yet) support such
7088 functions as e.g. arguments to builds() or inputs to @given.
7089 Thanks to Paul Ganssle for identifying and fixing this bug.
7091 4.18.1 - 2019-04-29
7093 This patch improves the performance of unique collections such as
7094 sets() when the elements are drawn from a sampled_from() strategy (‐
7095 issue #1115).
7096 4.18.0 - 2019-04-24
7098 This release adds the functions() strategy, which can be used to imi‐
7099 tate your 'real' function for callbacks.
7100 4.17.2 - 2019-04-19
7102 This release refactors stateful rule selection to share the new machin‐
7103 ery with sampled_from() instead of using the original independent im‐
7104 plementation.
7105 4.17.1 - 2019-04-16
7107 This patch allows Hypothesis to try a few more examples after finding
7108 the first bug, in hopes of reporting multiple distinct bugs. The
7109 heuristics described in issue #847 ensure that we avoid wasting time on
7110 fruitless searches, while still surfacing each bug as soon as possible.
7111 4.17.0 - 2019-04-16
7113 This release adds the strategy broadcastable_shapes(), which generates
7114 array shapes that are broadcast-compatible with a provided shape.
7115 4.16.0 - 2019-04-12
7117 This release allows register_type_strategy() to be used with
7118 python:typing.NewType instances. This may be useful to e.g. provide
7119 only positive integers for from_type(UserId) with a UserId = New‐
7120 Type('UserId', int) type.
7121 Thanks to PJCampi for suggesting and writing the patch!
7123 4.15.0 - 2019-04-09
7125 This release supports passing a timedelta as the deadline setting, so
7126 you no longer have to remember that the number is in milliseconds (‐
7127 issue #1900).
7128 Thanks to Damon Francisco for this change!
7130 4.14.7 - 2019-04-09
7132 This patch makes the type annotations on hypothesis.extra.dateutil com‐
7133 patible with mypy 0.700.
7134 4.14.6 - 2019-04-07
7136 This release fixes a bug introduced in Hypothesis 4.14.3 that would
7137 sometimes cause sampled_from(...).filter(...) to encounter an internal
7138 assertion failure when there are three or fewer elements, and every el‐
7139 ement is rejected by the filter.
7140 4.14.5 - 2019-04-05
7142 This patch takes the previous efficiency improvements to sam‐
7143 pled_from(...).filter(...) strategies that reject most elements, and
7144 generalises them to also apply to sampled_from(...).filter(...).fil‐
7145 ter(...) and longer chains of filters.
7146 4.14.4 - 2019-04-05
7148 This release fixes a bug that prevented random_module() from correctly
7149 restoring the previous state of the random module.
7150 The random state was instead being restored to a temporary determinis‐
7152 tic state, which accidentally caused subsequent tests to see the same
7153 random values across multiple test runs.
7154 4.14.3 - 2019-04-03
7156 This patch adds an internal special case to make sampled_from(...).fil‐
7157 ter(...) much more efficient when the filter rejects most elements (‐
7158 issue #1885).
7159 4.14.2 - 2019-03-31
7161 This patch improves the error message if the function f in s.flatmap(f)
7162 does not return a strategy.
7163 Thanks to Kai Chen for this change!
7165 4.14.1 - 2019-03-30
7167 This release modifies how Hypothesis selects operations to run during
7168 shrinking, by causing it to deprioritise previously useless classes of
7169 shrink until others have reached a fixed point.
7170 This avoids certain pathological cases where the shrinker gets very
7172 close to finishing and then takes a very long time to finish the last
7173 small changes because it tries many useless shrinks for each useful one
7174 towards the end. It also should cause a more modest improvement (prob‐
7175 ably no more than about 30%) in shrinking performance for most tests.
7176 4.14.0 - 2019-03-19
7178 This release blocks installation of Hypothesis on Python 3.4, which
7179 reached its end of life date on 2019-03-18.
7180 This should not be of interest to anyone but downstream maintainers -
7182 if you are affected, migrate to a secure version of Python as soon as
7183 possible or at least seek commercial support.
7184 4.13.0 - 2019-03-19
7186 This release makes it an explicit error to call floats(min_value=inf,
7187 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
7188 are no possible values that can be generated (issue #1859).
7189 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
7191 -0. and we could thus generate either if comparing by value, violating
7192 the sequence ordering of floats is a special case we don't want or
7193 need.
7194 4.12.1 - 2019-03-18
7196 This release should significantly reduce the amount of memory that Hy‐
7197 pothesis uses for representing large test cases, by storing information
7198 in a more compact representation and only unpacking it lazily when it
7199 is first needed.
7200 4.12.0 - 2019-03-18
7202 This update adds the report_multiple_bugs setting, which you can use to
7203 disable multi-bug reporting and only raise whichever bug had the small‐
7204 est minimal example. This is occasionally useful when using a debugger
7205 or tools that annotate tracebacks via introspection.
7206 4.11.7 - 2019-03-18
7208 This change makes a tiny improvement to the core engine's bookkeeping.
7209 There is no user-visible change.
7210 4.11.6 - 2019-03-15
7212 This release changes some of Hypothesis's internal shrinking behaviour
7213 in order to reduce memory usage and hopefully improve performance.
7214 4.11.5 - 2019-03-13
7216 This release adds a micro-optimisation to how Hypothesis handles debug
7217 reporting internally. Hard to shrink test may see a slight performance
7218 improvement, but in most common scenarios it is unlikely to be notice‐
7219 able.
7220 4.11.4 - 2019-03-13
7222 This release removes some redundant code that was no longer needed but
7223 was still running a significant amount of computation and allocation on
7224 the hot path. This should result in a modest speed improvement for
7225 most tests, especially those with large test cases.
7226 4.11.3 - 2019-03-13
7228 This release adds a micro-optimisation to how Hypothesis caches test
7229 cases. This will cause a small improvement in speed and memory usage
7230 for large test cases, but in most common scenarios it is unlikely to be
7231 noticeable.
7232 4.11.2 - 2019-03-13
7234 This release removes some internal code that populates a field that is
7235 no longer used anywhere. This should result in some modest performance
7236 and speed improvements and no other user visible effects.
7237 4.11.1 - 2019-03-13
7239 This is a formatting-only patch, enabled by a new version of isort.
7240 4.11.0 - 2019-03-12
7242 This release deprecates sampled_from() with empty sequences. This re‐
7243 turns nothing(), which gives a clear error if used directly... but sim‐
7244 ply vanishes if combined with another strategy.
7245 Tests that silently generate less than expected are a serious problem
7247 for anyone relying on them to find bugs, and we think reliability more
7248 important than convenience in this case.
7249 4.10.0 - 2019-03-11
7251 This release improves Hypothesis's to detect flaky tests, by noticing
7252 when the behaviour of the test changes between runs. In particular
7253 this will notice many new cases where data generation depends on exter‐
7254 nal state (e.g. external sources of randomness) and flag those as flaky
7255 sooner and more reliably.
7256 The basis of this feature is a considerable reengineering of how Hy‐
7258 pothesis stores its history of test cases, so on top of this its memory
7259 usage should be considerably reduced.
7260 4.9.0 - 2019-03-09
7262 This release adds the strategy valid_tuple_axes(), which generates tu‐
7263 ples of axis-indices that can be passed to the axis argument in NumPy's
7264 sequential functions (e.g. numpy:numpy.sum()).
7265 Thanks to Ryan Soklaski for this strategy.
7267 4.8.0 - 2019-03-06
7269 This release significantly tightens validation in hypothesis.settings.
7270 max_examples, buffer_size, and stateful_step_count must be positive in‐
7271 tegers; deadline must be a positive number or None; and derandomize
7272 must be either True or False.
7273 As usual, this replaces existing errors with a more helpful error and
7275 starts new validation checks as deprecation warnings.
7276 4.7.19 - 2019-03-04
7278 This release makes some micro-optimisations to certain calculations
7279 performed in the shrinker. These should particularly speed up large
7280 test cases where the shrinker makes many small changes. It will also
7281 reduce the amount allocated, but most of this is garbage that would
7282 have been immediately thrown away, so you probably won't see much ef‐
7283 fect specifically from that.
7284 4.7.18 - 2019-03-03
7286 This patch removes some overhead from arrays() with a constant shape
7287 and dtype. The resulting performance improvement is modest, but worth‐
7288 wile for small arrays.
7289 4.7.17 - 2019-03-01
7291 This release makes some micro-optimisations within Hypothesis's inter‐
7292 nal representation of test cases. This should cause heavily nested
7293 test cases to allocate less during generation and shrinking, which
7294 should speed things up slightly.
7295 4.7.16 - 2019-02-28
7297 This changes the order in which Hypothesis runs certain operations dur‐
7298 ing shrinking. This should significantly decrease memory usage and
7299 speed up shrinking of large examples.
7300 4.7.15 - 2019-02-28
7302 This release allows Hypothesis to calculate a number of attributes of
7303 generated test cases lazily. This should significantly reduce memory
7304 usage and modestly improve performance, especially for large test
7305 cases.
7306 4.7.14 - 2019-02-28
7308 This release reduces the number of operations the shrinker will try
7309 when reordering parts of a test case. This should in some circum‐
7310 stances significantly speed up shrinking. It may result in different
7311 final test cases, and if so usually slightly worse ones, but it should
7312 not generally have much impact on the end result as the operations re‐
7313 moved were typically useless.
7314 4.7.13 - 2019-02-27
7316 This release changes how Hypothesis reorders examples within a test
7317 case during shrinking. This should make shrinking considerably faster.
7318 4.7.12 - 2019-02-27
7320 This release slightly improves the shrinker's ability to replace parts
7321 of a test case with their minimal version, by allowing it to do so in
7322 bulk rather than one at a time. Where this is effective, shrinker per‐
7323 formance should be modestly improved.
7324 4.7.11 - 2019-02-25
7326 This release makes some micro-optimisations to common operations per‐
7327 formed during shrinking. Shrinking should now be slightly faster, es‐
7328 pecially for large examples with relatively fast test functions.
7329 4.7.10 - 2019-02-25
7331 This release is a purely internal refactoring of Hypothesis's API for
7332 representing test cases. There should be no user visible effect.
7333 4.7.9 - 2019-02-24
7335 This release changes certain shrink passes to make them more efficient
7336 when they aren't making progress.
7337 4.7.8 - 2019-02-23
7339 This patch removes some unused code, which makes the internals a bit
7340 easier to understand. There is no user-visible impact.
7341 4.7.7 - 2019-02-23
7343 This release reduces the number of operations the shrinker will try
7344 when reordering parts of a test case. This should in some circum‐
7345 stances significantly speed up shrinking. It may result in different
7346 final test cases, and if so usually slightly worse ones, but it should
7347 not generally have much impact on the end result as the operations re‐
7348 moved were typically useless.
7349 4.7.6 - 2019-02-23
7351 This patch removes some unused code from the shrinker. There is no
7352 user-visible change.
7353 4.7.5 - 2019-02-23
7355 This release changes certain shrink passes to make them adaptive - that
7356 is, in cases where they are successfully making progress they may now
7357 do so significantly faster.
7358 4.7.4 - 2019-02-22
7360 This is a docs-only patch, noting that because the lark-parser is under
7361 active development at version 0.x, hypothesis[lark] APIs may break in
7362 minor releases if necessary to keep up with the upstream package.
7363 4.7.3 - 2019-02-22
7365 This changes Hypothesis to no longer import various test frameworks by
7366 default (if they are installed). which will speed up the initial im‐
7367 port hypothesis call.
7368 4.7.2 - 2019-02-22
7370 This release changes Hypothesis's internal representation of a test
7371 case to calculate some expensive structural information on demand
7372 rather than eagerly. This should reduce memory usage a fair bit, and
7373 may make generation somewhat faster.
7374 4.7.1 - 2019-02-21
7376 This release refactors the internal representation of previously run
7377 test cases. The main thing you should see as a result is that Hypothe‐
7378 sis becomes somewhat less memory hungry.
7379 4.7.0 - 2019-02-21
7381 This patch allows array_shapes() to generate shapes with side-length or
7382 even dimension zero, though the minimum still defaults to one. These
7383 shapes are rare and have some odd behavior, but are particularly impor‐
7384 tant to test for just that reason!
7385 In a related bigfix, arrays() now supports generating zero-dimensional
7387 arrays with dtype=object and a strategy for iterable elements. Previ‐
7388 ously, the array element would incorrectly be set to the first item in
7389 the generated iterable.
7390 Thanks to Ryan Turner for continuing to improve our Numpy support.
7392 4.6.1 - 2019-02-19
7394 This release is a trivial micro-optimisation inside Hypothesis which
7395 should result in it using significantly less memory.
7396 4.6.0 - 2019-02-18
7398 This release changes some inconsistent behavior of arrays() from the
7399 Numpy extra when asked for an array of shape=(). arrays() will now al‐
7400 ways return a Numpy ndarray, and the array will always be of the re‐
7401 quested dtype.
7402 Thanks to Ryan Turner for this change.
7404 4.5.12 - 2019-02-18
7406 This release fixes a minor typo in an internal comment. There is no
7407 user-visible change.
7408 4.5.11 - 2019-02-15
7410 This release fixes issue #1813, a bug introduced in 3.59.1, which
7411 caused random_module() to no longer affect the body of the test: Al‐
7412 though Hypothesis would claim to be seeding the random module in fact
7413 tests would always run with a seed of zero.
7414 4.5.10 - 2019-02-14
7416 This patch fixes an off-by-one error in the maximum length of emails().
7417 Thanks to Krzysztof Jurewicz for pull request #1812.
7418 4.5.9 - 2019-02-14
7420 This patch removes some unused code from the shrinker. There is no
7421 user-visible change.
7422 4.5.8 - 2019-02-12
7424 This release fixes an internal IndexError in Hypothesis that could
7425 sometimes be triggered during shrinking.
7426 4.5.7 - 2019-02-11
7428 This release modifies the shrinker to interleave different types of re‐
7429 duction operations, e.g. switching between deleting data and lowering
7430 scalar values rather than trying entirely deletions then entirely low‐
7431 ering.
7432 This may slow things down somewhat in the typical case, but has the ma‐
7434 jor advantage that many previously difficult to shrink examples should
7435 become much faster, because the shrinker will no longer tend to stall
7436 when trying some ineffective changes to the shrink target but will in‐
7437 stead interleave it with other more effective operations.
7438 4.5.6 - 2019-02-11
7440 This release makes a number of internal changes to the implementation
7441 of hypothesis.extra.lark.from_lark(). These are primarily intended as
7442 a refactoring, but you may see some minor improvements to performance
7443 when generating large strings, and possibly to shrink quality.
7444 4.5.5 - 2019-02-10
7446 This patch prints an explanatory note when issue #1798 is triggered,
7447 because the error message from Numpy is too terse to locate the prob‐
7448 lem.
7449 4.5.4 - 2019-02-08
7451 In Python 2, long integers are not allowed in the shape argument to ar‐
7452 rays(). Thanks to Ryan Turner for fixing this.
7453 4.5.3 - 2019-02-08
7455 This release makes a small internal refactoring to clarify how Hypothe‐
7456 sis instructs tests to stop running when appropriate. There is no
7457 user-visible change.
7458 4.5.2 - 2019-02-06
7460 This release standardises all of the shrinker's internal operations on
7461 running in a random order.
7462 The main effect you will see from this that it should now be much less
7464 common for the shrinker to stall for a long time before making further
7465 progress. In some cases this will correspond to shrinking more slowly,
7466 but on average it should result in faster shrinking.
7467 4.5.1 - 2019-02-05
7469 This patch updates some docstrings, but has no runtime changes.
7470 4.5.0 - 2019-02-03
7472 This release adds exclude_min and exclude_max arguments to floats(), so
7473 that you can easily generate values from open or half-open intervals (‐
7474 issue #1622).
7475 4.4.6 - 2019-02-03
7477 This patch fixes a bug where from_regex() could throw an internal error
7478 if the python:re.IGNORECASE flag was used (issue #1786).
7479 4.4.5 - 2019-02-02
7481 This release removes two shrink passes that Hypothesis runs late in the
7482 process. These were very expensive when the test function was slow and
7483 often didn't do anything useful.
7484 Shrinking should get faster for most failing tests. If you see any re‐
7486 gression in example quality as a result of this release, please let us
7487 know.
7488 4.4.4 - 2019-02-02
7490 This release modifies the way that Hypothesis deletes data during
7491 shrinking. It will primarily be noticeable for very large examples,
7492 which should now shrink faster.
7493 The shrinker is now also able to perform some deletions that it could
7495 not previously, but this is unlikely to be very noticeable.
7496 4.4.3 - 2019-01-25
7498 This release fixes an open file leak that used to cause ResourceWarn‐
7499 ings.
7500 4.4.2 - 2019-01-24
7502 This release changes Hypothesis's internal approach to caching the re‐
7503 sults of executing test cases. The result should be that it is now
7504 significantly less memory hungry, especially when shrinking large test
7505 cases.
7506 Some tests may get slower or faster depending on whether the new or old
7508 caching strategy was well suited to them, but any change in speed in
7509 either direction should be minor.
7510 4.4.1 - 2019-01-24
7512 This patch tightens up some of our internal heuristics to deal with
7513 shrinking floating point numbers, which will now run in fewer circum‐
7514 stances.
7515 You are fairly unlikely to see much difference from this, but if you do
7517 you are likely to see shrinking become slightly faster and/or producing
7518 slightly worse results.
7519 4.4.0 - 2019-01-24
7521 This release adds the from_form() function, which allows automatic
7522 testing against Django forms. (issue #35)
7523 Thanks to Paul Stiverson for this feature, which resolves our oldest
7525 open issue!
7526 4.3.0 - 2019-01-24
7528 This release deprecates HealthCheck.hung_test and disables the associ‐
7529 ated runtime check for tests that ran for more than five minutes. Such
7530 a check is redundant now that we enforce the deadline and max_examples
7531 setting, which can be adjusted independently.
7532 4.2.0 - 2019-01-23
7534 This release adds a new module, hypothesis.extra.lark, which you can
7535 use to generate strings matching a context-free grammar.
7536 In this initial version, only lark-parser EBNF grammars are supported,
7538 by the new hypothesis.extra.lark.from_lark() function.
7539 4.1.2 - 2019-01-23
7541 This patch fixes a very rare overflow bug (issue #1748) which could
7542 raise an InvalidArgument error in complex_numbers() even though the ar‐
7543 guments were valid.
7544 4.1.1 - 2019-01-23
7546 This release makes some improvements to internal code organisation and
7547 documentation and has no impact on behaviour.
7548 4.1.0 - 2019-01-22
7550 This release adds register_random(), which registers random.Random in‐
7551 stances or compatible objects to be seeded and reset by Hypothesis to
7552 ensure that test cases are deterministic.
7553 We still recommend explicitly passing a random.Random instance from
7555 randoms() if possible, but registering a framework-global state for Hy‐
7556 pothesis to manage is better than flaky tests!
7557 4.0.2 - 2019-01-22
7559 This patch fixes issue #1387, where bounded integers() with a very
7560 large range would almost always generate very large numbers. Now, we
7561 usually use the same tuned distribution as unbounded integers().
7562 4.0.1 - 2019-01-16
7564 This release randomizes the order in which the shrinker tries some of
7565 its initial normalization operations. You are unlikely to see much
7566 difference as a result unless your generated examples are very large.
7567 In this case you may see some performance improvements in shrinking.
7568 4.0.0 - 2019-01-14
7570 Welcome to the next major version of Hypothesis!
7571 There are no new features here, as we release those in minor versions.
7573 Instead, 4.0 is a chance for us to remove deprecated features (many al‐
7574 ready converted into no-ops), and turn a variety of warnings into er‐
7575 rors.
7576 If you were running on the last version of Hypothesis 3.x without any
7578 Hypothesis deprecation warnings (or using private APIs), this will be a
7579 very boring upgrade. In fact, nothing will change for you at all. Per
7580 our deprecation policy, warnings added in the last six months (after
7581 2018-07-05) have not been converted to errors.
7582 Removals
7584 • hypothesis.extra.datetime has been removed, replaced by the core date
7585 and time strategies.
7586 • hypothesis.extra.fakefactory has been removed, replaced by general
7588 expansion of Hypothesis' strategies and the third-party ecosystem.
7589 • The SQLite example database backend has been removed.
7591 Settings
7593 • The deadline is now enforced by default, rather than just emitting a
7594 warning when the default (200 milliseconds per test case) deadline is
7595 exceeded.
7596 • The database_file setting has been removed; use database.
7598 • The perform_health_check setting has been removed; use sup‐
7600 press_health_check.
7601 • The max_shrinks setting has been removed; use phases to disable
7603 shrinking.
7604 • The min_satisfying_examples, max_iterations, strict, timeout, and
7606 use_coverage settings have been removed without user-configurable re‐
7607 placements.
7608 Strategies
7610 • The elements argument is now required for collection strategies.
7611 • The average_size argument was a no-op and has been removed.
7613 • Date and time strategies now only accept min_value and max_value for
7615 bounds.
7616 • builds() now requires that the thing to build is passed as the first
7618 positional argument.
7619 • Alphabet validation for text() raises errors, not warnings, as does
7621 category validation for characters().
7622 • The choices() strategy has been removed. Instead, you can use data()
7624 with sampled_from(), so choice(elements) becomes data.draw(sam‐
7625 pled_from(elements)).
7626 • The streaming() strategy has been removed. Instead, you can use
7628 data() and replace iterating over the stream with data.draw() calls.
7629 • sampled_from() and permutations() raise errors instead of warnings if
7631 passed a collection that is not a sequence.
7632 Miscellaneous
7634 • Applying @given to a test function multiple times was really ineffi‐
7635 cient, and now it's also an error.
7636 • Using the .example() method of a strategy (intended for interactive
7638 exploration) within another strategy or a test function always weak‐
7639 ened data generation and broke shrinking, and now it's an error too.
7640 • The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
7642 ported, as the database_file setting has been removed.
7643 • The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
7645 ported. You can use the --hypothesis-verbosity pytest argument in‐
7646 stead, or write your own setup code using the settings profile system
7647 to replace it.
7648 • Using @seed or derandomize=True now forces database=None to ensure
7650 results are in fact reproducible. If database is not None, doing so
7651 also emits a HypothesisWarning.
7652 • Unused exception types have been removed from hypothesis.errors;
7654 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
7655 ple, Timeout, and WrongFormat.
7656 Hypothesis 3.x
7658 3.88.3 - 2019-01-11
7659 This changes the order that the shrinker tries certain operations in
7660 its "emergency" phase which runs late in the process. The new order
7661 should be better at avoiding long stalls where the shrinker is failing
7662 to make progress, which may be helpful if you have difficult to shrink
7663 test cases. However this will not be noticeable in the vast majority
7664 of use cases.
7665 3.88.2 - 2019-01-11
7667 This is a pure refactoring release that extracts some logic from the
7668 core Hypothesis engine into its own class and file. It should have no
7669 user visible impact.
7670 3.88.1 - 2019-01-11
7672 This patch fixes some markup in our documentation.
7673 3.88.0 - 2019-01-10
7675 Introduces hypothesis.stateful.multiple(), which allows rules in rule
7676 based state machines to send multiple results at once to their target
7677 Bundle, or none at all.
7678 3.87.0 - 2019-01-10
7680 This release contains a massive cleanup of the Hypothesis for Django
7681 extra:
7682 • hypothesis.extra.django.models.models() is deprecated in favor of hy‐
7684 pothesis.extra.django.from_model().
7685 • hypothesis.extra.django.models.add_default_field_mapping() is depre‐
7687 cated in favor of hypothesis.extra.django.register_field_strategy().
7688 • from_model() does not infer a strategy for nullable fields or fields
7690 with a default unless passed infer, like builds(). models.models()
7691 would usually but not always infer, and a special default_value
7692 marker object was required to disable inference.
7693 3.86.9 - 2019-01-09
7695 This release improves some internal logic about when a test case in Hy‐
7696 pothesis's internal representation could lead to a valid test case. In
7697 some circumstances this can lead to a significant speed up during
7698 shrinking. It may have some minor negative impact on the quality of
7699 the final result due to certain shrink passes now having access to less
7700 information about test cases in some circumstances, but this should
7701 rarely matter.
7702 3.86.8 - 2019-01-09
7704 This release has no user visible changes but updates our URLs to use
7705 HTTPS.
7706 3.86.7 - 2019-01-08
7708 Hypothesis can now automatically generate values for Django models with
7709 a URLfield, thanks to a new provisional strategy for URLs (issue
7710 #1388).
7711 3.86.6 - 2019-01-07
7713 This release is a pure refactoring that extracts some internal code
7714 into its own file. It should have no user visible effect.
7715 3.86.5 - 2019-01-06
7717 This is a docs-only patch, which fixes some typos and removes a few hy‐
7718 perlinks for deprecated features.
7719 3.86.4 - 2019-01-04
7721 This release changes the order in which the shrinker tries to delete
7722 data. For large and slow tests this may significantly improve the per‐
7723 formance of shrinking.
7724 3.86.3 - 2019-01-04
7726 This release fixes a bug where certain places Hypothesis internal er‐
7727 rors could be raised during shrinking when a user exception occurred
7728 that suppressed an exception Hypothesis uses internally in its genera‐
7729 tion.
7730 The two known ways to trigger this problem were:
7732 • Errors raised in stateful tests' teardown function.
7734 • Errors raised in finally blocks that wrapped a call to data.draw.
7736 These cases will now be handled correctly.
7738 3.86.2 - 2019-01-04
7740 This patch is a docs-only change to fix a broken hyperlink.
7741 3.86.1 - 2019-01-04
7743 This patch fixes issue #1732, where integers() would always return long
7744 values on Python 2.
7745 3.86.0 - 2019-01-03
7747 This release ensures that infinite numbers are never generated by
7748 floats() with allow_infinity=False, which could previously happen in
7749 some cases where one bound was also provided.
7750 The trivially inconsistent min_value=inf, allow_infinity=False now
7752 raises an InvalidArgumentError, as does the inverse with max_value.
7753 You can still use just(inf) to generate inf without violating other
7754 constraints.
7755 3.85.3 - 2019-01-02
7757 Happy new year everyone! This release has no user visible changes but
7758 updates our copyright headers to include 2019.
7759 3.85.2 - 2018-12-31
7761 This release makes a small change to the way the shrinker works. You
7762 may see some improvements to speed of shrinking on especially large and
7763 hard to shrink examples, but most users are unlikely to see much dif‐
7764 ference.
7765 3.85.1 - 2018-12-30
7767 This patch fixes issue #1700, where a line that contained a Unicode
7768 character before a lambda definition would cause an internal exception.
7769 3.85.0 - 2018-12-29
7771 Introduces the hypothesis.stateful.consumes() function. When defining a
7772 rule in stateful testing, it can be used to mark bundles from which
7773 values should be consumed, i. e. removed after use in the rule. This
7774 has been proposed in issue #136.
7775 Thanks to Jochen Müller for this long-awaited feature.
7777 3.84.6 - 2018-12-28
7779 This patch makes a small internal change to fix an issue in Hypothe‐
7780 sis's own coverage tests (issue #1718).
7781 There is no user-visible change.
7783 3.84.5 - 2018-12-21
7785 This patch refactors the hypothesis.strategies module, so that private
7786 names should no longer appear in tab-completion lists. We previously
7787 relied on __all__ for this, but not all editors respect it.
7788 3.84.4 - 2018-12-21
7790 This is a follow-up patch to ensure that the deprecation date is auto‐
7791 matically recorded for any new deprecations. There is no user-visible
7792 effect.
7793 3.84.3 - 2018-12-20
7795 This patch updates the Hypothesis pytest plugin to avoid a recently
7796 deprecated hook interface. There is no user-visible change.
7797 3.84.2 - 2018-12-19
7799 This patch fixes the internals for integers() with one bound. Values
7800 from this strategy now always shrink towards zero instead of towards
7801 the bound, and should shrink much more efficiently too. On Python 2,
7802 providing a bound incorrectly excluded long integers, which can now be
7803 generated.
7804 3.84.1 - 2018-12-18
7806 This patch adds information about when features were deprecated, but
7807 this is only recorded internally and has no user-visible effect.
7808 3.84.0 - 2018-12-18
7810 This release changes the stateful testing backend from find() to use
7811 @given (issue #1300). This doesn't change how you create stateful
7812 tests, but does make them run more like other Hypothesis tests.
7813 @reproduce_failure and @seed now work for stateful tests.
7815 Stateful tests now respect the deadline and suppress_health_check set‐
7817 tings, though they are disabled by default. You can enable them by us‐
7818 ing @settings(...) as a class decorator with whatever arguments you
7819 prefer.
7820 3.83.2 - 2018-12-17
7822 Hypothesis has adopted Black as our code formatter (issue #1686).
7823 There are no functional changes to the source, but it's prettier!
7824 3.83.1 - 2018-12-13
7826 This patch increases the variety of examples generated by from_type().
7827 3.83.0 - 2018-12-12
7829 Our pytest plugin now warns you when strategy functions have been col‐
7830 lected as tests, which may happen when e.g. using the @composite deco‐
7831 rator when you should be using @given(st.data()) for inline draws.
7832 Such functions always pass when treated as tests, because the lazy cre‐
7833 ation of strategies mean that the function body is never actually exe‐
7834 cuted!
7835 3.82.6 - 2018-12-11
7837 Hypothesis can now show statistics when running under pytest-xdist.
7838 Previously, statistics were only reported when all tests were run in a
7839 single process (issue #700).
7840 3.82.5 - 2018-12-08
7842 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
7843 to integers() could cause errors on Python 3 due to internal rounding.
7844 3.82.4 - 2018-12-08
7846 Hypothesis now seeds and resets the global state of np.random for each
7847 test case, to ensure that tests are reproducible.
7848 This matches and complements the existing handling of the python:random
7850 module - Numpy simply maintains an independent PRNG for performance
7851 reasons.
7852 3.82.3 - 2018-12-08
7854 This is a no-op release to add the new Framework :: Hypothesis trove
7855 classifier to hypothesis on PyPI.
7856 You can use it as a filter to find Hypothesis-related packages such as
7858 extensions as they add the tag over the coming weeks, or simply visit
7859 our curated list.
7860 3.82.2 - 2018-12-08
7862 The Hypothesis for Pandas extension is now listed in setup.py, so you
7863 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
7864 tion.
7865 3.82.1 - 2018-10-29
7867 This patch fixes from_type() on Python 2 for classes where cls.__init__
7868 is object.__init__. Thanks to ccxcz for reporting issue #1656.
7869 3.82.0 - 2018-10-29
7871 The alphabet argument for text() now uses its default value of charac‐
7872 ters(blacklist_categories=('Cs',)) directly, instead of hiding that be‐
7873 hind alphabet=None and replacing it within the function. Passing None
7874 is therefore deprecated.
7875 3.81.0 - 2018-10-27
7877 GenericStateMachine and RuleBasedStateMachine now raise an explicit er‐
7878 ror when instances of settings are assigned to the classes' settings
7879 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
7880 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
7881 tor to handle this automatically.
7882 3.80.0 - 2018-10-25
7884 Since version 3.68.0, arrays() checks that values drawn from the ele‐
7885 ments and fill strategies can be safely cast to the dtype of the array,
7886 and emits a warning otherwise.
7887 This release expands the checks to cover overflow for finite complex64
7889 elements and string truncation caused by too-long elements or trailing
7890 null characters (issue #1591).
7891 3.79.4 - 2018-10-25
7893 Tests using @given now shrink errors raised from pytest helper func‐
7894 tions, instead of reporting the first example found.
7895 This was previously fixed in version 3.56.0, but only for stateful
7897 testing.
7898 3.79.3 - 2018-10-23
7900 Traceback elision is now disabled on Python 2, to avoid an import-time
7901 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
7902 3.79.2: issue #1648).
7903 We encourage all users to upgrade to Python 3 before the end of 2019.
7905 3.79.2 - 2018-10-23
7907 This patch shortens tracebacks from Hypothesis, so you can see exactly
7908 happened in your code without having to skip over irrelevant details
7909 about our internals (issue #848).
7910 In the example test (see pull request #1582), this reduces tracebacks
7912 from nine frames to just three - and for a test with multiple errors,
7913 from seven frames per error to just one!
7914 If you do want to see the internal details, you can disable frame eli‐
7916 sion by setting verbosity to debug.
7917 3.79.1 - 2018-10-22
7919 The abstract number classes Number, Complex, Real, Rational, and Inte‐
7920 gral are now supported by the from_type() strategy. Previously, you
7921 would have to use register_type_strategy() before they could be re‐
7922 solved (issue #1636)
7923 3.79.0 - 2018-10-18
7925 This release adds a CLI flag for verbosity --hypothesis-verbosity to
7926 the Hypothesis pytest plugin, applied after loading the profile speci‐
7927 fied by --hypothesis-profile. Valid options are the names of verbosity
7928 settings, quiet, normal, verbose or debug.
7929 Thanks to Bex Dunn for writing this patch at the PyCon Australia
7931 sprints!
7932 The pytest header now correctly reports the current profile if --hy‐
7934 pothesis-profile has been used.
7935 Thanks to Mathieu Paturel for the contribution at the Canberra Python
7937 Hacktoberfest.
7938 3.78.0 - 2018-10-16
7940 This release has deprecated the generation of integers, floats and
7941 fractions when the conversion of the upper and/ or lower bound is not
7942 100% exact, e.g. when an integer gets passed a bound that is not a
7943 whole number. (issue #1625)
7944 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
7946 3.77.0 - 2018-10-16
7948 This minor release adds functionality to settings allowing it to be
7949 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
7950 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
7952 3.76.1 - 2018-10-16
7954 This patch fixes some warnings added by recent releases of pydocstyle
7955 and mypy.
7956 3.76.0 - 2018-10-11
7958 This release deprecates using floats for min_size and max_size.
7959 The type hint for average_size arguments has been changed from Op‐
7961 tional[int] to None, because non-None values are always ignored and
7962 deprecated.
7963 3.75.4 - 2018-10-10
7965 This patch adds more internal comments to the core engine's se‐
7966 quence-length shrinker. There should be no user-visible change.
7967 3.75.3 - 2018-10-09
7969 This patch adds additional comments to some of the core engine's inter‐
7970 nal data structures. There is no user-visible change.
7971 3.75.2 - 2018-10-09
7973 This patch avoids caching a trivial case, fixing issue #493.
7974 3.75.1 - 2018-10-09
7976 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
7977 for this contribution!
7978 3.75.0 - 2018-10-08
7980 This release deprecates the use of min_size=None, setting the default
7981 min_size to 0 (issue #1618).
7982 3.74.3 - 2018-10-08
7984 This patch makes some small internal changes to comply with a new lint
7985 setting in the build. There should be no user-visible change.
7986 3.74.2 - 2018-10-03
7988 This patch fixes issue #1153, where time spent reifying a strategy was
7989 also counted in the time spent generating the first example. Strate‐
7990 gies are now fully constructed and validated before the timer is
7991 started.
7992 3.74.1 - 2018-10-03
7994 This patch fixes some broken formatting and links in the documentation.
7995 3.74.0 - 2018-10-01
7997 This release checks that the value of the print_blob setting is a
7998 PrintSettings instance.
7999 Being able to specify a boolean value was not intended, and is now dep‐
8001 recated. In addition, specifying True will now cause the blob to al‐
8002 ways be printed, instead of causing it to be suppressed.
8003 Specifying any value that is not a PrintSettings or a boolean is now an
8005 error.
8006 3.73.5 - 2018-10-01
8008 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
8009 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
8010 eter names min_value and max_value instead of the deprecated names
8011 3.73.4 - 2018-09-30
8013 This patch ensures that Hypothesis deprecation warnings display the
8014 code that emitted them when you're not running in -Werror mode (issue
8015 #652).
8016 3.73.3 - 2018-09-27
8018 Tracebacks involving @composite are now slightly shorter due to some
8019 internal refactoring.
8020 3.73.2 - 2018-09-26
8022 This patch fixes errors in the internal comments for one of the
8023 shrinker passes. There is no user-visible change.
8024 3.73.1 - 2018-09-25
8026 This patch substantially improves the distribution of data generated
8027 with recursive(), and fixes a rare internal error (issue #1502).
8028 3.73.0 - 2018-09-24
8030 This release adds the fulfill() function, which is designed for testing
8031 code that uses dpcontracts 0.4 or later for input validation. This
8032 provides some syntactic sugar around use of assume(), to automatically
8033 filter out and retry calls that cause a precondition check to fail (‐
8034 issue #1474).
8035 3.72.0 - 2018-09-24
8037 This release makes setting attributes of the hypothesis.settings class
8038 an explicit error. This has never had any effect, but could mislead
8039 users who confused it with the current settings instance hypothe‐
8040 sis.settings.default (which is also immutable). You can change the
8041 global settings with settings profiles.
8042 3.71.11 - 2018-09-24
8044 This patch factors out some common code in the shrinker for iterating
8045 over pairs of data blocks. There should be no user-visible change.
8046 3.71.10 - 2018-09-18
8048 This patch allows from_type() to handle the empty tuple type, typ‐
8049 ing.Tuple[()].
8050 3.71.9 - 2018-09-17
8052 This patch updates some internal comments for mypy. There is no
8053 user-visible effect, even for Mypy users.
8054 3.71.8 - 2018-09-17
8056 This patch fixes a rare bug that would cause a particular shrinker pass
8057 to raise an IndexError, if a shrink improvement changed the underlying
8058 data in an unexpected way.
8059 3.71.7 - 2018-09-17
8061 This release fixes the broken cross-references in our docs, and adds a
8062 CI check so we don't add new ones.
8063 3.71.6 - 2018-09-16
8065 This patch fixes two bugs (issue #944 and issue #1521), where messages
8066 about @seed did not check the current verbosity setting, and the wrong
8067 settings were active while executing explicit examples.
8068 3.71.5 - 2018-09-15
8070 This patch fixes a DeprecationWarning added in Python 3.8 (issue
8071 #1576).
8072 Thanks to tirkarthi for this contribution!
8074 3.71.4 - 2018-09-14
8076 This is a no-op release, which implements automatic DOI minting and
8077 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
8078 zon 2020 programme for providing this service!
8079 Check our CITATION file for details, or head right on over to
8081 doi.org/10.5281/zenodo.1412597
8082 3.71.3 - 2018-09-10
8084 This release adds the test name to some deprecation warnings, for eas‐
8085 ier debugging.
8086 Thanks to Sanyam Khurana for the patch!
8088 3.71.2 - 2018-09-10
8090 This release makes Hypothesis's memory usage substantially smaller for
8091 tests with many examples, by bounding the number of past examples it
8092 keeps around.
8093 You will not see much difference unless you are running tests with
8095 max_examples set to well over 1000, but if you do have such tests then
8096 you should see memory usage mostly plateau where previously it would
8097 have grown linearly with time.
8098 3.71.1 - 2018-09-09
8100 This patch adds internal comments to some tree traversals in the core
8101 engine. There is no user-visible change.
8102 3.71.0 - 2018-09-08
8104 This release deprecates the coverage-guided testing functionality, as
8105 it has proven brittle and does not really pull its weight.
8106 We intend to replace it with something more useful in the future, but
8108 the feature in its current form does not seem to be worth the cost of
8109 using, and whatever replaces it will likely look very different.
8110 3.70.4 - 2018-09-08
8112 This patch changes the behaviour of reproduce_failure() so that blobs
8113 are only printed in quiet mode when the print_blob setting is set to
8114 ALWAYS.
8115 Thanks to Cameron McGill for writing this patch at the PyCon Australia
8117 sprints!
8118 3.70.3 - 2018-09-03
8120 This patch removes some unnecessary code from the internals. There is
8121 no user-visible change.
8122 3.70.2 - 2018-09-03
8124 This patch fixes an internal bug where a corrupted argument to @repro‐
8125 duce_failure could raise the wrong type of error. Thanks again to
8126 Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
8127 tently provides excellent bug reports including issue #1558.
8128 3.70.1 - 2018-09-03
8130 This patch updates hypothesis to report its version and settings when
8131 run with pytest. (issue #1223).
8132 Thanks to Jack Massey for this feature.
8134 3.70.0 - 2018-09-01
8136 This release adds a fullmatch argument to from_regex(). When full‐
8137 match=True, the whole example will match the regex pattern as for
8138 python:re.fullmatch().
8139 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
8141 sprints!
8142 3.69.12 - 2018-08-30
8144 This release reverts the changes to logging handling in 3.69.11, which
8145 broke test that use the pytest caplog fixture internally because all
8146 logging was disabled (issue #1546).
8147 3.69.11 - 2018-08-29
8149 This patch will hide all logging messages produced by test cases before
8150 the final, minimal, failing test case (issue #356).
8151 Thanks to Gary Donovan for writing this patch at the PyCon Australia
8153 sprints!
8154 3.69.10 - 2018-08-29
8156 This patch fixes a bug that prevents coverage from reporting unexecuted
8157 Python files (issue #1085).
8158 Thanks to Gary Donovan for writing this patch at the PyCon Australia
8160 sprints!
8161 3.69.9 - 2018-08-28
8163 This patch improves the packaging of the Python package by adding LI‐
8164 CENSE.txt to the sdist (issue #1311), clarifying the minimum supported
8165 versions of pytz and dateutil (issue #1383), and adds keywords to the
8166 metadata (issue #1520).
8167 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
8169 tralia sprints!
8170 3.69.8 - 2018-08-28
8172 This is an internal change which replaces pickle with json to prevent
8173 possible security issues.
8174 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
8176 sprints!
8177 3.69.7 - 2018-08-28
8179 This patch ensures that note() prints the note for every test case when
8180 the verbosity setting is Verbosity.verbose. At normal verbosity it
8181 only prints from the final test case.
8182 Thanks to Tom McDermott for writing this patch at the PyCon Australia
8184 sprints!
8185 3.69.6 - 2018-08-27
8187 This patch improves the testing of some internal caching. It should
8188 have no user-visible effect.
8189 3.69.5 - 2018-08-27
8191 This change performs a small rename and refactoring in the core engine.
8192 There is no user-visible change.
8193 3.69.4 - 2018-08-27
8195 This change improves the core engine's ability to avoid unnecessary
8196 work, by consulting its cache of previously-tried inputs in more cases.
8197 3.69.3 - 2018-08-27
8199 This patch handles passing an empty python:enum.Enum to from_type() by
8200 returning nothing(), instead of raising an internal python:Assertion‐
8201 Error.
8202 Thanks to Paul Amazona for writing this patch at the PyCon Australia
8204 sprints!
8205 3.69.2 - 2018-08-23
8207 This patch fixes a small mistake in an internal comment. There is no
8208 user-visible change.
8209 3.69.1 - 2018-08-21
8211 This change fixes a small bug in how the core engine consults its cache
8212 of previously-tried inputs. There is unlikely to be any user-visible
8213 change.
8214 3.69.0 - 2018-08-20
8216 This release improves argument validation for stateful testing.
8217 • If the target or targets of a rule() are invalid, we now raise a use‐
8219 ful validation error rather than an internal exception.
8220 • Passing both the target and targets arguments is deprecated - append
8222 the target bundle to the targets tuple of bundles instead.
8223 • Passing the name of a Bundle rather than the Bundle itself is also
8225 deprecated.
8226 3.68.3 - 2018-08-20
8228 This is a docs-only patch, fixing some typos and formatting issues.
8229 3.68.2 - 2018-08-19
8231 This change fixes a small bug in how the core engine caches the results
8232 of previously-tried inputs. The effect is unlikely to be noticeable,
8233 but it might avoid unnecesary work in some cases.
8234 3.68.1 - 2018-08-18
8236 This patch documents the from_dtype() function, which infers a strategy
8237 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
8238 directly when creating e.g. Pandas objects.
8239 3.68.0 - 2018-08-15
8241 arrays() now checks that integer and float values drawn from elements
8242 and fill strategies can be safely cast to the dtype of the array, and
8243 emits a warning otherwise (issue #1385).
8244 Elements in the resulting array could previously violate constraints on
8246 the elements strategy due to floating-point overflow or truncation of
8247 integers to fit smaller types.
8248 3.67.1 - 2018-08-14
8250 This release contains a tiny refactoring of the internals. There is no
8251 user-visible change.
8252 3.67.0 - 2018-08-10
8254 This release adds a width argument to floats(), to generate lower-pre‐
8255 cision floating point numbers for e.g. Numpy arrays.
8256 The generated examples are always instances of Python's native float
8258 type, which is 64bit, but passing width=32 will ensure that all values
8259 can be exactly represented as 32bit floats. This can be useful to
8260 avoid overflow (to +/- infinity), and for efficiency of generation and
8261 shrinking.
8262 Half-precision floats (width=16) are also supported, but require Numpy
8264 if you are running Python 3.5 or earlier.
8265 3.66.33 - 2018-08-10
8267 This release fixes a bug in floats(), where setting allow_infin‐
8268 ity=False and exactly one of min_value and max_value would allow infi‐
8269 nite values to be generated.
8270 3.66.32 - 2018-08-09
8272 This release adds type hints to the example() and seed() decorators,
8273 and fixes the type hint on register_type_strategy(). The second argu‐
8274 ment to register_type_strategy() must either be a SearchStrategy, or a
8275 callable which takes a type and returns a SearchStrategy.
8276 3.66.31 - 2018-08-08
8278 Another set of changes designed to improve the performance of shrinking
8279 on large examples. In particular the shrinker should now spend consid‐
8280 erably less time running useless shrinks.
8281 3.66.30 - 2018-08-06
8283 "Bug fixes and performance improvements".
8284 This release is a fairly major overhaul of the shrinker designed to im‐
8286 prove its behaviour on large examples, especially around stateful test‐
8287 ing. You should hopefully see shrinking become much faster, with little
8288 to no quality degradation (in some cases quality may even improve).
8289 3.66.29 - 2018-08-05
8291 This release fixes two very minor bugs in the core engine:
8292 • it fixes a corner case that was missing in 3.66.28, which should
8294 cause shrinking to work slightly better.
8295 • it fixes some logic for how shrinking interacts with the database
8297 that was causing Hypothesis to be insufficiently aggressive about
8298 clearing out old keys.
8299 3.66.28 - 2018-08-05
8301 This release improves how Hypothesis handles reducing the size of inte‐
8302 gers' representation. This change should mostly be invisible as it's
8303 purely about the underlying representation and not the generated value,
8304 but it may result in some improvements to shrink performance.
8305 3.66.27 - 2018-08-05
8307 This release changes the order in which Hypothesis chooses parts of the
8308 test case to shrink. For typical usage this should be a significant
8309 performance improvement on large examples. It is unlikely to have a ma‐
8310 jor impact on example quality, but where it does change the result it
8311 should usually be an improvement.
8312 3.66.26 - 2018-08-05
8314 This release improves the debugging information that the shrinker emits
8315 about the operations it performs, giving better summary statistics
8316 about which passes resulted in test executions and whether they were
8317 successful.
8318 3.66.25 - 2018-08-05
8320 This release fixes several bugs that were introduced to the shrinker in
8321 3.66.24 which would have caused it to behave significantly less well
8322 than advertised. With any luck you should actually see the promised
8323 benefits now.
8324 3.66.24 - 2018-08-03
8326 This release changes how Hypothesis deletes data when shrinking in or‐
8327 der to better handle deletion of large numbers of contiguous sequences.
8328 Most tests should see little change, but this will hopefully provide a
8329 significant speed up for stateful testing.
8330 3.66.23 - 2018-08-02
8332 This release makes some internal changes to enable further improvements
8333 to the shrinker. You may see some changes in the final shrunk examples,
8334 but they are unlikely to be significant.
8335 3.66.22 - 2018-08-01
8337 This release adds some more internal caching to the shrinker. This
8338 should cause a significant speed up for shrinking, especially for
8339 stateful testing and large example sizes.
8340 3.66.21 - 2018-08-01
8342 This patch is for downstream packagers - our tests now pass under
8343 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
8344 of Hypothesis itself.
8345 3.66.20 - 2018-08-01
8347 This release removes some functionality from the shrinker that was tak‐
8348 ing a considerable amount of time and does not appear to be useful any
8349 more due to a number of quality improvements in the shrinker.
8350 You may see some degradation in shrink quality as a result of this, but
8352 mostly shrinking should just get much faster.
8353 3.66.19 - 2018-08-01
8355 This release slightly changes the format of some debugging information
8356 emitted during shrinking, and refactors some of the internal interfaces
8357 around that.
8358 3.66.18 - 2018-07-31
8360 This release is a very small internal refactoring which should have no
8361 user visible impact.
8362 3.66.17 - 2018-07-31
8364 This release fixes a bug that could cause an IndexError to be raised
8365 from inside Hypothesis during shrinking. It is likely that it was im‐
8366 possible to trigger this bug in practice - it was only made visible by
8367 some currently unreleased work.
8368 3.66.16 - 2018-07-31
8370 This release is a very small internal refactoring which should have no
8371 user visible impact.
8372 3.66.15 - 2018-07-31
8374 This release makes Hypothesis's shrinking faster by removing some re‐
8375 dundant work that it does when minimizing values in its internal repre‐
8376 sentation.
8377 3.66.14 - 2018-07-30
8379 This release expands the deprecation of timeout from 3.16.0 to also
8380 emit the deprecation warning in find or stateful testing.
8381 3.66.13 - 2018-07-30
8383 This release adds an additional shrink pass that is able to reduce the
8384 size of examples in some cases where the transformation is non-obvious.
8385 In particular this will improve the quality of some examples which
8386 would have regressed in 3.66.12.
8387 3.66.12 - 2018-07-28
8389 This release changes how we group data together for shrinking. It
8390 should result in improved shrinker performance, especially in stateful
8391 testing.
8392 3.66.11 - 2018-07-28
8394 This patch modifies how which rule to run is selected during rule based
8395 stateful testing. This should result in a slight performance increase
8396 during generation and a significant performance and quality improvement
8397 when shrinking.
8398 As a result of this change, some state machines which would previously
8400 have thrown an InvalidDefinition are no longer detected as invalid.
8401 3.66.10 - 2018-07-28
8403 This release weakens some minor functionality in the shrinker that had
8404 only modest benefit and made its behaviour much harder to reason about.
8405 This is unlikely to have much user visible effect, but it is possible
8407 that in some cases shrinking may get slightly slower. It is primarily
8408 to make it easier to work on the shrinker and pave the way for future
8409 work.
8410 3.66.9 - 2018-07-26
8412 This release improves the information that Hypothesis emits about its
8413 shrinking when verbosity is set to debug.
8414 3.66.8 - 2018-07-24
8416 This patch includes some minor fixes in the documentation, and updates
8417 the minimum version of pytest to 3.0 (released August 2016).
8418 3.66.7 - 2018-07-24
8420 This release fixes a bug where difficult to shrink tests could some‐
8421 times trigger an internal assertion error inside the shrinker.
8422 3.66.6 - 2018-07-23
8424 This patch ensures that Hypothesis fully supports Python 3.7, by up‐
8425 grading from_type() (issue #1264) and fixing some minor issues in our
8426 test suite (issue #1148).
8427 3.66.5 - 2018-07-22
8429 This patch fixes the online docs for various extras, by ensuring that
8430 their dependencies are installed on readthedocs.io (issue #1326).
8431 3.66.4 - 2018-07-20
8433 This release improves the shrinker's ability to reorder examples.
8434 For example, consider the following test:
8436 import hypothesis.strategies as st
8438 from hypothesis import given
8439 @given(st.text(), st.text())
8442 def test_non_equal(x, y):
8443 assert x != y
8444 Previously this could have failed with either of x="", y="0" or x="0",
8446 y="". Now it should always fail with x="", y="0".
8447 This will allow the shrinker to produce more consistent results, espe‐
8449 cially in cases where test cases contain some ordered collection whose
8450 actual order does not matter.
8451 3.66.3 - 2018-07-20
8453 This patch fixes inference in the builds() strategy with subtypes of
8454 python:typing.NamedTuple, where the __init__ method is not useful for
8455 introspection. We now use the field types instead - thanks to James
8456 Uther for identifying this bug.
8457 3.66.2 - 2018-07-19
8459 This release improves the shrinker's ability to handle situations where
8460 there is an additive constraint between two values.
8461 For example, consider the following test:
8463 import hypothesis.strategies as st
8465 from hypothesis import given
8466 @given(st.integers(), st.integers())
8469 def test_does_not_exceed_100(m, n):
8470 assert m + n < 100
8471 Previously this could have failed with almost any pair (m, n) with 0 <=
8473 m <= n and m + n == 100. Now it should almost always fail with m=0,
8474 n=100.
8475 This is a relatively niche specialisation, but can be useful in situa‐
8477 tions where e.g. a bug is triggered by an integer overflow.
8478 3.66.1 - 2018-07-09
8480 This patch fixes a rare bug where an incorrect percentage drawtime
8481 could be displayed for a test, when the system clock was changed during
8482 a test running under Python 2 (we use python:time.monotonic() where it
8483 is available to avoid such problems). It also fixes a possible
8484 zero-division error that can occur when the underlying C library dou‐
8485 ble-rounds an intermediate value in python:math.fsum() and gets the
8486 least significant bit wrong.
8487 3.66.0 - 2018-07-05
8489 This release improves validation of the alphabet argument to the text()
8490 strategy. The following misuses are now deprecated, and will be an er‐
8491 ror in a future version:
8492 • passing an unordered collection (such as set('abc')), which violates
8494 invariants about shrinking and reproducibility
8495 • passing an alphabet sequence with elements that are not strings
8497 • passing an alphabet sequence with elements that are not of length
8499 one, which violates any size constraints that may apply
8500 Thanks to Sushobhit for adding these warnings (issue #1329).
8502 3.65.3 - 2018-07-04
8504 This release fixes a mostly theoretical bug where certain usage of the
8505 internal API could trigger an assertion error inside Hypothesis. It is
8506 unlikely that this problem is even possible to trigger through the pub‐
8507 lic API.
8508 3.65.2 - 2018-07-04
8510 This release fixes dependency information for coverage. Previously Hy‐
8511 pothesis would allow installing coverage with any version, but it only
8512 works with coverage 4.0 or later.
8513 We now specify the correct metadata in our setup.py, so Hypothesis will
8515 only allow installation with compatible versions of coverage.
8516 3.65.1 - 2018-07-03
8518 This patch ensures that stateful tests which raise an error from a
8519 pytest helper still print the sequence of steps taken to reach that
8520 point (issue #1372). This reporting was previously broken because the
8521 helpers inherit directly from python:BaseException, and therefore re‐
8522 quire special handling to catch without breaking e.g. the use of ctrl-C
8523 to quit the test.
8524 3.65.0 - 2018-06-30
8526 This release deprecates the max_shrinks setting in favor of an internal
8527 heuristic. If you need to avoid shrinking examples, use the phases
8528 setting instead. (issue #1235)
8529 3.64.2 - 2018-06-27
8531 This release fixes a bug where an internal assertion error could some‐
8532 times be triggered while shrinking a failing test.
8533 3.64.1 - 2018-06-27
8535 This patch fixes type-checking errors in our vendored pretty-printer,
8536 which were ignored by our mypy config but visible for anyone else
8537 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
8538 3.64.0 - 2018-06-26
8540 This release adds an interface which can be used to insert a wrapper
8541 between the original test function and @given (issue #1257). This will
8542 be particularly useful for test runner extensions such as pytest-trio,
8543 but is not recommended for direct use by other users of Hypothesis.
8544 3.63.0 - 2018-06-26
8546 This release adds a new mechanism to infer strategies for classes de‐
8547 fined using attrs, based on the the type, converter, or validator of
8548 each attribute. This inference is now built in to builds() and
8549 from_type().
8550 On Python 2, from_type() no longer generates instances of int when
8552 passed long, or vice-versa.
8553 3.62.0 - 2018-06-26
8555 This release adds PEP 484 type hints to Hypothesis on a provisional ba‐
8556 sis, using the comment-based syntax for Python 2 compatibility. You
8557 can read more about our type hints here.
8558 It also adds the py.typed marker specified in PEP 561. After you pip
8560 install hypothesis, mypy 0.590 or later will therefore type-check your
8561 use of our public interface!
8562 3.61.0 - 2018-06-24
8564 This release deprecates the use of settings as a context manager, the
8565 use of which is somewhat ambiguous.
8566 Users should define settings with global state or with the @set‐
8568 tings(...) decorator.
8569 3.60.1 - 2018-06-20
8571 Fixed a bug in generating an instance of a Django model from a strategy
8572 where the primary key is generated as part of the strategy. See details
8573 here.
8574 Thanks to Tim Martin for this contribution.
8576 3.60.0 - 2018-06-20
8578 This release adds the @initialize decorator for stateful testing (orig‐
8579 inally discussed in issue #1216). All @initialize rules will be called
8580 once each in an arbitrary order before any normal rule is called.
8581 3.59.3 - 2018-06-19
8583 This is a no-op release to take into account some changes to the re‐
8584 lease process. It should have no user visible effect.
8585 3.59.2 - 2018-06-18
8587 This adds support for partially sorting examples which cannot be fully
8588 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
8589 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
8590 0].
8591 Thanks to Luke for contributing.
8593 3.59.1 - 2018-06-16
8595 This patch uses python:random.getstate() and python:random.setstate()
8596 to restore the PRNG state after @given runs deterministic tests. With‐
8597 out restoring state, you might have noticed problems such as issue
8598 #1266. The fix also applies to stateful testing (issue #702).
8599 3.59.0 - 2018-06-14
8601 This release adds the emails() strategy, which generates unicode
8602 strings representing an email address.
8603 Thanks to Sushobhit for moving this to the public API (issue #162).
8605 3.58.1 - 2018-06-13
8607 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
8608 1 2 3.
8609 Thanks to Luke for contributing.
8611 3.58.0 - 2018-06-13
8613 This adds a new extra timezones() strategy that generates dateutil
8614 timezones.
8615 Thanks to Conrad for contributing.
8617 3.57.0 - 2018-05-20
8619 Using an unordered collection with the permutations() strategy has been
8620 deprecated because the order in which e.g. a set shrinks is arbitrary.
8621 This may cause different results between runs.
8622 3.56.10 - 2018-05-16
8624 This release makes hypothesis.settings.define_setting a private method,
8625 which has the effect of hiding it from the documentation.
8626 3.56.9 - 2018-05-11
8628 This is another release with no functionality changes as part of
8629 changes to Hypothesis's new release tagging scheme.
8630 3.56.8 - 2018-05-10
8632 This is a release with no functionality changes that moves Hypothesis
8633 over to a new release tagging scheme.
8634 3.56.7 - 2018-05-10
8636 This release provides a performance improvement for most tests, but in
8637 particular users of sampled_from() who don't have numpy installed
8638 should see a significant performance improvement.
8639 3.56.6 - 2018-05-09
8641 This patch contains further internal work to support Mypy. There are
8642 no user-visible changes... yet.
8643 3.56.5 - 2018-04-22
8645 This patch contains some internal refactoring to run mypy in CI. There
8646 are no user-visible changes.
8647 3.56.4 - 2018-04-21
8649 This release involves some very minor internal clean up and should have
8650 no user visible effect at all.
8651 3.56.3 - 2018-04-20
8653 This release fixes a problem introduced in 3.56.0 where setting the hy‐
8654 pothesis home directory (through currently undocumented means) would no
8655 longer result in the default database location living in the new home
8656 directory.
8657 3.56.2 - 2018-04-20
8659 This release fixes a problem introduced in 3.56.0 where setting max_ex‐
8660 amples to 1 would result in tests failing with Unsatisfiable. This
8661 problem could also occur in other harder to trigger circumstances (e.g.
8662 by setting it to a low value, having a hard to satisfy assumption, and
8663 disabling health checks).
8664 3.56.1 - 2018-04-20
8666 This release fixes a problem that was introduced in 3.56.0: Use of the
8667 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
8668 cated, actually broken due to being read before various setup the dep‐
8669 recation path needed was done. It now works correctly (and emits a dep‐
8670 recation warning).
8671 3.56.0 - 2018-04-17
8673 This release deprecates several redundant or internally oriented set‐
8674 tings, working towards an orthogonal set of configuration options that
8675 are widely useful without requiring any knowledge of our internals (‐
8676 issue #535).
8677 • Deprecated settings that no longer have any effect are no longer
8679 shown in the __repr__ unless set to a non-default value.
8680 • hypothesis.settings.perform_health_check is deprecated, as it dupli‐
8682 cates suppress_health_check.
8683 • hypothesis.settings.max_iterations is deprecated and disabled, be‐
8685 cause we can usually get better behaviour from an internal heuristic
8686 than a user-controlled setting.
8687 • hypothesis.settings.min_satisfying_examples is deprecated and dis‐
8689 abled, due to overlap with the filter_too_much healthcheck and poor
8690 interaction with max_examples.
8691 • HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
8693 the profile system instead.
8694 • Examples tried by find() are now reported at debug verbosity level
8696 (as well as verbose level).
8697 3.55.6 - 2018-04-14
8699 This release fixes a somewhat obscure condition (issue #1230) under
8700 which you could occasionally see a failing test trigger an assertion
8701 error inside Hypothesis instead of failing normally.
8702 3.55.5 - 2018-04-14
8704 This patch fixes one possible cause of issue #966. When running Python
8705 2 with hash randomisation, passing a python:bytes object to python:ran‐
8706 dom.seed() would use version=1, which broke derandomize (because the
8707 seed depended on a randomised hash). If derandomize is still nondeter‐
8708 ministic for you, please open an issue.
8709 3.55.4 - 2018-04-13
8711 This patch makes a variety of minor improvements to the documentation,
8712 and improves a few validation messages for invalid inputs.
8713 3.55.3 - 2018-04-12
8715 This release updates the URL metadata associated with the PyPI package
8716 (again). It has no other user visible effects.
8717 3.55.2 - 2018-04-11
8719 This release updates the URL metadata associated with the PyPI package.
8720 It has no other user visible effects.
8721 3.55.1 - 2018-04-06
8723 This patch relaxes constraints in our tests on the expected values re‐
8724 turned by the standard library function hypot() and the internal helper
8725 function cathetus, to fix near-exact test failures on some 32-bit sys‐
8726 tems used by downstream packagers.
8727 3.55.0 - 2018-04-05
8729 This release includes several improvements to the handling of the data‐
8730 base setting.
8731 • The database_file setting was a historical artefact, and you should
8733 just use database directly.
8734 • The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
8736 favor of load_profile() and the database setting.
8737 • If you have not configured the example database at all and the de‐
8739 fault location is not usable (due to e.g. permissions issues), Hy‐
8740 pothesis will fall back to an in-memory database. This is not per‐
8741 sisted between sessions, but means that the defaults work on
8742 read-only filesystems.
8743 3.54.0 - 2018-04-04
8745 This release improves the complex_numbers() strategy, which now sup‐
8746 ports min_magnitude and max_magnitude arguments, along with allow_nan
8747 and allow_infinity like for floats().
8748 Thanks to J.J. Green for this feature.
8750 3.53.0 - 2018-04-01
8752 This release removes support for Django 1.8, which reached end of life
8753 on 2018-04-01. You can see Django's release and support schedule on
8754 the Django Project website.
8755 3.52.3 - 2018-04-01
8757 This patch fixes the min_satisfying_examples settings documentation, by
8758 explaining that example shrinking is tracked at the level of the under‐
8759 lying bytestream rather than the output value.
8760 The output from find() in verbose mode has also been adjusted - see the
8762 example session - to avoid duplicating lines when the example repr is
8763 constant, even if the underlying representation has been shrunken.
8764 3.52.2 - 2018-03-30
8766 This release improves the output of failures with rule based stateful
8767 testing in two ways:
8768 • The output from it is now usually valid Python code.
8770 • When the same value has two different names because it belongs to two
8772 different bundles, it will now display with the name associated with
8773 the correct bundle for a rule argument where it is used.
8774 3.52.1 - 2018-03-29
8776 This release improves the behaviour of stateful testing in two ways:
8777 • Previously some runs would run no steps (issue #376). This should no
8779 longer happen.
8780 • RuleBasedStateMachine tests which used bundles extensively would of‐
8782 ten shrink terribly. This should now be significantly improved,
8783 though there is likely a lot more room for improvement.
8784 This release also involves a low level change to how ranges of integers
8786 are handles which may result in other improvements to shrink quality in
8787 some cases.
8788 3.52.0 - 2018-03-24
8790 This release deprecates use of @settings(...) as a decorator, on func‐
8791 tions or methods that are not also decorated with @given. You can
8792 still apply these decorators in any order, though you should only do so
8793 once each.
8794 Applying @given twice was already deprecated, and applying @set‐
8796 tings(...) twice is deprecated in this release and will become an error
8797 in a future version. Neither could ever be used twice to good effect.
8798 Using @settings(...) as the sole decorator on a test is completely
8800 pointless, so this common usage error will become an error in a future
8801 version of Hypothesis.
8802 3.51.0 - 2018-03-24
8804 This release deprecates the average_size argument to lists() and other
8805 collection strategies. You should simply delete it wherever it was
8806 used in your tests, as it no longer has any effect.
8807 In early versions of Hypothesis, the average_size argument was treated
8809 as a hint about the distribution of examples from a strategy. Subse‐
8810 quent improvements to the conceptual model and the engine for generat‐
8811 ing and shrinking examples mean it is more effective to simply describe
8812 what constitutes a valid example, and let our internals handle the dis‐
8813 tribution.
8814 3.50.3 - 2018-03-24
8816 This patch contains some internal refactoring so that we can run with
8817 warnings as errors in CI.
8818 3.50.2 - 2018-03-20
8820 This has no user-visible changes except one slight formatting change to
8821 one docstring, to avoid a deprecation warning.
8822 3.50.1 - 2018-03-20
8824 This patch fixes an internal error introduced in 3.48.0, where a check
8825 for the Django test runner would expose import-time errors in Django
8826 configuration (issue #1167).
8827 3.50.0 - 2018-03-19
8829 This release improves validation of numeric bounds for some strategies.
8830 • integers() and floats() now raise InvalidArgument if passed a
8832 min_value or max_value which is not an instance of Real, instead of
8833 various internal errors.
8834 • floats() now converts its bounding values to the nearest float above
8836 or below the min or max bound respectively, instead of just casting
8837 to float. The old behaviour was incorrect in that you could generate
8838 float(min_value), even when this was less than min_value itself (pos‐
8839 sible with eg. fractions).
8840 • When both bounds are provided to floats() but there are no floats in
8842 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
8843 raised.
8844 • decimals() gives a more useful error message if passed a string that
8846 cannot be converted to Decimal in a context where this error is not
8847 trapped.
8848 Code that previously seemed to work may be explicitly broken if there
8850 were no floats between min_value and max_value (only possible with
8851 non-float bounds), or if a bound was not a Real number but still al‐
8852 lowed in python:math.isnan (some custom classes with a __float__
8853 method).
8854 3.49.1 - 2018-03-15
8856 This patch fixes our tests for Numpy dtype strategies on big-endian
8857 platforms, where the strategy behaved correctly but the test assumed
8858 that the native byte order was little-endian.
8859 There is no user impact unless you are running our test suite on
8861 big-endian platforms. Thanks to Graham Inggs for reporting issue
8862 #1164.
8863 3.49.0 - 2018-03-12
8865 This release deprecates passing elements=None to collection strategies,
8866 such as lists().
8867 Requiring lists(nothing()) or builds(list) instead of lists() means
8869 slightly more typing, but also improves the consistency and discover‐
8870 ability of our API - as well as showing how to compose or construct
8871 strategies in ways that still work in more complex situations.
8872 Passing a nonzero max_size to a collection strategy where the elements
8874 strategy contains no values is now deprecated, and will be an error in
8875 a future version. The equivalent with elements=None is already an er‐
8876 ror.
8877 3.48.1 - 2018-03-05
8879 This patch will minimize examples that would come out non-minimal in
8880 previous versions. Thanks to Kyle Reeve for this patch.
8881 3.48.0 - 2018-03-05
8883 This release improves some "unhappy paths" when using Hypothesis with
8884 the standard library python:unittest module:
8885 • Applying @given to a non-test method which is overridden from
8887 python:unittest.TestCase, such as setUp, raises a new health check.
8888 (issue #991)
8889 • Using subTest() within a test decorated with @given would leak inter‐
8891 mediate results when tests were run under the python:unittest test
8892 runner. Individual reporting of failing subtests is now disabled
8893 during a test using @given. (issue #1071)
8894 • @given is still not a class decorator, but the error message if you
8896 try using it on a class has been improved.
8897 As a related improvement, using django:django.test.TestCase with @given
8899 instead of hypothesis.extra.django.TestCase raises an explicit error
8900 instead of running all examples in a single database transaction.
8901 3.47.0 - 2018-03-02
8903 register_profile now accepts keyword arguments for specific settings,
8904 and the parent settings object is now optional. Using a name for a
8905 registered profile which is not a string was never suggested, but it is
8906 now also deprecated and will eventually be an error.
8907 3.46.2 - 2018-03-01
8909 This release removes an unnecessary branch from the code, and has no
8910 user-visible impact.
8911 3.46.1 - 2018-03-01
8913 This changes only the formatting of our docstrings and should have no
8914 user-visible effects.
8915 3.46.0 - 2018-02-26
8917 characters() has improved docs about what arguments are valid, and ad‐
8918 ditional validation logic to raise a clear error early (instead of e.g.
8919 silently ignoring a bad argument). Categories may be specified as the
8920 Unicode 'general category' (eg u'Nd'), or as the 'major category' (eg
8921 [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
8922 In previous versions, general categories were supported and all other
8924 input was silently ignored. Now, major categories are supported in ad‐
8925 dition to general categories (which may change the behaviour of some
8926 existing code), and all other input is deprecated.
8927 3.45.5 - 2018-02-26
8929 This patch improves strategy inference in hypothesis.extra.django to
8930 account for some validators in addition to field type - see issue #1116
8931 for ongoing work in this space.
8932 Specifically, if a CharField or TextField has an attached RegexValida‐
8934 tor, we now use from_regex() instead of text() as the underlying strat‐
8935 egy. This allows us to generate examples of the default User model,
8936 closing issue #1112.
8937 3.45.4 - 2018-02-25
8939 This patch improves some internal debugging information, fixes a typo
8940 in a validation error message, and expands the documentation for new
8941 contributors.
8942 3.45.3 - 2018-02-23
8944 This patch may improve example shrinking slightly for some strategies.
8945 3.45.2 - 2018-02-18
8947 This release makes our docstring style more consistent, thanks to
8948 flake8-docstrings. There are no user-visible changes.
8949 3.45.1 - 2018-02-17
8951 This fixes an indentation issue in docstrings for datetimes(), dates(),
8952 times(), and timedeltas().
8953 3.45.0 - 2018-02-13
8955 This release fixes builds() so that target can be used as a keyword ar‐
8956 gument for passing values to the target. The target itself can still be
8957 specified as a keyword argument, but that behavior is now deprecated.
8958 The target should be provided as the first positional argument.
8959 3.44.26 - 2018-02-06
8961 This release fixes some formatting issues in the Hypothesis source
8962 code. It should have no externally visible effects.
8963 3.44.25 - 2018-02-05
8965 This release changes the way in which Hypothesis tries to shrink the
8966 size of examples. It probably won't have much impact, but might make
8967 shrinking faster in some cases. It is unlikely but not impossible that
8968 it will change the resulting examples.
8969 3.44.24 - 2018-01-27
8971 This release fixes dependency information when installing Hypothesis
8972 from a binary "wheel" distribution.
8973 • The install_requires for enum34 is resolved at install time, rather
8975 than at build time (with potentially different results).
8976 • Django has fixed their python_requires for versions 2.0.0 onward,
8978 simplifying Python2-compatible constraints for downstream projects.
8979 3.44.23 - 2018-01-24
8981 This release improves shrinking in a class of pathological examples
8982 that you are probably never hitting in practice. If you are hitting
8983 them in practice this should be a significant speed up in shrinking. If
8984 you are not, you are very unlikely to notice any difference. You might
8985 see a slight slow down and/or slightly better falsifying examples.
8986 3.44.22 - 2018-01-23
8988 This release fixes a dependency problem. It was possible to install
8989 Hypothesis with an old version of attrs, which would throw a TypeError
8990 as soon as you tried to import hypothesis. Specifically, you need at‐
8991 trs 16.0.0 or newer.
8992 Hypothesis will now require the correct version of attrs when in‐
8994 stalling.
8995 3.44.21 - 2018-01-22
8997 This change adds some additional structural information that Hypothesis
8998 will use to guide its search.
8999 You mostly shouldn't see much difference from this. The two most likely
9001 effects you would notice are:
9002 1. Hypothesis stores slightly more examples in its database for passing
9004 tests.
9005 2. Hypothesis may find new bugs that it was previously missing, but it
9007 probably won't (this is a basic implementation of the feature that
9008 is intended to support future work. Although it is useful on its
9009 own, it's not very useful on its own).
9010 3.44.20 - 2018-01-21
9012 This is a small refactoring release that changes how Hypothesis tracks
9013 some information about the boundary of examples in its internal repre‐
9014 sentation.
9015 You are unlikely to see much difference in behaviour, but memory usage
9017 and run time may both go down slightly during normal test execution,
9018 and when failing Hypothesis might print its failing example slightly
9019 sooner.
9020 3.44.19 - 2018-01-21
9022 This changes how we compute the default average_size for all collection
9023 strategies. Previously setting a max_size without setting an aver‐
9024 age_size would have the seemingly paradoxical effect of making data
9025 generation slower, because it would raise the average size from its de‐
9026 fault. Now setting max_size will either leave the default unchanged or
9027 lower it from its default.
9028 If you are currently experiencing this problem, this may make your
9030 tests substantially faster. If you are not, this will likely have no
9031 effect on you.
9032 3.44.18 - 2018-01-20
9034 This is a small refactoring release that changes how Hypothesis detects
9035 when the structure of data generation depends on earlier values gener‐
9036 ated (e.g. when using flatmap or composite()). It should not have any
9037 observable effect on behaviour.
9038 3.44.17 - 2018-01-15
9040 This release fixes a typo in internal documentation, and has no
9041 user-visible impact.
9042 3.44.16 - 2018-01-13
9044 This release improves test case reduction for recursive data struc‐
9045 tures. Hypothesis now guarantees that whenever a strategy calls itself
9046 recursively (usually this will happen because you are using de‐
9047 ferred()), any recursive call may replace the top level value. e.g.
9048 given a tree structure, Hypothesis will always try replacing it with a
9049 subtree.
9050 Additionally this introduces a new heuristic that may in some circum‐
9052 stances significantly speed up test case reduction - Hypothesis should
9053 be better at immediately replacing elements drawn inside another strat‐
9054 egy with their minimal possible value.
9055 3.44.15 - 2018-01-13
9057 from_type() can now resolve recursive types such as binary trees (issue
9058 #1004). Detection of non-type arguments has also improved, leading to
9059 better error messages in many cases involving forward references.
9060 3.44.14 - 2018-01-08
9062 This release fixes a bug in the shrinker that prevented the optimisa‐
9063 tions in 3.44.6 from working in some cases. It would not have worked
9064 correctly when filtered examples were nested (e.g. with a set of inte‐
9065 gers in some range).
9066 This would not have resulted in any correctness problems, but shrinking
9068 may have been slower than it otherwise could be.
9069 3.44.13 - 2018-01-08
9071 This release changes the average bit length of values drawn from inte‐
9072 gers() to be much smaller. Additionally it changes the shrinking order
9073 so that now size is considered before sign - e.g. -1 will be preferred
9074 to +10.
9075 The new internal format for integers required some changes to the mini‐
9077 mizer to make work well, so you may also see some improvements to exam‐
9078 ple quality in unrelated areas.
9079 3.44.12 - 2018-01-07
9081 This changes Hypothesis's internal implementation of weighted sampling.
9082 This will affect example distribution and quality, but you shouldn't
9083 see any other effects.
9084 3.44.11 - 2018-01-06
9086 This is a change to some internals around how Hypothesis handles avoid‐
9087 ing generating duplicate examples and seeking out novel regions of the
9088 search space.
9089 You are unlikely to see much difference as a result of it, but it fixes
9091 a bug where an internal assertion could theoretically be triggered and
9092 has some minor effects on the distribution of examples so could poten‐
9093 tially find bugs that have previously been missed.
9094 3.44.10 - 2018-01-06
9096 This patch avoids creating debug statements when debugging is disabled.
9097 Profiling suggests this is a 5-10% performance improvement (pull re‐
9098 quest #1040).
9099 3.44.9 - 2018-01-06
9101 This patch blacklists null characters ('\x00') in automatically created
9102 strategies for Django CharField and TextField, due to a database issue
9103 which was recently fixed upstream (Hypothesis issue #1045).
9104 3.44.8 - 2018-01-06
9106 This release makes the Hypothesis shrinker slightly less greedy in or‐
9107 der to avoid local minima - when it gets stuck, it makes a small at‐
9108 tempt to search around the final example it would previously have re‐
9109 turned to find a new starting point to shrink from. This should improve
9110 example quality in some cases, especially ones where the test data has
9111 dependencies among parts of it that make it difficult for Hypothesis to
9112 proceed.
9113 3.44.7 - 2018-01-04
9115 This release adds support for Django 2 in the hypothesis-django extra.
9116 This release drops support for Django 1.10, as it is no longer sup‐
9118 ported by the Django team.
9119 3.44.6 - 2018-01-02
9121 This release speeds up test case reduction in many examples by being
9122 better at detecting large shrinks it can use to discard redundant parts
9123 of its input. This will be particularly noticeable in examples that
9124 make use of filtering and for some integer ranges.
9125 3.44.5 - 2018-01-02
9127 Happy new year!
9128 This is a no-op release that updates the year range on all of the copy‐
9130 right headers in our source to include 2018.
9131 3.44.4 - 2017-12-23
9133 This release fixes issue #1044, which slowed tests by up to 6% due to
9134 broken caching.
9135 3.44.3 - 2017-12-21
9137 This release improves the shrinker in cases where examples drawn ear‐
9138 lier can affect how much data is drawn later (e.g. when you draw a
9139 length parameter in a composite and then draw that many elements). Ex‐
9140 amples found in cases like this should now be much closer to minimal.
9141 3.44.2 - 2017-12-20
9143 This is a pure refactoring release which changes how Hypothesis manages
9144 its set of examples internally. It should have no externally visible
9145 effects.
9146 3.44.1 - 2017-12-18
9148 This release fixes issue #997, in which under some circumstances the
9149 body of tests run under Hypothesis would not show up when run under
9150 coverage even though the tests were run and the code they called out‐
9151 side of the test file would show up normally.
9152 3.44.0 - 2017-12-17
9154 This release adds a new feature: The @reproduce_failure decorator, de‐
9155 signed to make it easy to use Hypothesis's binary format for examples
9156 to reproduce a problem locally without having to share your example
9157 database between machines.
9158 This also changes when seeds are printed:
9160 • They will no longer be printed for normal falsifying examples, as
9162 there are now adequate ways of reproducing those for all cases, so it
9163 just contributes noise.
9164 • They will once again be printed when reusing examples from the data‐
9166 base, as health check failures should now be more reliable in this
9167 scenario so it will almost always work in this case.
9168 This work was funded by Smarkets.
9170 3.43.1 - 2017-12-17
9172 This release fixes a bug with Hypothesis's database management - exam‐
9173 ples that were found in the course of shrinking were saved in a way
9174 that indicated that they had distinct causes, and so they would all be
9175 retried on the start of the next test. The intended behaviour, which is
9176 now what is implemented, is that only a bounded subset of these exam‐
9177 ples would be retried.
9178 3.43.0 - 2017-12-17
9180 HypothesisDeprecationWarning now inherits from python:FutureWarning in‐
9181 stead of python:DeprecationWarning, as recommended by PEP 565 for
9182 user-facing warnings (issue #618). If you have not changed the default
9183 warnings settings, you will now see each distinct HypothesisDepreca‐
9184 tionWarning instead of only the first.
9185 3.42.2 - 2017-12-12
9187 This patch fixes issue #1017, where instances of a list or tuple sub‐
9188 type used as an argument to a strategy would be coerced to tuple.
9189 3.42.1 - 2017-12-10
9191 This release has some internal cleanup, which makes reading the code
9192 more pleasant and may shrink large examples slightly faster.
9193 3.42.0 - 2017-12-09
9195 This release deprecates faker-extra, which was designed as a transition
9196 strategy but does not support example shrinking or coverage-guided dis‐
9197 covery.
9198 3.41.0 - 2017-12-06
9200 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
9201 pling from multi-dimensional ndarrays still results in a deprecation
9202 warning. Thanks to Charlie Tanksley for this patch.
9203 3.40.1 - 2017-12-04
9205 This release makes two changes:
9206 • It makes the calculation of some of the metadata that Hypothesis uses
9208 for shrinking occur lazily. This should speed up performance of test
9209 case generation a bit because it no longer calculates information it
9210 doesn't need.
9211 • It improves the shrinker for certain classes of nested examples. e.g.
9213 when shrinking lists of lists, the shrinker is now able to concate‐
9214 nate two adjacent lists together into a single list. As a result of
9215 this change, shrinking may get somewhat slower when the minimal exam‐
9216 ple found is large.
9217 3.40.0 - 2017-12-02
9219 This release improves how various ways of seeding Hypothesis interact
9220 with the example database:
9221 • Using the example database with seed() is now deprecated. You should
9223 set database=None if you are doing that. This will only warn if you
9224 actually load examples from the database while using @seed.
9225 • The derandomize will behave the same way as @seed.
9227 • Using --hypothesis-seed will disable use of the database.
9229 • If a test used examples from the database, it will not suggest using
9231 a seed to reproduce it, because that won't work.
9232 This work was funded by Smarkets.
9234 3.39.0 - 2017-12-01
9236 This release adds a new health check that checks if the smallest "natu‐
9237 ral" possible example of your test case is very large - this will tend
9238 to cause Hypothesis to generate bad examples and be quite slow.
9239 This work was funded by Smarkets.
9241 3.38.9 - 2017-11-29
9243 This is a documentation release to improve the documentation of shrink‐
9244 ing behaviour for Hypothesis's strategies.
9245 3.38.8 - 2017-11-29
9247 This release improves the performance of characters() when using black‐
9248 list_characters and from_regex() when using negative character classes.
9249 The problems this fixes were found in the course of work funded by
9251 Smarkets.
9252 3.38.7 - 2017-11-29
9254 This is a patch release for from_regex(), which had a bug in handling
9255 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
9256 rectly when parsing regex.
9257 3.38.6 - 2017-11-28
9259 This patch changes a few byte-string literals from double to single
9260 quotes, thanks to an update in unify. There are no user-visible
9261 changes.
9262 3.38.5 - 2017-11-23
9264 This fixes the repr of strategies using lambda that are defined inside
9265 decorators to include the lambda source.
9266 This would mostly have been visible when using the statistics function‐
9268 ality - lambdas used for e.g. filtering would have shown up with a <un‐
9269 known> as their body. This can still happen, but it should happen less
9270 often now.
9271 3.38.4 - 2017-11-22
9273 This release updates the reported statistics so that they show approxi‐
9274 mately what fraction of your test run time is spent in data generation
9275 (as opposed to test execution).
9276 This work was funded by Smarkets.
9278 3.38.3 - 2017-11-21
9280 This is a documentation release, which ensures code examples are up to
9281 date by running them as doctests in CI (issue #711).
9282 3.38.2 - 2017-11-21
9284 This release changes the behaviour of the deadline setting when used
9285 with data(): Time spent inside calls to data.draw will no longer be
9286 counted towards the deadline time.
9287 As a side effect of some refactoring required for this work, the way
9289 flaky tests are handled has changed slightly. You are unlikely to see
9290 much difference from this, but some error messages will have changed.
9291 This work was funded by Smarkets.
9293 3.38.1 - 2017-11-21
9295 This patch has a variety of non-user-visible refactorings, removing
9296 various minor warts ranging from indirect imports to typos in comments.
9297 3.38.0 - 2017-11-18
9299 This release overhauls the health check system in a variety of small
9300 ways. It adds no new features, but is nevertheless a minor release be‐
9301 cause it changes which tests are likely to fail health checks.
9302 The most noticeable effect is that some tests that used to fail health
9304 checks will now pass, and some that used to pass will fail. These
9305 should all be improvements in accuracy. In particular:
9306 • New failures will usually be because they are now taking into account
9308 things like use of data() and assume() inside the test body.
9309 • New failures may also be because for some classes of example the way
9311 data generation performance was measured was artificially faster than
9312 real data generation (for most examples that are hitting performance
9313 health checks the opposite should be the case).
9314 • Tests that used to fail health checks and now pass do so because the
9316 health check system used to run in a way that was subtly different
9317 than the main Hypothesis data generation and lacked some of its sup‐
9318 port for e.g. large examples.
9319 If your data generation is especially slow, you may also see your tests
9321 get somewhat faster, as there is no longer a separate health check
9322 phase. This will be particularly noticeable when rerunning test fail‐
9323 ures.
9324 This work was funded by Smarkets.
9326 3.37.0 - 2017-11-12
9328 This is a deprecation release for some health check related features.
9329 The following are now deprecated:
9331 • Passing HealthCheck.exception_in_generation to suppress_health_check.
9333 This no longer does anything even when passed - All errors that oc‐
9334 cur during data generation will now be immediately reraised rather
9335 than going through the health check mechanism.
9336 • Passing HealthCheck.random_module to suppress_health_check. This
9338 hasn't done anything for a long time, but was never explicitly depre‐
9339 cated. Hypothesis always seeds the random module when running @given
9340 tests, so this is no longer an error and suppressing it doesn't do
9341 anything.
9342 • Passing non-HealthCheck values in suppress_health_check. This was
9344 previously allowed but never did anything useful.
9345 In addition, passing a non-iterable value as suppress_health_check will
9347 now raise an error immediately (it would never have worked correctly,
9348 but it would previously have failed later). Some validation error mes‐
9349 sages have also been updated.
9350 This work was funded by Smarkets.
9352 3.36.1 - 2017-11-10
9354 This is a yak shaving release, mostly concerned with our own tests.
9355 While getfullargspec() was documented as deprecated in Python 3.5, it
9357 never actually emitted a warning. Our code to silence this (nonexis‐
9358 tent) warning has therefore been removed.
9359 We now run our tests with DeprecationWarning as an error, and made some
9361 minor changes to our own tests as a result. This required similar up‐
9362 stream updates to coverage and execnet (a test-time dependency via
9363 pytest-xdist).
9364 There is no user-visible change in Hypothesis itself, but we encourage
9366 you to consider enabling deprecations as errors in your own tests.
9367 3.36.0 - 2017-11-06
9369 This release adds a setting to the public API, and does some internal
9370 cleanup:
9371 • The derandomize setting is now documented (issue #890)
9373 • Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
9375 #953)
9376 • Documented the strict setting as deprecated, and updated the build so
9378 our docs always match deprecations in the code.
9379 3.35.0 - 2017-11-06
9381 This minor release supports constraining uuids() to generate a particu‐
9382 lar version of UUID (issue #721).
9383 Thanks to Dion Misic for this feature.
9385 3.34.1 - 2017-11-02
9387 This patch updates the documentation to suggest builds(callable) in‐
9388 stead of just(callable()).
9389 3.34.0 - 2017-11-02
9391 Hypothesis now emits deprecation warnings if you apply @given more than
9392 once to a target.
9393 Applying @given repeatedly wraps the target multiple times. Each wrap‐
9395 per will search the space of of possible parameters separately. This
9396 is equivalent but will be much more inefficient than doing it with a
9397 single call to @given.
9398 For example, instead of @given(booleans()) @given(integers()), you
9400 could write @given(booleans(), integers())
9401 3.33.1 - 2017-11-02
9403 This is a bugfix release:
9404 • builds() would try to infer a strategy for required positional argu‐
9406 ments of the target from type hints, even if they had been given to
9407 builds() as positional arguments (issue #946). Now it only infers
9408 missing required arguments.
9409 • An internal introspection function wrongly reported self as a re‐
9411 quired argument for bound methods, which might also have affected
9412 builds(). Now it knows better.
9413 3.33.0 - 2017-10-16
9415 This release supports strategy inference for more Django field types -
9416 you can now omit an argument for Date, Time, Duration, Slug, IP Ad‐
9417 dress, and UUID fields. (issue #642)
9418 Strategy generation for fields with grouped choices now selects choices
9420 from each group, instead of selecting from the group names.
9421 3.32.2 - 2017-10-15
9423 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
9424 an experimental basis. It has never actually worked, and the new Hy‐
9425 pothesis example database is designed to make such a tool unnecessary.
9426 3.32.1 - 2017-10-13
9428 This patch has two improvements for strategies based on enumerations.
9429 • from_type() now handles enumerations correctly, delegating to sam‐
9431 pled_from(). Previously it noted that Enum.__init__ has no required
9432 arguments and therefore delegated to builds(), which would subse‐
9433 quently fail.
9434 • When sampling from an python:enum.Flag, we also generate combinations
9436 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
9437 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
9438 3.32.0 - 2017-10-09
9440 This changes the default value of the use_coverage setting to True when
9441 running on pypy (it was already True on CPython).
9442 It was previously set to False because we expected it to be too slow,
9444 but recent benchmarking shows that actually performance of the feature
9445 on pypy is fairly acceptable - sometimes it's slower than on CPython,
9446 sometimes it's faster, but it's generally within a factor of two either
9447 way.
9448 3.31.6 - 2017-10-08
9450 This patch improves the quality of strategies inferred from Numpy
9451 dtypes:
9452 • Integer dtypes generated examples with the upper half of their
9454 (non-sign) bits set to zero. The inferred strategies can now produce
9455 any representable integer.
9456 • Fixed-width unicode- and byte-string dtypes now cap the internal ex‐
9458 ample length, which should improve example and shrink quality.
9459 • Numpy arrays can only store fixed-size strings internally, and allow
9461 shorter strings by right-padding them with null bytes. Inferred
9462 string strategies no longer generate such values, as they can never
9463 be retrieved from an array. This improves shrinking performance by
9464 skipping useless values.
9465 This has already been useful in Hypothesis - we found an overflow bug
9467 in our Pandas support, and as a result indexes() and range_indexes()
9468 now check that min_size and max_size are at least zero.
9469 3.31.5 - 2017-10-08
9471 This release fixes a performance problem in tests where the use_cover‐
9472 age setting is True.
9473 Tests experience a slow-down proportionate to the amount of code they
9475 cover. This is still the case, but the factor is now low enough that
9476 it should be unnoticeable. Previously it was large and became much
9477 larger in 3.30.4.
9478 3.31.4 - 2017-10-08
9480 from_type() failed with a very confusing error if passed a NewType() (‐
9481 issue #901). These psudeo-types are now unwrapped correctly, and
9482 strategy inference works as expected.
9483 3.31.3 - 2017-10-06
9485 This release makes some small optimisations to our use of coverage that
9486 should reduce constant per-example overhead. This is probably only no‐
9487 ticeable on examples where the test itself is quite fast. On no-op
9488 tests that don't test anything you may see up to a fourfold speed in‐
9489 crease (which is still significantly slower than without coverage). On
9490 more realistic tests the speed up is likely to be less than that.
9491 3.31.2 - 2017-09-30
9493 This release fixes some formatting and small typos/grammar issues in
9494 the documentation, specifically the page docs/settings.rst, and the in‐
9495 line docs for the various settings.
9496 3.31.1 - 2017-09-30
9498 This release improves the handling of deadlines so that they act better
9499 with the shrinking process. This fixes issue #892.
9500 This involves two changes:
9502 1. The deadline is raised during the initial generation and shrinking,
9504 and then lowered to the set value for final replay. This restricts
9505 our attention to examples which exceed the deadline by a more sig‐
9506 nificant margin, which increases their reliability.
9507 2. When despite the above a test still becomes flaky because it is sig‐
9509 nificantly faster on rerun than it was on its first run, the error
9510 message is now more explicit about the nature of this problem, and
9511 includes both the initial test run time and the new test run time.
9512 In addition, this release also clarifies the documentation of the dead‐
9514 line setting slightly to be more explicit about where it applies.
9515 This work was funded by Smarkets.
9517 3.31.0 - 2017-09-29
9519 This release blocks installation of Hypothesis on Python 3.3, which
9520 reached its end of life date on 2017-09-29.
9521 This should not be of interest to anyone but downstream maintainers -
9523 if you are affected, migrate to a secure version of Python as soon as
9524 possible or at least seek commercial support.
9525 3.30.4 - 2017-09-27
9527 This release makes several changes:
9528 1. It significantly improves Hypothesis's ability to use coverage in‐
9530 formation to find interesting examples.
9531 2. It reduces the default max_examples setting from 200 to 100. This
9533 takes advantage of the improved algorithm meaning fewer examples are
9534 typically needed to get the same testing and is sufficiently better
9535 at covering interesting behaviour, and offsets some of the perfor‐
9536 mance problems of running under coverage.
9537 3. Hypothesis will always try to start its testing with an example that
9539 is near minimized.
9540 The new algorithm for 1 also makes some changes to Hypothesis's low
9542 level data generation which apply even with coverage turned off. They
9543 generally reduce the total amount of data generated, which should im‐
9544 prove test performance somewhat. Between this and 3 you should see a
9545 noticeable reduction in test runtime (how much so depends on your tests
9546 and how much example size affects their performance. On our benchmarks,
9547 where data generation dominates, we saw up to a factor of two perfor‐
9548 mance improvement, but it's unlikely to be that large.
9549 3.30.3 - 2017-09-25
9551 This release fixes some formatting and small typos/grammar issues in
9552 the documentation, specifically the page docs/details.rst, and some in‐
9553 line docs linked from there.
9554 3.30.2 - 2017-09-24
9556 This release changes Hypothesis's caching approach for functions in hy‐
9557 pothesis.strategies. Previously it would have cached extremely aggres‐
9558 sively and cache entries would never be evicted. Now it adopts a
9559 least-frequently used, least recently used key invalidation policy, and
9560 is somewhat more conservative about which strategies it caches.
9561 Workloads which create strategies based on dynamic values, e.g. by us‐
9563 ing flatmap or composite(), will use significantly less memory.
9564 3.30.1 - 2017-09-22
9566 This release fixes a bug where when running with the use_coverage=True
9567 setting inside an existing running instance of coverage, Hypothesis
9568 would frequently put files that the coveragerc excluded in the report
9569 for the enclosing coverage.
9570 3.30.0 - 2017-09-20
9572 This release introduces two new features:
9573 • When a test fails, either with a health check failure or a falsifying
9575 example, Hypothesis will print out a seed that led to that failure,
9576 if the test is not already running with a fixed seed. You can then
9577 recreate that failure using either the @seed decorator or (if you are
9578 running pytest) with --hypothesis-seed.
9579 • pytest users can specify a seed to use for @given based tests by
9581 passing the --hypothesis-seed command line argument.
9582 This work was funded by Smarkets.
9584 3.29.0 - 2017-09-19
9586 This release makes Hypothesis coverage aware. Hypothesis now runs all
9587 test bodies under coverage, and uses this information to guide its
9588 testing.
9589 The use_coverage setting can be used to disable this behaviour if you
9591 want to test code that is sensitive to coverage being enabled (either
9592 because of performance or interaction with the trace function).
9593 The main benefits of this feature are:
9595 • Hypothesis now observes when examples it discovers cover particular
9597 lines or branches and stores them in the database for later.
9598 • Hypothesis will make some use of this information to guide its explo‐
9600 ration of the search space and improve the examples it finds (this is
9601 currently used only very lightly and will likely improve signifi‐
9602 cantly in future releases).
9603 This also has the following side-effects:
9605 • Hypothesis now has an install time dependency on the coverage pack‐
9607 age.
9608 • Tests that are already running Hypothesis under coverage will likely
9610 get faster.
9611 • Tests that are not running under coverage now run their test bodies
9613 under coverage by default.
9614 This feature is only partially supported under pypy. It is signifi‐
9616 cantly slower than on CPython and is turned off by default as a result,
9617 but it should still work correctly if you want to use it.
9618 3.28.3 - 2017-09-18
9620 This release is an internal change that affects how Hypothesis handles
9621 calculating certain properties of strategies.
9622 The primary effect of this is that it fixes a bug where use of de‐
9624 ferred() could sometimes trigger an internal assertion error. However
9625 the fix for this bug involved some moderately deep changes to how Hy‐
9626 pothesis handles certain constructs so you may notice some additional
9627 knock-on effects.
9628 In particular the way Hypothesis handles drawing data from strategies
9630 that cannot generate any values has changed to bail out sooner than it
9631 previously did. This may speed up certain tests, but it is unlikely to
9632 make much of a difference in practice for tests that were not already
9633 failing with Unsatisfiable.
9634 3.28.2 - 2017-09-18
9636 This is a patch release that fixes a bug in the hypothesis.extra.pandas
9637 documentation where it incorrectly referred to column() instead of col‐
9638 umns().
9639 3.28.1 - 2017-09-16
9641 This is a refactoring release. It moves a number of internal uses of
9642 namedtuple() over to using attrs based classes, and removes a couple of
9643 internal namedtuple classes that were no longer in use.
9644 It should have no user visible impact.
9646 3.28.0 - 2017-09-15
9648 This release adds support for testing pandas via the hypothesis.ex‐
9649 tra.pandas module.
9650 It also adds a dependency on attrs.
9652 This work was funded by Stripe.
9654 3.27.1 - 2017-09-14
9656 This release fixes some formatting and broken cross-references in the
9657 documentation, which includes editing docstrings - and thus a patch re‐
9658 lease.
9659 3.27.0 - 2017-09-13
9661 This release introduces a deadline setting to Hypothesis.
9662 When set this turns slow tests into errors. By default it is unset but
9664 will warn if you exceed 200ms, which will become the default value in a
9665 future release.
9666 This work was funded by Smarkets.
9668 3.26.0 - 2017-09-12
9670 Hypothesis now emits deprecation warnings if you are using the legacy
9671 SQLite example database format, or the tool for merging them. These
9672 were already documented as deprecated, so this doesn't change their
9673 deprecation status, only that we warn about it.
9674 3.25.1 - 2017-09-12
9676 This release fixes a bug with generating numpy datetime and timedelta
9677 types: When inferring the strategy from the dtype, datetime and
9678 timedelta dtypes with sub-second precision would always produce exam‐
9679 ples with one second resolution. Inferring a strategy from a time
9680 dtype will now always produce example with the same precision.
9681 3.25.0 - 2017-09-12
9683 This release changes how Hypothesis shrinks and replays examples to
9684 take into account that it can encounter new bugs while shrinking the
9685 bug it originally found. Previously it would end up replacing the orig‐
9686 inally found bug with the new bug and show you only that one. Now it is
9687 (often) able to recognise when two bugs are distinct and when it finds
9688 more than one will show both.
9689 3.24.2 - 2017-09-11
9691 This release removes the (purely internal and no longer useful) strat‐
9692 egy_test_suite function and the corresponding strategytests module.
9693 3.24.1 - 2017-09-06
9695 This release improves the reduction of examples involving floating
9696 point numbers to produce more human readable examples.
9697 It also has some general purpose changes to the way the minimizer works
9699 internally, which may see some improvement in quality and slow down of
9700 test case reduction in cases that have nothing to do with floating
9701 point numbers.
9702 3.24.0 - 2017-09-05
9704 Hypothesis now emits deprecation warnings if you use some_strategy.ex‐
9705 ample() inside a test function or strategy definition (this was never
9706 intended to be supported, but is sufficiently widespread that it war‐
9707 rants a deprecation path).
9708 3.23.3 - 2017-09-05
9710 This is a bugfix release for decimals() with the places argument.
9711 • No longer fails health checks (issue #725, due to internal filtering)
9713 • Specifying a min_value and max_value without any decimals with places
9715 places between them gives a more useful error message.
9716 • Works for any valid arguments, regardless of the decimal precision
9718 context.
9719 3.23.2 - 2017-09-01
9721 This is a small refactoring release that removes a now-unused parameter
9722 to an internal API. It shouldn't have any user visible effect.
9723 3.23.1 - 2017-09-01
9725 Hypothesis no longer propagates the dynamic scope of settings into
9726 strategy definitions.
9727 This release is a small change to something that was never part of the
9729 public API and you will almost certainly not notice any effect unless
9730 you're doing something surprising, but for example the following code
9731 will now give a different answer in some circumstances:
9732 import hypothesis.strategies as st
9734 from hypothesis import settings
9735 CURRENT_SETTINGS = st.builds(lambda: settings.default)
9737 (We don't actually encourage you writing code like this)
9739 Previously this would have generated the settings that were in effect
9741 at the point of definition of CURRENT_SETTINGS. Now it will generate
9742 the settings that are used for the current test.
9743 It is very unlikely to be significant enough to be visible, but you may
9745 also notice a small performance improvement.
9746 3.23.0 - 2017-08-31
9748 This release adds a unique argument to arrays() which behaves the same
9749 ways as the corresponding one for lists(), requiring all of the ele‐
9750 ments in the generated array to be distinct.
9751 3.22.2 - 2017-08-29
9753 This release fixes an issue where Hypothesis would raise a TypeError
9754 when using the datetime-related strategies if running with PYTHONOPTI‐
9755 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
9756 3.22.1 - 2017-08-28
9758 Hypothesis now transparently handles problems with an internal unicode
9759 cache, including file truncation or read-only filesystems (issue #767).
9760 Thanks to Sam Hames for the patch.
9761 3.22.0 - 2017-08-26
9763 This release provides what should be a substantial performance improve‐
9764 ment to numpy arrays generated using provided numpy support, and adds a
9765 new fill_value argument to arrays() to control this behaviour.
9766 This work was funded by Stripe.
9768 3.21.3 - 2017-08-26
9770 This release fixes some extremely specific circumstances that probably
9771 have never occurred in the wild where users of deferred() might have
9772 seen a python:RuntimeError from too much recursion, usually in cases
9773 where no valid example could have been generated anyway.
9774 3.21.2 - 2017-08-25
9776 This release fixes some minor bugs in argument validation:
9777 • hypothesis.extra.numpy dtype strategies would raise an internal
9779 error instead of an InvalidArgument exception when passed an in‐
9780 valid endianness specification.
9781 • fractions() would raise an internal error instead of an InvalidAr‐
9783 gument if passed float("nan") as one of its bounds.
9784 • The error message for passing float("nan") as a bound to various
9786 strategies has been improved.
9787 • Various bound arguments will now raise InvalidArgument in cases
9789 where they would previously have raised an internal TypeError or
9790 ValueError from the relevant conversion function.
9791 • streaming() would not have emitted a deprecation warning when
9793 called with an invalid argument.
9794 3.21.1 - 2017-08-24
9796 This release fixes a bug where test failures that were the result of an
9797 @example would print an extra stack trace before re-raising the excep‐
9798 tion.
9799 3.21.0 - 2017-08-23
9801 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
9802 sis's deprecation warnings into errors. Similar functionality can be
9803 achieved by using simplefilter('error', HypothesisDeprecationWarning).
9804 3.20.0 - 2017-08-22
9806 This release renames the relevant arguments on the datetimes(),
9807 dates(), times(), and timedeltas() strategies to min_value and
9808 max_value, to make them consistent with the other strategies in the
9809 module.
9810 The old argument names are still supported but will emit a deprecation
9812 warning when used explicitly as keyword arguments. Arguments passed po‐
9813 sitionally will go to the new argument names and are not deprecated.
9814 3.19.3 - 2017-08-22
9816 This release provides a major overhaul to the internals of how Hypothe‐
9817 sis handles shrinking.
9818 This should mostly be visible in terms of getting better examples for
9820 tests which make heavy use of composite(), data() or flatmap where the
9821 data drawn depends a lot on previous choices, especially where size pa‐
9822 rameters are affected. Previously Hypothesis would have struggled to
9823 reliably produce good examples here. Now it should do much better. Per‐
9824 formance should also be better for examples with a non-zero min_size.
9825 You may see slight changes to example generation (e.g. improved example
9827 diversity) as a result of related changes to internals, but they are
9828 unlikely to be significant enough to notice.
9829 3.19.2 - 2017-08-21
9831 This release fixes two bugs in hypothesis.extra.numpy:
9832 • unicode_string_dtypes() didn't work at all due to an incorrect dtype
9834 specifier. Now it does.
9835 • Various impossible conditions would have been accepted but would er‐
9837 ror when they fail to produced any example. Now they raise an ex‐
9838 plicit InvalidArgument error.
9839 3.19.1 - 2017-08-21
9841 This is a bugfix release for issue #739, where bounds for fractions()
9842 or floating-point decimals() were not properly converted to integers
9843 before passing them to the integers strategy. This excluded some val‐
9844 ues that should have been possible, and could trigger internal errors
9845 if the bounds lay between adjacent integers.
9846 You can now bound fractions() with two arbitrarily close fractions.
9848 It is now an explicit error to supply a min_value, max_value, and
9850 max_denominator to fractions() where the value bounds do not include a
9851 fraction with denominator at most max_denominator.
9852 3.19.0 - 2017-08-20
9854 This release adds the from_regex() strategy, which generates strings
9855 that contain a match of a regular expression.
9856 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
9858 then helping to upstream it! (issue #662)
9859 3.18.5 - 2017-08-18
9861 This is a bugfix release for integers(). Previously the strategy would
9862 hit an internal assertion if passed non-integer bounds for min_value
9863 and max_value that had no integers between them. The strategy now
9864 raises InvalidArgument instead.
9865 3.18.4 - 2017-08-18
9867 Release to fix a bug where mocks can be used as test runners under cer‐
9868 tain conditions. Specifically, if a mock is injected into a test via
9869 pytest fixtures or patch decorators, and that mock is the first argu‐
9870 ment in the list, hypothesis will think it represents self and turns
9871 the mock into a test runner. If this happens, the affected test always
9872 passes because the mock is executed instead of the test body. Some‐
9873 times, it will also fail health checks.
9874 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
9876 for this bug fix.
9877 3.18.3 - 2017-08-17
9879 This release should improve the performance of some tests which experi‐
9880 enced a slow down as a result of the 3.13.0 release.
9881 Tests most likely to benefit from this are ones that make extensive use
9883 of min_size parameters, but others may see some improvement as well.
9884 3.18.2 - 2017-08-16
9886 This release fixes a bug introduced in 3.18.0. If the arguments
9887 whitelist_characters and blacklist_characters to characters() contained
9888 overlapping elements, then an InvalidArgument exception would be
9889 raised.
9890 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
9892 3.18.1 - 2017-08-14
9894 This is a bug fix release to fix issue #780, where sets() and similar
9895 would trigger health check errors if their element strategy could only
9896 produce one element (e.g. if it was just()).
9897 3.18.0 - 2017-08-13
9899 This is a feature release:
9900 • characters() now accepts whitelist_characters, particular characters
9902 which will be added to those it produces. (issue #668)
9903 • A bug fix for the internal function _union_interval_lists(), and a
9905 rename to _union_intervals(). It now correctly handles all cases
9906 where intervals overlap, and it always returns the result as a tuple
9907 for tuples.
9908 Thanks to Alex Willmer for these.
9910 3.17.0 - 2017-08-07
9912 This release documents the previously undocumented phases feature, mak‐
9913 ing it part of the public API. It also updates how the example database
9914 is used. Principally:
9915 • A Phases.reuse argument will now correctly control whether examples
9917 from the database are run (it previously did exactly the wrong thing
9918 and controlled whether examples would be saved).
9919 • Hypothesis will no longer try to rerun all previously failing exam‐
9921 ples. Instead it will replay the smallest previously failing example
9922 and a selection of other examples that are likely to trigger any
9923 other bugs that will found. This prevents a previous failure from
9924 dominating your tests unnecessarily.
9925 • As a result of the previous change, Hypothesis will be slower about
9927 clearing out old examples from the database that are no longer fail‐
9928 ing (because it can only clear out ones that it actually runs).
9929 3.16.1 - 2017-08-07
9931 This release makes an implementation change to how Hypothesis handles
9932 certain internal constructs.
9933 The main effect you should see is improvement to the behaviour and per‐
9935 formance of collection types, especially ones with a min_size parame‐
9936 ter. Many cases that would previously fail due to being unable to gen‐
9937 erate enough valid examples will now succeed, and other cases should
9938 run slightly faster.
9939 3.16.0 - 2017-08-04
9941 This release introduces a deprecation of the timeout feature. This re‐
9942 sults in the following changes:
9943 • Creating a settings object with an explicit timeout will emit a dep‐
9945 recation warning.
9946 • If your test stops because it hits the timeout (and has not found a
9948 bug) then it will emit a deprecation warning.
9949 • There is a new value unlimited which you can import from hypothesis.
9951 settings(timeout=unlimited) will not cause a deprecation warning.
9952 • There is a new health check, hung_test, which will trigger after a
9954 test has been running for five minutes if it is not suppressed.
9955 3.15.0 - 2017-08-04
9957 This release deprecates two strategies, choices() and streaming().
9958 Both of these are somewhat confusing to use and are entirely redundant
9960 since the introduction of the data() strategy for interactive drawing
9961 in tests, and their use should be replaced with direct use of data()
9962 instead.
9963 3.14.2 - 2017-08-03
9965 This fixes a bug where Hypothesis would not work correctly on Python
9966 2.7 if you had the python:typing module backport installed.
9967 3.14.1 - 2017-08-02
9969 This raises the maximum depth at which Hypothesis starts cutting off
9970 data generation to a more reasonable value which it is harder to hit by
9971 accident.
9972 This resolves (issue #751), in which some examples which previously
9974 worked would start timing out, but it will also likely improve the data
9975 generation quality for complex data types.
9976 3.14.0 - 2017-07-23
9978 Hypothesis now understands inline type annotations (issue #293):
9979 • If the target of builds() has type annotations, a default strategy
9981 for missing required arguments is selected based on the type.
9982 Type-based strategy selection will only override a default if you
9983 pass hypothesis.infer as a keyword argument.
9984 • If @given wraps a function with type annotations, you can pass infer
9986 as a keyword argument and the appropriate strategy will be substi‐
9987 tuted.
9988 • You can check what strategy will be inferred for a type with the new
9990 from_type() function.
9991 • register_type_strategy() teaches Hypothesis which strategy to infer
9993 for custom or unknown types. You can provide a strategy, or for more
9994 complex cases a function which takes the type and returns a strategy.
9995 3.13.1 - 2017-07-20
9997 This is a bug fix release for issue #514 - Hypothesis would continue
9998 running examples after a SkipTest exception was raised, including
9999 printing a falsifying example. Skip exceptions from the standard
10000 python:unittest module, and pytest, nose, or unittest2 modules now
10001 abort the test immediately without printing output.
10002 3.13.0 - 2017-07-16
10004 This release has two major aspects to it: The first is the introduction
10005 of deferred(), which allows more natural definition of recursive (in‐
10006 cluding mutually recursive) strategies.
10007 The second is a number of engine changes designed to support this sort
10009 of strategy better. These should have a knock-on effect of also improv‐
10010 ing the performance of any existing strategies that currently generate
10011 a lot of data or involve heavy nesting by reducing their typical exam‐
10012 ple size.
10013 3.12.0 - 2017-07-07
10015 This release makes some major internal changes to how Hypothesis repre‐
10016 sents data internally, as a prelude to some major engine changes that
10017 should improve data quality. There are no API changes, but it's a sig‐
10018 nificant enough internal change that a minor version bump seemed war‐
10019 ranted.
10020 User facing impact should be fairly mild, but includes:
10022 • All existing examples in the database will probably be invalidated.
10024 Hypothesis handles this automatically, so you don't need to do any‐
10025 thing, but if you see all your examples disappear that's why.
10026 • Almost all data distributions have changed significantly. Possibly
10028 for the better, possibly for the worse. This may result in new bugs
10029 being found, but it may also result in Hypothesis being unable to
10030 find bugs it previously did.
10031 • Data generation may be somewhat faster if your existing bottleneck
10033 was in draw_bytes (which is often the case for large examples).
10034 • Shrinking will probably be slower, possibly significantly.
10036 If you notice any effects you consider to be a significant regression,
10038 please open an issue about them.
10039 3.11.6 - 2017-06-19
10041 This release involves no functionality changes, but is the first to
10042 ship wheels as well as an sdist.
10043 3.11.5 - 2017-06-18
10045 This release provides a performance improvement to shrinking. For cases
10046 where there is some non-trivial "boundary" value (e.g. the bug happens
10047 for all values greater than some other value), shrinking should now be
10048 substantially faster. Other types of bug will likely see improvements
10049 too.
10050 This may also result in some changes to the quality of the final exam‐
10052 ples - it may sometimes be better, but is more likely to get slightly
10053 worse in some edge cases. If you see any examples where this happens in
10054 practice, please report them.
10055 3.11.4 - 2017-06-17
10057 This is a bugfix release: Hypothesis now prints explicit examples when
10058 running in verbose mode. (issue #313)
10059 3.11.3 - 2017-06-11
10061 This is a bugfix release: Hypothesis no longer emits a warning if you
10062 try to use sampled_from() with python:collections.OrderedDict. (issue
10063 #688)
10064 3.11.2 - 2017-06-10
10066 This is a documentation release. Several outdated snippets have been
10067 updated or removed, and many cross-references are now hyperlinks.
10068 3.11.1 - 2017-05-28
10070 This is a minor ergonomics release. Tracebacks shown by pytest no
10071 longer include Hypothesis internals for test functions decorated with
10072 @given.
10073 3.11.0 - 2017-05-23
10075 This is a feature release, adding datetime-related strategies to the
10076 core strategies.
10077 timezones() allows you to sample pytz timezones from the Olsen data‐
10079 base. Use directly in a recipe for tz-aware datetimes, or compose with
10080 none() to allow a mix of aware and naive output.
10081 The new dates(), times(), datetimes(), and timedeltas() strategies are
10083 all constrained by objects of their type. This means that you can gen‐
10084 erate dates bounded by a single day (i.e. a single date), or datetimes
10085 constrained to the microsecond.
10086 times() and datetimes() take an optional timezones= argument, which de‐
10088 faults to none() for naive times. You can use our extra strategy based
10089 on pytz, or roll your own timezones strategy with dateutil or even the
10090 standard library.
10091 The old dates, times, and datetimes strategies in hypothesis.ex‐
10093 tra.datetimes are deprecated in favor of the new core strategies, which
10094 are more flexible and have no dependencies.
10095 3.10.0 - 2017-05-22
10097 Hypothesis now uses python:inspect.getfullargspec() internally. On
10098 Python 2, there are no visible changes.
10099 On Python 3 @given and @composite now preserve PEP 3107 annotations on
10101 the decorated function. Keyword-only arguments are now either handled
10102 correctly (e.g. @composite), or caught in validation instead of
10103 silently discarded or raising an unrelated error later (e.g. @given).
10104 3.9.1 - 2017-05-22
10106 This is a bugfix release: the default field mapping for a DateTimeField
10107 in the Django extra now respects the USE_TZ setting when choosing a
10108 strategy.
10109 3.9.0 - 2017-05-19
10111 This is feature release, expanding the capabilities of the decimals()
10112 strategy.
10113 • The new (optional) places argument allows you to generate decimals
10115 with a certain number of places (e.g. cents, thousandths, satoshis).
10116 • If allow_infinity is None, setting min_bound no longer excludes posi‐
10118 tive infinity and setting max_value no longer excludes negative in‐
10119 finity.
10120 • All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
10122 True, or if allow_nan is None and min_value or max_value is None.
10123 • min_value and max_value may be given as decimal strings, e.g.
10125 "1.234".
10126 3.8.5 - 2017-05-16
10128 Hypothesis now imports python:sqlite3 when a SQLite database is used,
10129 rather than at module load, improving compatibility with Python imple‐
10130 mentations compiled without SQLite support (such as BSD or Jython).
10131 3.8.4 - 2017-05-16
10133 This is a compatibility bugfix release. sampled_from() no longer
10134 raises a deprecation warning when sampling from an python:enum.Enum, as
10135 all enums have a reliable iteration order.
10136 3.8.3 - 2017-05-09
10138 This release removes a version check for older versions of pytest when
10139 using the Hypothesis pytest plugin. The pytest plugin will now run un‐
10140 conditionally on all versions of pytest. This breaks compatibility with
10141 any version of pytest prior to 2.7.0 (which is more than two years
10142 old).
10143 The primary reason for this change is that the version check was a fre‐
10145 quent source of breakage when pytest change their versioning scheme. If
10146 you are not working on pytest itself and are not running a very old
10147 version of it, this release probably doesn't affect you.
10148 3.8.2 - 2017-04-26
10150 This is a code reorganisation release that moves some internal test
10151 helpers out of the main source tree so as to not have changes to them
10152 trigger releases in future.
10153 3.8.1 - 2017-04-26
10155 This is a documentation release. Almost all code examples are now
10156 doctests checked in CI, eliminating stale examples.
10157 3.8.0 - 2017-04-23
10159 This is a feature release, adding the iterables() strategy, equivalent
10160 to lists(...).map(iter) but with a much more useful repr. You can use
10161 this strategy to check that code doesn't accidentally depend on se‐
10162 quence properties such as indexing support or repeated iteration.
10163 3.7.4 - 2017-04-22
10165 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
10166 ture in the same test could cause the test to fail to fill the argu‐
10167 ments, and throw a TypeError.
10168 3.7.3 - 2017-04-21
10170 This release should include no user visible changes and is purely a
10171 refactoring release. This modularises the behaviour of the core given()
10172 function, breaking it up into smaller and more accessible parts, but
10173 its actual behaviour should remain unchanged.
10174 3.7.2 - 2017-04-21
10176 This reverts an undocumented change in 3.7.1 which broke installation
10177 on debian stable: The specifier for the hypothesis[django] extra_re‐
10178 quires had introduced a wild card, which was not supported on the de‐
10179 fault version of pip.
10180 3.7.1 - 2017-04-21
10182 This is a bug fix and internal improvements release.
10183 • In particular Hypothesis now tracks a tree of where it has already
10185 explored. This allows it to avoid some classes of duplicate exam‐
10186 ples, and significantly improves the performance of shrinking failing
10187 examples by allowing it to skip some shrinks that it can determine
10188 can't possibly work.
10189 • Hypothesis will no longer seed the global random arbitrarily unless
10191 you have asked it to using random_module()
10192 • Shrinking would previously have not worked correctly in some special
10194 cases on Python 2, and would have resulted in suboptimal examples.
10195 3.7.0 - 2017-03-20
10197 This is a feature release.
10198 New features:
10200 • Rule based stateful testing now has an @invariant decorator that
10202 specifies methods that are run after init and after every step, al‐
10203 lowing you to encode properties that should be true at all times.
10204 Thanks to Tom Prince for this feature.
10205 • The decimals() strategy now supports allow_nan and allow_infinity
10207 flags.
10208 • There are significantly more strategies available for numpy, includ‐
10210 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
10211 for this feature.
10212 • When using the data() strategy you can now add a label as an argument
10214 to draw(), which will be printed along with the value when an example
10215 fails. Thanks to Peter Inglesby for this feature.
10216 Bug fixes:
10218 • Bug fix: composite() now preserves functions' docstrings.
10220 • The build is now reproducible and doesn't depend on the path you
10222 build it from. Thanks to Chris Lamb for this feature.
10223 • numpy strategies for the void data type did not work correctly.
10225 Thanks to Zac Hatfield Dodds for this fix.
10226 There have also been a number of performance optimizations:
10228 • The permutations() strategy is now significantly faster to use for
10230 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
10231 • Shrinking of failing test cases should have got significantly faster
10233 in some circumstances where it was previously struggling for a long
10234 time.
10235 • Example generation now involves less indirection, which results in a
10237 small speedup in some cases (small enough that you won't really no‐
10238 tice it except in pathological cases).
10239 3.6.1 - 2016-12-20
10241 This release fixes a dependency problem and makes some small behind the
10242 scenes improvements.
10243 • The fake-factory dependency was renamed to faker. If you were depend‐
10245 ing on it through hypothesis[django] or hypothesis[fake-factory]
10246 without pinning it yourself then it would have failed to install
10247 properly. This release changes it so that hypothesis[fakefactory]
10248 (which can now also be installed as hypothesis[faker]) will install
10249 the renamed faker package instead.
10250 • This release also removed the dependency of hypothesis[django] on hy‐
10252 pothesis[fakefactory] - it was only being used for emails. These now
10253 use a custom strategy that isn't from fakefactory. As a result you
10254 should also see performance improvements of tests which generated
10255 User objects or other things with email fields, as well as better
10256 shrinking of email addresses.
10257 • The distribution of code using nested calls to one_of() or the | op‐
10259 erator for combining strategies has been improved, as branches are
10260 now flattened to give a more uniform distribution.
10261 • Examples using composite() or .flatmap should now shrink better. In
10263 particular this will affect things which work by first generating a
10264 length and then generating that many items, which have historically
10265 not shrunk very well.
10266 3.6.0 - 2016-10-31
10268 This release reverts Hypothesis to its old pretty printing of lambda
10269 functions based on attempting to extract the source code rather than
10270 decompile the bytecode. This is unfortunately slightly inferior in
10271 some cases and may result in you occasionally seeing things like lambda
10272 x: <unknown> in statistics reports and strategy reprs.
10273 This removes the dependencies on uncompyle6, xdis and spark-parser.
10275 The reason for this is that the new functionality was based on un‐
10277 compyle6, which turns out to introduce a hidden GPLed dependency - it
10278 in turn depended on xdis, and although the library was licensed under
10279 the MIT license, it contained some GPL licensed source code and thus
10280 should have been released under the GPL.
10281 My interpretation is that Hypothesis itself was never in violation of
10283 the GPL (because the license it is under, the Mozilla Public License
10284 v2, is fully compatible with being included in a GPL licensed work),
10285 but I have not consulted a lawyer on the subject. Regardless of the an‐
10286 swer to this question, adding a GPLed dependency will likely cause a
10287 lot of users of Hypothesis to inadvertently be in violation of the GPL.
10288 As a result, if you are running Hypothesis 3.5.x you really should up‐
10290 grade to this release immediately.
10291 3.5.3 - 2016-10-05
10293 This is a bug fix release.
10294 Bugs fixed:
10296 • If the same test was running concurrently in two processes and there
10298 were examples already in the test database which no longer failed,
10299 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
10300 Python 2) because an example it was trying to read was deleted before
10301 it was read. (issue #372).
10302 • Drawing from an integers() strategy with both a min_value and a
10304 max_value would reject too many examples needlessly. Now it repeat‐
10305 edly redraws until satisfied. (pull request #366. Thanks to Calen
10306 Pennington for the contribution).
10307 3.5.2 - 2016-09-24
10309 This is a bug fix release.
10310 • The Hypothesis pytest plugin broke pytest support for doctests. Now
10312 it doesn't.
10313 3.5.1 - 2016-09-23
10315 This is a bug fix release.
10316 • Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
10318 bytes and unicode.
10319 • python:unittest.TestCase tests would not have shown up in the new
10321 statistics mode. Now they do.
10322 • Similarly, stateful tests would not have shown up in statistics and
10324 now they do.
10325 • Statistics now print with pytest node IDs (the names you'd get in
10327 pytest verbose mode).
10328 3.5.0 - 2016-09-22
10330 This is a feature release.
10331 • fractions() and decimals() strategies now support min_value and
10333 max_value parameters. Thanks go to Anne Mulhern for the development
10334 of this feature.
10335 • The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
10337 tics parameter that gives detailed statistics about the tests that
10338 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
10339 funding the development of this feature.
10340 • There is a new event() function that can be used to add custom sta‐
10342 tistics.
10343 Additionally there have been some minor bug fixes:
10345 • In some cases Hypothesis should produce fewer duplicate examples
10347 (this will mostly only affect cases with a single parameter).
10348 • pytest command line parameters are now under an option group for Hy‐
10350 pothesis (thanks to David Keijser for fixing this)
10351 • Hypothesis would previously error if you used PEP 3107 function anno‐
10353 tations on your tests under Python 3.4.
10354 • The repr of many strategies using lambdas has been improved to in‐
10356 clude the lambda body (this was previously supported in many but not
10357 all cases).
10358 3.4.2 - 2016-07-13
10360 This is a bug fix release, fixing a number of problems with the set‐
10361 tings system:
10362 • Test functions defined using @given can now be called from other
10364 threads (issue #337)
10365 • Attempting to delete a settings property would previously have
10367 silently done the wrong thing. Now it raises an AttributeError.
10368 • Creating a settings object with a custom database_file parameter was
10370 silently getting ignored and the default was being used instead. Now
10371 it's not.
10372 3.4.1 - 2016-07-07
10374 This is a bug fix release for a single bug:
10375 • On Windows when running two Hypothesis processes in parallel (e.g.
10377 using pytest-xdist) they could race with each other and one would
10378 raise an exception due to the non-atomic nature of file renaming on
10379 Windows and the fact that you can't rename over an existing file.
10380 This is now fixed.
10381 3.4.0 - 2016-05-27
10383 This release is entirely provided by Lucas Wiman:
10384 Strategies constructed by the Django extra will now respect much more
10386 of Django's validations out of the box. Wherever possible,
10387 full_clean() should succeed.
10388 In particular:
10390 • The max_length, blank and choices kwargs are now respected.
10392 • Add support for DecimalField.
10394 • If a field includes validators, the list of validators are used to
10396 filter the field strategy.
10397 3.3.0 - 2016-05-27
10399 This release went wrong and is functionally equivalent to 3.2.0. Ignore
10400 it.
10401 3.2.0 - 2016-05-19
10403 This is a small single-feature release:
10404 • All tests using @given now fix the global random seed. This removes
10406 the health check for that. If a non-zero seed is required for the fi‐
10407 nal falsifying example, it will be reported. Otherwise Hypothesis
10408 will assume randomization was not a significant factor for the test
10409 and be silent on the subject. If you use random_module() this will
10410 continue to work and will always display the seed.
10411 3.1.3 - 2016-05-01
10413 Single bug fix release
10414 • Another charmap problem. In 3.1.2 text() and characters() would break
10416 on systems which had /tmp mounted on a different partition than the
10417 Hypothesis storage directory (usually in home). This fixes that.
10418 3.1.2 - 2016-04-30
10420 Single bug fix release:
10421 • Anything which used a text() or characters() strategy was broken on
10423 Windows and I hadn't updated appveyor to use the new repository loca‐
10424 tion so I didn't notice. This is now fixed and windows support should
10425 work correctly.
10426 3.1.1 - 2016-04-29
10428 Minor bug fix release.
10429 • Fix concurrency issue when running tests that use text() from multi‐
10431 ple processes at once (issue #302, thanks to Alex Chan).
10432 • Improve performance of code using lists() with max_size (thanks to
10434 Cristi Cobzarenco).
10435 • Fix install on Python 2 with ancient versions of pip so that it in‐
10437 stalls the enum34 backport (thanks to Donald Stufft for telling me
10438 how to do this).
10439 • Remove duplicated __all__ exports from hypothesis.strategies (thanks
10441 to Piët Delport).
10442 • Update headers to point to new repository location.
10444 • Allow use of strategies that can't be used in find() (e.g. choices())
10446 in stateful testing.
10447 3.1.0 - 2016-03-06
10449 • Add a nothing() strategy that never successfully generates values.
10450 • sampled_from() and one_of() can both now be called with an empty ar‐
10452 gument list, in which case they also never generate any values.
10453 • one_of() may now be called with a single argument that is a collec‐
10455 tion of strategies as well as as varargs.
10456 • Add a runner() strategy which returns the instance of the current
10458 test object if there is one.
10459 • 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
10461 can be used as such.
10462 • Tests using RuleBasedStateMachine should now shrink significantly
10464 better.
10465 • Hypothesis now uses a pretty-printing library internally, compatible
10467 with IPython's pretty printing protocol (actually using the same
10468 code). This may improve the quality of output in some cases.
10469 • Add a 'phases' setting that allows more fine grained control over
10471 which parts of the process Hypothesis runs
10472 • Add a suppress_health_check setting which allows you to turn off spe‐
10474 cific health checks in a fine grained manner.
10475 • Fix a bug where lists of non fixed size would always draw one more
10477 element than they included. This mostly didn't matter, but if would
10478 cause problems with empty strategies or ones with side effects.
10479 • Add a mechanism to the Django model generator to allow you to explic‐
10481 itly request the default value (thanks to Jeremy Thurgood for this
10482 one).
10483 3.0.5 - 2016-02-25
10485 • Fix a bug where Hypothesis would now error on pytest development ver‐
10486 sions.
10487 3.0.4 - 2016-02-24
10489 • Fix a bug where Hypothesis would error when running on Python 2.7.3
10490 or earlier because it was trying to pass a python:bytearray object to
10491 python:struct.unpack() (which is only supported since 2.7.4).
10492 3.0.3 - 2016-02-23
10494 • Fix version parsing of pytest to work with pytest release candidates
10495 • More general handling of the health check problem where things could
10497 fail because of a cache miss - now one "free" example is generated
10498 before the start of the health check run.
10499 3.0.2 - 2016-02-18
10501 • Under certain circumstances, strategies involving text() buried in‐
10502 side some other strategy (e.g. text().filter(...) or recur‐
10503 sive(text(), ...)) would cause a test to fail its health checks the
10504 first time it ran. This was caused by having to compute some related
10505 data and cache it to disk. On travis or anywhere else where the .hy‐
10506 pothesis directory was recreated this would have caused the tests to
10507 fail their health check on every run. This is now fixed for all the
10508 known cases, although there could be others lurking.
10509 3.0.1 - 2016-02-18
10511 • Fix a case where it was possible to trigger an "Unreachable" asser‐
10512 tion when running certain flaky stateful tests.
10513 • Improve shrinking of large stateful tests by eliminating a case where
10515 it was hard to delete early steps.
10516 • Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
10518 cantly by provide a custom implementation for fixed size blocks that
10519 can bypass a lot of machinery.
10520 • Set default home directory based on the current working directory at
10522 the point Hypothesis is imported, not whenever the function first
10523 happens to be called.
10524 3.0.0 - 2016-02-17
10526 Codename: This really should have been 2.1.
10527 Externally this looks like a very small release. It has one small
10529 breaking change that probably doesn't affect anyone at all (some behav‐
10530 iour that never really worked correctly is now outright forbidden) but
10531 necessitated a major version bump and one visible new feature.
10532 Internally this is a complete rewrite. Almost nothing other than the
10534 public API is the same.
10535 New features:
10537 • Addition of data() strategy which allows you to draw arbitrary data
10539 interactively within the test.
10540 • New "exploded" database format which allows you to more easily check
10542 the example database into a source repository while supporting merg‐
10543 ing.
10544 • Better management of how examples are saved in the database.
10546 • Health checks will now raise as errors when they fail. It was too
10548 easy to have the warnings be swallowed entirely.
10549 New limitations:
10551 • choices() and streaming() strategies may no longer be used with
10553 find(). Neither may data() (this is the change that necessitated a
10554 major version bump).
10555 Feature removal:
10557 • The ForkingTestCase executor has gone away. It may return in some
10559 more working form at a later date.
10560 Performance improvements:
10562 • A new model which allows flatmap, composite strategies and stateful
10564 testing to perform much better. They should also be more reliable.
10565 • Filtering may in some circumstances have improved significantly. This
10567 will help especially in cases where you have lots of values with in‐
10568 dividual filters on them, such as lists(x.filter(...)).
10569 • Modest performance improvements to the general test runner by avoid‐
10571 ing expensive operations
10572 In general your tests should have got faster. If they've instead got
10574 significantly slower, I'm interested in hearing about it.
10575 Data distribution:
10577 The data distribution should have changed significantly. This may un‐
10579 cover bugs the previous version missed. It may also miss bugs the pre‐
10580 vious version could have uncovered. Hypothesis is now producing less
10581 strongly correlated data than it used to, but the correlations are ex‐
10582 tended over more of the structure.
10583 Shrinking:
10585 Shrinking quality should have improved. In particular Hypothesis can
10587 now perform simultaneous shrinking of separate examples within a single
10588 test (previously it was only able to do this for elements of a single
10589 collection). In some cases performance will have improved, in some
10590 cases it will have got worse but generally shouldn't have by much.
10591 Older versions
10593 2.0.0 - 2016-01-10
10594 Codename: A new beginning
10595 This release cleans up all of the legacy that accrued in the course of
10597 Hypothesis 1.0. These are mostly things that were emitting deprecation
10598 warnings in 1.19.0, but there were a few additional changes.
10599 In particular:
10601 • non-strategy values will no longer be converted to strategies when
10603 used in given or find.
10604 • FailedHealthCheck is now an error and not a warning.
10606 • Handling of non-ascii reprs in user types have been simplified by us‐
10608 ing raw strings in more places in Python 2.
10609 • given no longer allows mixing positional and keyword arguments.
10611 • given no longer works with functions with defaults.
10613 • given no longer turns provided arguments into defaults - they will
10615 not appear in the argspec at all.
10616 • the basic() strategy no longer exists.
10618 • the n_ary_tree strategy no longer exists.
10620 • the average_list_length setting no longer exists. Note: If you're us‐
10622 ing using recursive() this will cause you a significant slow down.
10623 You should pass explicit average_size parameters to collections in
10624 recursive calls.
10625 • @rule can no longer be applied to the same method twice.
10627 • Python 2.6 and 3.3 are no longer officially supported, although in
10629 practice they still work fine.
10630 This also includes two non-deprecation changes:
10632 • given's keyword arguments no longer have to be the rightmost argu‐
10634 ments and can appear anywhere in the method signature.
10635 • The max_shrinks setting would sometimes not have been respected.
10637 1.19.0 - 2016-01-09
10639 Codename: IT COMES
10640 This release heralds the beginning of a new and terrible age of Hypoth‐
10642 esis 2.0.
10643 It's primary purpose is some final deprecations prior to said release.
10645 The goal is that if your code emits no warnings under this release then
10646 it will probably run unchanged under Hypothesis 2.0 (there are some
10647 caveats to this: 2.0 will drop support for some Python versions, and if
10648 you're using internal APIs then as usual that may break without warn‐
10649 ing).
10650 It does have two new features:
10652 • New @seed() decorator which allows you to manually seed a test. This
10654 may be harmlessly combined with and overrides the derandomize set‐
10655 ting.
10656 • settings objects may now be used as a decorator to fix those settings
10658 to a particular @given test.
10659 API changes (old usage still works but is deprecated):
10661 • Settings has been renamed to settings (lower casing) in order to make
10663 the decorator usage more natural.
10664 • Functions for the storage directory that were in hypothesis.settings
10666 are now in a new hypothesis.configuration module.
10667 Additional deprecations:
10669 • the average_list_length setting has been deprecated in favour of be‐
10671 ing explicit.
10672 • the basic() strategy has been deprecated as it is impossible to sup‐
10674 port it under a Conjecture based model, which will hopefully be im‐
10675 plemented at some point in the 2.x series.
10676 • the n_ary_tree strategy (which was never actually part of the public
10678 API) has been deprecated.
10679 • Passing settings or random as keyword arguments to given is depre‐
10681 cated (use the new functionality instead)
10682 Bug fixes:
10684 • No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
10686 tion in streaming() values.
10687 • When running in health check mode with non strict, don't print quite
10689 so many errors for an exception in reify.
10690 • When an assumption made in a test or a filter is flaky, tests will
10692 now raise Flaky instead of UnsatisfiedAssumption.
10693 1.18.1 - 2015-12-22
10695 Two behind the scenes changes:
10696 • Hypothesis will no longer write generated code to the file system.
10698 This will improve performance on some systems (e.g. if you're using
10699 PythonAnywhere which is running your code from NFS) and prevent some
10700 annoying interactions with auto-restarting systems.
10701 • Hypothesis will cache the creation of some strategies. This can sig‐
10703 nificantly improve performance for code that uses flatmap or compos‐
10704 ite and thus has to instantiate strategies a lot.
10705 1.18.0 - 2015-12-21
10707 Features:
10708 • Tests and find are now explicitly seeded off the global random mod‐
10710 ule. This means that if you nest one inside the other you will now
10711 get a health check error. It also means that you can control global
10712 randomization by seeding random.
10713 • There is a new random_module() strategy which seeds the global random
10715 module for you and handles things so that you don't get a health
10716 check warning if you use it inside your tests.
10717 • floats() now accepts two new arguments: allow_nan and allow_infinity.
10719 These default to the old behaviour, but when set to False will do
10720 what the names suggest.
10721 Bug fixes:
10723 • Fix a bug where tests that used text() on Python 3.4+ would not actu‐
10725 ally be deterministic even when explicitly seeded or using the deran‐
10726 domize mode, because generation depended on dictionary iteration or‐
10727 der which was affected by hash randomization.
10728 • Fix a bug where with complicated strategies the timing of the initial
10730 health check could affect the seeding of the subsequent test, which
10731 would also render supposedly deterministic tests non-deterministic in
10732 some scenarios.
10733 • In some circumstances flatmap() could get confused by two struc‐
10735 turally similar things it could generate and would produce a flaky
10736 test where the first time it produced an error but the second time it
10737 produced the other value, which was not an error. The same bug was
10738 presumably also possible in composite().
10739 • flatmap() and composite() initial generation should now be moderately
10741 faster. This will be particularly noticeable when you have many val‐
10742 ues drawn from the same strategy in a single run, e.g. constructs
10743 like lists(s.flatmap(f)). Shrinking performance may have suffered,
10744 but this didn't actually produce an interestingly worse result in any
10745 of the standard scenarios tested.
10746 1.17.1 - 2015-12-16
10748 A small bug fix release, which fixes the fact that the 'note' function
10749 could not be used on tests which used the @example decorator to provide
10750 explicit examples.
10751 1.17.0 - 2015-12-15
10753 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
10754 because it contains the following additional change that was not in‐
10755 tended to be in a patch release (it's perfectly stable, but is a
10756 larger change that should have required a minor version bump):
10757 • Hypothesis will now perform a series of "health checks" as part of
10759 running your tests. These detect and warn about some common error
10760 conditions that people often run into which wouldn't necessarily have
10761 caused the test to fail but would cause e.g. degraded performance or
10762 confusing results.
10763 1.16.1 - 2015-12-14
10765 Note: This release has been removed.
10766 A small bugfix release that allows bdists for Hypothesis to be built
10768 under 2.7 - the compat3.py file which had Python 3 syntax wasn't in‐
10769 tended to be loaded under Python 2, but when building a bdist it was.
10770 In particular this would break running setup.py test.
10771 1.16.0 - 2015-12-08
10773 There are no public API changes in this release but it includes a be‐
10774 haviour change that I wasn't comfortable putting in a patch release.
10775 • Functions from hypothesis.strategies will no longer raise InvalidAr‐
10777 gument on bad arguments. Instead the same errors will be raised when
10778 a test using such a strategy is run. This may improve startup time in
10779 some cases, but the main reason for it is so that errors in strate‐
10780 gies won't cause errors in loading, and it can interact correctly
10781 with things like pytest.mark.skipif.
10782 • Errors caused by accidentally invoking the legacy API are now much
10784 less confusing, although still throw NotImplementedError.
10785 • hypothesis.extra.django is 1.9 compatible.
10787 • When tests are run with max_shrinks=0 this will now still rerun the
10789 test on failure and will no longer print "Trying example:" before
10790 each run. Additionally note() will now work correctly when used with
10791 max_shrinks=0.
10792 1.15.0 - 2015-11-24
10794 A release with two new features.
10795 • A 'characters' strategy for more flexible generation of text with
10797 particular character ranges and types, kindly contributed by
10798 Alexander Shorin.
10799 • Add support for preconditions to the rule based stateful testing.
10801 Kindly contributed by Christopher Armstrong
10802 1.14.0 - 2015-11-01
10804 New features:
10805 • Add 'note' function which lets you include additional information in
10807 the final test run's output.
10808 • Add 'choices' strategy which gives you a choice function that emu‐
10810 lates random.choice.
10811 • Add 'uuid' strategy that generates UUIDs'
10813 • Add 'shared' strategy that lets you create a strategy that just gen‐
10815 erates a single shared value for each test run
10816 Bugs:
10818 • Using strategies of the form streaming(x.flatmap(f)) with find or in
10820 stateful testing would have caused InvalidArgument errors when the
10821 resulting values were used (because code that expected to only be
10822 called within a test context would be invoked).
10823 1.13.0 - 2015-10-29
10825 This is quite a small release, but deprecates some public API functions
10826 and removes some internal API functionality so gets a minor version
10827 bump.
10828 • All calls to the 'strategy' function are now deprecated, even ones
10830 which pass just a SearchStrategy instance (which is still a no-op).
10831 • Never documented hypothesis.extra entry_points mechanism has now been
10833 removed ( it was previously how hypothesis.extra packages were loaded
10834 and has been deprecated and unused for some time)
10835 • Some corner cases that could previously have produced an OverflowEr‐
10837 ror when simplifying failing cases using hypothesis.extra.datetimes
10838 (or dates or times) have now been fixed.
10839 • Hypothesis load time for first import has been significantly reduced
10841 - it used to be around 250ms (on my SSD laptop) and now is around
10842 100-150ms. This almost never matters but was slightly annoying when
10843 using it in the console.
10844 • hypothesis.strategies.randoms was previously missing from __all__.
10846 1.12.0 - 2015-10-18
10848 • Significantly improved performance of creating strategies using the
10849 functions from the hypothesis.strategies module by deferring the cal‐
10850 culation of their repr until it was needed. This is unlikely to have
10851 been an performance issue for you unless you were using flatmap, com‐
10852 posite or stateful testing, but for some cases it could be quite a
10853 significant impact.
10854 • A number of cases where the repr of strategies build from lambdas is
10856 improved
10857 • Add dates() and times() strategies to hypothesis.extra.datetimes
10859 • Add new 'profiles' mechanism to the settings system
10861 • Deprecates mutability of Settings, both the Settings.default top
10863 level property and individual settings.
10864 • A Settings object may now be directly initialized from a parent Set‐
10866 tings.
10867 • @given should now give a better error message if you attempt to use
10869 it with a function that uses destructuring arguments (it still won't
10870 work, but it will error more clearly),
10871 • A number of spelling corrections in error messages
10873 • pytest should no longer display the intermediate modules Hypothesis
10875 generates when running in verbose mode
10876 • Hypothesis should now correctly handle printing objects with
10878 non-ascii reprs on python 3 when running in a locale that cannot han‐
10879 dle ascii printing to stdout.
10880 • Add a unique=True argument to lists(). This is equivalent to
10882 unique_by=lambda x: x, but offers a more convenient syntax.
10883 1.11.4 - 2015-09-27
10885 • Hide modifications Hypothesis needs to make to sys.path by undoing
10886 them after we've imported the relevant modules. This is a workaround
10887 for issues cryptography experienced on windows.
10888 • Slightly improved performance of drawing from sampled_from on large
10890 lists of alternatives.
10891 • Significantly improved performance of drawing from one_of or strate‐
10893 gies using | (note this includes a lot of strategies internally -
10894 floats() and integers() both fall into this category). There turned
10895 out to be a massive performance regression introduced in 1.10.0 af‐
10896 fecting these which probably would have made tests using Hypothesis
10897 significantly slower than they should have been.
10898 1.11.3 - 2015-09-23
10900 • Better argument validation for datetimes() strategy - previously set‐
10901 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
10902 would not have raised an InvalidArgument error and instead would have
10903 behaved confusingly.
10904 • Compatibility with being run on pytest < 2.7 (achieved by disabling
10906 the plugin).
10907 1.11.2 - 2015-09-23
10909 Bug fixes:
10910 • Settings(database=my_db) would not be correctly inherited when used
10912 as a default setting, so that newly created settings would use the
10913 database_file setting and create an SQLite example database.
10914 • Settings.default.database = my_db would previously have raised an er‐
10916 ror and now works.
10917 • Timeout could sometimes be significantly exceeded if during simplifi‐
10919 cation there were a lot of examples tried that didn't trigger the
10920 bug.
10921 • When loading a heavily simplified example using a basic() strategy
10923 from the database this could cause Python to trigger a recursion er‐
10924 ror.
10925 • Remove use of deprecated API in pytest plugin so as to not emit warn‐
10927 ing
10928 Misc:
10930 • hypothesis-pytest is now part of hypothesis core. This should have no
10932 externally visible consequences, but you should update your dependen‐
10933 cies to remove hypothesis-pytest and depend on only Hypothesis.
10934 • Better repr for hypothesis.extra.datetimes() strategies.
10936 • Add .close() method to abstract base class for Backend (it was al‐
10938 ready present in the main implementation).
10939 1.11.1 - 2015-09-16
10941 Bug fixes:
10942 • When running Hypothesis tests in parallel (e.g. using pytest-xdist)
10944 there was a race condition caused by code generation.
10945 • Example databases are now cached per thread so as to not use sqlite
10947 connections from multiple threads. This should make Hypothesis now
10948 entirely thread safe.
10949 • floats() with only min_value or max_value set would have had a very
10951 bad distribution.
10952 • Running on 3.5, Hypothesis would have emitted deprecation warnings
10954 because of use of inspect.getargspec
10955 1.11.0 - 2015-08-31
10957 • text() with a non-string alphabet would have used the repr() of the
10958 the alphabet instead of its contexts. This is obviously silly. It now
10959 works with any sequence of things convertible to unicode strings.
10960 • @given will now work on methods whose definitions contains no ex‐
10962 plicit positional arguments, only varargs (issue #118). This may
10963 have some knock on effects because it means that @given no longer
10964 changes the argspec of functions other than by adding defaults.
10965 • Introduction of new @composite feature for more natural definition of
10967 strategies you'd previously have used flatmap for.
10968 1.10.6 - 2015-08-26
10970 Fix support for fixtures on Django 1.7.
10971 1.10.4 - 2015-08-21
10973 Tiny bug fix release:
10974 • If the database_file setting is set to None, this would have resulted
10976 in an error when running tests. Now it does the same as setting data‐
10977 base to None.
10978 1.10.3 - 2015-08-19
10980 Another small bug fix release.
10981 • lists(elements, unique_by=some_function, min_size=n) would have
10983 raised a ValidationError if n > Settings.default.average_list_length
10984 because it would have wanted to use an average list length shorter
10985 than the minimum size of the list, which is impossible. Now it in‐
10986 stead defaults to twice the minimum size in these circumstances.
10987 • basic() strategy would have only ever produced at most ten distinct
10989 values per run of the test (which is bad if you e.g. have it inside a
10990 list). This was obviously silly. It will now produce a much better
10991 distribution of data, both duplicated and non duplicated.
10992 1.10.2 - 2015-08-19
10994 This is a small bug fix release:
10995 • star imports from hypothesis should now work correctly.
10997 • example quality for examples using flatmap will be better, as the way
10999 it had previously been implemented was causing problems where Hypoth‐
11000 esis was erroneously labelling some examples as being duplicates.
11001 1.10.0 - 2015-08-04
11003 This is just a bugfix and performance release, but it changes some
11004 semi-public APIs, hence the minor version bump.
11005 • Significant performance improvements for strategies which are
11007 one_of() many branches. In particular this included recursive()
11008 strategies. This should take the case where you use one recursive()
11009 strategy as the base strategy of another from unusably slow (tens of
11010 seconds per generated example) to reasonably fast.
11011 • Better handling of just() and sampled_from() for values which have an
11013 incorrect __repr__ implementation that returns non-ASCII unicode on
11014 Python 2.
11015 • Better performance for flatmap from changing the internal morpher API
11017 to be significantly less general purpose.
11018 • Introduce a new semi-public BuildContext/cleanup API. This allows
11020 strategies to register cleanup activities that should run once the
11021 example is complete. Note that this will interact somewhat weirdly
11022 with find.
11023 • Better simplification behaviour for streaming strategies.
11025 • Don't error on lambdas which use destructuring arguments in Python 2.
11027 • Add some better reprs for a few strategies that were missing good
11029 ones.
11030 • The Random instances provided by randoms() are now copyable.
11032 • Slightly more debugging information about simplify when using a debug
11034 verbosity level.
11035 • Support using given for functions with varargs, but not passing argu‐
11037 ments to it as positional.
11038 1.9.0 - 2015-07-27
11040 Codename: The great bundling.
11041 This release contains two fairly major changes.
11043 The first is the deprecation of the hypothesis-extra mechanism. From
11045 now on all the packages that were previously bundled under it other
11046 than hypothesis-pytest (which is a different beast and will remain sep‐
11047 arate). The functionality remains unchanged and you can still import
11048 them from exactly the same location, they just are no longer separate
11049 packages.
11050 The second is that this introduces a new way of building strategies
11052 which lets you build up strategies recursively from other strategies.
11053 It also contains the minor change that calling .example() on a strategy
11055 object will give you examples that are more representative of the ac‐
11056 tual data you'll get. There used to be some logic in there to make the
11057 examples artificially simple but this proved to be a bad idea.
11058 1.8.5 - 2015-07-24
11060 This contains no functionality changes but fixes a mistake made with
11061 building the previous package that would have broken installation on
11062 Windows.
11063 1.8.4 - 2015-07-20
11065 Bugs fixed:
11066 • When a call to floats() had endpoints which were not floats but
11068 merely convertible to one (e.g. integers), these would be included in
11069 the generated data which would cause it to generate non-floats.
11070 • Splitting lambdas used in the definition of flatmap, map or filter
11072 over multiple lines would break the repr, which would in turn break
11073 their usage.
11074 1.8.3 - 2015-07-20
11076 "Falsifying example" would not have been printed when the failure came
11077 from an explicit example.
11078 1.8.2 - 2015-07-18
11080 Another small bugfix release:
11081 • When using ForkingTestCase you would usually not get the falsifying
11083 example printed if the process exited abnormally (e.g. due to
11084 os._exit).
11085 • Improvements to the distribution of characters when using text() with
11087 a default alphabet. In particular produces a better distribution of
11088 ascii and whitespace in the alphabet.
11089 1.8.1 - 2015-07-17
11091 This is a small release that contains a workaround for people who have
11092 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
11093 for Hypothesis per se because that's not a thing that is actually sup‐
11094 posed to work, but Hypothesis leans more heavily on repr than is typi‐
11095 cal so it's worth having a workaround for.
11096 1.8.0 - 2015-07-16
11098 New features:
11099 • Much more sensible reprs for strategies, especially ones that come
11101 from hypothesis.strategies. These should now have as reprs python
11102 code that would produce the same strategy.
11103 • lists() accepts a unique_by argument which forces the generated lists
11105 to be only contain elements unique according to some function key
11106 (which must return a hashable value).
11107 • Better error messages from flaky tests to help you debug things.
11109 Mostly invisible implementation details that may result in finding new
11111 bugs in your code:
11112 • Sets and dictionary generation should now produce a better range of
11114 results.
11115 • floats with bounds now focus more on 'critical values', trying to
11117 produce values at edge cases.
11118 • flatmap should now have better simplification for complicated cases,
11120 as well as generally being (I hope) more reliable.
11121 Bug fixes:
11123 • You could not previously use assume() if you were using the forking
11125 executor.
11126 1.7.2 - 2015-07-10
11128 This is purely a bug fix release:
11129 • When using floats() with stale data in the database you could some‐
11131 times get values in your tests that did not respect min_value or
11132 max_value.
11133 • When getting a Flaky error from an unreliable test it would have in‐
11135 correctly displayed the example that caused it.
11136 • 2.6 dependency on backports was incorrectly specified. This would
11138 only have caused you problems if you were building a universal wheel
11139 from Hypothesis, which is not how Hypothesis ships, so unless you're
11140 explicitly building wheels for your dependencies and support Python
11141 2.6 plus a later version of Python this probably would never have af‐
11142 fected you.
11143 • If you use flatmap in a way that the strategy on the right hand side
11145 depends sensitively on the left hand side you may have occasionally
11146 seen Flaky errors caused by producing unreliable examples when mini‐
11147 mizing a bug. This use case may still be somewhat fraught to be hon‐
11148 est. This code is due a major rearchitecture for 1.8, but in the
11149 meantime this release fixes the only source of this error that I'm
11150 aware of.
11151 1.7.1 - 2015-06-29
11153 Codename: There is no 1.7.0.
11154 A slight technical hitch with a premature upload means there's was a
11156 yanked 1.7.0 release. Oops.
11157 The major feature of this release is Python 2.6 support. Thanks to Jeff
11159 Meadows for doing most of the work there.
11160 Other minor features
11162 • strategies now has a permutations() function which returns a strategy
11164 yielding permutations of values from a given collection.
11165 • if you have a flaky test it will print the exception that it last saw
11167 before failing with Flaky, even if you do not have verbose reporting
11168 on.
11169 • Slightly experimental git merge script available as "python -m hy‐
11171 pothesis.tools.mergedbs". Instructions on how to use it in the doc‐
11172 string of that file.
11173 Bug fixes:
11175 • Better performance from use of filter. In particular tests which in‐
11177 volve large numbers of heavily filtered strategies should perform a
11178 lot better.
11179 • floats() with a negative min_value would not have worked correctly
11181 (worryingly, it would have just silently failed to run any examples).
11182 This is now fixed.
11183 • tests using sampled_from would error if the number of sampled ele‐
11185 ments was smaller than min_satisfying_examples.
11186 1.6.2 - 2015-06-08
11188 This is just a few small bug fixes:
11189 • Size bounds were not validated for values for a binary() strategy
11191 when reading examples from the database.
11192 • sampled_from is now in __all__ in hypothesis.strategies
11194 • floats no longer consider negative integers to be simpler than posi‐
11196 tive non-integers
11197 • Small floating point intervals now correctly count members, so if you
11199 have a floating point interval so narrow there are only a handful of
11200 values in it, this will no longer cause an error when Hypothesis runs
11201 out of values.
11202 1.6.1 - 2015-05-21
11204 This is a small patch release that fixes a bug where 1.6.0 broke the
11205 use of flatmap with the deprecated API and assumed the passed in func‐
11206 tion returned a SearchStrategy instance rather than converting it to a
11207 strategy.
11208 1.6.0 - 2015-05-21
11210 This is a smallish release designed to fix a number of bugs and smooth
11211 out some weird behaviours.
11212 • Fix a critical bug in flatmap where it would reuse old strategies. If
11214 all your flatmap code was pure you're fine. If it's not, I'm sur‐
11215 prised it's working at all. In particular if you want to use flatmap
11216 with django models, you desperately need to upgrade to this version.
11217 • flatmap simplification performance should now be better in some cases
11219 where it previously had to redo work.
11220 • Fix for a bug where invalid unicode data with surrogates could be
11222 generated during simplification (it was already filtered out during
11223 actual generation).
11224 • The Hypothesis database is now keyed off the name of the test instead
11226 of the type of data. This makes much more sense now with the new
11227 strategies API and is generally more robust. This means you will lose
11228 old examples on upgrade.
11229 • The database will now not delete values which fail to deserialize
11231 correctly, just skip them. This is to handle cases where multiple in‐
11232 compatible strategies share the same key.
11233 • find now also saves and loads values from the database, keyed off a
11235 hash of the function you're finding from.
11236 • Stateful tests now serialize and load values from the database. They
11238 should have before, really. This was a bug.
11239 • Passing a different verbosity level into a test would not have worked
11241 entirely correctly, leaving off some messages. This is now fixed.
11242 • Fix a bug where derandomized tests with unicode characters in the
11244 function body would error on Python 2.7.
11245 1.5.0 - 2015-05-14
11247 Codename: Strategic withdrawal.
11248 The purpose of this release is a radical simplification of the API for
11250 building strategies. Instead of the old approach of @strategy.extend
11251 and things that get converted to strategies, you just build strategies
11252 directly.
11253 The old method of defining strategies will still work until Hypothesis
11255 2.0, because it's a major breaking change, but will now emit depreca‐
11256 tion warnings.
11257 The new API is also a lot more powerful as the functions for defining
11259 strategies give you a lot of dials to turn. See the updated data sec‐
11260 tion for details.
11261 Other changes:
11263 • Mixing keyword and positional arguments in a call to @given is
11265 deprecated as well.
11266 • There is a new setting called 'strict'. When set to True, Hypothe‐
11268 sis will raise warnings instead of merely printing them. Turning
11269 it on by default is inadvisable because it means that Hypothesis
11270 minor releases can break your code, but it may be useful for mak‐
11271 ing sure you catch all uses of deprecated APIs.
11272 • max_examples in settings is now interpreted as meaning the maximum
11274 number of unique (ish) examples satisfying assumptions. A new set‐
11275 ting max_iterations which defaults to a larger value has the old
11276 interpretation.
11277 • Example generation should be significantly faster due to a new
11279 faster parameter selection algorithm. This will mostly show up for
11280 simple data types - for complex ones the parameter selection is
11281 almost certainly dominated.
11282 • Simplification has some new heuristics that will tend to cut down
11284 on cases where it could previously take a very long time.
11285 • timeout would previously not have been respected in cases where
11287 there were a lot of duplicate examples. You probably wouldn't have
11288 previously noticed this because max_examples counted duplicates,
11289 so this was very hard to hit in a way that mattered.
11290 • A number of internal simplifications to the SearchStrategy API.
11292 • You can now access the current Hypothesis version as hypothe‐
11294 sis.__version__.
11295 • A top level function is provided for running the stateful tests
11297 without the TestCase infrastructure.
11298 1.4.0 - 2015-05-04
11300 Codename: What a state.
11301 The big feature of this release is the new and slightly experimental
11303 stateful testing API. You can read more about that in the appropriate
11304 section.
11305 Two minor features the were driven out in the course of developing
11307 this:
11308 • You can now set settings.max_shrinks to limit the number of times Hy‐
11310 pothesis will try to shrink arguments to your test. If this is set to
11311 <= 0 then Hypothesis will not rerun your test and will just raise the
11312 failure directly. Note that due to technical limitations if
11313 max_shrinks is <= 0 then Hypothesis will print every example it calls
11314 your test with rather than just the failing one. Note also that I
11315 don't consider settings max_shrinks to zero a sensible way to run
11316 your tests and it should really be considered a debug feature.
11317 • There is a new debug level of verbosity which is even more verbose
11319 than verbose. You probably don't want this.
11320 Breakage of semi-public SearchStrategy API:
11322 • It is now a required invariant of SearchStrategy that if u simplifies
11324 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
11325 plifying should not increase the complexity even though it is not re‐
11326 quired to decrease it. Enforcing this invariant lead to finding some
11327 bugs where simplifying of integers, floats and sets was suboptimal.
11328 • Integers in basic data are now required to fit into 64 bits. As a re‐
11330 sult python integer types are now serialized as strings, and some
11331 types have stopped using quite so needlessly large random seeds.
11332 Hypothesis Stateful testing was then turned upon Hypothesis itself,
11334 which lead to an amazing number of minor bugs being found in Hypothesis
11335 itself.
11336 Bugs fixed (most but not all from the result of stateful testing) in‐
11338 clude:
11339 • Serialization of streaming examples was flaky in a way that you would
11341 probably never notice: If you generate a template, simplify it, seri‐
11342 alize it, deserialize it, serialize it again and then deserialize it
11343 you would get the original stream instead of the simplified one.
11344 • If you reduced max_examples below the number of examples already
11346 saved in the database, you would have got a ValueError. Additionally,
11347 if you had more than max_examples in the database all of them would
11348 have been considered.
11349 • @given will no longer count duplicate examples (which it never called
11351 your function with) towards max_examples. This may result in your
11352 tests running slower, but that's probably just because they're trying
11353 more examples.
11354 • General improvements to example search which should result in better
11356 performance and higher quality examples. In particular parameters
11357 which have a history of producing useless results will be more ag‐
11358 gressively culled. This is useful both because it decreases the
11359 chance of useless examples and also because it's much faster to not
11360 check parameters which we were unlikely to ever pick!
11361 • integers_from and lists of types with only one value (e.g. [None])
11363 would previously have had a very high duplication rate so you were
11364 probably only getting a handful of examples. They now have a much
11365 lower duplication rate, as well as the improvements to search making
11366 this less of a problem in the first place.
11367 • You would sometimes see simplification taking significantly longer
11369 than your defined timeout. This would happen because timeout was only
11370 being checked after each successful simplification, so if Hypothesis
11371 was spending a lot of time unsuccessfully simplifying things it
11372 wouldn't stop in time. The timeout is now applied for unsuccessful
11373 simplifications too.
11374 • In Python 2.7, integers_from strategies would have failed during sim‐
11376 plification with an OverflowError if their starting point was at or
11377 near to the maximum size of a 64-bit integer.
11378 • flatmap and map would have failed if called with a function without a
11380 __name__ attribute.
11381 • If max_examples was less than min_satisfying_examples this would al‐
11383 ways error. Now min_satisfying_examples is capped to max_examples.
11384 Note that if you have assumptions to satisfy here this will still
11385 cause an error.
11386 Some minor quality improvements:
11388 • Lists of streams, flatmapped strategies and basic strategies should
11390 now now have slightly better simplification.
11391 1.3.0 - 2015-05-22
11393 New features:
11394 • New verbosity level API for printing intermediate results and excep‐
11396 tions.
11397 • New specifier for strings generated from a specified alphabet.
11399 • Better error messages for tests that are failing because of a lack of
11401 enough examples.
11402 Bug fixes:
11404 • Fix error where use of ForkingTestCase would sometimes result in too
11406 many open files.
11407 • Fix error where saving a failing example that used flatmap could er‐
11409 ror.
11410 • Implement simplification for sampled_from, which apparently never
11412 supported it previously. Oops.
11413 General improvements:
11415 • Better range of examples when using one_of or sampled_from.
11417 • Fix some pathological performance issues when simplifying lists of
11419 complex values.
11420 • Fix some pathological performance issues when simplifying examples
11422 that require unicode strings with high codepoints.
11423 • Random will now simplify to more readable examples.
11425 1.2.1 - 2015-04-16
11427 A small patch release for a bug in the new executors feature. Tests
11428 which require doing something to their result in order to fail would
11429 have instead reported as flaky.
11430 1.2.0 - 2015-04-15
11432 Codename: Finders keepers.
11433 A bunch of new features and improvements.
11435 • Provide a mechanism for customizing how your tests are executed.
11437 • Provide a test runner that forks before running each example. This
11439 allows better support for testing native code which might trigger a
11440 segfault or a C level assertion failure.
11441 • Support for using Hypothesis to find examples directly rather than as
11443 just as a test runner.
11444 • New streaming type which lets you generate infinite lazily loaded
11446 streams of data - perfect for if you need a number of examples but
11447 don't know how many.
11448 • Better support for large integer ranges. You can now use inte‐
11450 gers_in_range with ranges of basically any size. Previously large
11451 ranges would have eaten up all your memory and taken forever.
11452 • Integers produce a wider range of data than before - previously they
11454 would only rarely produce integers which didn't fit into a machine
11455 word. Now it's much more common. This percolates to other numeric
11456 types which build on integers.
11457 • Better validation of arguments to @given. Some situations that would
11459 previously have caused silently wrong behaviour will now raise an er‐
11460 ror.
11461 • Include +/- sys.float_info.max in the set of floating point edge
11463 cases that Hypothesis specifically tries.
11464 • Fix some bugs in floating point ranges which happen when given +/-
11466 sys.float_info.max as one of the endpoints... (really any two floats
11467 that are sufficiently far apart so that x, y are finite but y - x is
11468 infinite). This would have resulted in generating infinite values
11469 instead of ones inside the range.
11470 1.1.1 - 2015-04-07
11472 Codename: Nothing to see here
11473 This is just a patch release put out because it fixed some internal
11475 bugs that would block the Django integration release but did not actu‐
11476 ally affect anything anyone could previously have been using. It also
11477 contained a minor quality fix for floats that I'd happened to have fin‐
11478 ished in time.
11479 • Fix some internal bugs with object lifecycle management that were im‐
11481 possible to hit with the previously released versions but broke hy‐
11482 pothesis-django.
11483 • Bias floating point numbers somewhat less aggressively towards very
11485 small numbers
11486 1.1.0 - 2015-04-06
11488 Codename: No-one mention the M word.
11489 • Unicode strings are more strongly biased towards ascii characters.
11491 Previously they would generate all over the space. This is mostly so
11492 that people who try to shape their unicode strings with assume() have
11493 less of a bad time.
11494 • A number of fixes to data deserialization code that could theoreti‐
11496 cally have caused mysterious bugs when using an old version of a Hy‐
11497 pothesis example database with a newer version. To the best of my
11498 knowledge a change that could have triggered this bug has never actu‐
11499 ally been seen in the wild. Certainly no-one ever reported a bug of
11500 this nature.
11501 • Out of the box support for Decimal and Fraction.
11503 • new dictionary specifier for dictionaries with variable keys.
11505 • Significantly faster and higher quality simplification, especially
11507 for collections of data.
11508 • New filter() and flatmap() methods on Strategy for better ways of
11510 building strategies out of other strategies.
11511 • New BasicStrategy class which allows you to define your own strate‐
11513 gies from scratch without needing an existing matching strategy or
11514 being exposed to the full horror or non-public nature of the Search‐
11515 Strategy interface.
11516 1.0.0 - 2015-03-27
11518 Codename: Blast-off!
11519 There are no code changes in this release. This is precisely the 0.9.2
11521 release with some updated documentation.
11522 0.9.2 - 2015-03-26
11524 Codename: T-1 days.
11525 • floats_in_range would not actually have produced floats_in_range un‐
11527 less that range happened to be (0, 1). Fix this.
11528 0.9.1 - 2015-03-25
11530 Codename: T-2 days.
11531 • Fix a bug where if you defined a strategy using map on a lambda then
11533 the results would not be saved in the database.
11534 • Significant performance improvements when simplifying examples using
11536 lists, strings or bounded integer ranges.
11537 0.9.0 - 2015-03-23
11539 Codename: The final countdown
11540 This release could also be called 1.0-RC1.
11542 It contains a teeny tiny bugfix, but the real point of this release is
11544 to declare feature freeze. There will be zero functionality changes be‐
11545 tween 0.9.0 and 1.0 unless something goes really really wrong. No new
11546 features will be added, no breaking API changes will occur, etc. This
11547 is the final shakedown before I declare Hypothesis stable and ready to
11548 use and throw a party to celebrate.
11549 Bug bounty for any bugs found between now and 1.0: I will buy you a
11551 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
11552 we ever find ourselves in the same city at the same time.
11553 The one tiny bugfix:
11555 • Under pypy, databases would fail to close correctly when garbage col‐
11557 lected, leading to a memory leak and a confusing error message if you
11558 were repeatedly creating databases and not closing them. It is very
11559 unlikely you were doing this and the chances of you ever having no‐
11560 ticed this bug are very low.
11561 0.7.2 - 2015-03-22
11563 Codename: Hygienic macros or bust
11564 • You can now name an argument to @given 'f' and it won't break (issue
11566 #38)
11567 • strategy_test_suite is now named strategy_test_suite as the documen‐
11569 tation claims and not in fact strategy_test_suitee
11570 • Settings objects can now be used as a context manager to temporarily
11572 override the default values inside their context.
11573 0.7.1 - 2015-03-21
11575 Codename: Point releases go faster
11576 • Better string generation by parametrizing by a limited alphabet
11578 • Faster string simplification - previously if simplifying a string
11580 with high range unicode characters it would try every unicode charac‐
11581 ter smaller than that. This was pretty pointless. Now it stops after
11582 it's a short range (it can still reach smaller ones through recursive
11583 calls because of other simplifying operations).
11584 • Faster list simplification by first trying a binary chop down the
11586 middle
11587 • Simultaneous simplification of identical elements in a list. So if a
11589 bug only triggers when you have duplicates but you drew e.g. [-17,
11590 -17], this will now simplify to [0, 0].
11591 0.7.0, - 2015-03-20
11593 Codename: Starting to look suspiciously real
11594 This is probably the last minor release prior to 1.0. It consists of
11596 stability improvements, a few usability things designed to make Hypoth‐
11597 esis easier to try out, and filing off some final rough edges from the
11598 API.
11599 • Significant speed and memory usage improvements
11601 • Add an example() method to strategy objects to give an example of the
11603 sort of data that the strategy generates.
11604 • Remove .descriptor attribute of strategies
11606 • Rename descriptor_test_suite to strategy_test_suite
11608 • Rename the few remaining uses of descriptor to specifier (descriptor
11610 already has a defined meaning in Python)
11611 0.6.0 - 2015-03-13
11613 Codename: I'm sorry, were you using that API?
11614 This is primarily a "simplify all the weird bits of the API" release.
11616 As a result there are a lot of breaking changes. If you just use @given
11617 with core types then you're probably fine.
11618 In particular:
11620 • Stateful testing has been removed from the API
11622 • The way the database is used has been rendered less useful (sorry).
11624 The feature for reassembling values saved from other tests doesn't
11625 currently work. This will probably be brought back in post 1.0.
11626 • SpecificationMapper is no longer a thing. Instead there is an
11628 ExtMethod called strategy which you extend to specify how to convert
11629 other types to strategies.
11630 • Settings are now extensible so you can add your own for configuring a
11632 strategy
11633 • MappedSearchStrategy no longer needs an unpack method
11635 • Basically all the SearchStrategy internals have changed massively. If
11637 you implemented SearchStrategy directly rather than using Mapped‐
11638 SearchStrategy talk to me about fixing it.
11639 • Change to the way extra packages work. You now specify the package.
11641 This must have a load() method. Additionally any modules in the pack‐
11642 age will be loaded in under hypothesis.extra
11643 Bug fixes:
11645 • Fix for a bug where calling falsify on a lambda with a non-ascii
11647 character in its body would error.
11648 Hypothesis Extra:
11650 hypothesis-fakefactory: An extension for using faker data in hypothe‐
11652 sis. Depends
11653 on fake-factory.
11654 0.5.0 - 2015-02-10
11656 Codename: Read all about it.
11657 Core hypothesis:
11659 • Add support back in for pypy and python 3.2
11661 • @given functions can now be invoked with some arguments explicitly
11663 provided. If all arguments that hypothesis would have provided are
11664 passed in then no falsification is run.
11665 • Related to the above, this means that you can now use pytest fixtures
11667 and mark.parametrize with Hypothesis without either interfering with
11668 the other.
11669 • Breaking change: @given no longer works for functions with varargs
11671 (varkwargs are fine). This might be added back in at a later date.
11672 • Windows is now fully supported. A limited version (just the tests
11674 with none of the extras) of the test suite is run on windows with
11675 each commit so it is now a first class citizen of the Hypothesis
11676 world.
11677 • Fix a bug for fuzzy equality of equal complex numbers with different
11679 reprs (this can happen when one coordinate is zero). This shouldn't
11680 affect users - that feature isn't used anywhere public facing.
11681 • Fix generation of floats on windows and 32-bit builds of python. I
11683 was using some struct.pack logic that only worked on certain word
11684 sizes.
11685 • When a test times out and hasn't produced enough examples this now
11687 raises a Timeout subclass of Unfalsifiable.
11688 • Small search spaces are better supported. Previously something like a
11690 @given(bool, bool) would have failed because it couldn't find enough
11691 examples. Hypothesis is now aware of the fact that these are small
11692 search spaces and will not error in this case.
11693 • Improvements to parameter search in the case of hard to satisfy as‐
11695 sume. Hypothesis will now spend less time exploring parameters that
11696 are unlikely to provide anything useful.
11697 • Increase chance of generating "nasty" floats
11699 • Fix a bug that would have caused unicode warnings if you had a sam‐
11701 pled_from that was mixing unicode and byte strings.
11702 • Added a standard test suite that you can use to validate a custom
11704 strategy you've defined is working correctly.
11705 Hypothesis extra:
11707 First off, introducing Hypothesis extra packages!
11709 These are packages that are separated out from core Hypothesis because
11711 they have one or more dependencies. Every hypothesis-extra package is
11712 pinned to a specific point release of Hypothesis and will have some
11713 version requirements on its dependency. They use entry_points so you
11714 will usually not need to explicitly import them, just have them in‐
11715 stalled on the path.
11716 This release introduces two of them:
11718 hypothesis-datetime:
11720 Does what it says on the tin: Generates datetimes for Hypothesis. Just
11722 install the package and datetime support will start working.
11723 Depends on pytz for timezone support
11725 hypothesis-pytest:
11727 A very rudimentary pytest plugin. All it does right now is hook the
11729 display of falsifying examples into pytest reporting.
11730 Depends on pytest.
11732 0.4.3 - 2015-02-05
11734 Codename: TIL narrow Python builds are a thing
11735 This just fixes the one bug.
11737 • Apparently there is such a thing as a "narrow python build" and OS X
11739 ships with these by default for python 2.7. These are builds where
11740 you only have two bytes worth of unicode. As a result, generating
11741 unicode was completely broken on OS X. Fix this by only generating
11742 unicode codepoints in the range supported by the system.
11743 0.4.2 - 2015-02-04
11745 Codename: O(dear)
11746 This is purely a bugfix release:
11748 • Provide sensible external hashing for all core types. This will sig‐
11750 nificantly improve performance of tracking seen examples which hap‐
11751 pens in literally every falsification run. For Hypothesis fixing this
11752 cut 40% off the runtime of the test suite. The behaviour is quadratic
11753 in the number of examples so if you're running the default configura‐
11754 tion this will be less extreme (Hypothesis's test suite runs at a
11755 higher number of examples than default), but you should still see a
11756 significant improvement.
11757 • Fix a bug in formatting of complex numbers where the string could get
11759 incorrectly truncated.
11760 0.4.1 - 2015-02-03
11762 Codename: Cruel and unusual edge cases
11763 This release is mostly about better test case generation.
11765 Enhancements:
11767 • Has a cool release name
11769 • text_type (str in python 3, unicode in python 2) example generation
11771 now actually produces interesting unicode instead of boring ascii
11772 strings.
11773 • floating point numbers are generated over a much wider range, with
11775 particular attention paid to generating nasty numbers - nan, infin‐
11776 ity, large and small values, etc.
11777 • examples can be generated using pieces of examples previously saved
11779 in the database. This allows interesting behaviour that has previ‐
11780 ously been discovered to be propagated to other examples.
11781 • improved parameter exploration algorithm which should allow it to
11783 more reliably hit interesting edge cases.
11784 • Timeout can now be disabled entirely by setting it to any value <= 0.
11786 Bug fixes:
11788 • The descriptor on a OneOfStrategy could be wrong if you had descrip‐
11790 tors which were equal but should not be coalesced. e.g. a strategy
11791 for one_of((frozenset({int}), {int})) would have reported its de‐
11792 scriptor as {int}. This is unlikely to have caused you any problems
11793 • If you had strategies that could produce NaN (which float previously
11795 couldn't but e.g. a Just(float('nan')) could) then this would have
11796 sent hypothesis into an infinite loop that would have only been ter‐
11797 minated when it hit the timeout.
11798 • Given elements that can take a long time to minimize, minimization of
11800 floats or tuples could be quadratic or worse in the that value. You
11801 should now see much better performance for simplification, albeit at
11802 some cost in quality.
11803 Other:
11805 • A lot of internals have been been rewritten. This shouldn't affect
11807 you at all, but it opens the way for certain of hypothesis's oddities
11808 to be a lot more extensible by users. Whether this is a good thing
11809 may be up for debate...
11810 0.4.0 - 2015-01-21
11812 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
11813 stores data in a local SQLite database and will reuse it for all tests
11814 of the same type.
11815 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
11817 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
11818 ther notice. The previous license remains applicable to all code prior
11819 to 0.4.0.
11820 Enhancements:
11822 • Printing of failing examples. I was finding that the pytest runner
11824 was not doing a good job of displaying these, and that Hypothesis it‐
11825 self could do much better.
11826 • Drop dependency on six for cross-version compatibility. It was easy
11828 enough to write the shim for the small set of features that we care
11829 about and this lets us avoid a moderately complex dependency.
11830 • Some improvements to statistical distribution of selecting from small
11832 (<= 3 elements)
11833 • Improvements to parameter selection for finding examples.
11835 Bugs fixed:
11837 • could_have_produced for lists, dicts and other collections would not
11839 have examined the elements and thus when using a union of different
11840 types of list this could result in Hypothesis getting confused and
11841 passing a value to the wrong strategy. This could potentially result
11842 in exceptions being thrown from within simplification.
11843 • sampled_from would not work correctly on a single element list.
11845 • Hypothesis could get very confused by values which are equal despite
11847 having different types being used in descriptors. Hypothesis now has
11848 its own more specific version of equality it uses for descriptors and
11849 tracking. It is always more fine grained than Python equality: Things
11850 considered != are not considered equal by hypothesis, but some things
11851 that are considered == are distinguished. If your test suite uses
11852 both frozenset and set tests this bug is probably affecting you.
11853 0.3.2 - 2015-01-16
11855 • Fix a bug where if you specified floats_in_range with integer argu‐
11856 ments Hypothesis would error in example simplification.
11857 • Improve the statistical distribution of the floats you get for the
11859 floats_in_range strategy. I'm not sure whether this will affect users
11860 in practice but it took my tests for various conditions from flaky to
11861 rock solid so it at the very least improves discovery of the artifi‐
11862 cial cases I'm looking for.
11863 • Improved repr() for strategies and RandomWithSeed instances.
11865 • Add detection for flaky test cases where hypothesis managed to find
11867 an example which breaks it but on the final invocation of the test it
11868 does not raise an error. This will typically happen with too much re‐
11869 cursion errors but could conceivably happen in other circumstances
11870 too.
11871 • Provide a "derandomized" mode. This allows you to run hypothesis with
11873 zero real randomization, making your build nice and deterministic.
11874 The tests run with a seed calculated from the function they're test‐
11875 ing so you should still get a good distribution of test cases.
11876 • Add a mechanism for more conveniently defining tests which just sam‐
11878 ple from some collection.
11879 • Fix for a really subtle bug deep in the internals of the strategy ta‐
11881 ble. In some circumstances if you were to define instance strategies
11882 for both a parent class and one or more of its subclasses you would
11883 under some circumstances get the strategy for the wrong superclass of
11884 an instance. It is very unlikely anyone has ever encountered this in
11885 the wild, but it is conceivably possible given that a mix of namedtu‐
11886 ple and tuple are used fairly extensively inside hypothesis which do
11887 exhibit this pattern of strategy.
11888 0.3.1 - 2015-01-13
11890 • Support for generation of frozenset and Random values
11891 • Correct handling of the case where a called function mutates it argu‐
11893 ment. This involved introducing a notion of a strategies knowing how
11894 to copy their argument. The default method should be entirely accept‐
11895 able and the worst case is that it will continue to have the old be‐
11896 haviour if you don't mark your strategy as mutable, so this shouldn't
11897 break anything.
11898 • Fix for a bug where some strategies did not correctly implement
11900 could_have_produced. It is very unlikely that any of these would have
11901 been seen in the wild, and the consequences if they had been would
11902 have been minor.
11903 • Re-export the @given decorator from the main hypothesis namespace.
11905 It's still available at the old location too.
11906 • Minor performance optimisation for simplifying long lists.
11908 0.3.0 - 2015-01-12
11910 • Complete redesign of the data generation system. Extreme breaking
11911 change for anyone who was previously writing their own SearchStrategy
11912 implementations. These will not work any more and you'll need to mod‐
11913 ify them.
11914 • New settings system allowing more global and modular control of Veri‐
11916 fier behaviour.
11917 • Decouple SearchStrategy from the StrategyTable. This leads to much
11919 more composable code which is a lot easier to understand.
11920 • A significant amount of internal API renaming and moving. This may
11922 also break your code.
11923 • Expanded available descriptors, allowing for generating integers or
11925 floats in a specific range.
11926 • Significantly more robust. A very large number of small bug fixes,
11928 none of which anyone is likely to have ever noticed.
11929 • Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
11931 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
11932 these to remain officially supported for a very long time. I would
11933 not be surprised to add pypy support back in later but I'm not going
11934 to do so until I know someone cares about it. In the meantime it will
11935 probably still work.
11936 0.2.2 - 2015-01-08
11938 • Fix an embarrassing complete failure of the installer caused by my
11939 being bad at version control
11940 0.2.1 - 2015-01-07
11942 • Fix a bug in the new stateful testing feature where you could make
11943 __init__ a @requires method. Simplification would not always work if
11944 the prune method was able to successfully shrink the test.
11945 0.2.0 - 2015-01-07
11947 • It's aliiive.
11948 • Improve python 3 support using six.
11950 • Distinguish between byte and unicode types.
11952 • Fix issues where FloatStrategy could raise.
11954 • Allow stateful testing to request constructor args.
11956 • Fix for issue where test annotations would timeout based on when the
11958 module was loaded instead of when the test started
11959 0.1.4 - 2013-12-14
11961 • Make verification runs time bounded with a configurable timeout
11962 0.1.3 - 2013-05-03
11964 • Bugfix: Stateful testing behaved incorrectly with subclassing.
11965 • Complex number support
11967 • support for recursive strategies
11969 • different error for hypotheses with unsatisfiable assumptions
11971 0.1.2 - 2013-03-24
11973 • Bugfix: Stateful testing was not minimizing correctly and could throw
11974 exceptions.
11975 • Better support for recursive strategies.
11977 • Support for named tuples.
11979 • Much faster integer generation.
11981 0.1.1 - 2013-03-24
11983 • Python 3.x support via 2to3.
11984 • Use new style classes (oops).
11986 0.1.0 - 2013-03-23
11988 • Introduce stateful testing.
11989 • Massive rewrite of internals to add flags and strategies.
11991 0.0.5 - 2013-03-13
11993 • No changes except trying to fix packaging
11994 0.0.4 - 2013-03-13
11996 • No changes except that I checked in a failing test case for 0.0.3 so
11997 had to replace the release. Doh
11998 0.0.3 - 2013-03-13
12000 • Improved a few internals.
12001 • Opened up creating generators from instances as a general API.
12003 • Test integration.
12005 0.0.2 - 2013-03-12
12007 • Starting to tighten up on the internals.
12008 • Change API to allow more flexibility in configuration.
12010 • More testing.
12012 0.0.1 - 2013-03-10
12014 • Initial release.
12015 • Basic working prototype. Demonstrates idea, probably shouldn't be
12017 used.
12018
12020 Hypothesis development is managed by me, David R. MacIver. I am the
12021 primary author of Hypothesis.
12022 However, I no longer do unpaid feature development on Hypothesis. My
12024 roles as leader of the project are:
12025 1. Helping other people do feature development on Hypothesis
12027 2. Fixing bugs and other code health issues
12029 3. Improving documentation
12031 4. General release management work
12033 5. Planning the general roadmap of the project
12035 6. Doing sponsored development on tasks that are too large or in depth
12037 for other people to take on
12038 So all new features must either be sponsored or implemented by someone
12040 else. That being said, the maintenance team takes an active role in
12041 shepherding pull requests and helping people write a new feature (see
12042 CONTRIBUTING.rst for details and pull request #154 for an example of
12043 how the process goes). This isn't "patches welcome", it's "we will help
12044 you write a patch".
12045 Release policy
12047 Hypothesis releases follow semantic versioning.
12048 We maintain backwards-compatibility wherever possible, and use depreca‐
12050 tion warnings to mark features that have been superseded by a newer al‐
12051 ternative. If you want to detect this, you can upgrade warnings to er‐
12052 rors in the usual ways.
12053 We use continuous deployment to ensure that you can always use our new‐
12055 est and shiniest features - every change to the source tree is automat‐
12056 ically built and published on PyPI as soon as it's merged onto master,
12057 after code review and passing our extensive test suite.
12058 Project roadmap
12060 Hypothesis does not have a long-term release plan. We respond to bug
12061 reports as they are made; new features are released as and when someone
12062 volunteers to write and maintain them.
12063
12065 For questions you are happy to ask in public, the Hypothesis community
12066 is a friendly place where I or others will be more than happy to help
12067 you out. You're also welcome to ask questions on Stack Overflow. If you
12068 do, please tag them with 'python-hypothesis' so someone sees them.
12069 For bugs and enhancements, please file an issue on the GitHub issue
12071 tracker. Note that as per the development policy, enhancements will
12072 probably not get implemented unless you're willing to pay for develop‐
12073 ment or implement them yourself (with assistance from me). Bugs will
12074 tend to get fixed reasonably promptly, though it is of course on a best
12075 effort basis.
12076 To see the versions of Python, optional dependencies, test runners, and
12078 operating systems Hypothesis supports (meaning incompatibility is
12079 treated as a bug), see supported.
12080 If you need to ask questions privately or want more of a guarantee of
12082 bugs being fixed promptly, please contact me on
12083 hypothesis-support@drmaciver.com to talk about availability of support
12084 contracts.
12085
12087 Downstream packagers often want to package Hypothesis. Here are some
12088 guidelines.
12089 The primary guideline is this: If you are not prepared to keep up with
12091 the Hypothesis release schedule, don't. You will annoy me and are doing
12092 your users a disservice.
12093 Hypothesis has a very frequent release schedule. It's rare that it goes
12095 a week without a release, and there are often multiple releases in a
12096 given week.
12097 If you are prepared to keep up with this schedule, you might find the
12099 rest of this document useful.
12100 Release tarballs
12102 These are available from the GitHub releases page. The tarballs on PyPI
12103 are intended for installation from a Python tool such as pip and should
12104 not be considered complete releases. Requests to include additional
12105 files in them will not be granted. Their absence is not a bug.
12106 Dependencies
12108 Python versions
12109 Hypothesis is designed to work with a range of Python versions - we
12110 support all versions of CPython with upstream support. We also support
12111 the latest versions of PyPy for Python 3.
12112 Other Python libraries
12114 Hypothesis has mandatory dependencies on the following libraries:
12115 • attrs
12117 • sortedcontainers
12119 Hypothesis has optional dependencies on the following libraries:
12121 extras_require = {
12123 "cli": ["click>=7.0", "black>=19.10b0"],
12124 "ghostwriter": ["black>=19.10b0"],
12125 "pytz": ["pytz>=2014.1"],
12126 "dateutil": ["python-dateutil>=1.4"],
12127 "lark": ["lark-parser>=0.6.5"],
12128 "numpy": ["numpy>=1.9.0"],
12129 "pandas": ["pandas>=0.25"],
12130 "pytest": ["pytest>=4.3"],
12131 "dpcontracts": ["dpcontracts>=0.4"],
12132 "redis": ["redis>=3.0.0"],
12133 # zoneinfo is an odd one: every dependency is conditional, because they're
12134 # only necessary on old versions of Python or Windows systems.
12135 "zoneinfo": [
12136 "tzdata>=2020.4 ; sys_platform == 'win32'",
12137 "backports.zoneinfo>=0.2.1 ; python_version<'3.9'",
12138 "importlib_resources>=3.3.0 ; python_version<'3.7'",
12139 ],
12140 # We only support Django versions with upstream support - see
12141 # https://www.djangoproject.com/download/#supported-versions
12142 "django": ["pytz>=2014.1", "django>=2.2"],
12143 }
12144 The way this works when installing Hypothesis normally is that these
12147 features become available if the relevant library is installed.
12148 Specifically for pytest, our plugin supports versions of pytest which
12150 have been out of upstream support for some time. Hypothesis tests can
12151 still be executed by even older versions of pytest - you just won't
12152 have the plugin to provide automatic marks, helpful usage warnings, and
12153 per-test statistics.
12154 Testing Hypothesis
12156 If you want to test Hypothesis as part of your packaging you will prob‐
12157 ably not want to use the mechanisms Hypothesis itself uses for running
12158 its tests, because it has a lot of logic for installing and testing
12159 against different versions of Python.
12160 The tests must be run with fairly recent tooling; check the
12162 requirements/ directory for details.
12163 The organisation of the tests is described in the
12165 hypothesis-python/tests/README.rst.
12166 Examples
12168 • arch linux
12169 • fedora
12171 • gentoo
12173
12175 One of the things that is often concerning for people using randomized
12176 testing is the question of how to reproduce failing test cases.
12177 NOTE:
12179 It is better to think about the data Hypothesis generates as being
12180 arbitrary, rather than random. We deliberately generate any valid
12181 data that seems likely to cause errors, so you shouldn't rely on any
12182 expected distribution of or relationships between generated data.
12183 You can read about "swarm testing" and "coverage guided fuzzing" if
12184 you're interested, because you don't need to know for Hypothesis!
12185 Fortunately Hypothesis has a number of features to support reproducing
12187 test failures. The one you will use most commonly when developing lo‐
12188 cally is the example database, which means that you shouldn't have to
12189 think about the problem at all for local use - test failures will just
12190 automatically reproduce without you having to do anything.
12191 The example database is perfectly suitable for sharing between ma‐
12193 chines, but there currently aren't very good work flows for that, so
12194 Hypothesis provides a number of ways to make examples reproducible by
12195 adding them to the source code of your tests. This is particularly use‐
12196 ful when e.g. you are trying to run an example that has failed on your
12197 CI, or otherwise share them between machines.
12198 Providing explicit examples
12200 The simplest way to reproduce a failed test is to ask Hypothesis to run
12201 the failing example it printed. For example, if Falsifying example:
12202 test(n=1) was printed you can decorate test with @example(n=1).
12203 @example can also be used to ensure a specific example is always exe‐
12205 cuted as a regression test or to cover some edge case - basically com‐
12206 bining a Hypothesis test and a traditional parametrized test.
12207 hypothesis.example(*args, **kwargs)
12209 A decorator which ensures a specific example is always tested.
12210 Hypothesis will run all examples you've asked for first. If any of them
12212 fail it will not go on to look for more examples.
12213 It doesn't matter whether you put the example decorator before or after
12215 given. Any permutation of the decorators in the above will do the same
12216 thing.
12217 Note that examples can be positional or keyword based. If they're posi‐
12219 tional then they will be filled in from the right when calling, so ei‐
12220 ther of the following styles will work as expected:
12221 @given(text())
12223 @example("Hello world")
12224 @example(x="Some very long string")
12225 def test_some_code(x):
12226 assert True
12227 from unittest import TestCase
12230 class TestThings(TestCase):
12233 @given(text())
12234 @example("Hello world")
12235 @example(x="Some very long string")
12236 def test_some_code(self, x):
12237 assert True
12238 As with @given, it is not permitted for a single example to be a mix of
12240 positional and keyword arguments. Either are fine, and you can use one
12241 in one example and the other in another example if for some reason you
12242 really want to, but a single example must be consistent.
12243 Reproducing a test run with @seed
12245 hypothesis.seed(seed)
12246 seed: Start the test execution from a specific seed.
12247 May be any hashable object. No exact meaning for seed is pro‐
12249 vided other than that for a fixed seed value Hypothesis will try
12250 the same actions (insofar as it can given external sources of
12251 non- determinism. e.g. timing and hash randomization).
12252 Overrides the derandomize setting, which is designed to enable
12254 deterministic builds rather than reproducing observed failures.
12255 When a test fails unexpectedly, usually due to a health check failure,
12257 Hypothesis will print out a seed that led to that failure, if the test
12258 is not already running with a fixed seed. You can then recreate that
12259 failure using either the @seed decorator or (if you are running pytest)
12260 with --hypothesis-seed. For example, the following test function and
12261 RuleBasedStateMachine will each check the same examples each time they
12262 are executed, thanks to @seed():
12263 @seed(1234)
12265 @given(x=...)
12266 def test(x):
12267 ...
12268 @seed(6789)
12271 class MyModel(RuleBasedStateMachine):
12272 ...
12273 The seed will not be printed if you could simply use @example instead.
12275 Reproducing an example with @reproduce_failure
12277 Hypothesis has an opaque binary representation that it uses for all ex‐
12278 amples it generates. This representation is not intended to be stable
12279 across versions or with respect to changes in the test, but can be used
12280 to to reproduce failures with the @reproduce_failure decorator.
12281 hypothesis.reproduce_failure(version, blob)
12283 Run the example that corresponds to this data blob in order to
12284 reproduce a failure.
12285 A test with this decorator always runs only one example and al‐
12287 ways fails. If the provided example does not cause a failure,
12288 or is in some way invalid for this test, then this will fail
12289 with a DidNotReproduce error.
12290 This decorator is not intended to be a permanent addition to
12292 your test suite. It's simply some code you can add to ease re‐
12293 production of a problem in the event that you don't have access
12294 to the test database. Because of this, no compatibility guaran‐
12295 tees are made between different versions of Hypothesis - its API
12296 may change arbitrarily from version to version.
12297 The intent is that you should never write this decorator by hand, but
12299 it is instead provided by Hypothesis. When a test fails with a falsi‐
12300 fying example, Hypothesis may print out a suggestion to use @repro‐
12301 duce_failure on the test to recreate the problem as follows:
12302 >>> from hypothesis import settings, given, PrintSettings
12304 >>> import hypothesis.strategies as st
12305 >>> @given(st.floats())
12306 ... @settings(print_blob=True)
12307 ... def test(f):
12308 ... assert f == f
12309 ...
12310 >>> try:
12311 ... test()
12312 ... except AssertionError:
12313 ... pass
12314 Falsifying example: test(f=nan)
12315 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
12317 Adding the suggested decorator to the test should reproduce the failure
12319 (as long as everything else is the same - changing the versions of
12320 Python or anything else involved, might of course affect the behaviour
12321 of the test! Note that changing the version of Hypothesis will result
12322 in a different error - each @reproduce_failure invocation is specific
12323 to a Hypothesis version).
12324 By default these messages are not printed. If you want to see these
12326 you must set the print_blob setting to True.
12327
12329 David R. MacIver
12330
12332 2013-2021, David R. MacIver
123335.43.9 Feb 01, 2021 HYPOTHESIS(1)
