1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
3
4
6 hypothesis - Hypothesis Documentation
7
8 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12
13 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15
16 Think of a normal unit test as being something like the following:
17
18 1. Set up some data.
19
20 2. Perform some operations on the data.
21
22 3. Assert something about the result.
23
24 Hypothesis lets you write tests which instead look like this:
25
26 1. For all data matching some specification.
27
28 2. Perform some operations on the data.
29
30 3. Assert something about the result.
31
32 This is often called property based testing, and was popularised by the
33 Haskell library Quickcheck.
34
35 It works by generating random data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the
41 future.
42
43 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47
48 · Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51
52 · If you delete an object, it is no longer visible.
53
54 · If you serialize and then deserialize a value, then you get the same
55 value back.
56
57 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69
70 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73
74 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77
78 def encode(input_string):
79 count = 1
80 prev = ''
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 else:
92 entry = (character, count)
93 lst.append(entry)
94 return lst
95
96
97 def decode(lst):
98 q = ''
99 for character, count in lst:
100 q += character * count
101 return q
102
103 We want to write a test for this that will check some invariant of
104 these functions.
105
106 The invariant one tends to try when you've got this sort of encoding /
107 decoding is that if you encode something and then decode it then you
108 get the same value back.
109
110 Lets see how you'd do that with Hypothesis:
111
112 from hypothesis import given
113 from hypothesis.strategies import text
114
115 @given(text())
116 def test_decode_inverts_encode(s):
117 assert decode(encode(s)) == s
118
119 (For this example we'll just let pytest discover and run the test.
120 We'll cover other ways you could have run it later).
121
122 The text function returns what Hypothesis calls a search strategy. An
123 object with methods that describe how to generate and simplify certain
124 kinds of values. The @given decorator then takes our test function and
125 turns it into a parametrized one which, when called, will run the test
126 function over a wide range of matching data from that strategy.
127
128 Anyway, this test immediately finds a bug in the code:
129
130 Falsifying example: test_decode_inverts_encode(s='')
131
132 UnboundLocalError: local variable 'character' referenced before assignment
133
134 Hypothesis correctly points out that this code is simply wrong if
135 called on an empty string.
136
137 If we fix that by just adding the following code to the beginning of
138 the function then Hypothesis tells us the code is correct (by doing
139 nothing as you'd expect a passing test to).
140
141 if not input_string:
142 return []
143
144 If we wanted to make sure this example was always checked we could add
145 it in explicitly:
146
147 from hypothesis import given, example
148 from hypothesis.strategies import text
149
150 @given(text())
151 @example('')
152 def test_decode_inverts_encode(s):
153 assert decode(encode(s)) == s
154
155 You don't have to do this, but it can be useful both for clarity pur‐
156 poses and for reliably hitting hard to find examples. Also in local
157 development Hypothesis will just remember and reuse the examples any‐
158 way, but there's not currently a very good workflow for sharing those
159 in your CI.
160
161 It's also worth noting that both example and given support keyword
162 arguments as well as positional. The following would have worked just
163 as well:
164
165 @given(s=text())
166 @example(s='')
167 def test_decode_inverts_encode(s):
168 assert decode(encode(s)) == s
169
170 Suppose we had a more interesting bug and forgot to reset the count
171 each time. Say we missed a line in our encode method:
172
173 def encode(input_string):
174 count = 1
175 prev = ''
176 lst = []
177 for character in input_string:
178 if character != prev:
179 if prev:
180 entry = (prev, count)
181 lst.append(entry)
182 # count = 1 # Missing reset operation
183 prev = character
184 else:
185 count += 1
186 else:
187 entry = (character, count)
188 lst.append(entry)
189 return lst
190
191 Hypothesis quickly informs us of the following example:
192
193 Falsifying example: test_decode_inverts_encode(s='001')
194
195 Note that the example provided is really quite simple. Hypothesis
196 doesn't just find any counter-example to your tests, it knows how to
197 simplify the examples it finds to produce small easy to understand
198 ones. In this case, two identical values are enough to set the count to
199 a number different from one, followed by another distinct value which
200 should have reset the count but in this case didn't.
201
202 The examples Hypothesis provides are valid Python code you can run. Any
203 arguments that you explicitly provide when calling the function are not
204 generated by Hypothesis, and if you explicitly provide all the argu‐
205 ments Hypothesis will just call the underlying function once rather
206 than running it multiple times.
207
208 Installing
209 Hypothesis is available on pypi as "hypothesis". You can install it
210 with:
211
212 pip install hypothesis
213
214 If you want to install directly from the source code (e.g. because you
215 want to make changes and install the changed version) you can do this
216 with:
217
218 pip install -e .
219
220 You should probably run the tests first to make sure nothing is broken.
221 You can do this with:
222
223 python setup.py test
224
225 Note that if they're not already installed this will try to install the
226 test dependencies.
227
228 You may wish to do all of this in a virtualenv. For example:
229
230 virtualenv venv
231 source venv/bin/activate
232 pip install hypothesis
233
234 Will create an isolated environment for you to try hypothesis out in
235 without affecting your system installed packages.
236
237 Running tests
238 In our example above we just let pytest discover and run our tests, but
239 we could also have run it explicitly ourselves:
240
241 if __name__ == '__main__':
242 test_decode_inverts_encode()
243
244 We could also have done this as a python:unittest.TestCase:
245
246 import unittest
247
248 class TestEncoding(unittest.TestCase):
249 @given(text())
250 def test_decode_inverts_encode(self, s):
251 self.assertEqual(decode(encode(s)), s)
252
253 if __name__ == '__main__':
254 unittest.main()
255
256 A detail: This works because Hypothesis ignores any arguments it hasn't
257 been told to provide (positional arguments start from the right), so
258 the self argument to the test is simply ignored and works as normal.
259 This also means that Hypothesis will play nicely with other ways of
260 parameterizing tests. e.g it works fine if you use pytest fixtures for
261 some arguments and Hypothesis for others.
262
263 Writing tests
264 A test in Hypothesis consists of two parts: A function that looks like
265 a normal test in your test framework of choice but with some additional
266 arguments, and a @given decorator that specifies how to provide those
267 arguments.
268
269 Here are some other examples of how you could use that:
270
271 from hypothesis import given
272 import hypothesis.strategies as st
273
274 @given(st.integers(), st.integers())
275 def test_ints_are_commutative(x, y):
276 assert x + y == y + x
277
278 @given(x=st.integers(), y=st.integers())
279 def test_ints_cancel(x, y):
280 assert (x + y) - y == x
281
282 @given(st.lists(st.integers()))
283 def test_reversing_twice_gives_same_list(xs):
284 # This will generate lists of arbitrary length (usually between 0 and
285 # 100 elements) whose elements are integers.
286 ys = list(xs)
287 ys.reverse()
288 ys.reverse()
289 assert xs == ys
290
291 @given(st.tuples(st.booleans(), st.text()))
292 def test_look_tuples_work_too(t):
293 # A tuple is generated as the one you provided, with the corresponding
294 # types in those positions.
295 assert len(t) == 2
296 assert isinstance(t[0], bool)
297 assert isinstance(t[1], str)
298
299 Note that as we saw in the above example you can pass arguments to
300 @given either as positional or as keywords.
301
302 Where to start
303 You should now know enough of the basics to write some tests for your
304 code using Hypothesis. The best way to learn is by doing, so go have a
305 try.
306
307 If you're stuck for ideas for how to use this sort of test for your
308 code, here are some good starting points:
309
310 1. Try just calling functions with appropriate random data and see if
311 they crash. You may be surprised how often this works. e.g. note
312 that the first bug we found in the encoding example didn't even get
313 as far as our assertion: It crashed because it couldn't handle the
314 data we gave it, not because it did the wrong thing.
315
316 2. Look for duplication in your tests. Are there any cases where you're
317 testing the same thing with multiple different examples? Can you
318 generalise that to a single test using Hypothesis?
319
320 3. This piece is designed for an F# implementation, but is still very
321 good advice which you may find helps give you good ideas for using
322 Hypothesis.
323
324 If you have any trouble getting started, don't feel shy about asking
325 for help.
326
328 This is an account of slightly less common Hypothesis features that you
329 don't need to get started but will nevertheless make your life easier.
330
331 Additional test output
332 Normally the output of a failing test will look something like:
333
334 Falsifying example: test_a_thing(x=1, y="foo")
335
336 With the repr of each keyword argument being printed.
337
338 Sometimes this isn't enough, either because you have values with a repr
339 that isn't very descriptive or because you need to see the output of
340 some intermediate steps of your test. That's where the note function
341 comes in:
342
343 >>> from hypothesis import given, note, strategies as st
344 >>> @given(st.lists(st.integers()), st.randoms())
345 ... def test_shuffle_is_noop(ls, r):
346 ... ls2 = list(ls)
347 ... r.shuffle(ls2)
348 ... note("Shuffle: %r" % (ls2))
349 ... assert ls == ls2
350 ...
351 >>> try:
352 ... test_shuffle_is_noop()
353 ... except AssertionError:
354 ... print('ls != ls2')
355 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(18))
356 Shuffle: [1, 0]
357 ls != ls2
358
359 The note is printed in the final run of the test in order to include
360 any additional information you might need in your test.
361
362 Test Statistics
363 If you are using pytest you can see a number of statistics about the
364 executed tests by passing the command line argument --hypothe‐
365 sis-show-statistics. This will include some general statistics about
366 the test:
367
368 For example if you ran the following with --hypothesis-show-statistics:
369
370 from hypothesis import given, strategies as st
371
372 @given(st.integers())
373 def test_integers(i):
374 pass
375
376 You would see:
377
378 test_integers:
379
380 - 100 passing examples, 0 failing examples, 0 invalid examples
381 - Typical runtimes: ~ 1ms
382 - Fraction of time spent in data generation: ~ 12%
383 - Stopped because settings.max_examples=100
384
385 The final "Stopped because" line is particularly important to note: It
386 tells you the setting value that determined when the test should stop
387 trying new examples. This can be useful for understanding the behaviour
388 of your tests. Ideally you'd always want this to be max_examples.
389
390 In some cases (such as filtered and recursive strategies) you will see
391 events mentioned which describe some aspect of the data generation:
392
393 from hypothesis import given, strategies as st
394
395 @given(st.integers().filter(lambda x: x % 2 == 0))
396 def test_even_integers(i):
397 pass
398
399 You would see something like:
400
401 test_even_integers:
402
403 - 100 passing examples, 0 failing examples, 36 invalid examples
404 - Typical runtimes: 0-1 ms
405 - Fraction of time spent in data generation: ~ 16%
406 - Stopped because settings.max_examples=100
407 - Events:
408 * 80.88%, Retried draw from integers().filter(lambda x: <unknown>) to satisfy filter
409 * 26.47%, Aborted test because unable to satisfy integers().filter(lambda x: <unknown>)
410
411 You can also mark custom events in a test using the event function:
412
413 hypothesis.event(value)
414 Record an event that occurred this test. Statistics on number of
415 test runs with each event will be reported at the end if you run
416 Hypothesis in statistics reporting mode.
417
418 Events should be strings or convertible to them.
419
420 from hypothesis import given, event, strategies as st
421
422 @given(st.integers().filter(lambda x: x % 2 == 0))
423 def test_even_integers(i):
424 event("i mod 3 = %d" % (i % 3,))
425
426 You will then see output like:
427
428 test_even_integers:
429
430 - 100 passing examples, 0 failing examples, 38 invalid examples
431 - Typical runtimes: 0-1 ms
432 - Fraction of time spent in data generation: ~ 16%
433 - Stopped because settings.max_examples=100
434 - Events:
435 * 80.43%, Retried draw from integers().filter(lambda x: <unknown>) to satisfy filter
436 * 31.88%, i mod 3 = 0
437 * 27.54%, Aborted test because unable to satisfy integers().filter(lambda x: <unknown>)
438 * 21.74%, i mod 3 = 1
439 * 18.84%, i mod 3 = 2
440
441 Arguments to event can be any hashable type, but two events will be
442 considered the same if they are the same when converted to a string
443 with python:str.
444
445 Making assumptions
446 Sometimes Hypothesis doesn't give you exactly the right sort of data
447 you want - it's mostly of the right shape, but some examples won't work
448 and you don't want to care about them. You can just ignore these by
449 aborting the test early, but this runs the risk of accidentally testing
450 a lot less than you think you are. Also it would be nice to spend less
451 time on bad examples - if you're running 100 examples per test (the
452 default) and it turns out 70 of those examples don't match your needs,
453 that's a lot of wasted time.
454
455 hypothesis.assume(condition)
456 Calling assume is like an assert that marks the example as bad,
457 rather than failing the test.
458
459 This allows you to specify properties that you assume will be
460 true, and let Hypothesis try to avoid similar examples in
461 future.
462
463 For example suppose you had the following test:
464
465 @given(floats())
466 def test_negation_is_self_inverse(x):
467 assert x == -(-x)
468
469 Running this gives us:
470
471 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
472 AssertionError
473
474 This is annoying. We know about NaN and don't really care about it, but
475 as soon as Hypothesis finds a NaN example it will get distracted by
476 that and tell us about it. Also the test will fail and we want it to
477 pass.
478
479 So lets block off this particular example:
480
481 from math import isnan
482
483 @given(floats())
484 def test_negation_is_self_inverse_for_non_nan(x):
485 assume(not isnan(x))
486 assert x == -(-x)
487
488 And this passes without a problem.
489
490 In order to avoid the easy trap where you assume a lot more than you
491 intended, Hypothesis will fail a test when it can't find enough exam‐
492 ples passing the assumption.
493
494 If we'd written:
495
496 @given(floats())
497 def test_negation_is_self_inverse_for_non_nan(x):
498 assume(False)
499 assert x == -(-x)
500
501 Then on running we'd have got the exception:
502
503 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
504
505 How good is assume?
506 Hypothesis has an adaptive exploration strategy to try to avoid things
507 which falsify assumptions, which should generally result in it still
508 being able to find examples in hard to find situations.
509
510 Suppose we had the following:
511
512 @given(lists(integers()))
513 def test_sum_is_positive(xs):
514 assert sum(xs) > 0
515
516 Unsurprisingly this fails and gives the falsifying example [].
517
518 Adding assume(xs) to this removes the trivial empty example and gives
519 us [0].
520
521 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
522 of positive integers is positive.
523
524 The reason that this should be surprising is not that it doesn't find a
525 counter-example, but that it finds enough examples at all.
526
527 In order to make sure something interesting is happening, suppose we
528 wanted to try this for long lists. e.g. suppose we added an
529 assume(len(xs) > 10) to it. This should basically never find an exam‐
530 ple: a naive strategy would find fewer than one in a thousand examples,
531 because if each element of the list is negative with probability
532 one-half, you'd have to have ten of these go the right way by chance.
533 In the default configuration Hypothesis gives up long before it's tried
534 1000 examples (by default it tries 200).
535
536 Here's what happens if we try to run this:
537
538 @given(lists(integers()))
539 def test_sum_is_positive(xs):
540 assume(len(xs) > 10)
541 assume(all(x > 0 for x in xs))
542 print(xs)
543 assert sum(xs) > 0
544
545 In: test_sum_is_positive()
546 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
547 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
548 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2...
549 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
550 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...
551 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
552
553 As you can see, Hypothesis doesn't find many examples here, but it
554 finds some - enough to keep it happy.
555
556 In general if you can shape your strategies better to your tests you
557 should - for example integers(1, 1000) is a lot better than assume(1 <=
558 x <= 1000), but assume will take you a long way if you can't.
559
560 Defining strategies
561 The type of object that is used to explore the examples given to your
562 test function is called a SearchStrategy. These are created using the
563 functions exposed in the hypothesis.strategies module.
564
565 Many of these strategies expose a variety of arguments you can use to
566 customize generation. For example for integers you can specify min and
567 max values of integers you want. If you want to see exactly what a
568 strategy produces you can ask for an example:
569
570 >>> integers(min_value=0, max_value=10).example()
571 1
572
573 Many strategies are built out of other strategies. For example, if you
574 want to define a tuple you need to say what goes in each element:
575
576 >>> from hypothesis.strategies import tuples
577 >>> tuples(integers(), integers()).example()
578 (-24597, 12566)
579
580 Further details are available in a separate document.
581
582 The gory details of given parameters
583 hypothesis.given(*given_arguments, **given_kwargs)
584 A decorator for turning a test function that accepts arguments
585 into a randomized test.
586
587 This is the main entry point to Hypothesis.
588
589 The @given decorator may be used to specify which arguments of a func‐
590 tion should be parametrized over. You can use either positional or key‐
591 word arguments or a mixture of the two.
592
593 For example all of the following are valid uses:
594
595 @given(integers(), integers())
596 def a(x, y):
597 pass
598
599 @given(integers())
600 def b(x, y):
601 pass
602
603 @given(y=integers())
604 def c(x, y):
605 pass
606
607 @given(x=integers())
608 def d(x, y):
609 pass
610
611 @given(x=integers(), y=integers())
612 def e(x, **kwargs):
613 pass
614
615 @given(x=integers(), y=integers())
616 def f(x, *args, **kwargs):
617 pass
618
619
620 class SomeTest(TestCase):
621 @given(integers())
622 def test_a_thing(self, x):
623 pass
624
625 The following are not:
626
627 @given(integers(), integers(), integers())
628 def g(x, y):
629 pass
630
631 @given(integers())
632 def h(x, *args):
633 pass
634
635 @given(integers(), x=integers())
636 def i(x, y):
637 pass
638
639 @given()
640 def j(x, y):
641 pass
642
643 The rules for determining what are valid uses of given are as follows:
644
645 1. You may pass any keyword argument to given.
646
647 2. Positional arguments to given are equivalent to the rightmost named
648 arguments for the test function.
649
650 3. Positional arguments may not be used if the underlying test function
651 has varargs, arbitrary keywords, or keyword-only arguments.
652
653 4. Functions tested with given may not have any defaults.
654
655 The reason for the "rightmost named arguments" behaviour is so that
656 using @given with instance methods works: self will be passed to the
657 function as normal and not be parametrized over.
658
659 The function returned by given has all the same arguments as the origi‐
660 nal test, minus those that are filled in by @given.
661
662 Custom function execution
663 Hypothesis provides you with a hook that lets you control how it runs
664 examples.
665
666 This lets you do things like set up and tear down around each example,
667 run examples in a subprocess, transform coroutine tests into normal
668 tests, etc. For example, TransactionTestCase in the Django extra runs
669 each example in a separate database transaction.
670
671 The way this works is by introducing the concept of an executor. An
672 executor is essentially a function that takes a block of code and run
673 it. The default executor is:
674
675 def default_executor(function):
676 return function()
677
678 You define executors by defining a method execute_example on a class.
679 Any test methods on that class with @given used on them will use
680 self.execute_example as an executor with which to run tests. For exam‐
681 ple, the following executor runs all its code twice:
682
683 from unittest import TestCase
684
685 class TestTryReallyHard(TestCase):
686 @given(integers())
687 def test_something(self, i):
688 perform_some_unreliable_operation(i)
689
690 def execute_example(self, f):
691 f()
692 return f()
693
694 Note: The functions you use in map, etc. will run inside the executor.
695 i.e. they will not be called until you invoke the function passed to
696 execute_example.
697
698 An executor must be able to handle being passed a function which
699 returns None, otherwise it won't be able to run normal test cases. So
700 for example the following executor is invalid:
701
702 from unittest import TestCase
703
704 class TestRunTwice(TestCase):
705 def execute_example(self, f):
706 return f()()
707
708 and should be rewritten as:
709
710 from unittest import TestCase
711
712 class TestRunTwice(TestCase):
713 def execute_example(self, f):
714 result = f()
715 if callable(result):
716 result = result()
717 return result
718
719 An alternative hook is provided for use by test runner extensions such
720 as pytest-trio, which cannot use the execute_example method. This is
721 not recommended for end-users - it is better to write a complete test
722 function directly, perhaps by using a decorator to perform the same
723 transformation before applying @given.
724
725 @given(x=integers())
726 @pytest.mark.trio
727 async def test(x):
728 ...
729 # Illustrative code, inside the pytest-trio plugin
730 test.hypothesis.inner_test = lambda x: trio.run(test, x)
731
732 For authors of test runners however, assigning to the inner_test
733 attribute of the hypothesis attribute of the test will replace the
734 interior test.
735
736 NOTE:
737 The new inner_test must accept and pass through all the *args and
738 **kwargs expected by the original test.
739
740 If the end user has also specified a custom executor using the exe‐
741 cute_example method, it - and all other execution-time logic - will be
742 applied to the new inner test assigned by the test runner.
743
744 Using Hypothesis to find values
745 You can use Hypothesis's data exploration features to find values sat‐
746 isfying some predicate. This is generally useful for exploring custom
747 strategies defined with @composite, or experimenting with conditions
748 for filtering data.
749
750 hypothesis.find(specifier, condition, settings=None, random=None, data‐
751 base_key=None)
752 Returns the minimal example from the given strategy specifier
753 that matches the predicate function condition.
754
755 >>> from hypothesis import find
756 >>> from hypothesis.strategies import sets, lists, integers
757 >>> find(lists(integers()), lambda x: sum(x) >= 10)
758 [10]
759 >>> find(lists(integers()), lambda x: sum(x) >= 10 and len(x) >= 3)
760 [0, 0, 10]
761 >>> find(sets(integers()), lambda x: sum(x) >= 10 and len(x) >= 3)
762 {0, 1, 9}
763
764 The first argument to find() describes data in the usual way for an
765 argument to given(), and supports all the same data types. The second
766 is a predicate it must satisfy.
767
768 Of course not all conditions are satisfiable. If you ask Hypothesis for
769 an example to a condition that is always false it will raise an error:
770
771 >>> find(integers(), lambda x: False)
772 Traceback (most recent call last):
773 ...
774 hypothesis.errors.NoSuchExample: No examples of condition lambda x: <unknown>
775
776 (The lambda x: unknown is because Hypothesis can't retrieve the source
777 code of lambdas from the interactive python console. It gives a better
778 error message most of the time which contains the actual condition)
779
780 Inferred Strategies
781 In some cases, Hypothesis can work out what to do when you omit argu‐
782 ments. This is based on introspection, not magic, and therefore has
783 well-defined limits.
784
785 builds() will check the signature of the target (using get‐
786 fullargspec()). If there are required arguments with type annotations
787 and no strategy was passed to builds(), from_type() is used to fill
788 them in. You can also pass the special value hypothesis.infer as a
789 keyword argument, to force this inference for arguments with a default
790 value.
791
792 >>> def func(a: int, b: str):
793 ... return [a, b]
794 >>> builds(func).example()
795 [-6993, '']
796
797 @given does not perform any implicit inference for required arguments,
798 as this would break compatibility with pytest fixtures. infer can be
799 used as a keyword argument to explicitly fill in an argument from its
800 type annotation.
801
802 @given(a=infer)
803 def test(a: int): pass
804 # is equivalent to
805 @given(a=integers())
806 def test(a): pass
807
808 Limitations
809 PEP 3107 type annotations are not supported on Python 2, and Hypothesis
810 does not inspect PEP 484 type comments at runtime. While from_type()
811 will work as usual, inference in builds() and @given will only work if
812 you manually create the __annotations__ attribute (e.g. by using @anno‐
813 tations(...) and @returns(...) decorators). The python:typing module
814 is fully supported on Python 2 if you have the backport installed.
815
816 The python:typing module is provisional and has a number of internal
817 changes between Python 3.5.0 and 3.6.1, including at minor versions.
818 These are all supported on a best-effort basis, but you may encounter
819 problems with an old version of the module. Please report them to us,
820 and consider updating to a newer version of Python as a workaround.
821
822 Type Annotations in Hypothesis
823 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
824 patible tool, the type checker should automatically pick up our type
825 hints.
826
827 NOTE:
828 Hypothesis' type hints may make breaking changes between minor
829 releases.
830
831 Upstream tools and conventions about type hints remain in flux - for
832 example the python:typing module itself is provisional, and Mypy has
833 not yet reached version 1.0 - and we plan to support the latest ver‐
834 sion of this ecosystem, as well as older versions where practical.
835
836 We may also find more precise ways to describe the type of various
837 interfaces, or change their type and runtime behaviour togther in a
838 way which is otherwise backwards-compatible. We often omit type
839 hints for deprecated features or arguments, as an additional form of
840 warning.
841
842 There are known issues inferring the type of examples generated by
843 deferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
844 ies(). We will fix these, and require correspondingly newer versions
845 of Mypy for type hinting, as the ecosystem improves.
846
847 Writing downstream type hints
848 Projects that provide Hypothesis strategies and use type hints may wish
849 to annotate their strategies too. This is a supported use-case, again
850 on a best-effort provisional basis. For example:
851
852 def foo_strategy() -> SearchStrategy[Foo]: ...
853
854 hypothesis.strategies.SearchStrategy is the type of all strategy
855 objects. It is a generic type, and covariant in the type of the exam‐
856 ples it creates. For example:
857
858 · integers() is of type SearchStrategy[int].
859
860 · lists(integers()) is of type SearchStrategy[List[int]].
861
862 · SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
863 a subtype of Animal (as seems likely).
864
865 WARNING:
866 SearchStrategy should only be used in type hints. Please do not
867 inherit from, compare to, or otherwise use it in any way outside of
868 type hints. The only supported way to construct objects of this
869 type is to use the functions provided by the hypothesis.strategies
870 module!
871
872 The Hypothesis pytest Plugin
873 Hypothesis includes a tiny plugin to improve integration with pytest,
874 which is activated by default (but does not affect other test runners).
875 It aims to improve the integration between Hypothesis and Pytest by
876 providing extra information and convenient access to config options.
877
878 · pytest --hypothesis-show-statistics can be used to display test and
879 data generation statistics.
880
881 · pytest --hypothesis-profile=<profile name> can be used to load a set‐
882 tings profile.
883
884 · pytest --hypothesis-seed=<an int> can be used to reproduce a failure
885 with a particular seed.
886
887 Finally, all tests that are defined with Hypothesis automatically have
888 @pytest.mark.hypothesis applied to them. See here for information on
889 working with markers.
890
891 NOTE:
892 Pytest will load the plugin automatically if Hypothesis is
893 installed. You don't need to do anything at all to use it.
894
896 Hypothesis tries to have good defaults for its behaviour, but sometimes
897 that's not enough and you need to tweak it.
898
899 The mechanism for doing this is the settings object. You can set up a
900 @given based test to use this using a settings decorator:
901
902 @given invocation is as follows:
903
904 from hypothesis import given, settings
905
906 @given(integers())
907 @settings(max_examples=500)
908 def test_this_thoroughly(x):
909 pass
910
911 This uses a settings object which causes the test to receive a much
912 larger set of examples than normal.
913
914 This may be applied either before or after the given and the results
915 are the same. The following is exactly equivalent:
916
917 from hypothesis import given, settings
918
919 @settings(max_examples=500)
920 @given(integers())
921 def test_this_thoroughly(x):
922 pass
923
924 Available settings
925 class hypothesis.settings(parent=None, **kwargs)
926 A settings object controls a variety of parameters that are used
927 in falsification. These may control both the falsification
928 strategy and the details of the data that is generated.
929
930 Default values are picked up from the settings.default object
931 and changes made there will be picked up in newly created set‐
932 tings.
933
934 buffer_size
935 The size of the underlying data used to generate exam‐
936 ples. If you need to generate really large examples you
937 may want to increase this, but it will make your tests
938 slower.
939
940 default value: 8192
941
942 database
943 An instance of hypothesis.database.ExampleDatabase that
944 will be used to save examples to and load previous exam‐
945 ples from. May be None in which case no storage will be
946 used, :memory: for an in-memory database, or any path for
947 a directory-based example database.
948
949 default value: (dynamically calculated)
950
951 database_file
952 The file or directory location to save and load previ‐
953 ously tried examples; :memory: for an in-memory cache or
954 None to disable caching entirely.
955
956 default value: (dynamically calculated)
957
958 The database_file setting is deprecated in favor of the
959 database setting, and will be removed in a future ver‐
960 sion. It only exists at all for complicated historical
961 reasons and you should just use database instead.
962
963 deadline
964 If set, a time in milliseconds (which may be a float to
965 express smaller units of time) that each individual exam‐
966 ple (i.e. each time your test function is called, not the
967 whole decorated test) within a test is not allowed to
968 exceed. Tests which take longer than that may be con‐
969 verted into errors (but will not necessarily be if close
970 to the deadline, to allow some variability in test run
971 time).
972
973 Set this to None to disable this behaviour entirely.
974
975 In future this will default to 200. For now, a Hypothe‐
976 sisDeprecationWarning will be emitted if you exceed that
977 default deadline and have not explicitly set a deadline
978 yourself.
979
980 default value: not_set
981
982 derandomize
983 If this is True then hypothesis will run in deterministic
984 mode where each falsification uses a random number gener‐
985 ator that is seeded based on the hypothesis to falsify,
986 which will be consistent across multiple runs. This has
987 the advantage that it will eliminate any randomness from
988 your tests, which may be preferable for some situations.
989 It does have the disadvantage of making your tests less
990 likely to find novel breakages.
991
992 default value: False
993
994 max_examples
995 Once this many satisfying examples have been considered
996 without finding any counter-example, falsification will
997 terminate.
998
999 default value: 100
1000
1001 max_iterations
1002 This doesn't actually do anything, but remains for com‐
1003 patibility reasons.
1004
1005 default value: not_set
1006
1007 The max_iterations setting has been disabled, as internal
1008 heuristics are more useful for this purpose than a user
1009 setting. It no longer has any effect.
1010
1011 max_shrinks
1012 Passing max_shrinks=0 disables the shrinking phase (see
1013 the phases setting), but any other value has no effect
1014 and uses a general heuristic.
1015
1016 default value: not_set
1017
1018 The max_shrinks setting has been disabled, as internal
1019 heuristics are more useful for this purpose than a user
1020 setting.
1021
1022 min_satisfying_examples
1023 This doesn't actually do anything, but remains for com‐
1024 patibility reasons.
1025
1026 default value: not_set
1027
1028 The min_satisfying_examples setting has been deprecated
1029 and disabled, due to overlap with the filter_too_much
1030 healthcheck and poor interaction with the max_examples
1031 setting.
1032
1033 perform_health_check
1034 If set to True, Hypothesis will run a preliminary health
1035 check before attempting to actually execute your test.
1036
1037 default value: not_set
1038
1039 This setting is deprecated, as perform_health_check=False
1040 duplicates the effect of sup‐
1041 press_health_check=HealthCheck.all(). Use that instead!
1042
1043 phases Control which phases should be run. See the full documen‐
1044 tation for more details
1045
1046 default value: (<Phase.explicit: 0>, <Phase.reuse: 1>,
1047 <Phase.generate: 2>, <Phase.shrink: 3>)
1048
1049 print_blob
1050 Determines whether to print blobs after tests that can be
1051 used to reproduce failures.
1052
1053 See the documentation on @reproduce_failure for more
1054 details of this behaviour.
1055
1056 default value: <PrintSettings.INFER: 1>
1057
1058 stateful_step_count
1059 Number of steps to run a stateful program for before giv‐
1060 ing up on it breaking.
1061
1062 default value: 50
1063
1064 strict Strict mode has been deprecated in favor of Python's
1065 standard warnings controls. Ironically, enabling it is
1066 therefore an error - it only exists so that users get the
1067 right type of error!
1068
1069 default value: False
1070
1071 Strict mode is deprecated and will go away in a future
1072 version of Hypothesis. To get the same behaviour, use
1073 warnings.simplefilter('error', HypothesisDeprecationWarn‐
1074 ing).
1075
1076 suppress_health_check
1077 A list of health checks to disable.
1078
1079 default value: ()
1080
1081 timeout
1082 Once this many seconds have passed, falsify will termi‐
1083 nate even if it has not found many examples. This is a
1084 soft rather than a hard limit - Hypothesis won't e.g.
1085 interrupt execution of the called function to stop it. If
1086 this value is <= 0 then no timeout will be applied.
1087
1088 default value: 60
1089
1090 The timeout setting is deprecated and will be removed in
1091 a future version of Hypothesis. To get the future behav‐
1092 iour set timeout=hypothesis.unlimited instead (which will
1093 remain valid for a further deprecation period after this
1094 setting has gone away).
1095
1096 use_coverage
1097 Whether to use coverage information to improve Hypothe‐
1098 sis's ability to find bugs.
1099
1100 You should generally leave this turned on unless your
1101 code performs poorly when run under coverage. If you turn
1102 it off, please file a bug report or add a comment to an
1103 existing one about the problem that prompted you to do
1104 so.
1105
1106 default value: True
1107
1108 verbosity
1109 Control the verbosity level of Hypothesis messages
1110
1111 default value: Verbosity.normal
1112
1113 Controlling What Runs
1114 Hypothesis divides tests into four logically distinct phases:
1115
1116 1. Running explicit examples provided with the @example decorator.
1117
1118 2. Rerunning a selection of previously failing examples to reproduce a
1119 previously seen error
1120
1121 3. Generating new examples.
1122
1123 4. Attempting to shrink an example found in phases 2 or 3 to a more
1124 manageable one (explicit examples cannot be shrunk).
1125
1126 The phases setting provides you with fine grained control over which of
1127 these run, with each phase corresponding to a value on the Phase enum:
1128
1129 1. Phase.explicit controls whether explicit examples are run.
1130
1131 2. Phase.reuse controls whether previous examples will be reused.
1132
1133 3. Phase.generate controls whether new examples will be generated.
1134
1135 4. Phase.shrink controls whether examples will be shrunk.
1136
1137 The phases argument accepts a collection with any subset of these. e.g.
1138 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1139 ples and shrink them, but will not run explicit examples or reuse pre‐
1140 vious failures, while settings(phases=[Phase.explicit]) will only run
1141 the explicit examples.
1142
1143 Seeing intermediate result
1144 To see what's going on while Hypothesis runs your tests, you can turn
1145 up the verbosity setting. This works with both find() and @given.
1146
1147 >>> from hypothesis import find, settings, Verbosity
1148 >>> from hypothesis.strategies import lists, integers
1149 >>> find(lists(integers()), any, settings=settings(verbosity=Verbosity.verbose))
1150 Tried non-satisfying example []
1151 Found satisfying example [-1198601713, -67, 116, -29578]
1152 Shrunk example to [-67, 116, -29578]
1153 Shrunk example to [116, -29578]
1154 Shrunk example to [-29578]
1155 Shrunk example to [-115]
1156 Shrunk example to [115]
1157 Shrunk example to [-57]
1158 Shrunk example to [29]
1159 Shrunk example to [-14]
1160 Shrunk example to [-7]
1161 Shrunk example to [4]
1162 Shrunk example to [2]
1163 Shrunk example to [1]
1164 [1]
1165
1166 The four levels are quiet, normal, verbose and debug. normal is the
1167 default, while in quiet mode Hypothesis will not print anything out,
1168 not even the final falsifying example. debug is basically verbose but a
1169 bit more so. You probably don't want it.
1170
1171 If you are using pytest, you may also need to disable output capturing
1172 for passing tests.
1173
1174 Building settings objects
1175 Settings can be created by calling settings with any of the available
1176 settings values. Any absent ones will be set to defaults:
1177
1178 >>> from hypothesis import settings
1179 >>> settings().max_examples
1180 100
1181 >>> settings(max_examples=10).max_examples
1182 10
1183
1184 You can also pass a 'parent' settings object as the first argument, and
1185 any settings you do not specify as keyword arguments will be copied
1186 from the parent settings:
1187
1188 >>> parent = settings(max_examples=10)
1189 >>> child = settings(parent, deadline=200)
1190 >>> parent.max_examples == child.max_examples == 10
1191 True
1192 >>> parent.deadline
1193 not_set
1194 >>> child.deadline
1195 200
1196
1197 Default settings
1198 At any given point in your program there is a current default settings,
1199 available as settings.default. As well as being a settings object in
1200 its own right, all newly created settings objects which are not explic‐
1201 itly based off another settings are based off the default, so will
1202 inherit any values that are not explicitly set from it.
1203
1204 You can change the defaults by using profiles.
1205
1206 settings Profiles
1207 Depending on your environment you may want different default settings.
1208 For example: during development you may want to lower the number of
1209 examples to speed up the tests. However, in a CI environment you may
1210 want more examples so you are more likely to find bugs.
1211
1212 Hypothesis allows you to define different settings profiles. These pro‐
1213 files can be loaded at any time.
1214
1215 Loading a profile changes the default settings but will not change the
1216 behavior of tests that explicitly change the settings.
1217
1218 >>> from hypothesis import settings
1219 >>> settings.register_profile("ci", max_examples=1000)
1220 >>> settings().max_examples
1221 100
1222 >>> settings.load_profile("ci")
1223 >>> settings().max_examples
1224 1000
1225
1226 Instead of loading the profile and overriding the defaults you can
1227 retrieve profiles for specific tests.
1228
1229 >>> with settings.get_profile("ci"):
1230 ... print(settings().max_examples)
1231 ...
1232 1000
1233
1234 Optionally, you may define the environment variable to load a profile
1235 for you. This is the suggested pattern for running your tests on CI.
1236 The code below should run in a conftest.py or any setup/initialization
1237 section of your test suite. If this variable is not defined the
1238 Hypothesis defined defaults will be loaded.
1239
1240 >>> import os
1241 >>> from hypothesis import settings, Verbosity
1242 >>> settings.register_profile("ci", max_examples=1000)
1243 >>> settings.register_profile("dev", max_examples=10)
1244 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1245 >>> settings.load_profile(os.getenv(u'HYPOTHESIS_PROFILE', 'default'))
1246
1247 If you are using the hypothesis pytest plugin and your profiles are
1248 registered by your conftest you can load one with the command line
1249 option --hypothesis-profile.
1250
1251 $ pytest tests --hypothesis-profile <profile-name>
1252
1253 Timeouts
1254 The timeout functionality of Hypothesis is being deprecated, and will
1255 eventually be removed. For the moment, the timeout setting can still be
1256 set and the old default timeout of one minute remains.
1257
1258 If you want to future proof your code you can get the future behaviour
1259 by setting it to the value hypothesis.unlimited.
1260
1261 from hypothesis import given, settings, unlimited
1262 from hypothesis import strategies as st
1263
1264 @settings(timeout=unlimited)
1265 @given(st.integers())
1266 def test_something_slow(i):
1267 ...
1268
1269 This will cause your code to run until it hits the normal Hypothesis
1270 example limits, regardless of how long it takes. timeout=unlimited will
1271 remain a valid setting after the timeout functionality has been depre‐
1272 cated (but will then have its own deprecation cycle).
1273
1274 There is however now a timing related health check which is designed to
1275 catch tests that run for ages by accident. If you really want your test
1276 to run forever, the following code will enable that:
1277
1278 from hypothesis import given, settings, unlimited, HealthCheck
1279 from hypothesis import strategies as st
1280
1281 @settings(timeout=unlimited, suppress_health_check=[
1282 HealthCheck.hung_test
1283 ])
1284 @given(st.integers())
1285 def test_something_slow(i):
1286 ...
1287
1289 Most things should be easy to generate and everything should be possi‐
1290 ble.
1291
1292 To support this principle Hypothesis provides strategies for most
1293 built-in types with arguments to constrain or adjust the output, as
1294 well as higher-order strategies that can be composed to generate more
1295 complex types.
1296
1297 This document is a guide to what strategies are available for generat‐
1298 ing data and how to build them. Strategies have a variety of other
1299 important internal features, such as how they simplify, but the data
1300 they can generate is the only public part of their API.
1301
1302 Functions for building strategies are all available in the hypothe‐
1303 sis.strategies module. The salient functions from it are as follows:
1304
1305 hypothesis.strategies.nothing()
1306 This strategy never successfully draws a value and will always
1307 reject on an attempt to draw.
1308
1309 Examples from this strategy do not shrink (because there are
1310 none).
1311
1312 hypothesis.strategies.just(value)
1313 Return a strategy which only generates value.
1314
1315 Note: value is not copied. Be wary of using mutable values.
1316
1317 If value is the result of a callable, you can use
1318 builds(callable) instead of just(callable()) to get a fresh
1319 value each time.
1320
1321 Examples from this strategy do not shrink (because there is only
1322 one).
1323
1324 hypothesis.strategies.none()
1325 Return a strategy which only generates None.
1326
1327 Examples from this strategy do not shrink (because there is only
1328 one).
1329
1330 hypothesis.strategies.one_of(*args)
1331 Return a strategy which generates values from any of the argu‐
1332 ment strategies.
1333
1334 This may be called with one iterable argument instead of multi‐
1335 ple strategy arguments. In which case one_of(x) and one_of(*x)
1336 are equivalent.
1337
1338 Examples from this strategy will generally shrink to ones that
1339 come from strategies earlier in the list, then shrink according
1340 to behaviour of the strategy that produced them. In order to get
1341 good shrinking behaviour, try to put simpler strategies first.
1342 e.g. one_of(none(), text()) is better than one_of(text(),
1343 none()).
1344
1345 This is especially important when using recursive strategies.
1346 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1347 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1348 st.none()) will shrink very badly indeed.
1349
1350 hypothesis.strategies.integers(min_value=None, max_value=None)
1351 Returns a strategy which generates integers (in Python 2 these
1352 may be ints or longs).
1353
1354 If min_value is not None then all values will be >= min_value.
1355 If max_value is not None then all values will be <= max_value
1356
1357 Examples from this strategy will shrink towards zero, and nega‐
1358 tive values will also shrink towards positive (i.e. -n may be
1359 replaced by +n).
1360
1361 hypothesis.strategies.booleans()
1362 Returns a strategy which generates instances of bool.
1363
1364 Examples from this strategy will shrink towards False (i.e.
1365 shrinking will try to replace True with False where possible).
1366
1367 hypothesis.strategies.floats(min_value=None, max_value=None,
1368 allow_nan=None, allow_infinity=None)
1369 Returns a strategy which generates floats.
1370
1371 · If min_value is not None, all values will be >= min_value.
1372
1373 · If max_value is not None, all values will be <= max_value.
1374
1375 · If min_value or max_value is not None, it is an error to
1376 enable allow_nan.
1377
1378 · If both min_value and max_value are not None, it is an error
1379 to enable allow_infinity.
1380
1381 Where not explicitly ruled out by the bounds, all of infinity,
1382 -infinity and NaN are possible values generated by this strat‐
1383 egy.
1384
1385 Examples from this strategy have a complicated and hard to
1386 explain shrinking behaviour, but it tries to improve "human
1387 readability". Finite numbers will be preferred to infinity and
1388 infinity will be preferred to NaN.
1389
1390 hypothesis.strategies.tuples(*args)
1391 Return a strategy which generates a tuple of the same length as
1392 args by generating the value at index i from args[i].
1393
1394 e.g. tuples(integers(), integers()) would generate a tuple of
1395 length two with both values an integer.
1396
1397 Examples from this strategy shrink by shrinking their component
1398 parts.
1399
1400 hypothesis.strategies.sampled_from(elements)
1401 Returns a strategy which generates any value present in ele‐
1402 ments.
1403
1404 Note that as with just(), values will not be copied and thus you
1405 should be careful of using mutable data.
1406
1407 sampled_from supports ordered collections, as well as Enum
1408 objects. Flag objects may also generate any combination of
1409 their members.
1410
1411 Examples from this strategy shrink by replacing them with values
1412 earlier in the list. So e.g. sampled_from((10, 1)) will shrink
1413 by trying to replace 1 values with 10, and sampled_from((1, 10))
1414 will shrink by trying to replace 10 values with 1.
1415
1416 hypothesis.strategies.lists(elements=None, min_size=None, aver‐
1417 age_size=None, max_size=None, unique_by=None, unique=False)
1418 Returns a list containing values drawn from elements with length
1419 in the interval [min_size, max_size] (no bounds in that direc‐
1420 tion if these are None). If max_size is 0 then elements may be
1421 None and only the empty list will be drawn.
1422
1423 The average_size argument is deprecated. Internal upgrades
1424 since Hypothesis 1.x mean we no longer needed this hint to gen‐
1425 erate useful data.
1426
1427 If unique is True (or something that evaluates to True), we com‐
1428 pare direct object equality, as if unique_by was lambda x: x.
1429 This comparison only works for hashable types.
1430
1431 if unique_by is not None it must be a function returning a hash‐
1432 able type when given a value drawn from elements. The resulting
1433 list will satisfy the condition that for i != j,
1434 unique_by(result[i]) != unique_by(result[j]).
1435
1436 Examples from this strategy shrink by trying to remove elements
1437 from the list, and by shrinking each individual element of the
1438 list.
1439
1440 hypothesis.strategies.sets(elements=None, min_size=None, aver‐
1441 age_size=None, max_size=None)
1442 This has the same behaviour as lists, but returns sets instead.
1443
1444 Note that Hypothesis cannot tell if values are drawn from ele‐
1445 ments are hashable until running the test, so you can define a
1446 strategy for sets of an unhashable type but it will fail at test
1447 time.
1448
1449 Examples from this strategy shrink by trying to remove elements
1450 from the set, and by shrinking each individual element of the
1451 set.
1452
1453 hypothesis.strategies.frozensets(elements=None, min_size=None, aver‐
1454 age_size=None, max_size=None)
1455 This is identical to the sets function but instead returns
1456 frozensets.
1457
1458 hypothesis.strategies.iterables(elements=None, min_size=None, aver‐
1459 age_size=None, max_size=None, unique_by=None, unique=False)
1460 This has the same behaviour as lists, but returns iterables
1461 instead.
1462
1463 Some iterables cannot be indexed (e.g. sets) and some do not
1464 have a fixed length (e.g. generators). This strategy produces
1465 iterators, which cannot be indexed and do not have a fixed
1466 length. This ensures that you do not accidentally depend on
1467 sequence behaviour.
1468
1469 hypothesis.strategies.fixed_dictionaries(mapping)
1470 Generates a dictionary of the same type as mapping with a fixed
1471 set of keys mapping to strategies. mapping must be a dict sub‐
1472 class.
1473
1474 Generated values have all keys present in mapping, with the cor‐
1475 responding values drawn from mapping[key]. If mapping is an
1476 instance of OrderedDict the keys will also be in the same order,
1477 otherwise the order is arbitrary.
1478
1479 Examples from this strategy shrink by shrinking each individual
1480 value in the generated dictionary.
1481
1482 hypothesis.strategies.dictionaries(keys, values, dict_class=<type
1483 'dict'>, min_size=None, average_size=None, max_size=None)
1484 Generates dictionaries of type dict_class with keys drawn from
1485 the keys argument and values drawn from the values argument.
1486
1487 The size parameters have the same interpretation as for lists.
1488
1489 Examples from this strategy shrink by trying to remove keys from
1490 the generated dictionary, and by shrinking each generated key
1491 and value.
1492
1493 hypothesis.strategies.streaming(elements)
1494 Generates an infinite stream of values where each value is drawn
1495 from elements.
1496
1497 The result is iterable (the iterator will never terminate) and
1498 indexable.
1499
1500 Examples from this strategy shrink by trying to shrink each
1501 value drawn.
1502
1503 Deprecated since version 3.15.0: Use data() instead.
1504
1505
1506 hypothesis.strategies.characters(whitelist_categories=None, black‐
1507 list_categories=None, blacklist_characters=None, min_codepoint=None,
1508 max_codepoint=None, whitelist_characters=None)
1509 Generates unicode text type (unicode on python 2, str on python
1510 3) characters following specified filtering rules.
1511
1512 · When no filtering rules are specifed, any character can be
1513 produced.
1514
1515 · If min_codepoint or max_codepoint is specifed, then only char‐
1516 acters having a codepoint in that range will be produced.
1517
1518 · If whitelist_categories is specified, then only characters
1519 from those Unicode categories will be produced. This is a fur‐
1520 ther restriction, characters must also satisfy min_codepoint
1521 and max_codepoint.
1522
1523 · If blacklist_categories is specified, then any character from
1524 those categories will not be produced. Any overlap between
1525 whitelist_categories and blacklist_categories will raise an
1526 exception, as each character can only belong to a single
1527 class.
1528
1529 · If whitelist_characters is specified, then any additional
1530 characters in that list will also be produced.
1531
1532 · If blacklist_characters is specified, then any characters in
1533 that list will be not be produced. Any overlap between
1534 whitelist_characters and blacklist_characters will raise an
1535 exception.
1536
1537 The _codepoint arguments must be integers between zero and
1538 python:sys.max_unicode. The _characters arguments must be col‐
1539 lections of length-one unicode strings, such as a unicode
1540 string.
1541
1542 The _categories arguments must be used to specify either the
1543 one-letter Unicode major category or the two-letter Unicode
1544 general category. For example, ('Nd', 'Lu') signifies "Number,
1545 decimal digit" and "Letter, uppercase". A single letter ('major
1546 category') can be given to match all corresponding categories,
1547 for example 'P' for characters in any punctuation category.
1548
1549 Examples from this strategy shrink towards the codepoint for
1550 '0', or the first allowable codepoint after it if '0' is
1551 excluded.
1552
1553 hypothesis.strategies.text(alphabet=None, min_size=None, aver‐
1554 age_size=None, max_size=None)
1555 Generates values of a unicode text type (unicode on python 2,
1556 str on python 3) with values drawn from alphabet, which should
1557 be an iterable of length one strings or a strategy generating
1558 such. If it is None it will default to generating the full uni‐
1559 code range (excluding surrogate characters). If it is an empty
1560 collection this will only generate empty strings.
1561
1562 min_size and max_size have the usual interpretations.
1563
1564 The average_size argument is deprecated. Internal upgrades
1565 since Hypothesis 1.x mean we no longer needed this hint to gen‐
1566 erate useful data.
1567
1568 Examples from this strategy shrink towards shorter strings, and
1569 with the characters in the text shrinking as per the alphabet
1570 strategy.
1571
1572 hypothesis.strategies.from_regex(regex)
1573 Generates strings that contain a match for the given regex (i.e.
1574 ones for which re.search() will return a non-None result).
1575
1576 regex may be a pattern or compiled regex. Both byte-strings and
1577 unicode strings are supported, and will generate examples of the
1578 same type.
1579
1580 You can use regex flags such as re.IGNORECASE, re.DOTALL or
1581 re.UNICODE to control generation. Flags can be passed either in
1582 compiled regex or inside the pattern with a (?iLmsux) group.
1583
1584 Some regular expressions are only partly supported - the under‐
1585 lying strategy checks local matching and relies on filtering to
1586 resolve context-dependent expressions. Using too many of these
1587 constructs may cause health-check errors as too many examples
1588 are filtered out. This mainly includes (positive or negative)
1589 lookahead and lookbehind groups.
1590
1591 If you want the generated string to match the whole regex you
1592 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1593 gle character string, while "." will return any string, and
1594 r"\A.$" will return a single character optionally followed by a
1595 "\n".
1596
1597 Examples from this strategy shrink towards shorter strings and
1598 lower character values.
1599
1600 hypothesis.strategies.binary(min_size=None, average_size=None,
1601 max_size=None)
1602 Generates the appropriate binary type (str in python 2, bytes in
1603 python 3).
1604
1605 min_size and max_size have the usual interpretations.
1606
1607 The average_size argument is deprecated. Internal upgrades
1608 since Hypothesis 1.x mean we no longer needed this hint to gen‐
1609 erate useful data.
1610
1611 Examples from this strategy shrink towards smaller strings and
1612 lower byte values.
1613
1614 hypothesis.strategies.randoms()
1615 Generates instances of Random (actually a Hypothesis specific
1616 RandomWithSeed class which displays what it was initially seeded
1617 with)
1618
1619 Examples from this strategy shrink to seeds closer to zero.
1620
1621 hypothesis.strategies.random_module()
1622 If your code depends on the global random module then you need
1623 to use this.
1624
1625 It will explicitly seed the random module at the start of your
1626 test so that tests are reproducible. The value it passes you is
1627 an opaque object whose only useful feature is that its repr dis‐
1628 plays the random seed. It is not itself a random number genera‐
1629 tor. If you want a random number generator you should use the
1630 randoms() strategy which will give you one.
1631
1632 Examples from these strategy shrink to seeds closer to zero.
1633
1634 hypothesis.strategies.builds(*callable_and_args, **kwargs)
1635 Generates values by drawing from args and kwargs and passing
1636 them to the callable (provided as the first positional argument)
1637 in the appropriate argument position.
1638
1639 e.g. builds(target, integers(), flag=booleans()) would draw an
1640 integer i and a boolean b and call target(i, flag=b).
1641
1642 If the callable has type annotations, they will be used to infer
1643 a strategy for required arguments that were not passed to
1644 builds. You can also tell builds to infer a strategy for an
1645 optional argument by passing the special value hypothesis.infer
1646 as a keyword argument to builds, instead of a strategy for that
1647 argument to the callable.
1648
1649 If the callable is a class defined with attrs, missing required
1650 arguments will be inferred from the attribute on a best-effort
1651 basis, e.g. by checking attrs standard validators.
1652
1653 Examples from this strategy shrink by shrinking the argument
1654 values to the callable.
1655
1656 hypothesis.strategies.from_type(thing)
1657 Looks up the appropriate search strategy for the given type.
1658
1659 from_type is used internally to fill in missing arguments to
1660 builds() and can be used interactively to explore what strate‐
1661 gies are available or to debug type resolution.
1662
1663 You can use register_type_strategy() to handle your custom
1664 types, or to globally redefine certain strategies - for example
1665 excluding NaN from floats, or use timezone-aware instead of
1666 naive time and datetime strategies.
1667
1668 The resolution logic may be changed in a future version, but
1669 currently tries these four options:
1670
1671 1. If thing is in the default lookup mapping or user-registered
1672 lookup, return the corresponding strategy. The default
1673 lookup covers all types with Hypothesis strategies, including
1674 extras where possible.
1675
1676 2. If thing is from the python:typing module, return the corre‐
1677 sponding strategy (special logic).
1678
1679 3. If thing has one or more subtypes in the merged lookup,
1680 return the union of the strategies for those types that are
1681 not subtypes of other elements in the lookup.
1682
1683 4. Finally, if thing has type annotations for all required argu‐
1684 ments, it is resolved via builds().
1685
1686 hypothesis.strategies.fractions(min_value=None, max_value=None,
1687 max_denominator=None)
1688 Returns a strategy which generates Fractions.
1689
1690 If min_value is not None then all generated values are no less
1691 than min_value. If max_value is not None then all generated
1692 values are no greater than max_value. min_value and max_value
1693 may be anything accepted by the Fraction constructor.
1694
1695 If max_denominator is not None then the denominator of any gen‐
1696 erated values is no greater than max_denominator. Note that
1697 max_denominator must be None or a positive integer.
1698
1699 Examples from this strategy shrink towards smaller denominators,
1700 then closer to zero.
1701
1702 hypothesis.strategies.decimals(min_value=None, max_value=None,
1703 allow_nan=None, allow_infinity=None, places=None)
1704 Generates instances of decimals.Decimal, which may be:
1705
1706 · A finite rational number, between min_value and max_value.
1707
1708 · Not a Number, if allow_nan is True. None means "allow NaN,
1709 unless min_value and max_value are not None".
1710
1711 · Positive or negative infinity, if max_value and min_value
1712 respectively are None, and allow_infinity is not False. None
1713 means "allow infinity, unless excluded by the min and max val‐
1714 ues".
1715
1716 Note that where floats have one NaN value, Decimals have four:
1717 signed, and either quiet or signalling. See the decimal module
1718 docs for more information on special values.
1719
1720 If places is not None, all finite values drawn from the strategy
1721 will have that number of digits after the decimal place.
1722
1723 Examples from this strategy do not have a well defined shrink
1724 order but try to maximize human readability when shrinking.
1725
1726 hypothesis.strategies.recursive(base, extend, max_leaves=100)
1727 base: A strategy to start from.
1728
1729 extend: A function which takes a strategy and returns a new
1730 strategy.
1731
1732 max_leaves: The maximum number of elements to be drawn from base
1733 on a given run.
1734
1735 This returns a strategy S such that S = extend(base | S). That
1736 is, values may be drawn from base, or from any strategy reach‐
1737 able by mixing applications of | and extend.
1738
1739 An example may clarify: recursive(booleans(), lists) would
1740 return a strategy that may return arbitrarily nested and mixed
1741 lists of booleans. So e.g. False, [True], [False, []], and
1742 [[[[True]]]] are all valid values to be drawn from that strat‐
1743 egy.
1744
1745 Examples from this strategy shrink by trying to reduce the
1746 amount of recursion and by shrinking according to the shrinking
1747 behaviour of base and the result of extend.
1748
1749 hypothesis.strategies.permutations(values)
1750 Return a strategy which returns permutations of the ordered col‐
1751 lection values.
1752
1753 Examples from this strategy shrink by trying to become closer to
1754 the original order of values.
1755
1756 hypothesis.strategies.datetimes(min_value=datetime.datetime(1, 1, 1, 0,
1757 0), max_value=datetime.datetime(9999, 12, 31, 23, 59, 59, 999999),
1758 timezones=none(), min_datetime=None, max_datetime=None)
1759 A strategy for generating datetimes, which may be time‐
1760 zone-aware.
1761
1762 This strategy works by drawing a naive datetime between
1763 min_datetime and max_datetime, which must both be naive (have no
1764 timezone).
1765
1766 timezones must be a strategy that generates tzinfo objects (or
1767 None, which is valid for naive datetimes). A value drawn from
1768 this strategy will be added to a naive datetime, and the result‐
1769 ing tz-aware datetime returned.
1770
1771 NOTE:
1772 tz-aware datetimes from this strategy may be ambiguous or
1773 non-existent due to daylight savings, leap seconds, timezone
1774 and calendar adjustments, etc. This is intentional, as mal‐
1775 formed timestamps are a common source of bugs.
1776
1777 hypothesis.extra.pytz.timezones() requires the pytz package, but
1778 provides all timezones in the Olsen database. If you want to
1779 allow naive datetimes, combine strategies like none() | time‐
1780 zones().
1781
1782 hypothesis.extra.dateutil.timezones() requires the
1783 python-dateutil package, and similarly provides all timezones
1784 there.
1785
1786 Alternatively, you can create a list of the timezones you wish
1787 to allow (e.g. from the standard library, datetutil, or pytz)
1788 and use sampled_from(). Ensure that simple values such as None
1789 or UTC are at the beginning of the list for proper minimisation.
1790
1791 Examples from this strategy shrink towards midnight on January
1792 1st 2000.
1793
1794 The following arguments have been renamed:
1795
1796 · min_datetime has been renamed to min_value
1797
1798 · max_datetime has been renamed to max_value
1799
1800 Use of the old names has been deprecated and will be removed in
1801 a future version of Hypothesis.
1802
1803 hypothesis.strategies.dates(min_value=datetime.date(1, 1, 1),
1804 max_value=datetime.date(9999, 12, 31), min_date=None, max_date=None)
1805 A strategy for dates between min_date and max_date.
1806
1807 Examples from this strategy shrink towards January 1st 2000.
1808
1809 The following arguments have been renamed:
1810
1811 · max_date has been renamed to max_value
1812
1813 · min_date has been renamed to min_value
1814
1815 Use of the old names has been deprecated and will be removed in
1816 a future version of Hypothesis.
1817
1818 hypothesis.strategies.times(min_value=datetime.time(0, 0),
1819 max_value=datetime.time(23, 59, 59, 999999), timezones=none(),
1820 min_time=None, max_time=None)
1821 A strategy for times between min_time and max_time.
1822
1823 The timezones argument is handled as for datetimes().
1824
1825 Examples from this strategy shrink towards midnight, with the
1826 timezone component shrinking as for the strategy that provided
1827 it.
1828
1829 The following arguments have been renamed:
1830
1831 · min_time has been renamed to min_value
1832
1833 · max_time has been renamed to max_value
1834
1835 Use of the old names has been deprecated and will be removed in
1836 a future version of Hypothesis.
1837
1838 hypothesis.strategies.timedeltas(min_value=date‐
1839 time.timedelta(-999999999), max_value=datetime.timedelta(999999999,
1840 86399, 999999), min_delta=None, max_delta=None)
1841 A strategy for timedeltas between min_value and max_value.
1842
1843 Examples from this strategy shrink towards zero.
1844
1845 The following arguments have been renamed:
1846
1847 · max_delta has been renamed to max_value
1848
1849 · min_delta has been renamed to min_value
1850
1851 Use of the old names has been deprecated and will be removed in
1852 a future version of Hypothesis.
1853
1854 hypothesis.strategies.composite(f)
1855 Defines a strategy that is built out of potentially arbitrarily
1856 many other strategies.
1857
1858 This is intended to be used as a decorator. See the full docu‐
1859 mentation for more details about how to use this function.
1860
1861 Examples from this strategy shrink by shrinking the output of
1862 each draw call.
1863
1864 hypothesis.strategies.complex_numbers(min_magnitude=0, max_magni‐
1865 tude=None, allow_infinity=None, allow_nan=None)
1866 Returns a strategy that generates complex numbers.
1867
1868 This strategy draws complex numbers with constrained magnitudes.
1869 The min_magnitude and max_magnitude parameters should be
1870 non-negative Real numbers; values of None correspond to zero and
1871 infinite values respectively.
1872
1873 If min_magnitude is positive or max_magnitude is finite, it is
1874 an error to enable allow_nan. If max_magnitude is finite, it is
1875 an error to enable allow_infinity.
1876
1877 The magnitude contraints are respected up to a relative error of
1878 (around) floating-point epsilon, due to implementation via the
1879 system sqrt function.
1880
1881 Examples from this strategy shrink by shrinking their real and
1882 imaginary parts, as floats().
1883
1884 If you need to generate complex numbers with particular real and
1885 imaginary parts or relationships between parts, consider using
1886 builds(complex, ...) <hypothesis.strategies.builds> or @compos‐
1887 ite <hypothesis.strategies.composite> respectively.
1888
1889 hypothesis.strategies.shared(base, key=None)
1890 Returns a strategy that draws a single shared value per run,
1891 drawn from base. Any two shared instances with the same key will
1892 share the same value, otherwise the identity of this strategy
1893 will be used. That is:
1894
1895 >>> s = integers() # or any other strategy
1896 >>> x = shared(s)
1897 >>> y = shared(s)
1898
1899 In the above x and y may draw different (or potentially the
1900 same) values. In the following they will always draw the same:
1901
1902 >>> x = shared(s, key="hi")
1903 >>> y = shared(s, key="hi")
1904
1905 Examples from this strategy shrink as per their base strategy.
1906
1907 hypothesis.strategies.choices()
1908 Strategy that generates a function that behaves like ran‐
1909 dom.choice.
1910
1911 Will note choices made for reproducibility.
1912
1913 Deprecated since version 3.15.0: Use data() with sampled_from()
1914 instead.
1915
1916
1917 Examples from this strategy shrink by making each choice func‐
1918 tion return an earlier value in the sequence passed to it.
1919
1920 hypothesis.strategies.uuids(version=None)
1921 Returns a strategy that generates UUIDs.
1922
1923 If the optional version argument is given, value is passed
1924 through to UUID and only UUIDs of that version will be gener‐
1925 ated.
1926
1927 All returned values from this will be unique, so e.g. if you do
1928 lists(uuids()) the resulting list will never contain duplicates.
1929
1930 Examples from this strategy don't have any meaningful shrink
1931 order.
1932
1933 hypothesis.strategies.runner(default=not_set)
1934 A strategy for getting "the current test runner", whatever that
1935 may be. The exact meaning depends on the entry point, but it
1936 will usually be the associated 'self' value for it.
1937
1938 If there is no current test runner and a default is provided,
1939 return that default. If no default is provided, raises Invali‐
1940 dArgument.
1941
1942 Examples from this strategy do not shrink (because there is only
1943 one).
1944
1945 hypothesis.strategies.data()
1946 This isn't really a normal strategy, but instead gives you an
1947 object which can be used to draw data interactively from other
1948 strategies.
1949
1950 It can only be used within @given, not find(). This is because
1951 the lifetime of the object cannot outlast the test body.
1952
1953 See the rest of the documentation for more complete information.
1954
1955 Examples from this strategy do not shrink (because there is only
1956 one), but the result of calls to each draw() call shrink as they
1957 normally would.
1958
1959 hypothesis.strategies.register_type_strategy(custom_type, strategy)
1960 Add an entry to the global type-to-strategy lookup.
1961
1962 This lookup is used in builds() and @given.
1963
1964 builds() will be used automatically for classes with type anno‐
1965 tations on __init__ , so you only need to register a strategy if
1966 one or more arguments need to be more tightly defined than their
1967 type-based default, or if you want to supply a strategy for an
1968 argument with a default value.
1969
1970 strategy may be a search strategy, or a function that takes a
1971 type and returns a strategy (useful for generic types).
1972
1973 hypothesis.strategies.deferred(definition)
1974 A deferred strategy allows you to write a strategy that refer‐
1975 ences other strategies that have not yet been defined. This
1976 allows for the easy definition of recursive and mutually recur‐
1977 sive strategies.
1978
1979 The definition argument should be a zero-argument function that
1980 returns a strategy. It will be evaluated the first time the
1981 strategy is used to produce an example.
1982
1983 Example usage:
1984
1985 >>> import hypothesis.strategies as st
1986 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1987 >>> x.example()
1988 (((False, (True, True)), (False, True)), (True, True))
1989 >>> x.example()
1990 True
1991
1992 Mutual recursion also works fine:
1993
1994 >>> a = st.deferred(lambda: st.booleans() | b)
1995 >>> b = st.deferred(lambda: st.tuples(a, a))
1996 >>> a.example()
1997 True
1998 >>> b.example()
1999 (False, (False, ((False, True), False)))
2000
2001 Examples from this strategy shrink as they normally would from
2002 the strategy returned by the definition.
2003
2004 hypothesis.strategies.emails()
2005 A strategy for generating email addresses as unicode strings.
2006 The address format is specific in RFC 5322#section-3.4.1. Values
2007 shrink towards shorter local-parts and host domains.
2008
2009 This strategy is useful for generating "user data" for tests, as
2010 mishandling of email addresses is a common source of bugs.
2011 Future updates will generate more complicated addresses allowed
2012 by the RFC.
2013
2014 Shrinking
2015 When using strategies it is worth thinking about how the data shrinks.
2016 Shrinking is the process by which Hypothesis tries to produce human
2017 readable examples when it finds a failure - it takes a complex example
2018 and turns it into a simpler one.
2019
2020 Each strategy defines an order in which it shrinks - you won't usually
2021 need to care about this much, but it can be worth being aware of as it
2022 can affect what the best way to write your own strategies is.
2023
2024 The exact shrinking behaviour is not a guaranteed part of the API, but
2025 it doesn't change that often and when it does it's usually because we
2026 think the new way produces nicer examples.
2027
2028 Possibly the most important one to be aware of is one_of(), which has a
2029 preference for values produced by strategies earlier in its argument
2030 list. Most of the others should largely "do the right thing" without
2031 you having to think about it.
2032
2033 Adapting strategies
2034 Often it is the case that a strategy doesn't produce exactly what you
2035 want it to and you need to adapt it. Sometimes you can do this in the
2036 test, but this hurts reuse because you then have to repeat the adaption
2037 in every test.
2038
2039 Hypothesis gives you ways to build strategies from other strategies
2040 given functions for transforming the data.
2041
2042 Mapping
2043 map is probably the easiest and most useful of these to use. If you
2044 have a strategy s and a function f, then an example s.map(f).example()
2045 is f(s.example()), i.e. we draw an example from s and then apply f to
2046 it.
2047
2048 e.g.:
2049
2050 >>> lists(integers()).map(sorted).example()
2051 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2052
2053 Note that many things that you might use mapping for can also be done
2054 with builds().
2055
2056 Filtering
2057 filter lets you reject some examples. s.filter(f).example() is some
2058 example of s such that f(example) is truthy.
2059
2060 >>> integers().filter(lambda x: x > 11).example()
2061 26126
2062 >>> integers().filter(lambda x: x > 11).example()
2063 23324
2064
2065 It's important to note that filter isn't magic and if your condition is
2066 too hard to satisfy then this can fail:
2067
2068 >>> integers().filter(lambda x: False).example()
2069 Traceback (most recent call last):
2070 ...
2071 hypothesis.errors.NoExamples: Could not find any valid examples in 20 tries
2072
2073 In general you should try to use filter only to avoid corner cases that
2074 you don't want rather than attempting to cut out a large chunk of the
2075 search space.
2076
2077 A technique that often works well here is to use map to first transform
2078 the data and then use filter to remove things that didn't work out. So
2079 for example if you wanted pairs of integers (x,y) such that x < y you
2080 could do the following:
2081
2082 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2083 [-8543729478746591815, 3760495307320535691]
2084
2085 Chaining strategies together
2086 Finally there is flatmap. flatmap draws an example, then turns that
2087 example into a strategy, then draws an example from that strategy.
2088
2089 It may not be obvious why you want this at first, but it turns out to
2090 be quite useful because it lets you generate different types of data
2091 with relationships to each other.
2092
2093 For example suppose we wanted to generate a list of lists of the same
2094 length:
2095
2096 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2097 ... lambda n: lists(lists(integers(), min_size=n, max_size=n)))
2098 >>> find(rectangle_lists, lambda x: True)
2099 []
2100 >>> find(rectangle_lists, lambda x: len(x) >= 10)
2101 [[], [], [], [], [], [], [], [], [], []]
2102 >>> find(rectangle_lists, lambda t: len(t) >= 3 and len(t[0]) >= 3)
2103 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2104 >>> find(rectangle_lists, lambda t: sum(len(s) for s in t) >= 10)
2105 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2106
2107 In this example we first choose a length for our tuples, then we build
2108 a strategy which generates lists containing lists precisely of that
2109 length. The finds show what simple examples for this look like.
2110
2111 Most of the time you probably don't want flatmap, but unlike filter and
2112 map which are just conveniences for things you could just do in your
2113 tests, flatmap allows genuinely new data generation that you wouldn't
2114 otherwise be able to easily do.
2115
2116 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2117 don't know Haskell, ignore everything in these parentheses. You do not
2118 need to understand anything about monads to use this, or anything else
2119 in Hypothesis).
2120
2121 Recursive data
2122 Sometimes the data you want to generate has a recursive definition.
2123 e.g. if you wanted to generate JSON data, valid JSON is:
2124
2125 1. Any float, any boolean, any unicode string.
2126
2127 2. Any list of valid JSON data
2128
2129 3. Any dictionary mapping unicode strings to valid JSON data.
2130
2131 The problem is that you cannot call a strategy recursively and expect
2132 it to not just blow up and eat all your memory. The other problem here
2133 is that not all unicode strings display consistently on different
2134 machines, so we'll restrict them in our doctest.
2135
2136 The way Hypothesis handles this is with the recursive() function which
2137 you pass in a base case and a function that, given a strategy for your
2138 data type, returns a new strategy for it. So for example:
2139
2140 >>> from string import printable; from pprint import pprint
2141 >>> json = recursive(none() | booleans() | floats() | text(printable),
2142 ... lambda children: lists(children) | dictionaries(text(printable), children))
2143 >>> pprint(json.example())
2144 ['dy',
2145 [None, True, 6.297399055778002e+16, False],
2146 {'a{h\\:694K~{mY>a1yA:#CmDYb': None},
2147 '\\kP!4',
2148 {'#1J1': '',
2149 'cx.': None,
2150 "jv'A?qyp_sB\n$62g": [],
2151 'qgnP': [False, -inf, 'la)']},
2152 [],
2153 {}]
2154 >>> pprint(json.example())
2155 {'': None,
2156 '(Rt)': 1.192092896e-07,
2157 ',': [],
2158 '6': 2.2250738585072014e-308,
2159 'HA=/': [],
2160 'YU]gy8': inf,
2161 'l': None,
2162 'nK': False}
2163 >>> pprint(json.example())
2164 []
2165
2166 That is, we start with our leaf data and then we augment it by allowing
2167 lists and dictionaries of anything we can generate as JSON data.
2168
2169 The size control of this works by limiting the maximum number of values
2170 that can be drawn from the base strategy. So for example if we wanted
2171 to only generate really small JSON we could do this as:
2172
2173 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2174 >>> small_lists.example()
2175 [False]
2176 >>> small_lists.example()
2177 True
2178 >>> small_lists.example()
2179 []
2180
2181 Composite strategies
2182 The @composite decorator lets you combine other strategies in more or
2183 less arbitrary ways. It's probably the main thing you'll want to use
2184 for complicated custom strategies.
2185
2186 The composite decorator works by converting a function that returns one
2187 example into a function that returns a strategy that produces such
2188 examples - which you can pass to @given, modify with .map or .filter,
2189 and generally use like any other strategy.
2190
2191 It does this by giving you a special function draw as the first argu‐
2192 ment, which can be used just like the corresponding method of the
2193 data() strategy within a test. In fact, the implementation is almost
2194 the same - but defining a strategy with @composite makes code reuse
2195 easier, and usually improves the display of failing examples.
2196
2197 For example, the following gives you a list and an index into it:
2198
2199 >>> @composite
2200 ... def list_and_index(draw, elements=integers()):
2201 ... xs = draw(lists(elements, min_size=1))
2202 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2203 ... return (xs, i)
2204
2205 draw(s) is a function that should be thought of as returning s.exam‐
2206 ple(), except that the result is reproducible and will minimize cor‐
2207 rectly. The decorated function has the initial argument removed from
2208 the list, but will accept all the others in the expected order.
2209 Defaults are preserved.
2210
2211 >>> list_and_index()
2212 list_and_index()
2213 >>> list_and_index().example()
2214 ([-21904], 0)
2215
2216 >>> list_and_index(booleans())
2217 list_and_index(elements=booleans())
2218 >>> list_and_index(booleans()).example()
2219 ([True], 0)
2220
2221 Note that the repr will work exactly like it does for all the built-in
2222 strategies: it will be a function that you can call to get the strategy
2223 in question, with values provided only if they do not match the
2224 defaults.
2225
2226 You can use assume inside composite functions:
2227
2228 @composite
2229 def distinct_strings_with_common_characters(draw):
2230 x = draw(text(), min_size=1)
2231 y = draw(text(alphabet=x))
2232 assume(x != y)
2233 return (x, y)
2234
2235 This works as assume normally would, filtering out any examples for
2236 which the passed in argument is falsey.
2237
2238 Drawing interactively in tests
2239 There is also the data() strategy, which gives you a means of using
2240 strategies interactively. Rather than having to specify everything up
2241 front in @given you can draw from strategies in the body of your test:
2242
2243 @given(data())
2244 def test_draw_sequentially(data):
2245 x = data.draw(integers())
2246 y = data.draw(integers(min_value=x))
2247 assert x < y
2248
2249 If the test fails, each draw will be printed with the falsifying exam‐
2250 ple. e.g. the above is wrong (it has a boundary condition error), so
2251 will print:
2252
2253 Falsifying example: test_draw_sequentially(data=data(...))
2254 Draw 1: 0
2255 Draw 2: 0
2256
2257 As you can see, data drawn this way is simplified as usual.
2258
2259 Test functions using the data() strategy do not support explicit @exam‐
2260 ple(...)s. In this case, the best option is usually to construct your
2261 data with @composite or the explicit example, and unpack this within
2262 the body of the test.
2263
2264 Optionally, you can provide a label to identify values generated by
2265 each call to data.draw(). These labels can be used to identify values
2266 in the output of a falsifying example.
2267
2268 For instance:
2269
2270 @given(data())
2271 def test_draw_sequentially(data):
2272 x = data.draw(integers(), label='First number')
2273 y = data.draw(integers(min_value=x), label='Second number')
2274 assert x < y
2275
2276 will produce the output:
2277
2278 Falsifying example: test_draw_sequentially(data=data(...))
2279 Draw 1 (First number): 0
2280 Draw 2 (Second number): 0
2281
2283 Hypothesis itself does not have any dependencies, but there are some
2284 packages that need additional things installed in order to work.
2285
2286 You can install these dependencies using the setuptools extra feature
2287 as e.g. pip install hypothesis[django]. This will check installation
2288 of compatible versions.
2289
2290 You can also just install hypothesis into a project using them, ignore
2291 the version constraints, and hope for the best.
2292
2293 In general "Which version is Hypothesis compatible with?" is a hard
2294 question to answer and even harder to regularly test. Hypothesis is
2295 always tested against the latest compatible version and each package
2296 will note the expected compatibility range. If you run into a bug with
2297 any of these please specify the dependency version.
2298
2299 There are separate pages for django and numpy.
2300
2301 hypothesis[pytz]
2302 This module provides pytz timezones.
2303
2304 You can use this strategy to make hypothesis.strategies.datetimes() and
2305 hypothesis.strategies.times() produce timezone-aware values.
2306
2307 hypothesis.extra.pytz.timezones()
2308 Any timezone in the Olsen database, as a pytz tzinfo object.
2309
2310 This strategy minimises to UTC, or the smallest possible fixed
2311 offset, and is designed for use with hypothesis.strategies.date‐
2312 times().
2313
2314 hypothesis[dateutil]
2315 hypothesis[datetime]
2316 This module provides deprecated time and date related strategies.
2317
2318 It depends on the pytz package, which is stable enough that almost any
2319 version should be compatible - most updates are for the timezone data‐
2320 base.
2321
2322 hypothesis.extra.datetime.datetimes(allow_naive=None, timezones=None,
2323 min_year=None, max_year=None)
2324 Return a strategy for generating datetimes.
2325
2326 Deprecated since version 3.9.0: use hypothesis.strategies.date‐
2327 times() instead.
2328
2329
2330 allow_naive=True will cause the values to sometimes be naive.
2331 timezones is the set of permissible timezones. If set to an
2332 empty collection all datetimes will be naive. If set to None all
2333 timezones available via pytz will be used.
2334
2335 All generated datetimes will be between min_year and max_year,
2336 inclusive.
2337
2338 hypothesis.extra.datetime.dates(min_year=None, max_year=None)
2339 Return a strategy for generating dates.
2340
2341 Deprecated since version 3.9.0: use hypothesis.strate‐
2342 gies.dates() instead.
2343
2344
2345 All generated dates will be between min_year and max_year,
2346 inclusive.
2347
2348 hypothesis.extra.datetime.times(allow_naive=None, timezones=None)
2349 Return a strategy for generating times.
2350
2351 Deprecated since version 3.9.0: use hypothesis.strate‐
2352 gies.times() instead.
2353
2354
2355 The allow_naive and timezones arguments act the same as the
2356 datetimes strategy above.
2357
2358 hypothesis[fakefactory]
2359 NOTE:
2360 This extra package is deprecated. We strongly recommend using
2361 native Hypothesis strategies, which are more effective at both find‐
2362 ing and shrinking failing examples for your tests.
2363
2364 The from_regex(), text() (with some specific alphabet), and sam‐
2365 pled_from() strategies may be particularly useful.
2366
2367 Faker (previously fake-factory) is a Python package that generates fake
2368 data for you. It's great for bootstraping your database, creating
2369 good-looking XML documents, stress-testing a database, or anonymizing
2370 production data. However, it's not designed for automated testing -
2371 data from Hypothesis looks less realistic, but produces minimal
2372 bug-triggering examples and uses coverage information to check more
2373 cases.
2374
2375 hypothesis.extra.fakefactory lets you use Faker generators to parame‐
2376 trize Hypothesis tests. This was only ever meant to ease your transi‐
2377 tion to Hypothesis, but we've improved Hypothesis enough since then
2378 that we no longer recommend using Faker for automated tests under any
2379 circumstances.
2380
2381 hypothesis.extra.fakefactory defines a function fake_factory which
2382 returns a strategy for producing text data from any Faker provider.
2383
2384 So for example the following will parametrize a test by an email
2385 address:
2386
2387 >>> fake_factory('email').example()
2388 'tnader@prosacco.info'
2389
2390 >>> fake_factory('name').example()
2391 'Zbyněk Černý CSc.'
2392
2393 You can explicitly specify the locale (otherwise it uses any of the
2394 available locales), either as a single locale or as several:
2395
2396 >>> fake_factory('name', locale='en_GB').example()
2397 'Antione Gerlach'
2398 >>> fake_factory('name', locales=['en_GB', 'cs_CZ']).example()
2399 'Miloš Šťastný'
2400 >>> fake_factory('name', locales=['en_GB', 'cs_CZ']).example()
2401 'Harm Sanford'
2402
2403 You can use custom Faker providers via the providers argument:
2404
2405 >>> from faker.providers import BaseProvider
2406 >>> class KittenProvider(BaseProvider):
2407 ... def meows(self):
2408 ... return 'meow %d' % (self.random_number(digits=10),)
2409 >>> fake_factory('meows', providers=[KittenProvider]).example()
2410 'meow 9139348419'
2411
2413 Hypothesis offers a number of features specific for Django testing,
2414 available in the hypothesis[django] extra. This is tested against each
2415 supported series with mainstream or extended support - if you're still
2416 getting security patches, you can test with Hypothesis.
2417
2418 Using it is quite straightforward: All you need to do is subclass
2419 hypothesis.extra.django.TestCase or hypothesis.extra.django.Transac‐
2420 tionTestCase and you can use @given as normal, and the transactions
2421 will be per example rather than per test function as they would be if
2422 you used @given with a normal django test suite (this is important
2423 because your test function will be called multiple times and you don't
2424 want them to interfere with each other). Test cases on these classes
2425 that do not use @given will be run as normal.
2426
2427 I strongly recommend not using TransactionTestCase unless you really
2428 have to. Because Hypothesis runs this in a loop the performance prob‐
2429 lems it normally has are significantly exacerbated and your tests will
2430 be really slow. If you are using TransactionTestCase, you may need to
2431 use @settings(suppress_health_check=[HealthCheck.too_slow]) to avoid
2432 errors due to slow example generation.
2433
2434 Having set up a test class, you can now pass @given a strategy for
2435 Django models:
2436
2437 For example, using the trivial django project I have for testing:
2438
2439 >>> from hypothesis.extra.django.models import models
2440 >>> from toystore.models import Customer
2441 >>> c = models(Customer).example()
2442 >>> c
2443 <Customer: Customer object>
2444 >>> c.email
2445 'jaime.urbina@gmail.com'
2446 >>> c.name
2447 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2448 >>> c.age
2449 -873375803
2450
2451 Hypothesis has just created this with whatever the relevant type of
2452 data is.
2453
2454 Obviously the customer's age is implausible, which is only possible
2455 because we have not used (eg) MinValueValidator to set the valid range
2456 for this field (or used a PositiveSmallIntegerField, which would only
2457 need a maximum value validator).
2458
2459 If you do have validators attached, Hypothesis will only generate exam‐
2460 ples that pass validation. Sometimes that will mean that we fail a
2461 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2462 egy to skip validation at the strategy level:
2463
2464 NOTE:
2465 Inference from validators will be much more powerful when issue
2466 #1116 is implemented, but there will always be some edge cases that
2467 require you to pass an explicit strategy.
2468
2469 >>> from hypothesis.strategies import integers
2470 >>> c = models(Customer, age=integers(min_value=0, max_value=120)).example()
2471 >>> c
2472 <Customer: Customer object>
2473 >>> c.age
2474 5
2475
2476 Tips and tricks
2477 Custom field types
2478 If you have a custom Django field type you can register it with Hypoth‐
2479 esis's model deriving functionality by registering a default strategy
2480 for it:
2481
2482 >>> from toystore.models import CustomishField, Customish
2483 >>> models(Customish).example()
2484 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2485 customish for model Customish
2486 >>> from hypothesis.extra.django.models import add_default_field_mapping
2487 >>> from hypothesis.strategies import just
2488 >>> add_default_field_mapping(CustomishField, just("hi"))
2489 >>> x = models(Customish).example()
2490 >>> x.customish
2491 'hi'
2492
2493 Note that this mapping is on exact type. Subtypes will not inherit it.
2494
2495 Generating child models
2496 For the moment there's no explicit support in hypothesis-django for
2497 generating dependent models. i.e. a Company model will generate no
2498 Shops. However if you want to generate some dependent models as well,
2499 you can emulate this by using the flatmap function as follows:
2500
2501 from hypothesis.strategies import lists, just
2502
2503 def generate_with_shops(company):
2504 return lists(models(Shop, company=just(company))).map(lambda _: company)
2505
2506 company_with_shops_strategy = models(Company).flatmap(generate_with_shops)
2507
2508 Lets unpack what this is doing:
2509
2510 The way flatmap works is that we draw a value from the original strat‐
2511 egy, then apply a function to it which gives us a new strategy. We then
2512 draw a value from that strategy. So in this case we're first drawing a
2513 company, and then we're drawing a list of shops belonging to that com‐
2514 pany: The just strategy is a strategy such that drawing it always pro‐
2515 duces the individual value, so models(Shop, company=just(company)) is a
2516 strategy that generates a Shop belonging to the original company.
2517
2518 So the following code would give us a list of shops all belonging to
2519 the same company:
2520
2521 models(Company).flatmap(lambda c: lists(models(Shop, company=just(c))))
2522
2523 The only difference from this and the above is that we want the com‐
2524 pany, not the shops. This is where the inner map comes in. We build the
2525 list of shops and then throw it away, instead returning the company we
2526 started for. This works because the models that Hypothesis generates
2527 are saved in the database, so we're essentially running the inner
2528 strategy purely for the side effect of creating those children in the
2529 database.
2530
2531 Using default field values
2532 Hypothesis ignores field defaults and always tries to generate values,
2533 even if it doesn't know how to. You can tell it to use the default
2534 value for a field instead of generating one by passing field‐
2535 name=default_value to models():
2536
2537 >>> from toystore.models import DefaultCustomish
2538 >>> models(DefaultCustomish).example()
2539 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2540 customish for model DefaultCustomish
2541 >>> from hypothesis.extra.django.models import default_value
2542 >>> x = models(DefaultCustomish, customish=default_value).example()
2543 >>> x.customish
2544 'b'
2545
2546 Generating primary key values
2547 If your model includes a custom primary key that you want to generate
2548 using a strategy (rather than a default auto-increment primary key)
2549 then Hypothesis has to deal with the possibility of a duplicate primary
2550 key.
2551
2552 If a model strategy generates a value for the primary key field,
2553 Hypothesis will create the model instance with update_or_create(),
2554 overwriting any existing instance in the database for this test case
2555 with the same primary key.
2556
2558 numpy
2559 Hypothesis offers a number of strategies for NumPy testing, available
2560 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
2561 package.
2562
2563 The centerpiece is the arrays() strategy, which generates arrays with
2564 any dtype, shape, and contents you can specify or give a strategy for.
2565 To make this as useful as possible, strategies are provided to generate
2566 array shapes and generate all kinds of fixed-size or compound dtypes.
2567
2568 pandas
2569 Hypothesis provides strategies for several of the core pandas data
2570 types: pandas.Index, pandas.Series and pandas.DataFrame.
2571
2572 The general approach taken by the pandas module is that there are mul‐
2573 tiple strategies for generating indexes, and all of the other strate‐
2574 gies take the number of entries they contain from their index strategy
2575 (with sensible defaults). So e.g. a Series is specified by specifying
2576 its numpy.dtype (and/or a strategy for generating elements for it).
2577
2578 Supported Versions
2579 There is quite a lot of variation between pandas versions. We only com‐
2580 mit to supporting the latest version of pandas, but older minor ver‐
2581 sions are supported on a "best effort" basis. Hypothesis is currently
2582 tested against and confirmed working with Pandas 0.19, 0.20, 0.21, and
2583 0.22.
2584
2585 Releases that are not the latest patch release of their minor version
2586 are not tested or officially supported, but will probably also work
2587 unless you hit a pandas bug.
2588
2590 Hypothesis tries to detect common mistakes and things that will cause
2591 difficulty at run time in the form of a number of 'health checks'.
2592
2593 These include detecting and warning about:
2594
2595 · Strategies with very slow data generation
2596
2597 · Strategies which filter out too much
2598
2599 · Recursive strategies which branch too much
2600
2601 · Tests that are unlikely to complete in a reasonable amount of time.
2602
2603 If any of these scenarios are detected, Hypothesis will emit a warning
2604 about them.
2605
2606 The general goal of these health checks is to warn you about things
2607 that you are doing that might appear to work but will either cause
2608 Hypothesis to not work correctly or to perform badly.
2609
2610 To selectively disable health checks, use the suppress_health_check
2611 setting. The argument for this parameter is a list with elements drawn
2612 from any of the class-level attributes of the HealthCheck class. Using
2613 a value of HealthCheck.all() will disable all health checks.
2614
2615 class hypothesis.HealthCheck
2616 Arguments for suppress_health_check.
2617
2618 Each member of this enum is a type of health check to suppress.
2619
2620 exception_in_generation = 0
2621 Deprecated and no longer does anything. It used to con‐
2622 vert errors in data generation into FailedHealthCheck
2623 error.
2624
2625 data_too_large = 1
2626 Check for when the typical size of the examples you are
2627 generating exceeds the maximum allowed size too often.
2628
2629 filter_too_much = 2
2630 Check for when the test is filtering out too many exam‐
2631 ples, either through use of assume() or filter(), or
2632 occasionally for Hypothesis internal reasons.
2633
2634 too_slow = 3
2635 Check for when your data generation is extremely slow and
2636 likely to hurt testing.
2637
2638 random_module = 4
2639 Deprecated and no longer does anything. It used to check
2640 for whether your tests used the global random module. Now
2641 @given tests automatically seed random so this is no
2642 longer an error.
2643
2644 return_value = 5
2645 Checks if your tests return a non-None value (which will
2646 be ignored and is unlikely to do what you want).
2647
2648 hung_test = 6
2649 Checks if your tests have been running for a very long
2650 time.
2651
2652 large_base_example = 7
2653 Checks if the natural example to shrink towards is very
2654 large.
2655
2656 not_a_test_method = 8
2657 Checks if @given has been applied to a method of
2658 unittest.TestCase.
2659
2661 When Hypothesis finds a bug it stores enough information in its data‐
2662 base to reproduce it. This enables you to have a classic testing work‐
2663 flow of find a bug, fix a bug, and be confident that this is actually
2664 doing the right thing because Hypothesis will start by retrying the
2665 examples that broke things last time.
2666
2667 Limitations
2668 The database is best thought of as a cache that you never need to
2669 invalidate: Information may be lost when you upgrade a Hypothesis ver‐
2670 sion or change your test, so you shouldn't rely on it for correctness -
2671 if there's an example you want to ensure occurs each time then there's
2672 a feature for including them in your source code - but it helps the
2673 development workflow considerably by making sure that the examples
2674 you've just found are reproduced.
2675
2676 The database also records examples that excercise less-used parts of
2677 your code, so the database may update even when no failing examples
2678 were found.
2679
2680 File locations
2681 The default storage format is as a fairly opaque directory structure.
2682 Each test corresponds to a directory, and each example to a file within
2683 that directory. The standard location for it is .hypothesis/examples
2684 in your current working directory. You can override this by setting the
2685 database setting.
2686
2687 There is also a legacy sqlite3 based format. This is mostly still sup‐
2688 ported for compatibility reasons, and support will be dropped in some
2689 future version of Hypothesis. If you use a database file name ending in
2690 .db, .sqlite or .sqlite3 that format will be used instead.
2691
2692 If you have not configured a database and the default location is unus‐
2693 able (e.g. because you do not have read/write permission), Hypothesis
2694 will issue a warning and then fall back to an in-memory database.
2695
2696 Upgrading Hypothesis and changing your tests
2697 The design of the Hypothesis database is such that you can put arbi‐
2698 trary data in the database and not get wrong behaviour. When you
2699 upgrade Hypothesis, old data might be invalidated, but this should hap‐
2700 pen transparently. It should never be the case that e.g. changing the
2701 strategy that generates an argument sometimes gives you data from the
2702 old strategy.
2703
2704 Sharing your example database
2705 NOTE:
2706 If specific examples are important for correctness you should use
2707 the @example decorator, as the example database may discard entries
2708 due to changes in your code or dependencies. For most users, we
2709 therefore recommend using the example database locally and possibly
2710 persisting it between CI builds, but not tracking it under version
2711 control.
2712
2713 The examples database can be shared simply by checking the directory
2714 into version control, for example with the following .gitignore:
2715
2716 # Ignore files cached by Hypothesis...
2717 .hypothesis/*
2718 # except for the examples directory
2719 !.hypothesis/examples/
2720
2721 Like everything under .hypothesis/, the examples directory will be
2722 transparently created on demand. Unlike the other subdirectories,
2723 examples/ is designed to handle merges, deletes, etc if you just add
2724 the directory into git, mercurial, or any similar version control sys‐
2725 tem.
2726
2728 With @given, your tests are still something that you mostly write your‐
2729 self, with Hypothesis providing some data. With Hypothesis's stateful
2730 testing, Hypothesis instead tries to generate not just data but entire
2731 tests. You specify a number of primitive actions that can be combined
2732 together, and then Hypothesis will try to find sequences of those
2733 actions that result in a failure.
2734
2735 NOTE:
2736 This style of testing is often called model-based testing, but in
2737 Hypothesis is called stateful testing (mostly for historical reasons
2738 - the original implementation of this idea in Hypothesis was more
2739 closely based on ScalaCheck's stateful testing where the name is
2740 more apt). Both of these names are somewhat misleading: You don't
2741 really need any sort of formal model of your code to use this, and
2742 it can be just as useful for pure APIs that don't involve any state
2743 as it is for stateful ones.
2744
2745 It's perhaps best to not take the name of this sort of testing too
2746 seriously. Regardless of what you call it, it is a powerful form of
2747 testing which is useful for most non-trivial APIs.
2748
2749 Hypothesis has two stateful testing APIs: A high level one, providing
2750 what we call rule based state machines, and a low level one, providing
2751 what we call generic state machines.
2752
2753 You probably want to use the rule based state machines - they provide a
2754 high level API for describing the sort of actions you want to perform,
2755 based on a structured representation of actions. However the generic
2756 state machines are more flexible, and are particularly useful if you
2757 want the set of currently possible actions to depend primarily on
2758 external state.
2759
2760 Rule based state machines
2761 Rule based state machines are the ones you're most likely to want to
2762 use. They're significantly more user friendly and should be good
2763 enough for most things you'd want to do.
2764
2765 The two main ingredients of a rule based state machine are rules and
2766 bundles.
2767
2768 A rule is very similar to a normal @given based test in that it takes
2769 values drawn from strategies and passes them to a user defined test
2770 function. The key difference is that where @given based tests must be
2771 independent, rules can be chained together - a single test run may
2772 involve multiple rule invocations, which may interact in various ways.
2773
2774 A Bundle is a named collection of generated values that can be reused
2775 by other operations in the test. They are populated with the results
2776 of rules, and may be used as arguments to rules, allowing data to flow
2777 from one rule to another, and rules to work on the results of previous
2778 computations or actions.
2779
2780 The following rule based state machine example is a simplified version
2781 of a test for Hypothesis's example database implementation. An example
2782 database maps keys to sets of values, and in this test we compare one
2783 implementation of it to a simplified in memory model of its behaviour,
2784 which just stores the same values in a Python dict. The test then runs
2785 operations against both the real database and the in-memory representa‐
2786 tion of it and looks for discrepancies in their behaviour.
2787
2788 import shutil
2789 import tempfile
2790
2791 from collections import defaultdict
2792 import hypothesis.strategies as st
2793 from hypothesis.database import DirectoryBasedExampleDatabase
2794 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
2795
2796
2797 class DatabaseComparison(RuleBasedStateMachine):
2798 def __init__(self):
2799 super(DatabaseComparison, self).__init__()
2800 self.tempd = tempfile.mkdtemp()
2801 self.database = DirectoryBasedExampleDatabase(self.tempd)
2802 self.model = defaultdict(set)
2803
2804 keys = Bundle('keys')
2805 values = Bundle('values')
2806
2807 @rule(target=keys, k=st.binary())
2808 def k(self, k):
2809 return k
2810
2811 @rule(target=values, v=st.binary())
2812 def v(self, v):
2813 return v
2814
2815 @rule(k=keys, v=values)
2816 def save(self, k, v):
2817 self.model[k].add(v)
2818 self.database.save(k, v)
2819
2820 @rule(k=keys, v=values)
2821 def delete(self, k, v):
2822 self.model[k].discard(v)
2823 self.database.delete(k, v)
2824
2825 @rule(k=keys)
2826 def values_agree(self, k):
2827 assert set(self.database.fetch(k)) == self.model[k]
2828
2829 def teardown(self):
2830 shutil.rmtree(self.tempd)
2831
2832
2833 TestDBComparison = DatabaseComparison.TestCase
2834
2835 In this we declare two bundles - one for keys, and one for values. We
2836 have two trivial rules which just populate them with data (k and v),
2837 and three non-trivial rules: save saves a value under a key and delete
2838 removes a value from a key, in both cases also updating the model of
2839 what should be in the database. values_agree then checks that the con‐
2840 tents of the database agrees with the model for a particular key.
2841
2842 We can then integrate this into our test suite by getting a unittest
2843 TestCase from it:
2844
2845 TestTrees = DatabaseComparison.TestCase
2846
2847 # Or just run with pytest's unittest support
2848 if __name__ == '__main__':
2849 unittest.main()
2850
2851 This test currently passes, but if we comment out the line where we
2852 call self.model[k].discard(v), we would see the following output when
2853 run under pytest:
2854
2855 AssertionError: assert set() == {b''}
2856
2857 ------------ Hypothesis ------------
2858
2859 state = DatabaseComparison()
2860 v1 = state.k(k=b'')
2861 v2 = state.v(v=v1)
2862 state.save(k=v1, v=v2)
2863 state.delete(k=v1, v=v2)
2864 state.values_agree(k=v1)
2865 state.teardown()
2866
2867 Note how it's printed out a very short program that will demonstrate
2868 the problem. The output from a rule based state machine should gener‐
2869 ally be pretty close to Python code - if you have custom repr implemen‐
2870 tations that don't return valid Python then it might not be, but most
2871 of the time you should just be able to copy and paste the code into a
2872 test to reproduce it.
2873
2874 You can control the detailed behaviour with a settings object on the
2875 TestCase (this is a normal hypothesis settings object using the
2876 defaults at the time the TestCase class was first referenced). For
2877 example if you wanted to run fewer examples with larger programs you
2878 could change the settings to:
2879
2880 DatabaseComparison.settings = settings(max_examples=50, stateful_step_count=100)
2881
2882 Which doubles the number of steps each program runs and halves the num‐
2883 ber of test cases that will be run.
2884
2885 Rules
2886 As said earlier, rules are the most common feature used in RuleBased‐
2887 StateMachine. They are defined by applying the rule() decorator on a
2888 function. Note that RuleBasedStateMachine must have at least one rule
2889 defined and that a single function cannot be used to define multiple
2890 rules (this to avoid having multiple rules doing the same things). Due
2891 to the stateful execution method, rules generally cannot take arguments
2892 from other sources such as fixtures or pytest.mark.parametrize - con‐
2893 sider providing them via a strategy such as sampled_from() instead.
2894
2895 hypothesis.stateful.rule(targets=(), target=None, **kwargs)
2896 Decorator for RuleBasedStateMachine. Any name present in target
2897 or targets will define where the end result of this function
2898 should go. If both are empty then the end result will be dis‐
2899 carded.
2900
2901 targets may either be a Bundle or the name of a Bundle.
2902
2903 kwargs then define the arguments that will be passed to the
2904 function invocation. If their value is a Bundle then values that
2905 have previously been produced for that bundle will be provided,
2906 if they are anything else it will be turned into a strategy and
2907 values from that will be provided.
2908
2909 Initializes
2910 Initializes are a special case of rules that are guaranteed to be run
2911 at most once at the beginning of a run (i.e. before any normal rule is
2912 called). Note if multiple initialize rules are defined, they may be
2913 called in any order, and that order will vary from run to run.
2914
2915 Initializes are typically useful to populate bundles:
2916
2917 hypothesis.stateful.initialize(targets=(), target=None, **kwargs)
2918 Decorator for RuleBasedStateMachine.
2919
2920 An initialize decorator behaves like a rule, but the decorated
2921 method is called at most once in a run. All initialize decorated
2922 methods will be called before any rule decorated methods, in an
2923 arbitrary order.
2924
2925 import hypothesis.strategies as st
2926 from hypothesis.stateful import RuleBasedStateMachine, Bundle, rule, initialize
2927
2928 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
2929
2930 class NumberModifier(RuleBasedStateMachine):
2931
2932 folders = Bundle('folders')
2933 files = Bundle('files')
2934
2935 @initialize(target=folders)
2936 def init_folders(self):
2937 return '/'
2938
2939 @rule(target=folders, name=name_strategy)
2940 def create_folder(self, parent, name):
2941 return '%s/%s' % (parent, name)
2942
2943 @rule(target=files, name=name_strategy)
2944 def create_file(self, parent, name):
2945 return '%s/%s' % (parent, name)
2946
2947 Preconditions
2948 While it's possible to use assume() in RuleBasedStateMachine rules, if
2949 you use it in only a few rules you can quickly run into a situation
2950 where few or none of your rules pass their assumptions. Thus, Hypothe‐
2951 sis provides a precondition() decorator to avoid this problem. The
2952 precondition() decorator is used on rule-decorated functions, and must
2953 be given a function that returns True or False based on the RuleBased‐
2954 StateMachine instance.
2955
2956 hypothesis.stateful.precondition(precond)
2957 Decorator to apply a precondition for rules in a RuleBased‐
2958 StateMachine. Specifies a precondition for a rule to be consid‐
2959 ered as a valid step in the state machine. The given function
2960 will be called with the instance of RuleBasedStateMachine and
2961 should return True or False. Usually it will need to look at
2962 attributes on that instance.
2963
2964 For example:
2965
2966 class MyTestMachine(RuleBasedStateMachine):
2967 state = 1
2968
2969 @precondition(lambda self: self.state != 0)
2970 @rule(numerator=integers())
2971 def divide_with(self, numerator):
2972 self.state = numerator / self.state
2973
2974 This is better than using assume in your rule since more valid
2975 rules should be able to be run.
2976
2977 from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
2978
2979 class NumberModifier(RuleBasedStateMachine):
2980
2981 num = 0
2982
2983 @rule()
2984 def add_one(self):
2985 self.num += 1
2986
2987 @precondition(lambda self: self.num != 0)
2988 @rule()
2989 def divide_with_one(self):
2990 self.num = 1 / self.num
2991
2992 By using precondition() here instead of assume(), Hypothesis can filter
2993 the inapplicable rules before running them. This makes it much more
2994 likely that a useful sequence of steps will be generated.
2995
2996 Note that currently preconditions can't access bundles; if you need to
2997 use preconditions, you should store relevant data on the instance
2998 instead.
2999
3000 Invariants
3001 Often there are invariants that you want to ensure are met after every
3002 step in a process. It would be possible to add these as rules that are
3003 run, but they would be run zero or multiple times between other rules.
3004 Hypothesis provides a decorator that marks a function to be run after
3005 every step.
3006
3007 hypothesis.stateful.invariant()
3008 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
3009 chine. The decorated function will be run after every rule and
3010 can raise an exception to indicate failed invariants.
3011
3012 For example:
3013
3014 class MyTestMachine(RuleBasedStateMachine):
3015 state = 1
3016
3017 @invariant()
3018 def is_nonzero(self):
3019 assert self.state != 0
3020
3021 from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
3022
3023 class NumberModifier(RuleBasedStateMachine):
3024
3025 num = 0
3026
3027 @rule()
3028 def add_two(self):
3029 self.num += 2
3030 if self.num > 50:
3031 self.num += 1
3032
3033 @invariant()
3034 def divide_with_one(self):
3035 assert self.num % 2 == 0
3036
3037 NumberTest = NumberModifier.TestCase
3038
3039 Invariants can also have precondition()s applied to them, in which case
3040 they will only be run if the precondition function returns true.
3041
3042 Note that currently invariants can't access bundles; if you need to use
3043 invariants, you should store relevant data on the instance instead.
3044
3045 Generic state machines
3046 The class GenericStateMachine is the underlying machinery of stateful
3047 testing in Hypothesis. Chances are you will want to use the rule based
3048 stateful testing for most things, but the generic state machine func‐
3049 tionality can be useful e.g. if you want to test things where the set
3050 of actions to be taken is more closely tied to the state of the system
3051 you are testing.
3052
3053 class hypothesis.stateful.GenericStateMachine
3054 A GenericStateMachine is the basic entry point into Hypothesis's
3055 approach to stateful testing.
3056
3057 The intent is for it to be subclassed to provide state machine
3058 descriptions
3059
3060 The way this is used is that Hypothesis will repeatedly execute
3061 something that looks something like:
3062
3063 x = MyStatemachineSubclass()
3064 x.check_invariants()
3065 try:
3066 for _ in range(n_steps):
3067 x.execute_step(x.steps().example())
3068 x.check_invariants()
3069 finally:
3070 x.teardown()
3071
3072 And if this ever produces an error it will shrink it down to a
3073 small sequence of example choices demonstrating that.
3074
3075 steps()
3076 Return a SearchStrategy instance the defines the avail‐
3077 able next steps.
3078
3079 execute_step(step)
3080 Execute a step that has been previously drawn from
3081 self.steps()
3082
3083 teardown()
3084 Called after a run has finished executing to clean up any
3085 necessary state.
3086
3087 Does nothing by default.
3088
3089 check_invariants()
3090 Called after initializing and after executing each step.
3091
3092 For example, here we use stateful testing as a sort of link checker, to
3093 test hypothesis.works for broken links or links that use HTTP instead
3094 of HTTPS.
3095
3096 from hypothesis.stateful import GenericStateMachine
3097 import hypothesis.strategies as st
3098 from requests_html import HTMLSession
3099
3100
3101 class LinkChecker(GenericStateMachine):
3102 def __init__(self):
3103 super(LinkChecker, self).__init__()
3104 self.session = HTMLSession()
3105 self.result = None
3106
3107 def steps(self):
3108 if self.result is None:
3109 # Always start on the home page
3110 return st.just("https://hypothesis.works/")
3111 else:
3112 return st.sampled_from([
3113 l
3114 for l in self.result.html.absolute_links
3115 # Don't try to crawl to other people's sites
3116 if l.startswith("https://hypothesis.works") and
3117 # Avoid Cloudflare's bot protection. We are a bot but we don't
3118 # care about the info it's hiding.
3119 '/cdn-cgi/' not in l
3120 ])
3121
3122 def execute_step(self, step):
3123 self.result = self.session.get(step)
3124
3125 assert self.result.status_code == 200
3126
3127 for l in self.result.html.absolute_links:
3128 # All links should be HTTPS
3129 assert "http://hypothesis.works" not in l
3130
3131
3132 TestLinks = LinkChecker.TestCase
3133
3134 Running this (at the time of writing this documentation) produced the
3135 following output:
3136
3137 AssertionError: assert 'http://hypothesis.works' not in 'http://hypoth...test-fixtures/'
3138 'http://hypothesis.works' is contained here:
3139 http://hypothesis.works/articles/hypothesis-pytest-fixtures/
3140 ? +++++++++++++++++++++++
3141
3142 ------------ Hypothesis ------------
3143
3144 Step #1: 'https://hypothesis.works/'
3145 Step #2: 'https://hypothesis.works/articles/'
3146
3147 More fine grained control
3148 If you want to bypass the TestCase infrastructure you can invoke these
3149 manually. The stateful module exposes the function
3150 run_state_machine_as_test, which takes an arbitrary function returning
3151 a GenericStateMachine and an optional settings parameter and does the
3152 same as the class based runTest provided.
3153
3154 In particular this may be useful if you wish to pass parameters to a
3155 custom __init__ in your subclass.
3156
3158 Hypothesis does its level best to be compatible with everything you
3159 could possibly need it to be compatible with. Generally you should just
3160 try it and expect it to work. If it doesn't, you can be surprised and
3161 check this document for the details.
3162
3163 Python versions
3164 Hypothesis is supported and tested on CPython 2.7 and CPython 3.4+.
3165
3166 Hypothesis also supports PyPy2, and will support PyPy3 when there is a
3167 stable release supporting Python 3.4+. Hypothesis does not currently
3168 work on Jython, though it probably could (issue #174). IronPython might
3169 work but hasn't been tested. 32-bit and narrow builds should work,
3170 though this is currently only tested on Windows.
3171
3172 In general Hypothesis does not officially support anything except the
3173 latest patch release of any version of Python it supports. Earlier
3174 releases should work and bugs in them will get fixed if reported, but
3175 they're not tested in CI and no guarantees are made.
3176
3177 Operating systems
3178 In theory Hypothesis should work anywhere that Python does. In practice
3179 it is only known to work and regularly tested on OS X, Windows and
3180 Linux, and you may experience issues running it elsewhere.
3181
3182 If you're using something else and it doesn't work, do get in touch and
3183 I'll try to help, but unless you can come up with a way for me to run a
3184 CI server on that operating system it probably won't stay fixed due to
3185 the inevitable march of time.
3186
3187 Testing frameworks
3188 In general Hypothesis goes to quite a lot of effort to generate things
3189 that look like normal Python test functions that behave as closely to
3190 the originals as possible, so it should work sensibly out of the box
3191 with every test framework.
3192
3193 If your testing relies on doing something other than calling a function
3194 and seeing if it raises an exception then it probably won't work out of
3195 the box. In particular things like tests which return generators and
3196 expect you to do something with them (e.g. nose's yield based tests)
3197 will not work. Use a decorator or similar to wrap the test to take this
3198 form, or ask the framework maintainer to support our hooks for insert‐
3199 ing such a wrapper later.
3200
3201 In terms of what's actually known to work:
3202
3203 · Hypothesis integrates as smoothly with py.test and unittest as we
3204 can make it, and this is verified as part of the CI. Note however
3205 that @given should only be used on tests, not
3206 python:unittest.TestCase setup or teardown methods.
3207
3208 · pytest fixtures work in the usual way for tests that have been
3209 decorated with @given - just avoid passing a strategy for each
3210 argument that will be supplied by a fixture. However, each fix‐
3211 ture will run once for the whole function, not once per example.
3212 Decorating a fixture function is meaningless.
3213
3214 · Nose works fine with hypothesis, and this is tested as part of the
3215 CI. yield based tests simply won't work.
3216
3217 · Integration with Django's testing requires use of the hypothe‐
3218 sis-django package. The issue is that in Django's tests' normal
3219 mode of execution it will reset the database once per test rather
3220 than once per example, which is not what you want.
3221
3222 · Coverage works out of the box with Hypothesis - we use it to guide
3223 example selection for user code, and Hypothesis has 100% branch
3224 coverage in its own tests.
3225
3226 Optional Packages
3227 The supported versions of optional packages, for strategies in hypothe‐
3228 sis.extra, are listed in the documentation for that extra. Our general
3229 goal is to support all versions that are supported upstream.
3230
3231 Regularly verifying this
3232 Everything mentioned above as explicitly supported is checked on every
3233 commit with Travis, Appveyor, and CircleCI. Our continous delivery
3234 pipeline runs all of these checks before publishing each release, so
3235 when we say they're supported we really mean it.
3236
3237 Hypothesis versions
3238 Backwards compatibility is better than backporting fixes, so we use
3239 semantic versioning and only support the most recent version of Hypoth‐
3240 esis. See support for more information.
3241
3243 This is a collection of examples of how to use Hypothesis in interest‐
3244 ing ways. It's small for now but will grow over time.
3245
3246 All of these examples are designed to be run under py.test (nose should
3247 probably work too).
3248
3249 How not to sort by a partial order
3250 The following is an example that's been extracted and simplified from a
3251 real bug that occurred in an earlier version of Hypothesis. The real
3252 bug was a lot harder to find.
3253
3254 Suppose we've got the following type:
3255
3256 class Node(object):
3257 def __init__(self, label, value):
3258 self.label = label
3259 self.value = tuple(value)
3260
3261 def __repr__(self):
3262 return "Node(%r, %r)" % (self.label, self.value)
3263
3264 def sorts_before(self, other):
3265 if len(self.value) >= len(other.value):
3266 return False
3267 return other.value[:len(self.value)] == self.value
3268
3269 Each node is a label and a sequence of some data, and we have the rela‐
3270 tionship sorts_before meaning the data of the left is an initial seg‐
3271 ment of the right. So e.g. a node with value [1, 2] will sort before a
3272 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
3273 before the other.
3274
3275 We have a list of nodes, and we want to topologically sort them with
3276 respect to this ordering. That is, we want to arrange the list so that
3277 if x.sorts_before(y) then x appears earlier in the list than y. We
3278 naively think that the easiest way to do this is to extend the partial
3279 order defined here to a total order by breaking ties arbitrarily and
3280 then using a normal sorting algorithm. So we define the following code:
3281
3282 from functools import total_ordering
3283
3284
3285 @total_ordering
3286 class TopoKey(object):
3287 def __init__(self, node):
3288 self.value = node
3289
3290 def __lt__(self, other):
3291 if self.value.sorts_before(other.value):
3292 return True
3293 if other.value.sorts_before(self.value):
3294 return False
3295
3296 return self.value.label < other.value.label
3297
3298
3299 def sort_nodes(xs):
3300 xs.sort(key=TopoKey)
3301
3302 This takes the order defined by sorts_before and extends it by breaking
3303 ties by comparing the node labels.
3304
3305 But now we want to test that it works.
3306
3307 First we write a function to verify that our desired outcome holds:
3308
3309 def is_prefix_sorted(xs):
3310 for i in range(len(xs)):
3311 for j in range(i+1, len(xs)):
3312 if xs[j].sorts_before(xs[i]):
3313 return False
3314 return True
3315
3316 This will return false if it ever finds a pair in the wrong order and
3317 return true otherwise.
3318
3319 Given this function, what we want to do with Hypothesis is assert that
3320 for all sequences of nodes, the result of calling sort_nodes on it is
3321 sorted.
3322
3323 First we need to define a strategy for Node:
3324
3325 from hypothesis import settings, strategies
3326 import hypothesis.strategies as s
3327
3328 NodeStrategy = s.builds(
3329 Node,
3330 s.integers(),
3331 s.lists(s.booleans(), average_size=5, max_size=10))
3332
3333 We want to generate short lists of values so that there's a decent
3334 chance of one being a prefix of the other (this is also why the choice
3335 of bool as the elements). We then define a strategy which builds a node
3336 out of an integer and one of those short lists of booleans.
3337
3338 We can now write a test:
3339
3340 from hypothesis import given
3341
3342 @given(s.lists(NodeStrategy))
3343 def test_sorting_nodes_is_prefix_sorted(xs):
3344 sort_nodes(xs)
3345 assert is_prefix_sorted(xs)
3346
3347 this immediately fails with the following example:
3348
3349 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
3350
3351 The reason for this is that because False is not a prefix of (True,
3352 True) nor vice versa, sorting things the first two nodes are equal
3353 because they have equal labels. This makes the whole order non-transi‐
3354 tive and produces basically nonsense results.
3355
3356 But this is pretty unsatisfying. It only works because they have the
3357 same label. Perhaps we actually wanted our labels to be unique. Lets
3358 change the test to do that.
3359
3360 def deduplicate_nodes_by_label(nodes):
3361 table = {node.label: node for node in nodes}
3362 return list(table.values())
3363
3364 We define a function to deduplicate nodes by labels, and can now map
3365 that over a strategy for lists of nodes to give us a strategy for lists
3366 of nodes with unique labels:
3367
3368 @given(s.lists(NodeStrategy).map(deduplicate_nodes_by_label))
3369 def test_sorting_nodes_is_prefix_sorted(xs):
3370 sort_nodes(xs)
3371 assert is_prefix_sorted(xs)
3372
3373 Hypothesis quickly gives us an example of this still being wrong:
3374
3375 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))])
3376
3377 Now this is a more interesting example. None of the nodes will sort
3378 equal. What is happening here is that the first node is strictly less
3379 than the last node because (False,) is a prefix of (False, False). This
3380 is in turn strictly less than the middle node because neither is a pre‐
3381 fix of the other and -2 < -1. The middle node is then less than the
3382 first node because -1 < 0.
3383
3384 So, convinced that our implementation is broken, we write a better one:
3385
3386 def sort_nodes(xs):
3387 for i in hrange(1, len(xs)):
3388 j = i - 1
3389 while j >= 0:
3390 if xs[j].sorts_before(xs[j+1]):
3391 break
3392 xs[j], xs[j+1] = xs[j+1], xs[j]
3393 j -= 1
3394
3395 This is just insertion sort slightly modified - we swap a node back‐
3396 wards until swapping it further would violate the order constraints.
3397 The reason this works is because our order is a partial order already
3398 (this wouldn't produce a valid result for a general topological sorting
3399 - you need the transitivity).
3400
3401 We now run our test again and it passes, telling us that this time
3402 we've successfully managed to sort some nodes without getting it com‐
3403 pletely wrong. Go us.
3404
3405 Time zone arithmetic
3406 This is an example of some tests for pytz which check that various
3407 timezone conversions behave as you would expect them to. These tests
3408 should all pass, and are mostly a demonstration of some useful sorts of
3409 thing to test with Hypothesis, and how the datetimes() strategy works.
3410
3411 >>> from datetime import timedelta
3412 >>> from hypothesis.extra.pytz import timezones
3413 >>> from hypothesis.strategies import datetimes
3414
3415 >>> # The datetimes strategy is naive by default, so tell it to use timezones
3416 >>> aware_datetimes = datetimes(timezones=timezones())
3417
3418 >>> @given(aware_datetimes, timezones(), timezones())
3419 ... def test_convert_via_intermediary(dt, tz1, tz2):
3420 ... """Test that converting between timezones is not affected
3421 ... by a detour via another timezone.
3422 ... """
3423 ... assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
3424
3425 >>> @given(aware_datetimes, timezones())
3426 ... def test_convert_to_and_fro(dt, tz2):
3427 ... """If we convert to a new timezone and back to the old one
3428 ... this should leave the result unchanged.
3429 ... """
3430 ... tz1 = dt.tzinfo
3431 ... assert dt == dt.astimezone(tz2).astimezone(tz1)
3432
3433 >>> @given(aware_datetimes, timezones())
3434 ... def test_adding_an_hour_commutes(dt, tz):
3435 ... """When converting between timezones it shouldn't matter
3436 ... if we add an hour here or add an hour there.
3437 ... """
3438 ... an_hour = timedelta(hours=1)
3439 ... assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
3440
3441 >>> @given(aware_datetimes, timezones())
3442 ... def test_adding_a_day_commutes(dt, tz):
3443 ... """When converting between timezones it shouldn't matter
3444 ... if we add a day here or add a day there.
3445 ... """
3446 ... a_day = timedelta(days=1)
3447 ... assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
3448
3449 >>> # And we can check that our tests pass
3450 >>> test_convert_via_intermediary()
3451 >>> test_convert_to_and_fro()
3452 >>> test_adding_an_hour_commutes()
3453 >>> test_adding_a_day_commutes()
3454
3455 Condorcet's Paradox
3456 A classic paradox in voting theory, called Condorcet's paradox, is that
3457 majority preferences are not transitive. That is, there is a population
3458 and a set of three candidates A, B and C such that the majority of the
3459 population prefer A to B, B to C and C to A.
3460
3461 Wouldn't it be neat if we could use Hypothesis to provide an example of
3462 this?
3463
3464 Well as you can probably guess from the presence of this section, we
3465 can! This is slightly surprising because it's not really obvious how we
3466 would generate an election given the types that Hypothesis knows about.
3467
3468 The trick here turns out to be twofold:
3469
3470 1. We can generate a type that is much larger than an election, extract
3471 an election out of that, and rely on minimization to throw away all
3472 the extraneous detail.
3473
3474 2. We can use assume and rely on Hypothesis's adaptive exploration to
3475 focus on the examples that turn out to generate interesting elec‐
3476 tions
3477
3478 Without further ado, here is the code:
3479
3480 from hypothesis import given, assume
3481 from hypothesis.strategies import integers, lists
3482 from collections import Counter
3483
3484
3485 def candidates(votes):
3486 return {candidate for vote in votes for candidate in vote}
3487
3488
3489 def build_election(votes):
3490 """
3491 Given a list of lists we extract an election out of this. We do this
3492 in two phases:
3493
3494 1. First of all we work out the full set of candidates present in all
3495 votes and throw away any votes that do not have that whole set.
3496 2. We then take each vote and make it unique, keeping only the first
3497 instance of any candidate.
3498
3499 This gives us a list of total orderings of some set. It will usually
3500 be a lot smaller than the starting list, but that's OK.
3501 """
3502 all_candidates = candidates(votes)
3503 votes = list(filter(lambda v: set(v) == all_candidates, votes))
3504 if not votes:
3505 return []
3506 rebuilt_votes = []
3507 for vote in votes:
3508 rv = []
3509 for v in vote:
3510 if v not in rv:
3511 rv.append(v)
3512 assert len(rv) == len(all_candidates)
3513 rebuilt_votes.append(rv)
3514 return rebuilt_votes
3515
3516
3517 @given(lists(lists(integers(min_value=1, max_value=5))))
3518 def test_elections_are_transitive(election):
3519 election = build_election(election)
3520 # Small elections are unlikely to be interesting
3521 assume(len(election) >= 3)
3522 all_candidates = candidates(election)
3523 # Elections with fewer than three candidates certainly can't exhibit
3524 # intransitivity
3525 assume(len(all_candidates) >= 3)
3526
3527 # Now we check if the election is transitive
3528
3529 # First calculate the pairwise counts of how many prefer each candidate
3530 # to the other
3531 counts = Counter()
3532 for vote in election:
3533 for i in range(len(vote)):
3534 for j in range(i+1, len(vote)):
3535 counts[(vote[i], vote[j])] += 1
3536
3537 # Now look at which pairs of candidates one has a majority over the
3538 # other and store that.
3539 graph = {}
3540 all_candidates = candidates(election)
3541 for i in all_candidates:
3542 for j in all_candidates:
3543 if counts[(i, j)] > counts[(j, i)]:
3544 graph.setdefault(i, set()).add(j)
3545
3546 # Now for each triple assert that it is transitive.
3547 for x in all_candidates:
3548 for y in graph.get(x, ()):
3549 for z in graph.get(y, ()):
3550 assert x not in graph.get(z, ())
3551
3552 The example Hypothesis gives me on my first run (your mileage may of
3553 course vary) is:
3554
3555 [[3, 1, 4], [4, 3, 1], [1, 4, 3]]
3556
3557 Which does indeed do the job: The majority (votes 0 and 1) prefer 3 to
3558 1, the majority (votes 0 and 2) prefer 1 to 4 and the majority (votes 1
3559 and 2) prefer 4 to 3. This is in fact basically the canonical example
3560 of the voting paradox, modulo variations on the names of candidates.
3561
3562 Fuzzing an HTTP API
3563 Hypothesis's support for testing HTTP services is somewhat nascent.
3564 There are plans for some fully featured things around this, but right
3565 now they're probably quite far down the line.
3566
3567 But you can do a lot yourself without any explicit support! Here's a
3568 script I wrote to throw random data against the API for an entirely
3569 fictitious service called Waspfinder (this is only lightly obfuscated
3570 and you can easily figure out who I'm actually talking about, but I
3571 don't want you to run this code and hammer their API without their per‐
3572 mission).
3573
3574 All this does is use Hypothesis to generate random JSON data matching
3575 the format their API asks for and check for 500 errors. More advanced
3576 tests which then use the result and go on to do other things are defi‐
3577 nitely also possible. The swagger-conformance package provides an
3578 excellent example of this!
3579
3580 import unittest
3581 from hypothesis import given, assume, settings, strategies as st
3582 from collections import namedtuple
3583 import requests
3584 import os
3585 import random
3586 import time
3587 import math
3588
3589 # These tests will be quite slow because we have to talk to an external
3590 # service. Also we'll put in a sleep between calls so as to not hammer it.
3591 # As a result we reduce the number of test cases and turn off the timeout.
3592 settings.default.max_examples = 100
3593 settings.default.timeout = -1
3594
3595 Goal = namedtuple("Goal", ("slug",))
3596
3597
3598 # We just pass in our API credentials via environment variables.
3599 waspfinder_token = os.getenv('WASPFINDER_TOKEN')
3600 waspfinder_user = os.getenv('WASPFINDER_USER')
3601 assert waspfinder_token is not None
3602 assert waspfinder_user is not None
3603
3604 GoalData = st.fixed_dictionaries({
3605 'title': st.text(),
3606 'goal_type': st.sampled_from([
3607 "hustler", "biker", "gainer", "fatloser", "inboxer",
3608 "drinker", "custom"]),
3609 'goaldate': st.one_of(st.none(), st.floats()),
3610 'goalval': st.one_of(st.none(), st.floats()),
3611 'rate': st.one_of(st.none(), st.floats()),
3612 'initval': st.floats(),
3613 'panic': st.floats(),
3614 'secret': st.booleans(),
3615 'datapublic': st.booleans(),
3616 })
3617
3618
3619 needs2 = ['goaldate', 'goalval', 'rate']
3620
3621
3622 class WaspfinderTest(unittest.TestCase):
3623
3624 @given(GoalData)
3625 def test_create_goal_dry_run(self, data):
3626 # We want slug to be unique for each run so that multiple test runs
3627 # don't interfere with each other. If for some reason some slugs trigger
3628 # an error and others don't we'll get a Flaky error, but that's OK.
3629 slug = hex(random.getrandbits(32))[2:]
3630
3631 # Use assume to guide us through validation we know about, otherwise
3632 # we'll spend a lot of time generating boring examples.
3633
3634 # Title must not be empty
3635 assume(data["title"])
3636
3637 # Exactly two of these values should be not None. The other will be
3638 # inferred by the API.
3639
3640 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
3641 for v in data.values():
3642 if isinstance(v, float):
3643 assume(not math.isnan(v))
3644 data["slug"] = slug
3645
3646 # The API nicely supports a dry run option, which means we don't have
3647 # to worry about the user account being spammed with lots of fake goals
3648 # Otherwise we would have to make sure we cleaned up after ourselves
3649 # in this test.
3650 data["dryrun"] = True
3651 data["auth_token"] = waspfinder_token
3652 for d, v in data.items():
3653 if v is None:
3654 data[d] = "null"
3655 else:
3656 data[d] = str(v)
3657 result = requests.post(
3658 "https://waspfinder.example.com/api/v1/users/"
3659 "%s/goals.json" % (waspfinder_user,), data=data)
3660
3661 # Lets not hammer the API too badly. This will of course make the
3662 # tests even slower than they otherwise would have been, but that's
3663 # life.
3664 time.sleep(1.0)
3665
3666 # For the moment all we're testing is that this doesn't generate an
3667 # internal error. If we didn't use the dry run option we could have
3668 # then tried doing more with the result, but this is a good start.
3669 self.assertNotEqual(result.status_code, 500)
3670
3671 if __name__ == '__main__':
3672 unittest.main()
3673
3675 The Hypothesis community is small for the moment but is full of excel‐
3676 lent people who can answer your questions and help you out. Please do
3677 join us.
3678
3679 The two major places for community discussion are:
3680
3681 · The mailing list.
3682
3683 · An IRC channel, #hypothesis on freenode, which is more active than
3684 the mailing list.
3685
3686 Feel free to use these to ask for help, provide feedback, or discuss
3687 anything remotely Hypothesis related at all.
3688
3689 Please note that the Hypothesis code of conduct applies in all Hypothe‐
3690 sis community spaces.
3691
3692 If you would like to cite Hypothesis, please consider our sugested
3693 citation.
3694
3695 If you like repo badges, we suggest the following badge, which you can
3696 add with reStructuredText or Markdown, respectively: [image]
3697
3698 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
3699 :alt: Tested with Hypothesis
3700 :target: https://hypothesis.readthedocs.io
3701
3702 [![Tested with Hypothesis](https://img.shields.io/badge/hypothesis-tested-brightgreen.svg)](https://hypothesis.readthedocs.io/)
3703
3705 What is Hypothesis for?
3706
3707 From the perspective of a user, the purpose of Hypothesis is to make it
3708 easier for you to write better tests.
3709
3710 From my perspective as the author, that is of course also a purpose of
3711 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
3712 mania for a moment), the larger purpose of Hypothesis is to drag the
3713 world kicking and screaming into a new and terrifying age of high qual‐
3714 ity software.
3715
3716 Software is, as they say, eating the world. Software is also terrible.
3717 It's buggy, insecure and generally poorly thought out. This combination
3718 is clearly a recipe for disaster.
3719
3720 And the state of software testing is even worse. Although it's fairly
3721 uncontroversial at this point that you should be testing your code, can
3722 you really say with a straight face that most projects you've worked on
3723 are adequately tested?
3724
3725 A lot of the problem here is that it's too hard to write good tests.
3726 Your tests encode exactly the same assumptions and fallacies that you
3727 had when you wrote the code, so they miss exactly the same bugs that
3728 you missed when you wrote the code.
3729
3730 Meanwhile, there are all sorts of tools for making testing better that
3731 are basically unused. The original Quickcheck is from 1999 and the
3732 majority of developers have not even heard of it, let alone used it.
3733 There are a bunch of half-baked implementations for most languages, but
3734 very few of them are worth using.
3735
3736 The goal of Hypothesis is to bring advanced testing techniques to the
3737 masses, and to provide an implementation that is so high quality that
3738 it is easier to use them than it is not to use them. Where I can, I
3739 will beg, borrow and steal every good idea I can find that someone has
3740 had to make software testing better. Where I can't, I will invent new
3741 ones.
3742
3743 Quickcheck is the start, but I also plan to integrate ideas from fuzz
3744 testing (a planned future feature is to use coverage information to
3745 drive example selection, and the example saving database is already
3746 inspired by the workflows people use for fuzz testing), and am open to
3747 and actively seeking out other suggestions and ideas.
3748
3749 The plan is to treat the social problem of people not using these ideas
3750 as a bug to which there is a technical solution: Does property-based
3751 testing not match your workflow? That's a bug, let's fix it by figur‐
3752 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
3753 tom data for your application? That's a bug. Let's fix it by figuring
3754 out how to make it easier, or how to take something you're already
3755 using to specify your data and derive a generator from that automati‐
3756 cally. Find the explanations of these advanced ideas hopelessly obtuse
3757 and hard to follow? That's a bug. Let's provide you with an easy API
3758 that lets you test your code better without a PhD in software verifica‐
3759 tion.
3760
3761 Grand ambitions, I know, and I expect ultimately the reality will be
3762 somewhat less grand, but so far in about three months of development,
3763 Hypothesis has become the most solid implementation of Quickcheck ever
3764 seen in a mainstream language (as long as we don't count Scala as main‐
3765 stream yet), and at the same time managed to significantly push forward
3766 the state of the art, so I think there's reason to be optimistic.
3767
3769 This is a page for listing people who are using Hypothesis and how
3770 excited they are about that. If that's you and your name is not on the
3771 list, this file is in Git and I'd love it if you sent me a pull request
3772 to fix that.
3773
3774 Stripe
3775 At Stripe we use Hypothesis to test every piece of our machine learning
3776 model training pipeline (powered by scikit). Before we migrated, our
3777 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
3778 resentative at all of our actual very complex data. Because we needed
3779 to craft examples for each test, we took the easy way out and lived
3780 with extremely low test coverage.
3781
3782 Hypothesis changed all that. Once we had our strategies for generating
3783 Dataframes of features it became trivial to slightly customize each
3784 strategy for new tests. Our coverage is now close to 90%.
3785
3786 Full-stop, property-based testing is profoundly more powerful - and has
3787 caught or prevented far more bugs - than our old style of example-based
3788 testing.
3789
3790 Kristian Glass - Director of Technology at LaterPay GmbH
3791 Hypothesis has been brilliant for expanding the coverage of our test
3792 cases, and also for making them much easier to read and understand, so
3793 we're sure we're testing the things we want in the way we want.
3794
3795 Seth Morton
3796 When I first heard about Hypothesis, I knew I had to include it in my
3797 two open-source Python libraries, natsort and fastnumbers . Quite
3798 frankly, I was a little appalled at the number of bugs and "holes" I
3799 found in the code. I can now say with confidence that my libraries are
3800 more robust to "the wild." In addition, Hypothesis gave me the confi‐
3801 dence to expand these libraries to fully support Unicode input, which I
3802 never would have had the stomach for without such thorough testing
3803 capabilities. Thanks!
3804
3805 Sixty North
3806 At Sixty North we use Hypothesis for testing Segpy an open source
3807 Python library for shifting data between Python data structures and SEG
3808 Y files which contain geophysical data from the seismic reflection sur‐
3809 veys used in oil and gas exploration.
3810
3811 This is our first experience of property-based testing – as opposed to
3812 example-based testing. Not only are our tests more powerful, they are
3813 also much better explanations of what we expect of the production code.
3814 In fact, the tests are much closer to being specifications. Hypothesis
3815 has located real defects in our code which went undetected by tradi‐
3816 tional test cases, simply because Hypothesis is more relentlessly devi‐
3817 ous about test case generation than us mere humans! We found Hypothe‐
3818 sis particularly beneficial for Segpy because SEG Y is an antiquated
3819 format that uses legacy text encodings (EBCDIC) and even a legacy
3820 floating point format we implemented from scratch in Python.
3821
3822 Hypothesis is sure to find a place in most of our future Python code‐
3823 bases and many existing ones too.
3824
3825 mulkieran
3826 Just found out about this excellent QuickCheck for Python implementa‐
3827 tion and ran up a few tests for my bytesize package last night. Refuted
3828 a few hypotheses in the process.
3829
3830 Looking forward to using it with a bunch of other projects as well.
3831
3832 Adam Johnson
3833 I have written a small library to serialize dicts to MariaDB's dynamic
3834 columns binary format, mariadb-dyncol. When I first developed it, I
3835 thought I had tested it really well - there were hundreds of test
3836 cases, some of them even taken from MariaDB's test suite itself. I was
3837 ready to release.
3838
3839 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
3840 Wow! It found bug after bug after bug. Even after a first release, I
3841 thought of a way to make the tests do more validation, which revealed a
3842 further round of bugs! Most impressively, Hypothesis found a compli‐
3843 cated off-by-one error in a condition with 4095 versus 4096 bytes of
3844 data - something that I would never have found.
3845
3846 Long live Hypothesis! (Or at least, property-based testing).
3847
3848 Josh Bronson
3849 Adopting Hypothesis improved bidict's test coverage and significantly
3850 increased our ability to make changes to the code with confidence that
3851 correct behavior would be preserved. Thank you, David, for the great
3852 testing tool.
3853
3854 Cory Benfield
3855 Hypothesis is the single most powerful tool in my toolbox for working
3856 with algorithmic code, or any software that produces predictable output
3857 from a wide range of sources. When using it with Priority, Hypothesis
3858 consistently found errors in my assumptions and extremely subtle bugs
3859 that would have taken months of real-world use to locate. In some
3860 cases, Hypothesis found subtle deviations from the correct output of
3861 the algorithm that may never have been noticed at all.
3862
3863 When it comes to validating the correctness of your tools, nothing
3864 comes close to the thoroughness and power of Hypothesis.
3865
3866 Jon Moore
3867 One extremely satisfied user here. Hypothesis is a really solid imple‐
3868 mentation of property-based testing, adapted well to Python, and with
3869 good features such as failure-case shrinkers. I first used it on a
3870 project where we needed to verify that a vendor's Python and non-Python
3871 implementations of an algorithm matched, and it found about a dozen
3872 cases that previous example-based testing and code inspections had not.
3873 Since then I've been evangelizing for it at our firm.
3874
3875 Russel Winder
3876 I am using Hypothesis as an integral part of my Python workshops. Test‐
3877 ing is an integral part of Python programming and whilst unittest and,
3878 better, py.test can handle example-based testing, property-based test‐
3879 ing is increasingly far more important than example-base testing, and
3880 Hypothesis fits the bill.
3881
3882 Wellfire Interactive
3883 We've been using Hypothesis in a variety of client projects, from test‐
3884 ing Django-related functionality to domain-specific calculations. It
3885 both speeds up and simplifies the testing process since there's so much
3886 less tedious and error-prone work to do in identifying edge cases. Test
3887 coverage is nice but test depth is even nicer, and it's much easier to
3888 get meaningful test depth using Hypothesis.
3889
3890 Cody Kochmann
3891 Hypothesis is being used as the engine for random object generation
3892 with my open source function fuzzer battle_tested which maps all behav‐
3893 iors of a function allowing you to minimize the chance of unexpected
3894 crashes when running code in production.
3895
3896 With how efficient Hypothesis is at generating the edge cases that
3897 cause unexpected behavior occur, battle_tested is able to map out the
3898 entire behavior of most functions in less than a few seconds.
3899
3900 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
3901 ing it.
3902
3903 Merchise Autrement
3904 Just minutes after our first use of hypothesis we uncovered a subtle
3905 bug in one of our most used library. Since then, we have increasingly
3906 used hypothesis to improve the quality of our testing in libraries and
3907 applications as well.
3908
3909 Florian Kromer
3910 At Roboception GmbH I use Hypothesis to implement fully automated
3911 stateless and stateful reliability tests for the 3D sensor rc_visard
3912 and robotic software components .
3913
3914 Thank you very much for creating the (probably) most powerful prop‐
3915 erty-based testing framework.
3916
3917 Your name goes here
3918 I know there are many more, because I keep finding out about new people
3919 I'd never even heard of using Hypothesis. If you're looking to way to
3920 give back to a tool you love, adding your name here only takes a moment
3921 and would really help a lot. As per instructions at the top, just send
3922 me a pull request and I'll add you to the list.
3923
3925 The following is a non-exhaustive list of open source projects I know
3926 are using Hypothesis. If you're aware of any others please add them to
3927 the list! The only inclusion criterion right now is that if it's a
3928 Python library then it should be available on pypi.
3929
3930 You can find hundreds more from the Hypothesis page at libraries.io,
3931 and over a thousand on GitHub.
3932
3933 · aur
3934
3935 · argon2_cffi
3936
3937 · attrs
3938
3939 · axelrod
3940
3941 · bidict
3942
3943 · binaryornot
3944
3945 · brotlipy
3946
3947 · chardet
3948
3949 · cmph-cffi
3950
3951 · cryptography
3952
3953 · dbus-signature-pyparsing
3954
3955 · fastnumbers
3956
3957 · flocker
3958
3959 · flownetpy
3960
3961 · funsize
3962
3963 · fusion-index
3964
3965 · hyper-h2
3966
3967 · into-dbus-python
3968
3969 · justbases
3970
3971 · justbytes
3972
3973 · loris
3974
3975 · mariadb-dyncol
3976
3977 · mercurial
3978
3979 · natsort
3980
3981 · pretext
3982
3983 · priority
3984
3985 · PyCEbox
3986
3987 · PyPy
3988
3989 · pyrsistent
3990
3991 · python-humble-utils
3992
3993 · pyudev
3994
3995 · qutebrowser
3996
3997 · RubyMarshal
3998
3999 · Segpy
4000
4001 · simoa
4002
4003 · srt
4004
4005 · tchannel
4006
4007 · vdirsyncer
4008
4009 · wcag-contrast-ratio
4010
4011 · yacluster
4012
4013 · yturl
4014
4016 Hypothesis has been eagerly used and extended by the open source commu‐
4017 nity. This page lists extensions and applications; you can find more
4018 or newer packages by searching PyPI or libraries.io.
4019
4020 If there's something missing which you think should be here, let us
4021 know!
4022
4023 NOTE:
4024 Being listed on this page does not imply that the Hypothesis main‐
4025 tainers endorse a package.
4026
4027 External Strategies
4028 Some packages provide strategies directly:
4029
4030 · hypothesis-fspaths - strategy to generate filesystem paths.
4031
4032 · hypothesis-geojson - strategy to generate GeoJson.
4033
4034 · hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
4035
4036 · hypothesis_sqlalchemy - strategies to generate SQLAlchemy objects.
4037
4038 · hypothesis-ros - strategies to generate messages and parameters for
4039 the Robot Operating System.
4040
4041 Others provide a function to infer a strategy from some other schema:
4042
4043 · lollipop-hypothesis - infer strategies from lollipop schemas.
4044
4045 · hypothesis-drf - infer strategies from a djangorestframework seri‐
4046 aliser.
4047
4048 · hypothesis-mongoengine - infer strategies from a mongoengine model.
4049
4050 · hypothesis-pb - infer strategies from Protocol Buffer schemas.
4051
4052 Other Cool Things
4053 swagger-conformance is powered by Hypothesis and pyswagger. Based on a
4054 Swagger specification, it can build and run an entire test suite to
4055 check that the implementation matches the spec. The command-line ver‐
4056 sion can test apps written in any language, simply by passing the file
4057 or URL path to the schema to check!
4058
4059 Trio is an async framework with "an obsessive focus on usability and
4060 correctness", so naturally it works with Hypothesis! pytest-trio
4061 includes a custom hook that allows @given(...) to work with Trio-style
4062 async test functions, and hypothesis-trio includes stateful testing
4063 extensions to support concurrent programs.
4064
4065 libarchimedes makes it easy to use Hypothesis in the Hy language, a
4066 Lisp embedded in Python.
4067
4068 battle_tested is a fuzzing tool that will show you how your code can
4069 fail - by trying all kinds of inputs and reporting whatever happens.
4070
4071 pytest-subtesthack functions as a workaround for issue #377.
4072
4073 Writing an Extension
4074 See CONTRIBUTING.rst for more information.
4075
4076 New strategies can be added to Hypothesis, or published as an external
4077 package on PyPI - either is fine for most strategies. If in doubt, ask!
4078
4079 It's generally much easier to get things working outside, because
4080 there's more freedom to experiment and fewer requirements in stability
4081 and API style. We're happy to review and help with external packages as
4082 well as pull requests!
4083
4084 If you're thinking about writing an extension, please name it hypothe‐
4085 sis-{something} - a standard prefix makes the community more visible
4086 and searching for extensions easier.
4087
4088 On the other hand, being inside gets you access to some deeper imple‐
4089 mentation features (if you need them) and better long-term guarantees
4090 about maintenance. We particularly encourage pull requests for new
4091 composable primitives that make implementing other strategies easier,
4092 or for widely used types in the standard library. Strategies for other
4093 things are also welcome; anything with external dependencies just goes
4094 in hypothesis.extra.
4095
4097 This is a record of all past Hypothesis releases and what went into
4098 them, in reverse chronological order. All previous releases should
4099 still be available on pip.
4100
4101 Hypothesis APIs come in three flavours:
4102
4103 · Public: Hypothesis releases since 1.0 are semantically versioned with
4104 respect to these parts of the API. These will not break except
4105 between major version bumps. All APIs mentioned in this documentation
4106 are public unless explicitly noted otherwise.
4107
4108 · Semi-public: These are APIs that are considered ready to use but are
4109 not wholly nailed down yet. They will not break in patch releases and
4110 will usually not break in minor releases, but when necessary minor
4111 releases may break semi-public APIs.
4112
4113 · Internal: These may break at any time and you really should not use
4114 them at all.
4115
4116 You should generally assume that an API is internal unless you have
4117 specific information to the contrary.
4118
4119 3.66.11 - 2018-07-28
4120 This patch modifies how which rule to run is selected during rule based
4121 stateful testing. This should result in a slight performance increase
4122 during generation and a significant performance and quality improvement
4123 when shrinking.
4124
4125 As a result of this change, some state machines which would previously
4126 have thrown an InvalidDefinition are no longer detected as invalid.
4127
4128 3.66.10 - 2018-07-28
4129 This release weakens some minor functionality in the shrinker that had
4130 only modest benefit and made its behaviour much harder to reason about.
4131
4132 This is unlikely to have much user visible effect, but it is possible
4133 that in some cases shrinking may get slightly slower. It is primarily
4134 to make it easier to work on the shrinker and pave the way for future
4135 work.
4136
4137 3.66.9 - 2018-07-26
4138 This release improves the information that Hypothesis emits about its
4139 shrinking when verbosity is set to debug.
4140
4141 3.66.8 - 2018-07-24
4142 This patch includes some minor fixes in the documentation, and updates
4143 the minimum version of pytest to 3.0 (released August 2016).
4144
4145 3.66.7 - 2018-07-24
4146 This release fixes a bug where difficult to shrink tests could some‐
4147 times trigger an internal assertion error inside the shrinker.
4148
4149 3.66.6 - 2018-07-23
4150 This patch ensures that Hypothesis fully supports Python 3.7, by
4151 upgrading from_type() (issue #1264) and fixing some minor issues in our
4152 test suite (issue #1148).
4153
4154 3.66.5 - 2018-07-22
4155 This patch fixes the online docs for various extras, by ensuring that
4156 their dependencies are installed on readthedocs.io (issue #1326).
4157
4158 3.66.4 - 2018-07-20
4159 This release improves the shrinker's ability to reorder examples.
4160
4161 For example, consider the following test:
4162
4163 import hypothesis.strategies as st
4164 from hypothesis import given
4165
4166 @given(st.text(), st.text())
4167 def test_non_equal(x, y):
4168 assert x != y
4169
4170 Previously this could have failed with either of x="", y="0" or x="0",
4171 y="". Now it should always fail with x="", y="0".
4172
4173 This will allow the shrinker to produce more consistent results, espe‐
4174 cially in cases where test cases contain some ordered collection whose
4175 actual order does not matter.
4176
4177 3.66.3 - 2018-07-20
4178 This patch fixes inference in the builds() strategy with subtypes of
4179 python:typing.NamedTuple, where the __init__ method is not useful for
4180 introspection. We now use the field types instead - thanks to James
4181 Uther for identifying this bug.
4182
4183 3.66.2 - 2018-07-19
4184 This release improves the shrinker's ability to handle situations where
4185 there is an additive constraint between two values.
4186
4187 For example, consider the following test:
4188
4189 import hypothesis.strategies as st
4190 from hypothesis import given
4191
4192 @given(st.integers(), st.integers())
4193 def test_does_not_exceed_100(m, n):
4194 assert m + n < 100
4195
4196 Previously this could have failed with almost any pair (m, n) with 0 <=
4197 m <= n and m + n == 100. Now it should almost always fail with m=0,
4198 n=100.
4199
4200 This is a relatively niche specialisation, but can be useful in situa‐
4201 tions where e.g. a bug is triggered by an integer overflow.
4202
4203 3.66.1 - 2018-07-09
4204 This patch fixes a rare bug where an incorrect percentage drawtime
4205 could be displayed for a test, when the system clock was changed during
4206 a test running under Python 2 (we use python:time.monotonic() where it
4207 is available to avoid such problems). It also fixes a possible
4208 zero-division error that can occur when the underlying C library dou‐
4209 ble-rounds an intermediate value in python:math.fsum() and gets the
4210 least significant bit wrong.
4211
4212 3.66.0 - 2018-07-05
4213 This release improves validation of the alphabet argument to the text()
4214 strategy. The following misuses are now deprecated, and will be an
4215 error in a future version:
4216
4217 · passing an unordered collection (such as set('abc')), which violates
4218 invariants about shrinking and reproducibility
4219
4220 · passing an alphabet sequence with elements that are not strings
4221
4222 · passing an alphabet sequence with elements that are not of length
4223 one, which violates any size constraints that may apply
4224
4225 Thanks to Sushobhit for adding these warnings (issue #1329).
4226
4227 3.65.3 - 2018-07-04
4228 This release fixes a mostly theoretical bug where certain usage of the
4229 internal API could trigger an assertion error inside Hypothesis. It is
4230 unlikely that this problem is even possible to trigger through the pub‐
4231 lic API.
4232
4233 3.65.2 - 2018-07-04
4234 This release fixes dependency information for coverage. Previously
4235 Hypothesis would allow installing coverage with any version, but it
4236 only works with coverage 4.0 or later.
4237
4238 We now specify the correct metadata in our setup.py, so Hypothesis will
4239 only allow installation with compatible versions of coverage.
4240
4241 3.65.1 - 2018-07-03
4242 This patch ensures that stateful tests which raise an error from a
4243 pytest helper still print the sequence of steps taken to reach that
4244 point (issue #1372). This reporting was previously broken because the
4245 helpers inherit directly from BaseException, and therefore require spe‐
4246 cial handling to catch without breaking e.g. the use of ctrl-C to quit
4247 the test.
4248
4249 3.65.0 - 2018-06-30
4250 This release deprecates the max_shrinks setting in favor of an internal
4251 heuristic. If you need to avoid shrinking examples, use the phases
4252 setting instead. (issue #1235)
4253
4254 3.64.2 - 2018-06-27
4255 This release fixes a bug where an internal assertion error could some‐
4256 times be triggered while shrinking a failing test.
4257
4258 3.64.1 - 2018-06-27
4259 This patch fixes type-checking errors in our vendored pretty-printer,
4260 which were ignored by our mypy config but visible for anyone else
4261 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
4262
4263 3.64.0 - 2018-06-26
4264 This release adds an interface which can be used to insert a wrapper
4265 between the original test function and @given (issue #1257). This will
4266 be particularly useful for test runner extensions such as pytest-trio,
4267 but is not recommended for direct use by other users of Hypothesis.
4268
4269 3.63.0 - 2018-06-26
4270 This release adds a new mechanism to infer strategies for classes
4271 defined using attrs, based on the the type, converter, or validator of
4272 each attribute. This inference is now built in to builds() and
4273 from_type().
4274
4275 On Python 2, from_type() no longer generates instances of int when
4276 passed long, or vice-versa.
4277
4278 3.62.0 - 2018-06-26
4279 This release adds PEP 484 type hints to Hypothesis on a provisional
4280 basis, using the comment-based syntax for Python 2 compatibility. You
4281 can read more about our type hints here.
4282
4283 It also adds the py.typed marker specified in PEP 561. After you pip
4284 install hypothesis, mypy 0.590 or later will therefore type-check your
4285 use of our public interface!
4286
4287 3.61.0 - 2018-06-24
4288 This release deprecates the use of settings as a context manager, the
4289 use of which is somewhat ambiguous.
4290
4291 Users should define settings with global state or with the @set‐
4292 tings(...) decorator.
4293
4294 3.60.1 - 2018-06-20
4295 Fixed a bug in generating an instance of a Django model from a strategy
4296 where the primary key is generated as part of the strategy. See details
4297 here.
4298
4299 Thanks to Tim Martin for this contribution.
4300
4301 3.60.0 - 2018-06-20
4302 This release add initialize decorator for stateful testing (originally
4303 discussed in issue #1216). initialize act as a special rule that is
4304 only called once, and all initialize rules are guaranteed to be called
4305 before any normal rule is called.
4306
4307 3.59.3 - 2018-06-19
4308 This is a no-op release to take into account some changes to the
4309 release process. It should have no user visible effect.
4310
4311 3.59.2 - 2018-06-18
4312 This adds support for partially sorting examples which cannot be fully
4313 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
4314 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
4315 0].
4316
4317 Thanks to Luke for contributing.
4318
4319 3.59.1 - 2018-06-16
4320 This patch uses python:random.getstate() and python:random.setstate()
4321 to restore the PRNG state after @given runs deterministic tests. With‐
4322 out restoring state, you might have noticed problems such as issue
4323 #1266. The fix also applies to stateful testing (issue #702).
4324
4325 3.59.0 - 2018-06-14
4326 This release adds the emails() strategy, which generates unicode
4327 strings representing an email address.
4328
4329 Thanks to Sushobhit for moving this to the public API (issue #162).
4330
4331 3.58.1 - 2018-06-13
4332 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
4333 1 2 3.
4334
4335 Thanks to Luke for contributing.
4336
4337 3.58.0 - 2018-06-13
4338 This adds a new extra timezones() strategy that generates dateutil
4339 timezones.
4340
4341 Depends on python-dateutil.
4342
4343 Thanks to Conrad for contributing.
4344
4345 3.57.0 - 2018-05-20
4346 Using an unordered collection with the permutations() strategy has been
4347 deprecated because the order in which e.g. a set shrinks is arbitrary.
4348 This may cause different results between runs.
4349
4350 3.56.10 - 2018-05-16
4351 This release makes define_setting a private method, which has the
4352 effect of hiding it from the documentation.
4353
4354 3.56.9 - 2018-05-11
4355 This is another release with no functionality changes as part of
4356 changes to Hypothesis's new release tagging scheme.
4357
4358 3.56.8 - 2018-05-10
4359 This is a release with no functionality changes that moves Hypothesis
4360 over to a new release tagging scheme.
4361
4362 3.56.7 - 2018-05-10
4363 This release provides a performance improvement for most tests, but in
4364 particular users of sampled_from who don't have numpy installed should
4365 see a significant performance improvement.
4366
4367 3.56.6 - 2018-05-09
4368 This patch contains further internal work to support Mypy. There are
4369 no user-visible changes... yet.
4370
4371 3.56.5 - 2018-04-22
4372 This patch contains some internal refactoring to run mypy in CI. There
4373 are no user-visible changes.
4374
4375 3.56.4 - 2018-04-21
4376 This release involves some very minor internal clean up and should have
4377 no user visible effect at all.
4378
4379 3.56.3 - 2018-04-20
4380 This release fixes a problem introduced in 3.56.0 where setting the
4381 hypothesis home directory (through currently undocumented means) would
4382 no longer result in the default database location living in the new
4383 home directory.
4384
4385 3.56.2 - 2018-04-20
4386 This release fixes a problem introduced in 3.56.0 where setting
4387 max_examples to 1 would result in tests failing with Unsatisfiable.
4388 This problem could also occur in other harder to trigger circumstances
4389 (e.g. by setting it to a low value, having a hard to satisfy assump‐
4390 tion, and disabling health checks).
4391
4392 3.56.1 - 2018-04-20
4393 This release fixes a problem that was introduced in 3.56.0: Use of the
4394 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
4395 cated, actually broken due to being read before various setup the dep‐
4396 recation path needed was done. It now works correctly (and emits a dep‐
4397 recation warning).
4398
4399 3.56.0 - 2018-04-17
4400 This release deprecates several redundant or internally oriented set‐
4401 tings, working towards an orthogonal set of configuration options that
4402 are widely useful without requiring any knowledge of our internals (‐
4403 issue #535).
4404
4405 · Deprecated settings that no longer have any effect are no longer
4406 shown in the __repr__ unless set to a non-default value.
4407
4408 · perform_health_check is deprecated, as it duplicates sup‐
4409 press_health_check.
4410
4411 · max_iterations is deprecated and disabled, because we can usually get
4412 better behaviour from an internal heuristic than a user-controlled
4413 setting.
4414
4415 · min_satisfying_examples is deprecated and disabled, due to overlap
4416 with the filter_too_much healthcheck and poor interaction with
4417 max_examples.
4418
4419 · HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
4420 the profile system instead.
4421
4422 · Examples tried by find() are now reported at debug verbosity level
4423 (as well as verbose level).
4424
4425 3.55.6 - 2018-04-14
4426 This release fixes a somewhat obscure condition (issue #1230) under
4427 which you could occasionally see a failing test trigger an assertion
4428 error inside Hypothesis instead of failing normally.
4429
4430 3.55.5 - 2018-04-14
4431 This patch fixes one possible cause of issue #966. When running Python
4432 2 with hash randomisation, passing a python:bytes object to python:ran‐
4433 dom.seed() would use version=1, which broke derandomize (because the
4434 seed depended on a randomised hash). If derandomize is still nondeter‐
4435 ministic for you, please open an issue.
4436
4437 3.55.4 - 2018-04-13
4438 This patch makes a variety of minor improvements to the documentation,
4439 and improves a few validation messages for invalid inputs.
4440
4441 3.55.3 - 2018-04-12
4442 This release updates the URL metadata associated with the PyPI package
4443 (again). It has no other user visible effects.
4444
4445 3.55.2 - 2018-04-11
4446 This release updates the URL metadata associated with the PyPI package.
4447 It has no other user visible effects.
4448
4449 3.55.1 - 2018-04-06
4450 This patch relaxes constraints on the expected values returned by the
4451 standard library function hypot() and the internal helper function
4452 cathetus(), this to fix near-exact test-failures on some 32-bit sys‐
4453 tems.
4454
4455 3.55.0 - 2018-04-05
4456 This release includes several improvements to the handling of the data‐
4457 base setting.
4458
4459 · The database_file setting was a historical artefact, and you should
4460 just use database directly.
4461
4462 · The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
4463 favor of load_profile() and the database setting.
4464
4465 · If you have not configured the example database at all and the
4466 default location is not usable (due to e.g. permissions issues),
4467 Hypothesis will fall back to an in-memory database. This is not per‐
4468 sisted between sessions, but means that the defaults work on
4469 read-only filesystems.
4470
4471 3.54.0 - 2018-04-04
4472 This release improves the complex_numbers() strategy, which now sup‐
4473 ports min_magnitude and max_magnitude arguments, along with allow_nan
4474 and allow_infinity like for floats().
4475
4476 Thanks to J.J. Green for this feature.
4477
4478 3.53.0 - 2018-04-01
4479 This release removes support for Django 1.8, which reached end of life
4480 on 2018-04-01. You can see Django's release and support schedule on
4481 the Django Project website.
4482
4483 3.52.3 - 2018-04-01
4484 This patch fixes the min_satisfying_examples settings documentation, by
4485 explaining that example shrinking is tracked at the level of the under‐
4486 lying bytestream rather than the output value.
4487
4488 The output from find() in verbose mode has also been adjusted - see the
4489 example session - to avoid duplicating lines when the example repr is
4490 constant, even if the underlying representation has been shrunken.
4491
4492 3.52.2 - 2018-03-30
4493 This release improves the output of failures with rule based stateful
4494 testing in two ways:
4495
4496 · The output from it is now usually valid Python code.
4497
4498 · When the same value has two different names because it belongs to two
4499 different bundles, it will now display with the name associated with
4500 the correct bundle for a rule argument where it is used.
4501
4502 3.52.1 - 2018-03-29
4503 This release improves the behaviour of stateful testing in two ways:
4504
4505 · Previously some runs would run no steps (issue #376). This should no
4506 longer happen.
4507
4508 · RuleBasedStateMachine tests which used bundles extensively would
4509 often shrink terribly. This should now be significantly improved,
4510 though there is likely a lot more room for improvement.
4511
4512 This release also involves a low level change to how ranges of integers
4513 are handles which may result in other improvements to shrink quality in
4514 some cases.
4515
4516 3.52.0 - 2018-03-24
4517 This release deprecates use of @settings(...) as a decorator, on func‐
4518 tions or methods that are not also decorated with @given. You can
4519 still apply these decorators in any order, though you should only do so
4520 once each.
4521
4522 Applying @given twice was already deprecated, and applying @set‐
4523 tings(...) twice is deprecated in this release and will become an error
4524 in a future version. Neither could ever be used twice to good effect.
4525
4526 Using @settings(...) as the sole decorator on a test is completely
4527 pointless, so this common usage error will become an error in a future
4528 version of Hypothesis.
4529
4530 3.51.0 - 2018-03-24
4531 This release deprecates the average_size argument to lists() and other
4532 collection strategies. You should simply delete it wherever it was
4533 used in your tests, as it no longer has any effect.
4534
4535 In early versions of Hypothesis, the average_size argument was treated
4536 as a hint about the distribution of examples from a strategy. Subse‐
4537 quent improvements to the conceptual model and the engine for generat‐
4538 ing and shrinking examples mean it is more effective to simply describe
4539 what constitutes a valid example, and let our internals handle the dis‐
4540 tribution.
4541
4542 3.50.3 - 2018-03-24
4543 This patch contains some internal refactoring so that we can run with
4544 warnings as errors in CI.
4545
4546 3.50.2 - 2018-03-20
4547 This has no user-visible changes except one slight formatting change to
4548 one docstring, to avoid a deprecation warning.
4549
4550 3.50.1 - 2018-03-20
4551 This patch fixes an internal error introduced in 3.48.0, where a check
4552 for the Django test runner would expose import-time errors in Django
4553 configuration (issue #1167).
4554
4555 3.50.0 - 2018-03-19
4556 This release improves validation of numeric bounds for some strategies.
4557
4558 · integers() and floats() now raise InvalidArgument if passed a
4559 min_value or max_value which is not an instance of Real, instead of
4560 various internal errors.
4561
4562 · floats() now converts its bounding values to the nearest float above
4563 or below the min or max bound respectively, instead of just casting
4564 to float. The old behaviour was incorrect in that you could generate
4565 float(min_value), even when this was less than min_value itself (pos‐
4566 sible with eg. fractions).
4567
4568 · When both bounds are provided to floats() but there are no floats in
4569 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
4570 raised.
4571
4572 · decimals() gives a more useful error message if passed a string that
4573 cannot be converted to Decimal in a context where this error is not
4574 trapped.
4575
4576 Code that previously seemed to work may be explicitly broken if there
4577 were no floats between min_value and max_value (only possible with
4578 non-float bounds), or if a bound was not a Real number but still
4579 allowed in python:math.isnan (some custom classes with a __float__
4580 method).
4581
4582 3.49.1 - 2018-03-15
4583 This patch fixes our tests for Numpy dtype strategies on big-endian
4584 platforms, where the strategy behaved correctly but the test assumed
4585 that the native byte order was little-endian.
4586
4587 There is no user impact unless you are running our test suite on
4588 big-endian platforms. Thanks to Graham Inggs for reporting issue
4589 #1164.
4590
4591 3.49.0 - 2018-03-12
4592 This release deprecates passing elements=None to collection strategies,
4593 such as lists().
4594
4595 Requiring lists(nothing()) or builds(list) instead of lists() means
4596 slightly more typing, but also improves the consistency and discover‐
4597 ability of our API - as well as showing how to compose or construct
4598 strategies in ways that still work in more complex situations.
4599
4600 Passing a nonzero max_size to a collection strategy where the elements
4601 strategy contains no values is now deprecated, and will be an error in
4602 a future version. The equivalent with elements=None is already an
4603 error.
4604
4605 3.48.1 - 2018-03-05
4606 This patch will minimize examples that would come out non-minimal in
4607 previous versions. Thanks to Kyle Reeve for this patch.
4608
4609 3.48.0 - 2018-03-05
4610 This release improves some "unhappy paths" when using Hypothesis with
4611 the standard library python:unittest module:
4612
4613 · Applying @given to a non-test method which is overridden from
4614 python:unittest.TestCase, such as setUp, raises a new health check.
4615 (issue #991)
4616
4617 · Using subTest() within a test decorated with @given would leak inter‐
4618 mediate results when tests were run under the python:unittest test
4619 runner. Individual reporting of failing subtests is now disabled
4620 during a test using @given. (issue #1071)
4621
4622 · @given is still not a class decorator, but the error message if you
4623 try using it on a class has been improved.
4624
4625 As a related improvement, using django:django.test.TestCase with @given
4626 instead of hypothesis.extra.django.TestCase raises an explicit error
4627 instead of running all examples in a single database transaction.
4628
4629 3.47.0 - 2018-03-02
4630 register_profile now accepts keyword arguments for specific settings,
4631 and the parent settings object is now optional. Using a name for a
4632 registered profile which is not a string was never suggested, but it is
4633 now also deprecated and will eventually be an error.
4634
4635 3.46.2 - 2018-03-01
4636 This release removes an unnecessary branch from the code, and has no
4637 user-visible impact.
4638
4639 3.46.1 - 2018-03-01
4640 This changes only the formatting of our docstrings and should have no
4641 user-visible effects.
4642
4643 3.46.0 - 2018-02-26
4644 characters() has improved docs about what arguments are valid, and
4645 additional validation logic to raise a clear error early (instead of
4646 e.g. silently ignoring a bad argument). Categories may be specified as
4647 the Unicode 'general category' (eg u'Nd'), or as the 'major category'
4648 (eg [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
4649
4650 In previous versions, general categories were supported and all other
4651 input was silently ignored. Now, major categories are supported in
4652 addition to general categories (which may change the behaviour of some
4653 existing code), and all other input is deprecated.
4654
4655 3.45.5 - 2018-02-26
4656 This patch improves strategy inference in hypothesis.extra.django to
4657 account for some validators in addition to field type - see issue #1116
4658 for ongoing work in this space.
4659
4660 Specifically, if a CharField or TextField has an attached RegexValida‐
4661 tor, we now use from_regex() instead of text() as the underlying strat‐
4662 egy. This allows us to generate examples of the default User model,
4663 closing issue #1112.
4664
4665 3.45.4 - 2018-02-25
4666 This patch improves some internal debugging information, fixes a typo
4667 in a validation error message, and expands the documentation for new
4668 contributors.
4669
4670 3.45.3 - 2018-02-23
4671 This patch may improve example shrinking slightly for some strategies.
4672
4673 3.45.2 - 2018-02-18
4674 This release makes our docstring style more consistent, thanks to
4675 flake8-docstrings. There are no user-visible changes.
4676
4677 3.45.1 - 2018-02-17
4678 This fixes an indentation issue in docstrings for datetimes(), dates(),
4679 times(), and timedeltas().
4680
4681 3.45.0 - 2018-02-13
4682 This release fixes builds() so that target can be used as a keyword
4683 argument for passing values to the target. The target itself can still
4684 be specified as a keyword argument, but that behavior is now depre‐
4685 cated. The target should be provided as the first positional argument.
4686
4687 3.44.26 - 2018-02-06
4688 This release fixes some formatting issues in the Hypothesis source
4689 code. It should have no externally visible effects.
4690
4691 3.44.25 - 2018-02-05
4692 This release changes the way in which Hypothesis tries to shrink the
4693 size of examples. It probably won't have much impact, but might make
4694 shrinking faster in some cases. It is unlikely but not impossible that
4695 it will change the resulting examples.
4696
4697 3.44.24 - 2018-01-27
4698 This release fixes dependency information when installing Hypothesis
4699 from a binary "wheel" distribution.
4700
4701 · The install_requires for enum34 is resolved at install time, rather
4702 than at build time (with potentially different results).
4703
4704 · Django has fixed their python_requires for versions 2.0.0 onward,
4705 simplifying Python2-compatible constraints for downstream projects.
4706
4707 3.44.23 - 2018-01-24
4708 This release improves shrinking in a class of pathological examples
4709 that you are probably never hitting in practice. If you are hitting
4710 them in practice this should be a significant speed up in shrinking. If
4711 you are not, you are very unlikely to notice any difference. You might
4712 see a slight slow down and/or slightly better falsifying examples.
4713
4714 3.44.22 - 2018-01-23
4715 This release fixes a dependency problem. It was possible to install
4716 Hypothesis with an old version of attrs, which would throw a TypeError
4717 as soon as you tried to import hypothesis. Specifically, you need
4718 attrs 16.0.0 or newer.
4719
4720 Hypothesis will now require the correct version of attrs when
4721 installing.
4722
4723 3.44.21 - 2018-01-22
4724 This change adds some additional structural information that Hypothesis
4725 will use to guide its search.
4726
4727 You mostly shouldn't see much difference from this. The two most likely
4728 effects you would notice are:
4729
4730 1. Hypothesis stores slightly more examples in its database for passing
4731 tests.
4732
4733 2. Hypothesis may find new bugs that it was previously missing, but it
4734 probably won't (this is a basic implementation of the feature that
4735 is intended to support future work. Although it is useful on its
4736 own, it's not very useful on its own).
4737
4738 3.44.20 - 2018-01-21
4739 This is a small refactoring release that changes how Hypothesis tracks
4740 some information about the boundary of examples in its internal repre‐
4741 sentation.
4742
4743 You are unlikely to see much difference in behaviour, but memory usage
4744 and run time may both go down slightly during normal test execution,
4745 and when failing Hypothesis might print its failing example slightly
4746 sooner.
4747
4748 3.44.19 - 2018-01-21
4749 This changes how we compute the default average_size for all collection
4750 strategies. Previously setting a max_size without setting an aver‐
4751 age_size would have the seemingly paradoxical effect of making data
4752 generation slower, because it would raise the average size from its
4753 default. Now setting max_size will either leave the default unchanged
4754 or lower it from its default.
4755
4756 If you are currently experiencing this problem, this may make your
4757 tests substantially faster. If you are not, this will likely have no
4758 effect on you.
4759
4760 3.44.18 - 2018-01-20
4761 This is a small refactoring release that changes how Hypothesis detects
4762 when the structure of data generation depends on earlier values gener‐
4763 ated (e.g. when using flatmap or composite()). It should not have any
4764 observable effect on behaviour.
4765
4766 3.44.17 - 2018-01-15
4767 This release fixes a typo in internal documentation, and has no
4768 user-visible impact.
4769
4770 3.44.16 - 2018-01-13
4771 This release improves test case reduction for recursive data struc‐
4772 tures. Hypothesis now guarantees that whenever a strategy calls itself
4773 recursively (usually this will happen because you are using
4774 deferred()), any recursive call may replace the top level value. e.g.
4775 given a tree structure, Hypothesis will always try replacing it with a
4776 subtree.
4777
4778 Additionally this introduces a new heuristic that may in some circum‐
4779 stances significantly speed up test case reduction - Hypothesis should
4780 be better at immediately replacing elements drawn inside another strat‐
4781 egy with their minimal possible value.
4782
4783 3.44.15 - 2018-01-13
4784 from_type() can now resolve recursive types such as binary trees (issue
4785 #1004). Detection of non-type arguments has also improved, leading to
4786 better error messages in many cases involving forward references.
4787
4788 3.44.14 - 2018-01-08
4789 This release fixes a bug in the shrinker that prevented the optimisa‐
4790 tions in 3.44.6 from working in some cases. It would not have worked
4791 correctly when filtered examples were nested (e.g. with a set of inte‐
4792 gers in some range).
4793
4794 This would not have resulted in any correctness problems, but shrinking
4795 may have been slower than it otherwise could be.
4796
4797 3.44.13 - 2018-01-08
4798 This release changes the average bit length of values drawn from inte‐
4799 gers() to be much smaller. Additionally it changes the shrinking order
4800 so that now size is considered before sign - e.g. -1 will be preferred
4801 to +10.
4802
4803 The new internal format for integers required some changes to the mini‐
4804 mizer to make work well, so you may also see some improvements to exam‐
4805 ple quality in unrelated areas.
4806
4807 3.44.12 - 2018-01-07
4808 This changes Hypothesis's internal implementation of weighted sampling.
4809 This will affect example distribution and quality, but you shouldn't
4810 see any other effects.
4811
4812 3.44.11 - 2018-01-06
4813 This is a change to some internals around how Hypothesis handles avoid‐
4814 ing generating duplicate examples and seeking out novel regions of the
4815 search space.
4816
4817 You are unlikely to see much difference as a result of it, but it fixes
4818 a bug where an internal assertion could theoretically be triggered and
4819 has some minor effects on the distribution of examples so could poten‐
4820 tially find bugs that have previously been missed.
4821
4822 3.44.10 - 2018-01-06
4823 This patch avoids creating debug statements when debugging is disabled.
4824 Profiling suggests this is a 5-10% performance improvement (issue
4825 #1040).
4826
4827 3.44.9 - 2018-01-06
4828 This patch blacklists null characters ('\x00') in automatically created
4829 strategies for Django CharField and TextField, due to a database issue
4830 which was recently fixed upstream (Hypothesis issue #1045).
4831
4832 3.44.8 - 2018-01-06
4833 This release makes the Hypothesis shrinker slightly less greedy in
4834 order to avoid local minima - when it gets stuck, it makes a small
4835 attempt to search around the final example it would previously have
4836 returned to find a new starting point to shrink from. This should
4837 improve example quality in some cases, especially ones where the test
4838 data has dependencies among parts of it that make it difficult for
4839 Hypothesis to proceed.
4840
4841 3.44.7 - 2018-01-04
4842 This release adds support for Django 2 in the hypothesis-django extra.
4843
4844 This release drops support for Django 1.10, as it is no longer sup‐
4845 ported by the Django team.
4846
4847 3.44.6 - 2018-01-02
4848 This release speeds up test case reduction in many examples by being
4849 better at detecting large shrinks it can use to discard redundant parts
4850 of its input. This will be particularly noticeable in examples that
4851 make use of filtering and for some integer ranges.
4852
4853 3.44.5 - 2018-01-02
4854 Happy new year!
4855
4856 This is a no-op release that updates the year range on all of the copy‐
4857 right headers in our source to include 2018.
4858
4859 3.44.4 - 2017-12-23
4860 This release fixes issue #1044, which slowed tests by up to 6% due to
4861 broken caching.
4862
4863 3.44.3 - 2017-12-21
4864 This release improves the shrinker in cases where examples drawn ear‐
4865 lier can affect how much data is drawn later (e.g. when you draw a
4866 length parameter in a composite and then draw that many elements).
4867 Examples found in cases like this should now be much closer to minimal.
4868
4869 3.44.2 - 2017-12-20
4870 This is a pure refactoring release which changes how Hypothesis manages
4871 its set of examples internally. It should have no externally visible
4872 effects.
4873
4874 3.44.1 - 2017-12-18
4875 This release fixes issue #997, in which under some circumstances the
4876 body of tests run under Hypothesis would not show up when run under
4877 coverage even though the tests were run and the code they called out‐
4878 side of the test file would show up normally.
4879
4880 3.44.0 - 2017-12-17
4881 This release adds a new feature: The @reproduce_failure decorator,
4882 designed to make it easy to use Hypothesis's binary format for examples
4883 to reproduce a problem locally without having to share your example
4884 database between machines.
4885
4886 This also changes when seeds are printed:
4887
4888 · They will no longer be printed for normal falsifying examples, as
4889 there are now adequate ways of reproducing those for all cases, so it
4890 just contributes noise.
4891
4892 · They will once again be printed when reusing examples from the data‐
4893 base, as health check failures should now be more reliable in this
4894 scenario so it will almost always work in this case.
4895
4896 This work was funded by Smarkets.
4897
4898 3.43.1 - 2017-12-17
4899 This release fixes a bug with Hypothesis's database management - exam‐
4900 ples that were found in the course of shrinking were saved in a way
4901 that indicated that they had distinct causes, and so they would all be
4902 retried on the start of the next test. The intended behaviour, which is
4903 now what is implemented, is that only a bounded subset of these exam‐
4904 ples would be retried.
4905
4906 3.43.0 - 2017-12-17
4907 HypothesisDeprecationWarning now inherits from python:FutureWarning
4908 instead of python:DeprecationWarning, as recommended by PEP 565 for
4909 user-facing warnings (issue #618). If you have not changed the default
4910 warnings settings, you will now see each distinct HypothesisDepreca‐
4911 tionWarning instead of only the first.
4912
4913 3.42.2 - 2017-12-12
4914 This patch fixes issue #1017, where instances of a list or tuple sub‐
4915 type used as an argument to a strategy would be coerced to tuple.
4916
4917 3.42.1 - 2017-12-10
4918 This release has some internal cleanup, which makes reading the code
4919 more pleasant and may shrink large examples slightly faster.
4920
4921 3.42.0 - 2017-12-09
4922 This release deprecates faker-extra, which was designed as a transition
4923 strategy but does not support example shrinking or coverage-guided dis‐
4924 covery.
4925
4926 3.41.0 - 2017-12-06
4927 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
4928 pling from multi-dimensional ndarrays still results in a deprecation
4929 warning. Thanks to Charlie Tanksley for this patch.
4930
4931 3.40.1 - 2017-12-04
4932 This release makes two changes:
4933
4934 · It makes the calculation of some of the metadata that Hypothesis uses
4935 for shrinking occur lazily. This should speed up performance of test
4936 case generation a bit because it no longer calculates information it
4937 doesn't need.
4938
4939 · It improves the shrinker for certain classes of nested examples. e.g.
4940 when shrinking lists of lists, the shrinker is now able to concate‐
4941 nate two adjacent lists together into a single list. As a result of
4942 this change, shrinking may get somewhat slower when the minimal exam‐
4943 ple found is large.
4944
4945 3.40.0 - 2017-12-02
4946 This release improves how various ways of seeding Hypothesis interact
4947 with the example database:
4948
4949 · Using the example database with seed() is now deprecated. You should
4950 set database=None if you are doing that. This will only warn if you
4951 actually load examples from the database while using @seed.
4952
4953 · The derandomize will behave the same way as @seed.
4954
4955 · Using --hypothesis-seed will disable use of the database.
4956
4957 · If a test used examples from the database, it will not suggest using
4958 a seed to reproduce it, because that won't work.
4959
4960 This work was funded by Smarkets.
4961
4962 3.39.0 - 2017-12-01
4963 This release adds a new health check that checks if the smallest "natu‐
4964 ral" possible example of your test case is very large - this will tend
4965 to cause Hypothesis to generate bad examples and be quite slow.
4966
4967 This work was funded by Smarkets.
4968
4969 3.38.9 - 2017-11-29
4970 This is a documentation release to improve the documentation of shrink‐
4971 ing behaviour for Hypothesis's strategies.
4972
4973 3.38.8 - 2017-11-29
4974 This release improves the performance of characters() when using black‐
4975 list_characters and from_regex() when using negative character classes.
4976
4977 The problems this fixes were found in the course of work funded by
4978 Smarkets.
4979
4980 3.38.7 - 2017-11-29
4981 This is a patch release for from_regex(), which had a bug in handling
4982 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
4983 rectly when parsing regex.
4984
4985 3.38.6 - 2017-11-28
4986 This patch changes a few byte-string literals from double to single
4987 quotes, thanks to an update in unify. There are no user-visible
4988 changes.
4989
4990 3.38.5 - 2017-11-23
4991 This fixes the repr of strategies using lambda that are defined inside
4992 decorators to include the lambda source.
4993
4994 This would mostly have been visible when using the statistics function‐
4995 ality - lambdas used for e.g. filtering would have shown up with a
4996 <unknown> as their body. This can still happen, but it should happen
4997 less often now.
4998
4999 3.38.4 - 2017-11-22
5000 This release updates the reported statistics so that they show approxi‐
5001 mately what fraction of your test run time is spent in data generation
5002 (as opposed to test execution).
5003
5004 This work was funded by Smarkets.
5005
5006 3.38.3 - 2017-11-21
5007 This is a documentation release, which ensures code examples are up to
5008 date by running them as doctests in CI (issue #711).
5009
5010 3.38.2 - 2017-11-21
5011 This release changes the behaviour of the deadline setting when used
5012 with data(): Time spent inside calls to data.draw will no longer be
5013 counted towards the deadline time.
5014
5015 As a side effect of some refactoring required for this work, the way
5016 flaky tests are handled has changed slightly. You are unlikely to see
5017 much difference from this, but some error messages will have changed.
5018
5019 This work was funded by Smarkets.
5020
5021 3.38.1 - 2017-11-21
5022 This patch has a variety of non-user-visible refactorings, removing
5023 various minor warts ranging from indirect imports to typos in comments.
5024
5025 3.38.0 - 2017-11-18
5026 This release overhauls the health check system in a variety of small
5027 ways. It adds no new features, but is nevertheless a minor release
5028 because it changes which tests are likely to fail health checks.
5029
5030 The most noticeable effect is that some tests that used to fail health
5031 checks will now pass, and some that used to pass will fail. These
5032 should all be improvements in accuracy. In particular:
5033
5034 · New failures will usually be because they are now taking into account
5035 things like use of data() and assume() inside the test body.
5036
5037 · New failures may also be because for some classes of example the way
5038 data generation performance was measured was artificially faster than
5039 real data generation (for most examples that are hitting performance
5040 health checks the opposite should be the case).
5041
5042 · Tests that used to fail health checks and now pass do so because the
5043 health check system used to run in a way that was subtly different
5044 than the main Hypothesis data generation and lacked some of its sup‐
5045 port for e.g. large examples.
5046
5047 If your data generation is especially slow, you may also see your tests
5048 get somewhat faster, as there is no longer a separate health check
5049 phase. This will be particularly noticeable when rerunning test fail‐
5050 ures.
5051
5052 This work was funded by Smarkets.
5053
5054 3.37.0 - 2017-11-12
5055 This is a deprecation release for some health check related features.
5056
5057 The following are now deprecated:
5058
5059 · Passing exception_in_generation to suppress_health_check. This no
5060 longer does anything even when passed - All errors that occur during
5061 data generation will now be immediately reraised rather than going
5062 through the health check mechanism.
5063
5064 · Passing random_module to suppress_health_check. This hasn't done any‐
5065 thing for a long time, but was never explicitly deprecated. Hypothe‐
5066 sis always seeds the random module when running @given tests, so this
5067 is no longer an error and suppressing it doesn't do anything.
5068
5069 · Passing non-HealthCheck values in suppress_health_check. This was
5070 previously allowed but never did anything useful.
5071
5072 In addition, passing a non-iterable value as suppress_health_check will
5073 now raise an error immediately (it would never have worked correctly,
5074 but it would previously have failed later). Some validation error mes‐
5075 sages have also been updated.
5076
5077 This work was funded by Smarkets.
5078
5079 3.36.1 - 2017-11-10
5080 This is a yak shaving release, mostly concerned with our own tests.
5081
5082 While getfullargspec() was documented as deprecated in Python 3.5, it
5083 never actually emitted a warning. Our code to silence this (nonexis‐
5084 tent) warning has therefore been removed.
5085
5086 We now run our tests with DeprecationWarning as an error, and made some
5087 minor changes to our own tests as a result. This required similar
5088 upstream updates to coverage and execnet (a test-time dependency via
5089 pytest-xdist).
5090
5091 There is no user-visible change in Hypothesis itself, but we encourage
5092 you to consider enabling deprecations as errors in your own tests.
5093
5094 3.36.0 - 2017-11-06
5095 This release adds a setting to the public API, and does some internal
5096 cleanup:
5097
5098 · The derandomize setting is now documented (issue #890)
5099
5100 · Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
5101 #953)
5102
5103 · Documented the strict setting as deprecated, and updated the build so
5104 our docs always match deprecations in the code.
5105
5106 3.35.0 - 2017-11-06
5107 This minor release supports constraining uuids() to generate a particu‐
5108 lar version of UUID (issue #721).
5109
5110 Thanks to Dion Misic for this feature.
5111
5112 3.34.1 - 2017-11-02
5113 This patch updates the documentation to suggest builds(callable)
5114 instead of just(callable()).
5115
5116 3.34.0 - 2017-11-02
5117 Hypothesis now emits deprecation warnings if you apply @given more than
5118 once to a target.
5119
5120 Applying @given repeatedly wraps the target multiple times. Each wrap‐
5121 per will search the space of of possible parameters separately. This
5122 is equivalent but will be much more inefficient than doing it with a
5123 single call to @given.
5124
5125 For example, instead of @given(booleans()) @given(integers()), you
5126 could write @given(booleans(), integers())
5127
5128 3.33.1 - 2017-11-02
5129 This is a bugfix release:
5130
5131 · builds() would try to infer a strategy for required positional argu‐
5132 ments of the target from type hints, even if they had been given to
5133 builds() as positional arguments (issue #946). Now it only infers
5134 missing required arguments.
5135
5136 · An internal introspection function wrongly reported self as a
5137 required argument for bound methods, which might also have affected
5138 builds(). Now it knows better.
5139
5140 3.33.0 - 2017-10-16
5141 This release supports strategy inference for more field types in Django
5142 models() - you can now omit an argument for Date, Time, Duration, Slug,
5143 IP Address, and UUID fields. (issue #642)
5144
5145 Strategy generation for fields with grouped choices now selects choices
5146 from each group, instead of selecting from the group names.
5147
5148 3.32.2 - 2017-10-15
5149 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
5150 an experimental basis. It has never actually worked, and the new
5151 Hypothesis example database is designed to make such a tool unneces‐
5152 sary.
5153
5154 3.32.1 - 2017-10-13
5155 This patch has two improvements for strategies based on enumerations.
5156
5157 · from_type() now handles enumerations correctly, delegating to sam‐
5158 pled_from(). Previously it noted that Enum.__init__ has no required
5159 arguments and therefore delegated to builds(), which would subse‐
5160 quently fail.
5161
5162 · When sampling from an python:enum.Flag, we also generate combinations
5163 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
5164 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
5165
5166 3.32.0 - 2017-10-09
5167 This changes the default value of use_coverage=True to True when run‐
5168 ning on pypy (it was already True on CPython).
5169
5170 It was previously set to False because we expected it to be too slow,
5171 but recent benchmarking shows that actually performance of the feature
5172 on pypy is fairly acceptable - sometimes it's slower than on CPython,
5173 sometimes it's faster, but it's generally within a factor of two either
5174 way.
5175
5176 3.31.6 - 2017-10-08
5177 This patch improves the quality of strategies inferred from Numpy
5178 dtypes:
5179
5180 · Integer dtypes generated examples with the upper half of their
5181 (non-sign) bits set to zero. The inferred strategies can now produce
5182 any representable integer.
5183
5184 · Fixed-width unicode- and byte-string dtypes now cap the internal
5185 example length, which should improve example and shrink quality.
5186
5187 · Numpy arrays can only store fixed-size strings internally, and allow
5188 shorter strings by right-padding them with null bytes. Inferred
5189 string strategies no longer generate such values, as they can never
5190 be retrieved from an array. This improves shrinking performance by
5191 skipping useless values.
5192
5193 This has already been useful in Hypothesis - we found an overflow bug
5194 in our Pandas support, and as a result indexes() and range_indexes()
5195 now check that min_size and max_size are at least zero.
5196
5197 3.31.5 - 2017-10-08
5198 This release fixes a performance problem in tests where use_coverage is
5199 set to True.
5200
5201 Tests experience a slow-down proportionate to the amount of code they
5202 cover. This is still the case, but the factor is now low enough that
5203 it should be unnoticeable. Previously it was large and became much
5204 larger in 3.30.4.
5205
5206 3.31.4 - 2017-10-08
5207 from_type() failed with a very confusing error if passed a NewType() (‐
5208 issue #901). These psudeo-types are now unwrapped correctly, and
5209 strategy inference works as expected.
5210
5211 3.31.3 - 2017-10-06
5212 This release makes some small optimisations to our use of coverage that
5213 should reduce constant per-example overhead. This is probably only
5214 noticeable on examples where the test itself is quite fast. On no-op
5215 tests that don't test anything you may see up to a fourfold speed
5216 increase (which is still significantly slower than without coverage).
5217 On more realistic tests the speed up is likely to be less than that.
5218
5219 3.31.2 - 2017-09-30
5220 This release fixes some formatting and small typos/grammar issues in
5221 the documentation, specifically the page docs/settings.rst, and the
5222 inline docs for the various settings.
5223
5224 3.31.1 - 2017-09-30
5225 This release improves the handling of deadlines so that they act better
5226 with the shrinking process. This fixes issue #892.
5227
5228 This involves two changes:
5229
5230 1. The deadline is raised during the initial generation and shrinking,
5231 and then lowered to the set value for final replay. This restricts
5232 our attention to examples which exceed the deadline by a more sig‐
5233 nificant margin, which increases their reliability.
5234
5235 2. When despite the above a test still becomes flaky because it is sig‐
5236 nificantly faster on rerun than it was on its first run, the error
5237 message is now more explicit about the nature of this problem, and
5238 includes both the initial test run time and the new test run time.
5239
5240 In addition, this release also clarifies the documentation of the dead‐
5241 line setting slightly to be more explicit about where it applies.
5242
5243 This work was funded by Smarkets.
5244
5245 3.31.0 - 2017-09-29
5246 This release blocks installation of Hypothesis on Python 3.3, which
5247 reached its end of life date on 2017-09-29.
5248
5249 This should not be of interest to anyone but downstream maintainers -
5250 if you are affected, migrate to a secure version of Python as soon as
5251 possible or at least seek commercial support.
5252
5253 3.30.4 - 2017-09-27
5254 This release makes several changes:
5255
5256 1. It significantly improves Hypothesis's ability to use coverage
5257 information to find interesting examples.
5258
5259 2. It reduces the default max_examples setting from 200 to 100. This
5260 takes advantage of the improved algorithm meaning fewer examples are
5261 typically needed to get the same testing and is sufficiently better
5262 at covering interesting behaviour, and offsets some of the perfor‐
5263 mance problems of running under coverage.
5264
5265 3. Hypothesis will always try to start its testing with an example that
5266 is near minimized.
5267
5268 The new algorithm for 1 also makes some changes to Hypothesis's low
5269 level data generation which apply even with coverage turned off. They
5270 generally reduce the total amount of data generated, which should
5271 improve test performance somewhat. Between this and 3 you should see a
5272 noticeable reduction in test runtime (how much so depends on your tests
5273 and how much example size affects their performance. On our benchmarks,
5274 where data generation dominates, we saw up to a factor of two perfor‐
5275 mance improvement, but it's unlikely to be that large.
5276
5277 3.30.3 - 2017-09-25
5278 This release fixes some formatting and small typos/grammar issues in
5279 the documentation, specifically the page docs/details.rst, and some
5280 inline docs linked from there.
5281
5282 3.30.2 - 2017-09-24
5283 This release changes Hypothesis's caching approach for functions in
5284 hypothesis.strategies. Previously it would have cached extremely
5285 aggressively and cache entries would never be evicted. Now it adopts a
5286 least-frequently used, least recently used key invalidation policy, and
5287 is somewhat more conservative about which strategies it caches.
5288
5289 Workloads which create strategies based on dynamic values, e.g. by
5290 using flatmap or composite(), will use significantly less memory.
5291
5292 3.30.1 - 2017-09-22
5293 This release fixes a bug where when running with use_coverage=True
5294 inside an existing running instance of coverage, Hypothesis would fre‐
5295 quently put files that the coveragerc excluded in the report for the
5296 enclosing coverage.
5297
5298 3.30.0 - 2017-09-20
5299 This release introduces two new features:
5300
5301 · When a test fails, either with a health check failure or a falsifying
5302 example, Hypothesis will print out a seed that led to that failure,
5303 if the test is not already running with a fixed seed. You can then
5304 recreate that failure using either the @seed decorator or (if you are
5305 running pytest) with --hypothesis-seed.
5306
5307 · pytest users can specify a seed to use for @given based tests by
5308 passing the --hypothesis-seed command line argument.
5309
5310 This work was funded by Smarkets.
5311
5312 3.29.0 - 2017-09-19
5313 This release makes Hypothesis coverage aware. Hypothesis now runs all
5314 test bodies under coverage, and uses this information to guide its
5315 testing.
5316
5317 The use_coverage setting can be used to disable this behaviour if you
5318 want to test code that is sensitive to coverage being enabled (either
5319 because of performance or interaction with the trace function).
5320
5321 The main benefits of this feature are:
5322
5323 · Hypothesis now observes when examples it discovers cover particular
5324 lines or branches and stores them in the database for later.
5325
5326 · Hypothesis will make some use of this information to guide its explo‐
5327 ration of the search space and improve the examples it finds (this is
5328 currently used only very lightly and will likely improve signifi‐
5329 cantly in future releases).
5330
5331 This also has the following side-effects:
5332
5333 · Hypothesis now has an install time dependency on the coverage pack‐
5334 age.
5335
5336 · Tests that are already running Hypothesis under coverage will likely
5337 get faster.
5338
5339 · Tests that are not running under coverage now run their test bodies
5340 under coverage by default.
5341
5342 This feature is only partially supported under pypy. It is signifi‐
5343 cantly slower than on CPython and is turned off by default as a result,
5344 but it should still work correctly if you want to use it.
5345
5346 3.28.3 - 2017-09-18
5347 This release is an internal change that affects how Hypothesis handles
5348 calculating certain properties of strategies.
5349
5350 The primary effect of this is that it fixes a bug where use of
5351 deferred() could sometimes trigger an internal assertion error. However
5352 the fix for this bug involved some moderately deep changes to how
5353 Hypothesis handles certain constructs so you may notice some additional
5354 knock-on effects.
5355
5356 In particular the way Hypothesis handles drawing data from strategies
5357 that cannot generate any values has changed to bail out sooner than it
5358 previously did. This may speed up certain tests, but it is unlikely to
5359 make much of a difference in practice for tests that were not already
5360 failing with Unsatisfiable.
5361
5362 3.28.2 - 2017-09-18
5363 This is a patch release that fixes a bug in the hypothesis.extra.pandas
5364 documentation where it incorrectly referred to column() instead of col‐
5365 umns().
5366
5367 3.28.1 - 2017-09-16
5368 This is a refactoring release. It moves a number of internal uses of
5369 namedtuple() over to using attrs based classes, and removes a couple of
5370 internal namedtuple classes that were no longer in use.
5371
5372 It should have no user visible impact.
5373
5374 3.28.0 - 2017-09-15
5375 This release adds support for testing pandas via the hypothe‐
5376 sis.extra.pandas module.
5377
5378 It also adds a dependency on attrs.
5379
5380 This work was funded by Stripe.
5381
5382 3.27.1 - 2017-09-14
5383 This release fixes some formatting and broken cross-references in the
5384 documentation, which includes editing docstrings - and thus a patch
5385 release.
5386
5387 3.27.0 - 2017-09-13
5388 This release introduces a deadline setting to Hypothesis.
5389
5390 When set this turns slow tests into errors. By default it is unset but
5391 will warn if you exceed 200ms, which will become the default value in a
5392 future release.
5393
5394 This work was funded by Smarkets.
5395
5396 3.26.0 - 2017-09-12
5397 Hypothesis now emits deprecation warnings if you are using the legacy
5398 SQLite example database format, or the tool for merging them. These
5399 were already documented as deprecated, so this doesn't change their
5400 deprecation status, only that we warn about it.
5401
5402 3.25.1 - 2017-09-12
5403 This release fixes a bug with generating numpy datetime and timedelta
5404 types: When inferring the strategy from the dtype, datetime and
5405 timedelta dtypes with sub-second precision would always produce exam‐
5406 ples with one second resolution. Inferring a strategy from a time
5407 dtype will now always produce example with the same precision.
5408
5409 3.25.0 - 2017-09-12
5410 This release changes how Hypothesis shrinks and replays examples to
5411 take into account that it can encounter new bugs while shrinking the
5412 bug it originally found. Previously it would end up replacing the orig‐
5413 inally found bug with the new bug and show you only that one. Now it is
5414 (often) able to recognise when two bugs are distinct and when it finds
5415 more than one will show both.
5416
5417 3.24.2 - 2017-09-11
5418 This release removes the (purely internal and no longer useful) strat‐
5419 egy_test_suite function and the corresponding strategytests module.
5420
5421 3.24.1 - 2017-09-06
5422 This release improves the reduction of examples involving floating
5423 point numbers to produce more human readable examples.
5424
5425 It also has some general purpose changes to the way the minimizer works
5426 internally, which may see some improvement in quality and slow down of
5427 test case reduction in cases that have nothing to do with floating
5428 point numbers.
5429
5430 3.24.0 - 2017-09-05
5431 Hypothesis now emits deprecation warnings if you use some_strat‐
5432 egy.example() inside a test function or strategy definition (this was
5433 never intended to be supported, but is sufficiently widespread that it
5434 warrants a deprecation path).
5435
5436 3.23.3 - 2017-09-05
5437 This is a bugfix release for decimals() with the places argument.
5438
5439 · No longer fails health checks (issue #725, due to internal filtering)
5440
5441 · Specifying a min_value and max_value without any decimals with places
5442 places between them gives a more useful error message.
5443
5444 · Works for any valid arguments, regardless of the decimal precision
5445 context.
5446
5447 3.23.2 - 2017-09-01
5448 This is a small refactoring release that removes a now-unused parameter
5449 to an internal API. It shouldn't have any user visible effect.
5450
5451 3.23.1 - 2017-09-01
5452 Hypothesis no longer propagates the dynamic scope of settings into
5453 strategy definitions.
5454
5455 This release is a small change to something that was never part of the
5456 public API and you will almost certainly not notice any effect unless
5457 you're doing something surprising, but for example the following code
5458 will now give a different answer in some circumstances:
5459
5460 import hypothesis.strategies as st
5461 from hypothesis import settings
5462
5463 CURRENT_SETTINGS = st.builds(lambda: settings.default)
5464
5465 (We don't actually encourage you writing code like this)
5466
5467 Previously this would have generated the settings that were in effect
5468 at the point of definition of CURRENT_SETTINGS. Now it will generate
5469 the settings that are used for the current test.
5470
5471 It is very unlikely to be significant enough to be visible, but you may
5472 also notice a small performance improvement.
5473
5474 3.23.0 - 2017-08-31
5475 This release adds a unique argument to arrays() which behaves the same
5476 ways as the corresponding one for lists(), requiring all of the ele‐
5477 ments in the generated array to be distinct.
5478
5479 3.22.2 - 2017-08-29
5480 This release fixes an issue where Hypothesis would raise a TypeError
5481 when using the datetime-related strategies if running with PYTHONOPTI‐
5482 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
5483
5484 3.22.1 - 2017-08-28
5485 Hypothesis now transparently handles problems with an internal unicode
5486 cache, including file truncation or read-only filesystems (issue #767).
5487 Thanks to Sam Hames for the patch.
5488
5489 3.22.0 - 2017-08-26
5490 This release provides what should be a substantial performance improve‐
5491 ment to numpy arrays generated using provided numpy support, and adds a
5492 new fill_value argument to arrays() to control this behaviour.
5493
5494 This work was funded by Stripe.
5495
5496 3.21.3 - 2017-08-26
5497 This release fixes some extremely specific circumstances that probably
5498 have never occurred in the wild where users of deferred() might have
5499 seen a RuntimeError from too much recursion, usually in cases where no
5500 valid example could have been generated anyway.
5501
5502 3.21.2 - 2017-08-25
5503 This release fixes some minor bugs in argument validation:
5504
5505 · hypothesis.extra.numpy dtype strategies would raise an internal
5506 error instead of an InvalidArgument exception when passed an
5507 invalid endianness specification.
5508
5509 · fractions() would raise an internal error instead of an InvalidAr‐
5510 gument if passed float("nan") as one of its bounds.
5511
5512 · The error message for passing float("nan") as a bound to various
5513 strategies has been improved.
5514
5515 · Various bound arguments will now raise InvalidArgument in cases
5516 where they would previously have raised an internal TypeError or
5517 ValueError from the relevant conversion function.
5518
5519 · streaming() would not have emitted a deprecation warning when
5520 called with an invalid argument.
5521
5522 3.21.1 - 2017-08-24
5523 This release fixes a bug where test failures that were the result of an
5524 @example would print an extra stack trace before re-raising the excep‐
5525 tion.
5526
5527 3.21.0 - 2017-08-23
5528 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
5529 sis's deprecation warnings into errors. Similar functionality can be
5530 achieved by using simplefilter('error', HypothesisDeprecationWarning).
5531
5532 3.20.0 - 2017-08-22
5533 This release renames the relevant arguments on the datetimes(),
5534 dates(), times(), and timedeltas() strategies to min_value and
5535 max_value, to make them consistent with the other strategies in the
5536 module.
5537
5538 The old argument names are still supported but will emit a deprecation
5539 warning when used explicitly as keyword arguments. Arguments passed
5540 positionally will go to the new argument names and are not deprecated.
5541
5542 3.19.3 - 2017-08-22
5543 This release provides a major overhaul to the internals of how Hypothe‐
5544 sis handles shrinking.
5545
5546 This should mostly be visible in terms of getting better examples for
5547 tests which make heavy use of composite(), data() or flatmap where the
5548 data drawn depends a lot on previous choices, especially where size
5549 parameters are affected. Previously Hypothesis would have struggled to
5550 reliably produce good examples here. Now it should do much better. Per‐
5551 formance should also be better for examples with a non-zero min_size.
5552
5553 You may see slight changes to example generation (e.g. improved example
5554 diversity) as a result of related changes to internals, but they are
5555 unlikely to be significant enough to notice.
5556
5557 3.19.2 - 2017-08-21
5558 This release fixes two bugs in hypothesis.extra.numpy:
5559
5560 · unicode_string_dtypes() didn't work at all due to an incorrect dtype
5561 specifier. Now it does.
5562
5563 · Various impossible conditions would have been accepted but would
5564 error when they fail to produced any example. Now they raise an
5565 explicit InvalidArgument error.
5566
5567 3.19.1 - 2017-08-21
5568 This is a bugfix release for issue #739, where bounds for fractions()
5569 or floating-point decimals() were not properly converted to integers
5570 before passing them to the integers strategy. This excluded some val‐
5571 ues that should have been possible, and could trigger internal errors
5572 if the bounds lay between adjacent integers.
5573
5574 You can now bound fractions() with two arbitrarily close fractions.
5575
5576 It is now an explicit error to supply a min_value, max_value, and
5577 max_denominator to fractions() where the value bounds do not include a
5578 fraction with denominator at most max_denominator.
5579
5580 3.19.0 - 2017-08-20
5581 This release adds the from_regex() strategy, which generates strings
5582 that contain a match of a regular expression.
5583
5584 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
5585 then helping to upstream it! (issue #662)
5586
5587 3.18.5 - 2017-08-18
5588 This is a bugfix release for integers(). Previously the strategy would
5589 hit an internal assertion if passed non-integer bounds for min_value
5590 and max_value that had no integers between them. The strategy now
5591 raises InvalidArgument instead.
5592
5593 3.18.4 - 2017-08-18
5594 Release to fix a bug where mocks can be used as test runners under cer‐
5595 tain conditions. Specifically, if a mock is injected into a test via
5596 pytest fixtures or patch decorators, and that mock is the first argu‐
5597 ment in the list, hypothesis will think it represents self and turns
5598 the mock into a test runner. If this happens, the affected test always
5599 passes because the mock is executed instead of the test body. Some‐
5600 times, it will also fail health checks.
5601
5602 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
5603 for this bug fix.
5604
5605 3.18.3 - 2017-08-17
5606 This release should improve the performance of some tests which experi‐
5607 enced a slow down as a result of the 3.13.0 release.
5608
5609 Tests most likely to benefit from this are ones that make extensive use
5610 of min_size parameters, but others may see some improvement as well.
5611
5612 3.18.2 - 2017-08-16
5613 This release fixes a bug introduced in 3.18.0. If the arguments
5614 whitelist_characters and blacklist_characters to characters() both con‐
5615 tained elements, then an InvalidArgument exception would be raised.
5616
5617 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
5618
5619 3.18.1 - 2017-08-14
5620 This is a bug fix release to fix issue #780, where sets() and similar
5621 would trigger health check errors if their element strategy could only
5622 produce one element (e.g. if it was just()).
5623
5624 3.18.0 - 2017-08-13
5625 This is a feature release:
5626
5627 · characters() now accepts whitelist_characters, particular characters
5628 which will be added to those it produces. (issue #668)
5629
5630 · A bug fix for the internal function _union_interval_lists(), and a
5631 rename to _union_intervals(). It now correctly handles all cases
5632 where intervals overlap, and it always returns the result as a tuple
5633 for tuples.
5634
5635 Thanks to Alex Willmer for these.
5636
5637 3.17.0 - 2017-08-07
5638 This release documents the previously undocumented phases feature, mak‐
5639 ing it part of the public API. It also updates how the example database
5640 is used. Principally:
5641
5642 · A Phases.reuse argument will now correctly control whether examples
5643 from the database are run (it previously did exactly the wrong thing
5644 and controlled whether examples would be saved).
5645
5646 · Hypothesis will no longer try to rerun all previously failing exam‐
5647 ples. Instead it will replay the smallest previously failing example
5648 and a selection of other examples that are likely to trigger any
5649 other bugs that will found. This prevents a previous failure from
5650 dominating your tests unnecessarily.
5651
5652 · As a result of the previous change, Hypothesis will be slower about
5653 clearing out old examples from the database that are no longer fail‐
5654 ing (because it can only clear out ones that it actually runs).
5655
5656 3.16.1 - 2017-08-07
5657 This release makes an implementation change to how Hypothesis handles
5658 certain internal constructs.
5659
5660 The main effect you should see is improvement to the behaviour and per‐
5661 formance of collection types, especially ones with a min_size parame‐
5662 ter. Many cases that would previously fail due to being unable to gen‐
5663 erate enough valid examples will now succeed, and other cases should
5664 run slightly faster.
5665
5666 3.16.0 - 2017-08-04
5667 This release introduces a deprecation of the timeout feature. This
5668 results in the following changes:
5669
5670 · Creating a settings object with an explicit timeout will emit a dep‐
5671 recation warning.
5672
5673 · If your test stops because it hits the timeout (and has not found a
5674 bug) then it will emit a deprecation warning.
5675
5676 · There is a new value unlimited which you can import from hypothesis.
5677 settings(timeout=unlimited) will not cause a deprecation warning.
5678
5679 · There is a new health check, hung_test, which will trigger after a
5680 test has been running for five minutes if it is not suppressed.
5681
5682 3.15.0 - 2017-08-04
5683 This release deprecates two strategies, choices() and streaming().
5684
5685 Both of these are somewhat confusing to use and are entirely redundant
5686 since the introduction of the data() strategy for interactive drawing
5687 in tests, and their use should be replaced with direct use of data()
5688 instead.
5689
5690 3.14.2 - 2017-08-03
5691 This fixes a bug where Hypothesis would not work correctly on Python
5692 2.7 if you had the python:typing module backport installed.
5693
5694 3.14.1 - 2017-08-02
5695 This raises the maximum depth at which Hypothesis starts cutting off
5696 data generation to a more reasonable value which it is harder to hit by
5697 accident.
5698
5699 This resolves (issue #751), in which some examples which previously
5700 worked would start timing out, but it will also likely improve the data
5701 generation quality for complex data types.
5702
5703 3.14.0 - 2017-07-23
5704 Hypothesis now understands inline type annotations (issue #293):
5705
5706 · If the target of builds() has type annotations, a default strategy
5707 for missing required arguments is selected based on the type.
5708 Type-based strategy selection will only override a default if you
5709 pass hypothesis.infer as a keyword argument.
5710
5711 · If @given wraps a function with type annotations, you can pass infer
5712 as a keyword argument and the appropriate strategy will be substi‐
5713 tuted.
5714
5715 · You can check what strategy will be inferred for a type with the new
5716 from_type() function.
5717
5718 · register_type_strategy() teaches Hypothesis which strategy to infer
5719 for custom or unknown types. You can provide a strategy, or for more
5720 complex cases a function which takes the type and returns a strategy.
5721
5722 3.13.1 - 2017-07-20
5723 This is a bug fix release for issue #514 - Hypothesis would continue
5724 running examples after a SkipTest exception was raised, including
5725 printing a falsifying example. Skip exceptions from the standard
5726 python:unittest module, and pytest, nose, or unittest2 modules now
5727 abort the test immediately without printing output.
5728
5729 3.13.0 - 2017-07-16
5730 This release has two major aspects to it: The first is the introduction
5731 of deferred(), which allows more natural definition of recursive
5732 (including mutually recursive) strategies.
5733
5734 The second is a number of engine changes designed to support this sort
5735 of strategy better. These should have a knock-on effect of also improv‐
5736 ing the performance of any existing strategies that currently generate
5737 a lot of data or involve heavy nesting by reducing their typical exam‐
5738 ple size.
5739
5740 3.12.0 - 2017-07-07
5741 This release makes some major internal changes to how Hypothesis repre‐
5742 sents data internally, as a prelude to some major engine changes that
5743 should improve data quality. There are no API changes, but it's a sig‐
5744 nificant enough internal change that a minor version bump seemed war‐
5745 ranted.
5746
5747 User facing impact should be fairly mild, but includes:
5748
5749 · All existing examples in the database will probably be invalidated.
5750 Hypothesis handles this automatically, so you don't need to do any‐
5751 thing, but if you see all your examples disappear that's why.
5752
5753 · Almost all data distributions have changed significantly. Possibly
5754 for the better, possibly for the worse. This may result in new bugs
5755 being found, but it may also result in Hypothesis being unable to
5756 find bugs it previously did.
5757
5758 · Data generation may be somewhat faster if your existing bottleneck
5759 was in draw_bytes (which is often the case for large examples).
5760
5761 · Shrinking will probably be slower, possibly significantly.
5762
5763 If you notice any effects you consider to be a significant regression,
5764 please open an issue about them.
5765
5766 3.11.6 - 2017-06-19
5767 This release involves no functionality changes, but is the first to
5768 ship wheels as well as an sdist.
5769
5770 3.11.5 - 2017-06-18
5771 This release provides a performance improvement to shrinking. For cases
5772 where there is some non-trivial "boundary" value (e.g. the bug happens
5773 for all values greater than some other value), shrinking should now be
5774 substantially faster. Other types of bug will likely see improvements
5775 too.
5776
5777 This may also result in some changes to the quality of the final exam‐
5778 ples - it may sometimes be better, but is more likely to get slightly
5779 worse in some edge cases. If you see any examples where this happens in
5780 practice, please report them.
5781
5782 3.11.4 - 2017-06-17
5783 This is a bugfix release: Hypothesis now prints explicit examples when
5784 running in verbose mode. (issue #313)
5785
5786 3.11.3 - 2017-06-11
5787 This is a bugfix release: Hypothesis no longer emits a warning if you
5788 try to use sampled_from() with python:collections.OrderedDict. (issue
5789 #688)
5790
5791 3.11.2 - 2017-06-10
5792 This is a documentation release. Several outdated snippets have been
5793 updated or removed, and many cross-references are now hyperlinks.
5794
5795 3.11.1 - 2017-05-28
5796 This is a minor ergonomics release. Tracebacks shown by pytest no
5797 longer include Hypothesis internals for test functions decorated with
5798 @given.
5799
5800 3.11.0 - 2017-05-23
5801 This is a feature release, adding datetime-related strategies to the
5802 core strategies.
5803
5804 timezones() allows you to sample pytz timezones from the Olsen data‐
5805 base. Use directly in a recipe for tz-aware datetimes, or compose with
5806 none() to allow a mix of aware and naive output.
5807
5808 The new dates(), times(), datetimes(), and timedeltas() strategies are
5809 all constrained by objects of their type. This means that you can gen‐
5810 erate dates bounded by a single day (i.e. a single date), or datetimes
5811 constrained to the microsecond.
5812
5813 times() and datetimes() take an optional timezones= argument, which
5814 defaults to none() for naive times. You can use our extra strategy
5815 based on pytz, or roll your own timezones strategy with dateutil or
5816 even the standard library.
5817
5818 The old dates, times, and datetimes strategies in hypothe‐
5819 sis.extra.datetimes are deprecated in favor of the new core strategies,
5820 which are more flexible and have no dependencies.
5821
5822 3.10.0 - 2017-05-22
5823 Hypothesis now uses python:inspect.getfullargspec() internally. On
5824 Python 2, there are no visible changes.
5825
5826 On Python 3 @given and @composite now preserve PEP 3107 annotations on
5827 the decorated function. Keyword-only arguments are now either handled
5828 correctly (e.g. @composite), or caught in validation instead of
5829 silently discarded or raising an unrelated error later (e.g. @given).
5830
5831 3.9.1 - 2017-05-22
5832 This is a bugfix release: the default field mapping for a DateTimeField
5833 in the Django extra now respects the USE_TZ setting when choosing a
5834 strategy.
5835
5836 3.9.0 - 2017-05-19
5837 This is feature release, expanding the capabilities of the decimals()
5838 strategy.
5839
5840 · The new (optional) places argument allows you to generate decimals
5841 with a certain number of places (e.g. cents, thousandths, satoshis).
5842
5843 · If allow_infinity is None, setting min_bound no longer excludes posi‐
5844 tive infinity and setting max_value no longer excludes negative
5845 infinity.
5846
5847 · All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
5848 True, or if allow_nan is None and min_value or max_value is None.
5849
5850 · min_value and max_value may be given as decimal strings, e.g.
5851 "1.234".
5852
5853 3.8.5 - 2017-05-16
5854 Hypothesis now imports python:sqlite3 when a SQLite database is used,
5855 rather than at module load, improving compatibility with Python imple‐
5856 mentations compiled without SQLite support (such as BSD or Jython).
5857
5858 3.8.4 - 2017-05-16
5859 This is a compatibility bugfix release. sampled_from no longer raises
5860 a deprecation warning when sampling from an Enum, as all enums have a
5861 reliable iteration order.
5862
5863 3.8.3 - 2017-05-09
5864 This release removes a version check for older versions of pytest when
5865 using the Hypothesis pytest plugin. The pytest plugin will now run
5866 unconditionally on all versions of pytest. This breaks compatibility
5867 with any version of pytest prior to 2.7.0 (which is more than two years
5868 old).
5869
5870 The primary reason for this change is that the version check was a fre‐
5871 quent source of breakage when pytest change their versioning scheme. If
5872 you are not working on pytest itself and are not running a very old
5873 version of it, this release probably doesn't affect you.
5874
5875 3.8.2 - 2017-04-26
5876 This is a code reorganisation release that moves some internal test
5877 helpers out of the main source tree so as to not have changes to them
5878 trigger releases in future.
5879
5880 3.8.1 - 2017-04-26
5881 This is a documentation release. Almost all code examples are now
5882 doctests checked in CI, eliminating stale examples.
5883
5884 3.8.0 - 2017-04-23
5885 This is a feature release, adding the iterables() strategy, equivalent
5886 to lists(...).map(iter) but with a much more useful repr. You can use
5887 this strategy to check that code doesn't accidentally depend on
5888 sequence properties such as indexing support or repeated iteration.
5889
5890 3.7.4 - 2017-04-22
5891 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
5892 ture in the same test could cause the test to fail to fill the argu‐
5893 ments, and throw a TypeError.
5894
5895 3.7.3 - 2017-04-21
5896 This release should include no user visible changes and is purely a
5897 refactoring release. This modularises the behaviour of the core given()
5898 function, breaking it up into smaller and more accessible parts, but
5899 its actual behaviour should remain unchanged.
5900
5901 3.7.2 - 2017-04-21
5902 This reverts an undocumented change in 3.7.1 which broke installation
5903 on debian stable: The specifier for the hypothesis[django]
5904 extra_requires had introduced a wild card, which was not supported on
5905 the default version of pip.
5906
5907 3.7.1 - 2017-04-21
5908 This is a bug fix and internal improvements release.
5909
5910 · In particular Hypothesis now tracks a tree of where it has already
5911 explored. This allows it to avoid some classes of duplicate exam‐
5912 ples, and significantly improves the performance of shrinking failing
5913 examples by allowing it to skip some shrinks that it can determine
5914 can't possibly work.
5915
5916 · Hypothesis will no longer seed the global random arbitrarily unless
5917 you have asked it to using random_module()
5918
5919 · Shrinking would previously have not worked correctly in some special
5920 cases on Python 2, and would have resulted in suboptimal examples.
5921
5922 3.7.0 - 2017-03-20
5923 This is a feature release.
5924
5925 New features:
5926
5927 · Rule based stateful testing now has an @invariant decorator that
5928 specifies methods that are run after init and after every step,
5929 allowing you to encode properties that should be true at all times.
5930 Thanks to Tom Prince for this feature.
5931
5932 · The decimals() strategy now supports allow_nan and allow_infinity
5933 flags.
5934
5935 · There are significantly more strategies available for numpy, includ‐
5936 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
5937 for this feature.
5938
5939 · When using the data() strategy you can now add a label as an argument
5940 to draw(), which will be printed along with the value when an example
5941 fails. Thanks to Peter Inglesby for this feature.
5942
5943 Bug fixes:
5944
5945 · Bug fix: composite() now preserves functions' docstrings.
5946
5947 · The build is now reproducible and doesn't depend on the path you
5948 build it from. Thanks to Chris Lamb for this feature.
5949
5950 · numpy strategies for the void data type did not work correctly.
5951 Thanks to Zac Hatfield Dodds for this fix.
5952
5953 There have also been a number of performance optimizations:
5954
5955 · The permutations() strategy is now significantly faster to use for
5956 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
5957
5958 · Shrinking of failing test cases should have got significantly faster
5959 in some circumstances where it was previously struggling for a long
5960 time.
5961
5962 · Example generation now involves less indirection, which results in a
5963 small speedup in some cases (small enough that you won't really
5964 notice it except in pathological cases).
5965
5966 3.6.1 - 2016-12-20
5967 This release fixes a dependency problem and makes some small behind the
5968 scenes improvements.
5969
5970 · The fake-factory dependency was renamed to faker. If you were depend‐
5971 ing on it through hypothesis[django] or hypothesis[fake-factory]
5972 without pinning it yourself then it would have failed to install
5973 properly. This release changes it so that hypothesis[fakefactory]
5974 (which can now also be installed as hypothesis[faker]) will install
5975 the renamed faker package instead.
5976
5977 · This release also removed the dependency of hypothesis[django] on
5978 hypothesis[fakefactory] - it was only being used for emails. These
5979 now use a custom strategy that isn't from fakefactory. As a result
5980 you should also see performance improvements of tests which generated
5981 User objects or other things with email fields, as well as better
5982 shrinking of email addresses.
5983
5984 · The distribution of code using nested calls to one_of() or the |
5985 operator for combining strategies has been improved, as branches are
5986 now flattened to give a more uniform distribution.
5987
5988 · Examples using composite() or .flatmap should now shrink better. In
5989 particular this will affect things which work by first generating a
5990 length and then generating that many items, which have historically
5991 not shrunk very well.
5992
5993 3.6.0 - 2016-10-31
5994 This release reverts Hypothesis to its old pretty printing of lambda
5995 functions based on attempting to extract the source code rather than
5996 decompile the bytecode. This is unfortunately slightly inferior in
5997 some cases and may result in you occasionally seeing things like lambda
5998 x: <unknown> in statistics reports and strategy reprs.
5999
6000 This removes the dependencies on uncompyle6, xdis and spark-parser.
6001
6002 The reason for this is that the new functionality was based on
6003 uncompyle6, which turns out to introduce a hidden GPLed dependency - it
6004 in turn depended on xdis, and although the library was licensed under
6005 the MIT license, it contained some GPL licensed source code and thus
6006 should have been released under the GPL.
6007
6008 My interpretation is that Hypothesis itself was never in violation of
6009 the GPL (because the license it is under, the Mozilla Public License
6010 v2, is fully compatible with being included in a GPL licensed work),
6011 but I have not consulted a lawyer on the subject. Regardless of the
6012 answer to this question, adding a GPLed dependency will likely cause a
6013 lot of users of Hypothesis to inadvertently be in violation of the GPL.
6014
6015 As a result, if you are running Hypothesis 3.5.x you really should
6016 upgrade to this release immediately.
6017
6018 3.5.3 - 2016-10-05
6019 This is a bug fix release.
6020
6021 Bugs fixed:
6022
6023 · If the same test was running concurrently in two processes and there
6024 were examples already in the test database which no longer failed,
6025 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
6026 Python 2) because an example it was trying to read was deleted before
6027 it was read. (issue #372).
6028
6029 · Drawing from an integers() strategy with both a min_value and a
6030 max_value would reject too many examples needlessly. Now it repeat‐
6031 edly redraws until satisfied. (pull request #366. Thanks to Calen
6032 Pennington for the contribution).
6033
6034 3.5.2 - 2016-09-24
6035 This is a bug fix release.
6036
6037 · The Hypothesis pytest plugin broke pytest support for doctests. Now
6038 it doesn't.
6039
6040 3.5.1 - 2016-09-23
6041 This is a bug fix release.
6042
6043 · Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
6044 bytes and unicode.
6045
6046 · python:unittest.TestCase tests would not have shown up in the new
6047 statistics mode. Now they do.
6048
6049 · Similarly, stateful tests would not have shown up in statistics and
6050 now they do.
6051
6052 · Statistics now print with pytest node IDs (the names you'd get in
6053 pytest verbose mode).
6054
6055 3.5.0 - 2016-09-22
6056 This is a feature release.
6057
6058 · fractions() and decimals() strategies now support min_value and
6059 max_value parameters. Thanks go to Anne Mulhern for the development
6060 of this feature.
6061
6062 · The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
6063 tics parameter that gives detailed statistics about the tests that
6064 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
6065 funding the development of this feature.
6066
6067 · There is a new event() function that can be used to add custom sta‐
6068 tistics.
6069
6070 Additionally there have been some minor bug fixes:
6071
6072 · In some cases Hypothesis should produce fewer duplicate examples
6073 (this will mostly only affect cases with a single parameter).
6074
6075 · py.test command line parameters are now under an option group for
6076 Hypothesis (thanks to David Keijser for fixing this)
6077
6078 · Hypothesis would previously error if you used PEP 3107 function anno‐
6079 tations on your tests under Python 3.4.
6080
6081 · The repr of many strategies using lambdas has been improved to
6082 include the lambda body (this was previously supported in many but
6083 not all cases).
6084
6085 3.4.2 - 2016-07-13
6086 This is a bug fix release, fixing a number of problems with the set‐
6087 tings system:
6088
6089 · Test functions defined using @given can now be called from other
6090 threads (issue #337)
6091
6092 · Attempting to delete a settings property would previously have
6093 silently done the wrong thing. Now it raises an AttributeError.
6094
6095 · Creating a settings object with a custom database_file parameter was
6096 silently getting ignored and the default was being used instead. Now
6097 it's not.
6098
6099 3.4.1 - 2016-07-07
6100 This is a bug fix release for a single bug:
6101
6102 · On Windows when running two Hypothesis processes in parallel (e.g.
6103 using pytest-xdist) they could race with each other and one would
6104 raise an exception due to the non-atomic nature of file renaming on
6105 Windows and the fact that you can't rename over an existing file.
6106 This is now fixed.
6107
6108 3.4.0 - 2016-05-27
6109 This release is entirely provided by Lucas Wiman:
6110
6111 Strategies constructed by models() will now respect much more of
6112 Django's validations out of the box. Wherever possible full_clean()
6113 should succeed.
6114
6115 In particular:
6116
6117 · The max_length, blank and choices kwargs are now respected.
6118
6119 · Add support for DecimalField.
6120
6121 · If a field includes validators, the list of validators are used to
6122 filter the field strategy.
6123
6124 3.3.0 - 2016-05-27
6125 This release went wrong and is functionally equivalent to 3.2.0. Ignore
6126 it.
6127
6128 3.2.0 - 2016-05-19
6129 This is a small single-feature release:
6130
6131 · All tests using @given now fix the global random seed. This removes
6132 the health check for that. If a non-zero seed is required for the
6133 final falsifying example, it will be reported. Otherwise Hypothesis
6134 will assume randomization was not a significant factor for the test
6135 and be silent on the subject. If you use random_module() this will
6136 continue to work and will always display the seed.
6137
6138 3.1.3 - 2016-05-01
6139 Single bug fix release
6140
6141 · Another charmap problem. In 3.1.2 text() and characters() would break
6142 on systems which had /tmp mounted on a different partition than the
6143 Hypothesis storage directory (usually in home). This fixes that.
6144
6145 3.1.2 - 2016-04-30
6146 Single bug fix release:
6147
6148 · Anything which used a text() or characters() strategy was broken on
6149 Windows and I hadn't updated appveyor to use the new repository loca‐
6150 tion so I didn't notice. This is now fixed and windows support should
6151 work correctly.
6152
6153 3.1.1 - 2016-04-29
6154 Minor bug fix release.
6155
6156 · Fix concurrency issue when running tests that use text() from multi‐
6157 ple processes at once (issue #302, thanks to Alex Chan).
6158
6159 · Improve performance of code using lists() with max_size (thanks to
6160 Cristi Cobzarenco).
6161
6162 · Fix install on Python 2 with ancient versions of pip so that it
6163 installs the enum34 backport (thanks to Donald Stufft for telling me
6164 how to do this).
6165
6166 · Remove duplicated __all__ exports from hypothesis.strategies (thanks
6167 to Piët Delport).
6168
6169 · Update headers to point to new repository location.
6170
6171 · Allow use of strategies that can't be used in find() (e.g. choices())
6172 in stateful testing.
6173
6174 3.1.0 - 2016-03-06
6175 · Add a nothing() strategy that never successfully generates values.
6176
6177 · sampled_from() and one_of() can both now be called with an empty
6178 argument list, in which case they also never generate any values.
6179
6180 · one_of() may now be called with a single argument that is a collec‐
6181 tion of strategies as well as as varargs.
6182
6183 · Add a runner() strategy which returns the instance of the current
6184 test object if there is one.
6185
6186 · 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
6187 can be used as such.
6188
6189 · Tests using RuleBasedStateMachine should now shrink significantly
6190 better.
6191
6192 · Hypothesis now uses a pretty-printing library internally, compatible
6193 with IPython's pretty printing protocol (actually using the same
6194 code). This may improve the quality of output in some cases.
6195
6196 · Add a 'phases' setting that allows more fine grained control over
6197 which parts of the process Hypothesis runs
6198
6199 · Add a suppress_health_check setting which allows you to turn off spe‐
6200 cific health checks in a fine grained manner.
6201
6202 · Fix a bug where lists of non fixed size would always draw one more
6203 element than they included. This mostly didn't matter, but if would
6204 cause problems with empty strategies or ones with side effects.
6205
6206 · Add a mechanism to the Django model generator to allow you to explic‐
6207 itly request the default value (thanks to Jeremy Thurgood for this
6208 one).
6209
6210 3.0.5 - 2016-02-25
6211 · Fix a bug where Hypothesis would now error on py.test development
6212 versions.
6213
6214 3.0.4 - 2016-02-24
6215 · Fix a bug where Hypothesis would error when running on Python 2.7.3
6216 or earlier because it was trying to pass a python:bytearray object to
6217 python:struct.unpack() (which is only supported since 2.7.4).
6218
6219 3.0.3 - 2016-02-23
6220 · Fix version parsing of py.test to work with py.test release candi‐
6221 dates
6222
6223 · More general handling of the health check problem where things could
6224 fail because of a cache miss - now one "free" example is generated
6225 before the start of the health check run.
6226
6227 3.0.2 - 2016-02-18
6228 · Under certain circumstances, strategies involving text() buried
6229 inside some other strategy (e.g. text().filter(...) or recur‐
6230 sive(text(), ...)) would cause a test to fail its health checks the
6231 first time it ran. This was caused by having to compute some related
6232 data and cache it to disk. On travis or anywhere else where the
6233 .hypothesis directory was recreated this would have caused the tests
6234 to fail their health check on every run. This is now fixed for all
6235 the known cases, although there could be others lurking.
6236
6237 3.0.1 - 2016-02-18
6238 · Fix a case where it was possible to trigger an "Unreachable" asser‐
6239 tion when running certain flaky stateful tests.
6240
6241 · Improve shrinking of large stateful tests by eliminating a case where
6242 it was hard to delete early steps.
6243
6244 · Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
6245 cantly by provide a custom implementation for fixed size blocks that
6246 can bypass a lot of machinery.
6247
6248 · Set default home directory based on the current working directory at
6249 the point Hypothesis is imported, not whenever the function first
6250 happens to be called.
6251
6252 3.0.0 - 2016-02-17
6253 Codename: This really should have been 2.1.
6254
6255 Externally this looks like a very small release. It has one small
6256 breaking change that probably doesn't affect anyone at all (some behav‐
6257 iour that never really worked correctly is now outright forbidden) but
6258 necessitated a major version bump and one visible new feature.
6259
6260 Internally this is a complete rewrite. Almost nothing other than the
6261 public API is the same.
6262
6263 New features:
6264
6265 · Addition of data() strategy which allows you to draw arbitrary data
6266 interactively within the test.
6267
6268 · New "exploded" database format which allows you to more easily check
6269 the example database into a source repository while supporting merg‐
6270 ing.
6271
6272 · Better management of how examples are saved in the database.
6273
6274 · Health checks will now raise as errors when they fail. It was too
6275 easy to have the warnings be swallowed entirely.
6276
6277 New limitations:
6278
6279 · choices() and streaming() strategies may no longer be used with
6280 find(). Neither may data() (this is the change that necessitated a
6281 major version bump).
6282
6283 Feature removal:
6284
6285 · The ForkingTestCase executor has gone away. It may return in some
6286 more working form at a later date.
6287
6288 Performance improvements:
6289
6290 · A new model which allows flatmap, composite strategies and stateful
6291 testing to perform much better. They should also be more reliable.
6292
6293 · Filtering may in some circumstances have improved significantly. This
6294 will help especially in cases where you have lots of values with
6295 individual filters on them, such as lists(x.filter(...)).
6296
6297 · Modest performance improvements to the general test runner by avoid‐
6298 ing expensive operations
6299
6300 In general your tests should have got faster. If they've instead got
6301 significantly slower, I'm interested in hearing about it.
6302
6303 Data distribution:
6304
6305 The data distribution should have changed significantly. This may
6306 uncover bugs the previous version missed. It may also miss bugs the
6307 previous version could have uncovered. Hypothesis is now producing less
6308 strongly correlated data than it used to, but the correlations are
6309 extended over more of the structure.
6310
6311 Shrinking:
6312
6313 Shrinking quality should have improved. In particular Hypothesis can
6314 now perform simultaneous shrinking of separate examples within a single
6315 test (previously it was only able to do this for elements of a single
6316 collection). In some cases performance will have improved, in some
6317 cases it will have got worse but generally shouldn't have by much.
6318
6319 2.0.0 - 2016-01-10
6320 Codename: A new beginning
6321
6322 This release cleans up all of the legacy that accrued in the course of
6323 Hypothesis 1.0. These are mostly things that were emitting deprecation
6324 warnings in 1.19.0, but there were a few additional changes.
6325
6326 In particular:
6327
6328 · non-strategy values will no longer be converted to strategies when
6329 used in given or find.
6330
6331 · FailedHealthCheck is now an error and not a warning.
6332
6333 · Handling of non-ascii reprs in user types have been simplified by
6334 using raw strings in more places in Python 2.
6335
6336 · given no longer allows mixing positional and keyword arguments.
6337
6338 · given no longer works with functions with defaults.
6339
6340 · given no longer turns provided arguments into defaults - they will
6341 not appear in the argspec at all.
6342
6343 · the basic() strategy no longer exists.
6344
6345 · the n_ary_tree strategy no longer exists.
6346
6347 · the average_list_length setting no longer exists. Note: If you're
6348 using using recursive() this will cause you a significant slow down.
6349 You should pass explicit average_size parameters to collections in
6350 recursive calls.
6351
6352 · @rule can no longer be applied to the same method twice.
6353
6354 · Python 2.6 and 3.3 are no longer officially supported, although in
6355 practice they still work fine.
6356
6357 This also includes two non-deprecation changes:
6358
6359 · given's keyword arguments no longer have to be the rightmost argu‐
6360 ments and can appear anywhere in the method signature.
6361
6362 · The max_shrinks setting would sometimes not have been respected.
6363
6364 1.19.0 - 2016-01-09
6365 Codename: IT COMES
6366
6367 This release heralds the beginning of a new and terrible age of Hypoth‐
6368 esis 2.0.
6369
6370 It's primary purpose is some final deprecations prior to said release.
6371 The goal is that if your code emits no warnings under this release then
6372 it will probably run unchanged under Hypothesis 2.0 (there are some
6373 caveats to this: 2.0 will drop support for some Python versions, and if
6374 you're using internal APIs then as usual that may break without warn‐
6375 ing).
6376
6377 It does have two new features:
6378
6379 · New @seed() decorator which allows you to manually seed a test. This
6380 may be harmlessly combined with and overrides the derandomize set‐
6381 ting.
6382
6383 · settings objects may now be used as a decorator to fix those settings
6384 to a particular @given test.
6385
6386 API changes (old usage still works but is deprecated):
6387
6388 · Settings has been renamed to settings (lower casing) in order to make
6389 the decorator usage more natural.
6390
6391 · Functions for the storage directory that were in hypothesis.settings
6392 are now in a new hypothesis.configuration module.
6393
6394 Additional deprecations:
6395
6396 · the average_list_length setting has been deprecated in favour of
6397 being explicit.
6398
6399 · the basic() strategy has been deprecated as it is impossible to sup‐
6400 port it under a Conjecture based model, which will hopefully be
6401 implemented at some point in the 2.x series.
6402
6403 · the n_ary_tree strategy (which was never actually part of the public
6404 API) has been deprecated.
6405
6406 · Passing settings or random as keyword arguments to given is depre‐
6407 cated (use the new functionality instead)
6408
6409 Bug fixes:
6410
6411 · No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
6412 tion in streaming() values.
6413
6414 · When running in health check mode with non strict, don't print quite
6415 so many errors for an exception in reify.
6416
6417 · When an assumption made in a test or a filter is flaky, tests will
6418 now raise Flaky instead of UnsatisfiedAssumption.
6419
6420 1.18.1 - 2015-12-22
6421 Two behind the scenes changes:
6422
6423 · Hypothesis will no longer write generated code to the file system.
6424 This will improve performance on some systems (e.g. if you're using
6425 PythonAnywhere which is running your code from NFS) and prevent some
6426 annoying interactions with auto-restarting systems.
6427
6428 · Hypothesis will cache the creation of some strategies. This can sig‐
6429 nificantly improve performance for code that uses flatmap or compos‐
6430 ite and thus has to instantiate strategies a lot.
6431
6432 1.18.0 - 2015-12-21
6433 Features:
6434
6435 · Tests and find are now explicitly seeded off the global random mod‐
6436 ule. This means that if you nest one inside the other you will now
6437 get a health check error. It also means that you can control global
6438 randomization by seeding random.
6439
6440 · There is a new random_module() strategy which seeds the global random
6441 module for you and handles things so that you don't get a health
6442 check warning if you use it inside your tests.
6443
6444 · floats() now accepts two new arguments: allow_nan and allow_infinity.
6445 These default to the old behaviour, but when set to False will do
6446 what the names suggest.
6447
6448 Bug fixes:
6449
6450 · Fix a bug where tests that used text() on Python 3.4+ would not actu‐
6451 ally be deterministic even when explicitly seeded or using the deran‐
6452 domize mode, because generation depended on dictionary iteration
6453 order which was affected by hash randomization.
6454
6455 · Fix a bug where with complicated strategies the timing of the initial
6456 health check could affect the seeding of the subsequent test, which
6457 would also render supposedly deterministic tests non-deterministic in
6458 some scenarios.
6459
6460 · In some circumstances flatmap() could get confused by two struc‐
6461 turally similar things it could generate and would produce a flaky
6462 test where the first time it produced an error but the second time it
6463 produced the other value, which was not an error. The same bug was
6464 presumably also possible in composite().
6465
6466 · flatmap() and composite() initial generation should now be moderately
6467 faster. This will be particularly noticeable when you have many val‐
6468 ues drawn from the same strategy in a single run, e.g. constructs
6469 like lists(s.flatmap(f)). Shrinking performance may have suffered,
6470 but this didn't actually produce an interestingly worse result in any
6471 of the standard scenarios tested.
6472
6473 1.17.1 - 2015-12-16
6474 A small bug fix release, which fixes the fact that the 'note' function
6475 could not be used on tests which used the @example decorator to provide
6476 explicit examples.
6477
6478 1.17.0 - 2015-12-15
6479 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
6480 because it contains the following additional change that was not
6481 intended to be in a patch release (it's perfectly stable, but is a
6482 larger change that should have required a minor version bump):
6483
6484 · Hypothesis will now perform a series of "health checks" as part of
6485 running your tests. These detect and warn about some common error
6486 conditions that people often run into which wouldn't necessarily have
6487 caused the test to fail but would cause e.g. degraded performance or
6488 confusing results.
6489
6490 1.16.1 - 2015-12-14
6491 Note: This release has been removed.
6492
6493 A small bugfix release that allows bdists for Hypothesis to be built
6494 under 2.7 - the compat3.py file which had Python 3 syntax wasn't
6495 intended to be loaded under Python 2, but when building a bdist it was.
6496 In particular this would break running setup.py test.
6497
6498 1.16.0 - 2015-12-08
6499 There are no public API changes in this release but it includes a be‐
6500 haviour change that I wasn't comfortable putting in a patch release.
6501
6502 · Functions from hypothesis.strategies will no longer raise InvalidAr‐
6503 gument on bad arguments. Instead the same errors will be raised when
6504 a test using such a strategy is run. This may improve startup time in
6505 some cases, but the main reason for it is so that errors in strate‐
6506 gies won't cause errors in loading, and it can interact correctly
6507 with things like pytest.mark.skipif.
6508
6509 · Errors caused by accidentally invoking the legacy API are now much
6510 less confusing, although still throw NotImplementedError.
6511
6512 · hypothesis.extra.django is 1.9 compatible.
6513
6514 · When tests are run with max_shrinks=0 this will now still rerun the
6515 test on failure and will no longer print "Trying example:" before
6516 each run. Additionally note() will now work correctly when used with
6517 max_shrinks=0.
6518
6519 1.15.0 - 2015-11-24
6520 A release with two new features.
6521
6522 · A 'characters' strategy for more flexible generation of text with
6523 particular character ranges and types, kindly contributed by
6524 Alexander Shorin.
6525
6526 · Add support for preconditions to the rule based stateful testing.
6527 Kindly contributed by Christopher Armstrong
6528
6529 1.14.0 - 2015-11-01
6530 New features:
6531
6532 · Add 'note' function which lets you include additional information in
6533 the final test run's output.
6534
6535 · Add 'choices' strategy which gives you a choice function that emu‐
6536 lates random.choice.
6537
6538 · Add 'uuid' strategy that generates UUIDs'
6539
6540 · Add 'shared' strategy that lets you create a strategy that just gen‐
6541 erates a single shared value for each test run
6542
6543 Bugs:
6544
6545 · Using strategies of the form streaming(x.flatmap(f)) with find or in
6546 stateful testing would have caused InvalidArgument errors when the
6547 resulting values were used (because code that expected to only be
6548 called within a test context would be invoked).
6549
6550 1.13.0 - 2015-10-29
6551 This is quite a small release, but deprecates some public API functions
6552 and removes some internal API functionality so gets a minor version
6553 bump.
6554
6555 · All calls to the 'strategy' function are now deprecated, even ones
6556 which pass just a SearchStrategy instance (which is still a no-op).
6557
6558 · Never documented hypothesis.extra entry_points mechanism has now been
6559 removed ( it was previously how hypothesis.extra packages were loaded
6560 and has been deprecated and unused for some time)
6561
6562 · Some corner cases that could previously have produced an OverflowEr‐
6563 ror when simplifying failing cases using hypothesis.extra.datetimes
6564 (or dates or times) have now been fixed.
6565
6566 · Hypothesis load time for first import has been significantly reduced
6567 - it used to be around 250ms (on my SSD laptop) and now is around
6568 100-150ms. This almost never matters but was slightly annoying when
6569 using it in the console.
6570
6571 · hypothesis.strategies.randoms was previously missing from __all__.
6572
6573 1.12.0 - 2015-10-18
6574 · Significantly improved performance of creating strategies using the
6575 functions from the hypothesis.strategies module by deferring the cal‐
6576 culation of their repr until it was needed. This is unlikely to have
6577 been an performance issue for you unless you were using flatmap, com‐
6578 posite or stateful testing, but for some cases it could be quite a
6579 significant impact.
6580
6581 · A number of cases where the repr of strategies build from lambdas is
6582 improved
6583
6584 · Add dates() and times() strategies to hypothesis.extra.datetimes
6585
6586 · Add new 'profiles' mechanism to the settings system
6587
6588 · Deprecates mutability of Settings, both the Settings.default top
6589 level property and individual settings.
6590
6591 · A Settings object may now be directly initialized from a parent Set‐
6592 tings.
6593
6594 · @given should now give a better error message if you attempt to use
6595 it with a function that uses destructuring arguments (it still won't
6596 work, but it will error more clearly),
6597
6598 · A number of spelling corrections in error messages
6599
6600 · py.test should no longer display the intermediate modules Hypothesis
6601 generates when running in verbose mode
6602
6603 · Hypothesis should now correctly handle printing objects with
6604 non-ascii reprs on python 3 when running in a locale that cannot han‐
6605 dle ascii printing to stdout.
6606
6607 · Add a unique=True argument to lists(). This is equivalent to
6608 unique_by=lambda x: x, but offers a more convenient syntax.
6609
6610 1.11.4 - 2015-09-27
6611 · Hide modifications Hypothesis needs to make to sys.path by undoing
6612 them after we've imported the relevant modules. This is a workaround
6613 for issues cryptography experienced on windows.
6614
6615 · Slightly improved performance of drawing from sampled_from on large
6616 lists of alternatives.
6617
6618 · Significantly improved performance of drawing from one_of or strate‐
6619 gies using | (note this includes a lot of strategies internally -
6620 floats() and integers() both fall into this category). There turned
6621 out to be a massive performance regression introduced in 1.10.0
6622 affecting these which probably would have made tests using Hypothesis
6623 significantly slower than they should have been.
6624
6625 1.11.3 - 2015-09-23
6626 · Better argument validation for datetimes() strategy - previously set‐
6627 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
6628 would not have raised an InvalidArgument error and instead would have
6629 behaved confusingly.
6630
6631 · Compatibility with being run on pytest < 2.7 (achieved by disabling
6632 the plugin).
6633
6634 1.11.2 - 2015-09-23
6635 Bug fixes:
6636
6637 · Settings(database=my_db) would not be correctly inherited when used
6638 as a default setting, so that newly created settings would use the
6639 database_file setting and create an SQLite example database.
6640
6641 · Settings.default.database = my_db would previously have raised an
6642 error and now works.
6643
6644 · Timeout could sometimes be significantly exceeded if during simplifi‐
6645 cation there were a lot of examples tried that didn't trigger the
6646 bug.
6647
6648 · When loading a heavily simplified example using a basic() strategy
6649 from the database this could cause Python to trigger a recursion
6650 error.
6651
6652 · Remove use of deprecated API in pytest plugin so as to not emit warn‐
6653 ing
6654
6655 Misc:
6656
6657 · hypothesis-pytest is now part of hypothesis core. This should have no
6658 externally visible consequences, but you should update your dependen‐
6659 cies to remove hypothesis-pytest and depend on only Hypothesis.
6660
6661 · Better repr for hypothesis.extra.datetimes() strategies.
6662
6663 · Add .close() method to abstract base class for Backend (it was
6664 already present in the main implementation).
6665
6666 1.11.1 - 2015-09-16
6667 Bug fixes:
6668
6669 · When running Hypothesis tests in parallel (e.g. using pytest-xdist)
6670 there was a race condition caused by code generation.
6671
6672 · Example databases are now cached per thread so as to not use sqlite
6673 connections from multiple threads. This should make Hypothesis now
6674 entirely thread safe.
6675
6676 · floats() with only min_value or max_value set would have had a very
6677 bad distribution.
6678
6679 · Running on 3.5, Hypothesis would have emitted deprecation warnings
6680 because of use of inspect.getargspec
6681
6682 1.11.0 - 2015-08-31
6683 · text() with a non-string alphabet would have used the repr() of the
6684 the alphabet instead of its contexts. This is obviously silly. It now
6685 works with any sequence of things convertible to unicode strings.
6686
6687 · @given will now work on methods whose definitions contains no
6688 explicit positional arguments, only varargs (issue #118). This may
6689 have some knock on effects because it means that @given no longer
6690 changes the argspec of functions other than by adding defaults.
6691
6692 · Introduction of new @composite feature for more natural definition of
6693 strategies you'd previously have used flatmap for.
6694
6695 1.10.6 - 2015-08-26
6696 Fix support for fixtures on Django 1.7.
6697
6698 1.10.4 - 2015-08-21
6699 Tiny bug fix release:
6700
6701 · If the database_file setting is set to None, this would have resulted
6702 in an error when running tests. Now it does the same as setting data‐
6703 base to None.
6704
6705 1.10.3 - 2015-08-19
6706 Another small bug fix release.
6707
6708 · lists(elements, unique_by=some_function, min_size=n) would have
6709 raised a ValidationError if n > Settings.default.average_list_length
6710 because it would have wanted to use an average list length shorter
6711 than the minimum size of the list, which is impossible. Now it
6712 instead defaults to twice the minimum size in these circumstances.
6713
6714 · basic() strategy would have only ever produced at most ten distinct
6715 values per run of the test (which is bad if you e.g. have it inside a
6716 list). This was obviously silly. It will now produce a much better
6717 distribution of data, both duplicated and non duplicated.
6718
6719 1.10.2 - 2015-08-19
6720 This is a small bug fix release:
6721
6722 · star imports from hypothesis should now work correctly.
6723
6724 · example quality for examples using flatmap will be better, as the way
6725 it had previously been implemented was causing problems where Hypoth‐
6726 esis was erroneously labelling some examples as being duplicates.
6727
6728 1.10.0 - 2015-08-04
6729 This is just a bugfix and performance release, but it changes some
6730 semi-public APIs, hence the minor version bump.
6731
6732 · Significant performance improvements for strategies which are
6733 one_of() many branches. In particular this included recursive()
6734 strategies. This should take the case where you use one recursive()
6735 strategy as the base strategy of another from unusably slow (tens of
6736 seconds per generated example) to reasonably fast.
6737
6738 · Better handling of just() and sampled_from() for values which have an
6739 incorrect __repr__ implementation that returns non-ASCII unicode on
6740 Python 2.
6741
6742 · Better performance for flatmap from changing the internal morpher API
6743 to be significantly less general purpose.
6744
6745 · Introduce a new semi-public BuildContext/cleanup API. This allows
6746 strategies to register cleanup activities that should run once the
6747 example is complete. Note that this will interact somewhat weirdly
6748 with find.
6749
6750 · Better simplification behaviour for streaming strategies.
6751
6752 · Don't error on lambdas which use destructuring arguments in Python 2.
6753
6754 · Add some better reprs for a few strategies that were missing good
6755 ones.
6756
6757 · The Random instances provided by randoms() are now copyable.
6758
6759 · Slightly more debugging information about simplify when using a debug
6760 verbosity level.
6761
6762 · Support using given for functions with varargs, but not passing argu‐
6763 ments to it as positional.
6764
6765 1.9.0 - 2015-07-27
6766 Codename: The great bundling.
6767
6768 This release contains two fairly major changes.
6769
6770 The first is the deprecation of the hypothesis-extra mechanism. From
6771 now on all the packages that were previously bundled under it other
6772 than hypothesis-pytest (which is a different beast and will remain sep‐
6773 arate). The functionality remains unchanged and you can still import
6774 them from exactly the same location, they just are no longer separate
6775 packages.
6776
6777 The second is that this introduces a new way of building strategies
6778 which lets you build up strategies recursively from other strategies.
6779
6780 It also contains the minor change that calling .example() on a strategy
6781 object will give you examples that are more representative of the
6782 actual data you'll get. There used to be some logic in there to make
6783 the examples artificially simple but this proved to be a bad idea.
6784
6785 1.8.5 - 2015-07-24
6786 This contains no functionality changes but fixes a mistake made with
6787 building the previous package that would have broken installation on
6788 Windows.
6789
6790 1.8.4 - 2015-07-20
6791 Bugs fixed:
6792
6793 · When a call to floats() had endpoints which were not floats but
6794 merely convertible to one (e.g. integers), these would be included in
6795 the generated data which would cause it to generate non-floats.
6796
6797 · Splitting lambdas used in the definition of flatmap, map or filter
6798 over multiple lines would break the repr, which would in turn break
6799 their usage.
6800
6801 1.8.3 - 2015-07-20
6802 "Falsifying example" would not have been printed when the failure came
6803 from an explicit example.
6804
6805 1.8.2 - 2015-07-18
6806 Another small bugfix release:
6807
6808 · When using ForkingTestCase you would usually not get the falsifying
6809 example printed if the process exited abnormally (e.g. due to
6810 os._exit).
6811
6812 · Improvements to the distribution of characters when using text() with
6813 a default alphabet. In particular produces a better distribution of
6814 ascii and whitespace in the alphabet.
6815
6816 1.8.1 - 2015-07-17
6817 This is a small release that contains a workaround for people who have
6818 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
6819 for Hypothesis per se because that's not a thing that is actually sup‐
6820 posed to work, but Hypothesis leans more heavily on repr than is typi‐
6821 cal so it's worth having a workaround for.
6822
6823 1.8.0 - 2015-07-16
6824 New features:
6825
6826 · Much more sensible reprs for strategies, especially ones that come
6827 from hypothesis.strategies. These should now have as reprs python
6828 code that would produce the same strategy.
6829
6830 · lists() accepts a unique_by argument which forces the generated lists
6831 to be only contain elements unique according to some function key
6832 (which must return a hashable value).
6833
6834 · Better error messages from flaky tests to help you debug things.
6835
6836 Mostly invisible implementation details that may result in finding new
6837 bugs in your code:
6838
6839 · Sets and dictionary generation should now produce a better range of
6840 results.
6841
6842 · floats with bounds now focus more on 'critical values', trying to
6843 produce values at edge cases.
6844
6845 · flatmap should now have better simplification for complicated cases,
6846 as well as generally being (I hope) more reliable.
6847
6848 Bug fixes:
6849
6850 · You could not previously use assume() if you were using the forking
6851 executor.
6852
6853 1.7.2 - 2015-07-10
6854 This is purely a bug fix release:
6855
6856 · When using floats() with stale data in the database you could some‐
6857 times get values in your tests that did not respect min_value or
6858 max_value.
6859
6860 · When getting a Flaky error from an unreliable test it would have
6861 incorrectly displayed the example that caused it.
6862
6863 · 2.6 dependency on backports was incorrectly specified. This would
6864 only have caused you problems if you were building a universal wheel
6865 from Hypothesis, which is not how Hypothesis ships, so unless you're
6866 explicitly building wheels for your dependencies and support Python
6867 2.6 plus a later version of Python this probably would never have
6868 affected you.
6869
6870 · If you use flatmap in a way that the strategy on the right hand side
6871 depends sensitively on the left hand side you may have occasionally
6872 seen Flaky errors caused by producing unreliable examples when mini‐
6873 mizing a bug. This use case may still be somewhat fraught to be hon‐
6874 est. This code is due a major rearchitecture for 1.8, but in the
6875 meantime this release fixes the only source of this error that I'm
6876 aware of.
6877
6878 1.7.1 - 2015-06-29
6879 Codename: There is no 1.7.0.
6880
6881 A slight technical hitch with a premature upload means there's was a
6882 yanked 1.7.0 release. Oops.
6883
6884 The major feature of this release is Python 2.6 support. Thanks to Jeff
6885 Meadows for doing most of the work there.
6886
6887 Other minor features
6888
6889 · strategies now has a permutations() function which returns a strategy
6890 yielding permutations of values from a given collection.
6891
6892 · if you have a flaky test it will print the exception that it last saw
6893 before failing with Flaky, even if you do not have verbose reporting
6894 on.
6895
6896 · Slightly experimental git merge script available as "python -m
6897 hypothesis.tools.mergedbs". Instructions on how to use it in the doc‐
6898 string of that file.
6899
6900 Bug fixes:
6901
6902 · Better performance from use of filter. In particular tests which
6903 involve large numbers of heavily filtered strategies should perform a
6904 lot better.
6905
6906 · floats() with a negative min_value would not have worked correctly
6907 (worryingly, it would have just silently failed to run any examples).
6908 This is now fixed.
6909
6910 · tests using sampled_from would error if the number of sampled ele‐
6911 ments was smaller than min_satisfying_examples.
6912
6913 1.6.2 - 2015-06-08
6914 This is just a few small bug fixes:
6915
6916 · Size bounds were not validated for values for a binary() strategy
6917 when reading examples from the database.
6918
6919 · sampled_from is now in __all__ in hypothesis.strategies
6920
6921 · floats no longer consider negative integers to be simpler than posi‐
6922 tive non-integers
6923
6924 · Small floating point intervals now correctly count members, so if you
6925 have a floating point interval so narrow there are only a handful of
6926 values in it, this will no longer cause an error when Hypothesis runs
6927 out of values.
6928
6929 1.6.1 - 2015-05-21
6930 This is a small patch release that fixes a bug where 1.6.0 broke the
6931 use of flatmap with the deprecated API and assumed the passed in func‐
6932 tion returned a SearchStrategy instance rather than converting it to a
6933 strategy.
6934
6935 1.6.0 - 2015-05-21
6936 This is a smallish release designed to fix a number of bugs and smooth
6937 out some weird behaviours.
6938
6939 · Fix a critical bug in flatmap where it would reuse old strategies. If
6940 all your flatmap code was pure you're fine. If it's not, I'm sur‐
6941 prised it's working at all. In particular if you want to use flatmap
6942 with django models, you desperately need to upgrade to this version.
6943
6944 · flatmap simplification performance should now be better in some cases
6945 where it previously had to redo work.
6946
6947 · Fix for a bug where invalid unicode data with surrogates could be
6948 generated during simplification (it was already filtered out during
6949 actual generation).
6950
6951 · The Hypothesis database is now keyed off the name of the test instead
6952 of the type of data. This makes much more sense now with the new
6953 strategies API and is generally more robust. This means you will lose
6954 old examples on upgrade.
6955
6956 · The database will now not delete values which fail to deserialize
6957 correctly, just skip them. This is to handle cases where multiple
6958 incompatible strategies share the same key.
6959
6960 · find now also saves and loads values from the database, keyed off a
6961 hash of the function you're finding from.
6962
6963 · Stateful tests now serialize and load values from the database. They
6964 should have before, really. This was a bug.
6965
6966 · Passing a different verbosity level into a test would not have worked
6967 entirely correctly, leaving off some messages. This is now fixed.
6968
6969 · Fix a bug where derandomized tests with unicode characters in the
6970 function body would error on Python 2.7.
6971
6972 1.5.0 - 2015-05-14
6973 Codename: Strategic withdrawal.
6974
6975 The purpose of this release is a radical simplification of the API for
6976 building strategies. Instead of the old approach of @strategy.extend
6977 and things that get converted to strategies, you just build strategies
6978 directly.
6979
6980 The old method of defining strategies will still work until Hypothesis
6981 2.0, because it's a major breaking change, but will now emit depreca‐
6982 tion warnings.
6983
6984 The new API is also a lot more powerful as the functions for defining
6985 strategies give you a lot of dials to turn. See the updated data sec‐
6986 tion for details.
6987
6988 Other changes:
6989
6990 · Mixing keyword and positional arguments in a call to @given is
6991 deprecated as well.
6992
6993 · There is a new setting called 'strict'. When set to True, Hypothe‐
6994 sis will raise warnings instead of merely printing them. Turning
6995 it on by default is inadvisable because it means that Hypothesis
6996 minor releases can break your code, but it may be useful for mak‐
6997 ing sure you catch all uses of deprecated APIs.
6998
6999 · max_examples in settings is now interpreted as meaning the maximum
7000 number of unique (ish) examples satisfying assumptions. A new set‐
7001 ting max_iterations which defaults to a larger value has the old
7002 interpretation.
7003
7004 · Example generation should be significantly faster due to a new
7005 faster parameter selection algorithm. This will mostly show up for
7006 simple data types - for complex ones the parameter selection is
7007 almost certainly dominated.
7008
7009 · Simplification has some new heuristics that will tend to cut down
7010 on cases where it could previously take a very long time.
7011
7012 · timeout would previously not have been respected in cases where
7013 there were a lot of duplicate examples. You probably wouldn't have
7014 previously noticed this because max_examples counted duplicates,
7015 so this was very hard to hit in a way that mattered.
7016
7017 · A number of internal simplifications to the SearchStrategy API.
7018
7019 · You can now access the current Hypothesis version as hypothe‐
7020 sis.__version__.
7021
7022 · A top level function is provided for running the stateful tests
7023 without the TestCase infrastructure.
7024
7025 1.4.0 - 2015-05-04
7026 Codename: What a state.
7027
7028 The big feature of this release is the new and slightly experimental
7029 stateful testing API. You can read more about that in the appropriate
7030 section.
7031
7032 Two minor features the were driven out in the course of developing
7033 this:
7034
7035 · You can now set settings.max_shrinks to limit the number of times
7036 Hypothesis will try to shrink arguments to your test. If this is set
7037 to <= 0 then Hypothesis will not rerun your test and will just raise
7038 the failure directly. Note that due to technical limitations if
7039 max_shrinks is <= 0 then Hypothesis will print every example it calls
7040 your test with rather than just the failing one. Note also that I
7041 don't consider settings max_shrinks to zero a sensible way to run
7042 your tests and it should really be considered a debug feature.
7043
7044 · There is a new debug level of verbosity which is even more verbose
7045 than verbose. You probably don't want this.
7046
7047 Breakage of semi-public SearchStrategy API:
7048
7049 · It is now a required invariant of SearchStrategy that if u simplifies
7050 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
7051 plifying should not increase the complexity even though it is not
7052 required to decrease it. Enforcing this invariant lead to finding
7053 some bugs where simplifying of integers, floats and sets was subopti‐
7054 mal.
7055
7056 · Integers in basic data are now required to fit into 64 bits. As a
7057 result python integer types are now serialized as strings, and some
7058 types have stopped using quite so needlessly large random seeds.
7059
7060 Hypothesis Stateful testing was then turned upon Hypothesis itself,
7061 which lead to an amazing number of minor bugs being found in Hypothesis
7062 itself.
7063
7064 Bugs fixed (most but not all from the result of stateful testing)
7065 include:
7066
7067 · Serialization of streaming examples was flaky in a way that you would
7068 probably never notice: If you generate a template, simplify it, seri‐
7069 alize it, deserialize it, serialize it again and then deserialize it
7070 you would get the original stream instead of the simplified one.
7071
7072 · If you reduced max_examples below the number of examples already
7073 saved in the database, you would have got a ValueError. Additionally,
7074 if you had more than max_examples in the database all of them would
7075 have been considered.
7076
7077 · @given will no longer count duplicate examples (which it never called
7078 your function with) towards max_examples. This may result in your
7079 tests running slower, but that's probably just because they're trying
7080 more examples.
7081
7082 · General improvements to example search which should result in better
7083 performance and higher quality examples. In particular parameters
7084 which have a history of producing useless results will be more
7085 aggressively culled. This is useful both because it decreases the
7086 chance of useless examples and also because it's much faster to not
7087 check parameters which we were unlikely to ever pick!
7088
7089 · integers_from and lists of types with only one value (e.g. [None])
7090 would previously have had a very high duplication rate so you were
7091 probably only getting a handful of examples. They now have a much
7092 lower duplication rate, as well as the improvements to search making
7093 this less of a problem in the first place.
7094
7095 · You would sometimes see simplification taking significantly longer
7096 than your defined timeout. This would happen because timeout was only
7097 being checked after each successful simplification, so if Hypothesis
7098 was spending a lot of time unsuccessfully simplifying things it
7099 wouldn't stop in time. The timeout is now applied for unsuccessful
7100 simplifications too.
7101
7102 · In Python 2.7, integers_from strategies would have failed during sim‐
7103 plification with an OverflowError if their starting point was at or
7104 near to the maximum size of a 64-bit integer.
7105
7106 · flatmap and map would have failed if called with a function without a
7107 __name__ attribute.
7108
7109 · If max_examples was less than min_satisfying_examples this would
7110 always error. Now min_satisfying_examples is capped to max_examples.
7111 Note that if you have assumptions to satisfy here this will still
7112 cause an error.
7113
7114 Some minor quality improvements:
7115
7116 · Lists of streams, flatmapped strategies and basic strategies should
7117 now now have slightly better simplification.
7118
7119 1.3.0 - 2015-05-22
7120 New features:
7121
7122 · New verbosity level API for printing intermediate results and excep‐
7123 tions.
7124
7125 · New specifier for strings generated from a specified alphabet.
7126
7127 · Better error messages for tests that are failing because of a lack of
7128 enough examples.
7129
7130 Bug fixes:
7131
7132 · Fix error where use of ForkingTestCase would sometimes result in too
7133 many open files.
7134
7135 · Fix error where saving a failing example that used flatmap could
7136 error.
7137
7138 · Implement simplification for sampled_from, which apparently never
7139 supported it previously. Oops.
7140
7141 General improvements:
7142
7143 · Better range of examples when using one_of or sampled_from.
7144
7145 · Fix some pathological performance issues when simplifying lists of
7146 complex values.
7147
7148 · Fix some pathological performance issues when simplifying examples
7149 that require unicode strings with high codepoints.
7150
7151 · Random will now simplify to more readable examples.
7152
7153 1.2.1 - 2015-04-16
7154 A small patch release for a bug in the new executors feature. Tests
7155 which require doing something to their result in order to fail would
7156 have instead reported as flaky.
7157
7158 1.2.0 - 2015-04-15
7159 Codename: Finders keepers.
7160
7161 A bunch of new features and improvements.
7162
7163 · Provide a mechanism for customizing how your tests are executed.
7164
7165 · Provide a test runner that forks before running each example. This
7166 allows better support for testing native code which might trigger a
7167 segfault or a C level assertion failure.
7168
7169 · Support for using Hypothesis to find examples directly rather than as
7170 just as a test runner.
7171
7172 · New streaming type which lets you generate infinite lazily loaded
7173 streams of data - perfect for if you need a number of examples but
7174 don't know how many.
7175
7176 · Better support for large integer ranges. You can now use inte‐
7177 gers_in_range with ranges of basically any size. Previously large
7178 ranges would have eaten up all your memory and taken forever.
7179
7180 · Integers produce a wider range of data than before - previously they
7181 would only rarely produce integers which didn't fit into a machine
7182 word. Now it's much more common. This percolates to other numeric
7183 types which build on integers.
7184
7185 · Better validation of arguments to @given. Some situations that would
7186 previously have caused silently wrong behaviour will now raise an
7187 error.
7188
7189 · Include +/- sys.float_info.max in the set of floating point edge
7190 cases that Hypothesis specifically tries.
7191
7192 · Fix some bugs in floating point ranges which happen when given +/-
7193 sys.float_info.max as one of the endpoints... (really any two floats
7194 that are sufficiently far apart so that x, y are finite but y - x is
7195 infinite). This would have resulted in generating infinite values
7196 instead of ones inside the range.
7197
7198 1.1.1 - 2015-04-07
7199 Codename: Nothing to see here
7200
7201 This is just a patch release put out because it fixed some internal
7202 bugs that would block the Django integration release but did not actu‐
7203 ally affect anything anyone could previously have been using. It also
7204 contained a minor quality fix for floats that I'd happened to have fin‐
7205 ished in time.
7206
7207 · Fix some internal bugs with object lifecycle management that were
7208 impossible to hit with the previously released versions but broke
7209 hypothesis-django.
7210
7211 · Bias floating point numbers somewhat less aggressively towards very
7212 small numbers
7213
7214 1.1.0 - 2015-04-06
7215 Codename: No-one mention the M word.
7216
7217 · Unicode strings are more strongly biased towards ascii characters.
7218 Previously they would generate all over the space. This is mostly so
7219 that people who try to shape their unicode strings with assume() have
7220 less of a bad time.
7221
7222 · A number of fixes to data deserialization code that could theoreti‐
7223 cally have caused mysterious bugs when using an old version of a
7224 Hypothesis example database with a newer version. To the best of my
7225 knowledge a change that could have triggered this bug has never actu‐
7226 ally been seen in the wild. Certainly no-one ever reported a bug of
7227 this nature.
7228
7229 · Out of the box support for Decimal and Fraction.
7230
7231 · new dictionary specifier for dictionaries with variable keys.
7232
7233 · Significantly faster and higher quality simplification, especially
7234 for collections of data.
7235
7236 · New filter() and flatmap() methods on Strategy for better ways of
7237 building strategies out of other strategies.
7238
7239 · New BasicStrategy class which allows you to define your own strate‐
7240 gies from scratch without needing an existing matching strategy or
7241 being exposed to the full horror or non-public nature of the Search‐
7242 Strategy interface.
7243
7244 1.0.0 - 2015-03-27
7245 Codename: Blast-off!
7246
7247 There are no code changes in this release. This is precisely the 0.9.2
7248 release with some updated documentation.
7249
7250 0.9.2 - 2015-03-26
7251 Codename: T-1 days.
7252
7253 · floats_in_range would not actually have produced floats_in_range
7254 unless that range happened to be (0, 1). Fix this.
7255
7256 0.9.1 - 2015-03-25
7257 Codename: T-2 days.
7258
7259 · Fix a bug where if you defined a strategy using map on a lambda then
7260 the results would not be saved in the database.
7261
7262 · Significant performance improvements when simplifying examples using
7263 lists, strings or bounded integer ranges.
7264
7265 0.9.0 - 2015-03-23
7266 Codename: The final countdown
7267
7268 This release could also be called 1.0-RC1.
7269
7270 It contains a teeny tiny bugfix, but the real point of this release is
7271 to declare feature freeze. There will be zero functionality changes
7272 between 0.9.0 and 1.0 unless something goes really really wrong. No new
7273 features will be added, no breaking API changes will occur, etc. This
7274 is the final shakedown before I declare Hypothesis stable and ready to
7275 use and throw a party to celebrate.
7276
7277 Bug bounty for any bugs found between now and 1.0: I will buy you a
7278 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
7279 we ever find ourselves in the same city at the same time.
7280
7281 The one tiny bugfix:
7282
7283 · Under pypy, databases would fail to close correctly when garbage col‐
7284 lected, leading to a memory leak and a confusing error message if you
7285 were repeatedly creating databases and not closing them. It is very
7286 unlikely you were doing this and the chances of you ever having
7287 noticed this bug are very low.
7288
7289 0.7.2 - 2015-03-22
7290 Codename: Hygienic macros or bust
7291
7292 · You can now name an argument to @given 'f' and it won't break (issue
7293 #38)
7294
7295 · strategy_test_suite is now named strategy_test_suite as the documen‐
7296 tation claims and not in fact strategy_test_suitee
7297
7298 · Settings objects can now be used as a context manager to temporarily
7299 override the default values inside their context.
7300
7301 0.7.1 - 2015-03-21
7302 Codename: Point releases go faster
7303
7304 · Better string generation by parametrizing by a limited alphabet
7305
7306 · Faster string simplification - previously if simplifying a string
7307 with high range unicode characters it would try every unicode charac‐
7308 ter smaller than that. This was pretty pointless. Now it stops after
7309 it's a short range (it can still reach smaller ones through recursive
7310 calls because of other simplifying operations).
7311
7312 · Faster list simplification by first trying a binary chop down the
7313 middle
7314
7315 · Simultaneous simplification of identical elements in a list. So if a
7316 bug only triggers when you have duplicates but you drew e.g. [-17,
7317 -17], this will now simplify to [0, 0].
7318
7319 0.7.0, - 2015-03-20
7320 Codename: Starting to look suspiciously real
7321
7322 This is probably the last minor release prior to 1.0. It consists of
7323 stability improvements, a few usability things designed to make Hypoth‐
7324 esis easier to try out, and filing off some final rough edges from the
7325 API.
7326
7327 · Significant speed and memory usage improvements
7328
7329 · Add an example() method to strategy objects to give an example of the
7330 sort of data that the strategy generates.
7331
7332 · Remove .descriptor attribute of strategies
7333
7334 · Rename descriptor_test_suite to strategy_test_suite
7335
7336 · Rename the few remaining uses of descriptor to specifier (descriptor
7337 already has a defined meaning in Python)
7338
7339 0.6.0 - 2015-03-13
7340 Codename: I'm sorry, were you using that API?
7341
7342 This is primarily a "simplify all the weird bits of the API" release.
7343 As a result there are a lot of breaking changes. If you just use @given
7344 with core types then you're probably fine.
7345
7346 In particular:
7347
7348 · Stateful testing has been removed from the API
7349
7350 · The way the database is used has been rendered less useful (sorry).
7351 The feature for reassembling values saved from other tests doesn't
7352 currently work. This will probably be brought back in post 1.0.
7353
7354 · SpecificationMapper is no longer a thing. Instead there is an
7355 ExtMethod called strategy which you extend to specify how to convert
7356 other types to strategies.
7357
7358 · Settings are now extensible so you can add your own for configuring a
7359 strategy
7360
7361 · MappedSearchStrategy no longer needs an unpack method
7362
7363 · Basically all the SearchStrategy internals have changed massively. If
7364 you implemented SearchStrategy directly rather than using Mapped‐
7365 SearchStrategy talk to me about fixing it.
7366
7367 · Change to the way extra packages work. You now specify the package.
7368 This must have a load() method. Additionally any modules in the pack‐
7369 age will be loaded in under hypothesis.extra
7370
7371 Bug fixes:
7372
7373 · Fix for a bug where calling falsify on a lambda with a non-ascii
7374 character in its body would error.
7375
7376 Hypothesis Extra:
7377
7378 hypothesis-fakefactory: An extension for using faker data in hypothe‐
7379 sis. Depends
7380 on fake-factory.
7381
7382 0.5.0 - 2015-02-10
7383 Codename: Read all about it.
7384
7385 Core hypothesis:
7386
7387 · Add support back in for pypy and python 3.2
7388
7389 · @given functions can now be invoked with some arguments explicitly
7390 provided. If all arguments that hypothesis would have provided are
7391 passed in then no falsification is run.
7392
7393 · Related to the above, this means that you can now use pytest fixtures
7394 and mark.parametrize with Hypothesis without either interfering with
7395 the other.
7396
7397 · Breaking change: @given no longer works for functions with varargs
7398 (varkwargs are fine). This might be added back in at a later date.
7399
7400 · Windows is now fully supported. A limited version (just the tests
7401 with none of the extras) of the test suite is run on windows with
7402 each commit so it is now a first class citizen of the Hypothesis
7403 world.
7404
7405 · Fix a bug for fuzzy equality of equal complex numbers with different
7406 reprs (this can happen when one coordinate is zero). This shouldn't
7407 affect users - that feature isn't used anywhere public facing.
7408
7409 · Fix generation of floats on windows and 32-bit builds of python. I
7410 was using some struct.pack logic that only worked on certain word
7411 sizes.
7412
7413 · When a test times out and hasn't produced enough examples this now
7414 raises a Timeout subclass of Unfalsifiable.
7415
7416 · Small search spaces are better supported. Previously something like a
7417 @given(bool, bool) would have failed because it couldn't find enough
7418 examples. Hypothesis is now aware of the fact that these are small
7419 search spaces and will not error in this case.
7420
7421 · Improvements to parameter search in the case of hard to satisfy
7422 assume. Hypothesis will now spend less time exploring parameters that
7423 are unlikely to provide anything useful.
7424
7425 · Increase chance of generating "nasty" floats
7426
7427 · Fix a bug that would have caused unicode warnings if you had a sam‐
7428 pled_from that was mixing unicode and byte strings.
7429
7430 · Added a standard test suite that you can use to validate a custom
7431 strategy you've defined is working correctly.
7432
7433 Hypothesis extra:
7434
7435 First off, introducing Hypothesis extra packages!
7436
7437 These are packages that are separated out from core Hypothesis because
7438 they have one or more dependencies. Every hypothesis-extra package is
7439 pinned to a specific point release of Hypothesis and will have some
7440 version requirements on its dependency. They use entry_points so you
7441 will usually not need to explicitly import them, just have them
7442 installed on the path.
7443
7444 This release introduces two of them:
7445
7446 hypothesis-datetime:
7447
7448 Does what it says on the tin: Generates datetimes for Hypothesis. Just
7449 install the package and datetime support will start working.
7450
7451 Depends on pytz for timezone support
7452
7453 hypothesis-pytest:
7454
7455 A very rudimentary pytest plugin. All it does right now is hook the
7456 display of falsifying examples into pytest reporting.
7457
7458 Depends on pytest.
7459
7460 0.4.3 - 2015-02-05
7461 Codename: TIL narrow Python builds are a thing
7462
7463 This just fixes the one bug.
7464
7465 · Apparently there is such a thing as a "narrow python build" and OS X
7466 ships with these by default for python 2.7. These are builds where
7467 you only have two bytes worth of unicode. As a result, generating
7468 unicode was completely broken on OS X. Fix this by only generating
7469 unicode codepoints in the range supported by the system.
7470
7471 0.4.2 - 2015-02-04
7472 Codename: O(dear)
7473
7474 This is purely a bugfix release:
7475
7476 · Provide sensible external hashing for all core types. This will sig‐
7477 nificantly improve performance of tracking seen examples which hap‐
7478 pens in literally every falsification run. For Hypothesis fixing this
7479 cut 40% off the runtime of the test suite. The behaviour is quadratic
7480 in the number of examples so if you're running the default configura‐
7481 tion this will be less extreme (Hypothesis's test suite runs at a
7482 higher number of examples than default), but you should still see a
7483 significant improvement.
7484
7485 · Fix a bug in formatting of complex numbers where the string could get
7486 incorrectly truncated.
7487
7488 0.4.1 - 2015-02-03
7489 Codename: Cruel and unusual edge cases
7490
7491 This release is mostly about better test case generation.
7492
7493 Enhancements:
7494
7495 · Has a cool release name
7496
7497 · text_type (str in python 3, unicode in python 2) example generation
7498 now actually produces interesting unicode instead of boring ascii
7499 strings.
7500
7501 · floating point numbers are generated over a much wider range, with
7502 particular attention paid to generating nasty numbers - nan, infin‐
7503 ity, large and small values, etc.
7504
7505 · examples can be generated using pieces of examples previously saved
7506 in the database. This allows interesting behaviour that has previ‐
7507 ously been discovered to be propagated to other examples.
7508
7509 · improved parameter exploration algorithm which should allow it to
7510 more reliably hit interesting edge cases.
7511
7512 · Timeout can now be disabled entirely by setting it to any value <= 0.
7513
7514 Bug fixes:
7515
7516 · The descriptor on a OneOfStrategy could be wrong if you had descrip‐
7517 tors which were equal but should not be coalesced. e.g. a strategy
7518 for one_of((frozenset({int}), {int})) would have reported its
7519 descriptor as {int}. This is unlikely to have caused you any problems
7520
7521 · If you had strategies that could produce NaN (which float previously
7522 couldn't but e.g. a Just(float('nan')) could) then this would have
7523 sent hypothesis into an infinite loop that would have only been ter‐
7524 minated when it hit the timeout.
7525
7526 · Given elements that can take a long time to minimize, minimization of
7527 floats or tuples could be quadratic or worse in the that value. You
7528 should now see much better performance for simplification, albeit at
7529 some cost in quality.
7530
7531 Other:
7532
7533 · A lot of internals have been been rewritten. This shouldn't affect
7534 you at all, but it opens the way for certain of hypothesis's oddities
7535 to be a lot more extensible by users. Whether this is a good thing
7536 may be up for debate...
7537
7538 0.4.0 - 2015-01-21
7539 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
7540 stores data in a local SQLite database and will reuse it for all tests
7541 of the same type.
7542
7543 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
7544 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
7545 ther notice. The previous license remains applicable to all code prior
7546 to 0.4.0.
7547
7548 Enhancements:
7549
7550 · Printing of failing examples. I was finding that the pytest runner
7551 was not doing a good job of displaying these, and that Hypothesis
7552 itself could do much better.
7553
7554 · Drop dependency on six for cross-version compatibility. It was easy
7555 enough to write the shim for the small set of features that we care
7556 about and this lets us avoid a moderately complex dependency.
7557
7558 · Some improvements to statistical distribution of selecting from small
7559 (<= 3 elements)
7560
7561 · Improvements to parameter selection for finding examples.
7562
7563 Bugs fixed:
7564
7565 · could_have_produced for lists, dicts and other collections would not
7566 have examined the elements and thus when using a union of different
7567 types of list this could result in Hypothesis getting confused and
7568 passing a value to the wrong strategy. This could potentially result
7569 in exceptions being thrown from within simplification.
7570
7571 · sampled_from would not work correctly on a single element list.
7572
7573 · Hypothesis could get very confused by values which are equal despite
7574 having different types being used in descriptors. Hypothesis now has
7575 its own more specific version of equality it uses for descriptors and
7576 tracking. It is always more fine grained than Python equality: Things
7577 considered != are not considered equal by hypothesis, but some things
7578 that are considered == are distinguished. If your test suite uses
7579 both frozenset and set tests this bug is probably affecting you.
7580
7581 0.3.2 - 2015-01-16
7582 · Fix a bug where if you specified floats_in_range with integer argu‐
7583 ments Hypothesis would error in example simplification.
7584
7585 · Improve the statistical distribution of the floats you get for the
7586 floats_in_range strategy. I'm not sure whether this will affect users
7587 in practice but it took my tests for various conditions from flaky to
7588 rock solid so it at the very least improves discovery of the artifi‐
7589 cial cases I'm looking for.
7590
7591 · Improved repr() for strategies and RandomWithSeed instances.
7592
7593 · Add detection for flaky test cases where hypothesis managed to find
7594 an example which breaks it but on the final invocation of the test it
7595 does not raise an error. This will typically happen with too much
7596 recursion errors but could conceivably happen in other circumstances
7597 too.
7598
7599 · Provide a "derandomized" mode. This allows you to run hypothesis with
7600 zero real randomization, making your build nice and deterministic.
7601 The tests run with a seed calculated from the function they're test‐
7602 ing so you should still get a good distribution of test cases.
7603
7604 · Add a mechanism for more conveniently defining tests which just sam‐
7605 ple from some collection.
7606
7607 · Fix for a really subtle bug deep in the internals of the strategy ta‐
7608 ble. In some circumstances if you were to define instance strategies
7609 for both a parent class and one or more of its subclasses you would
7610 under some circumstances get the strategy for the wrong superclass of
7611 an instance. It is very unlikely anyone has ever encountered this in
7612 the wild, but it is conceivably possible given that a mix of namedtu‐
7613 ple and tuple are used fairly extensively inside hypothesis which do
7614 exhibit this pattern of strategy.
7615
7616 0.3.1 - 2015-01-13
7617 · Support for generation of frozenset and Random values
7618
7619 · Correct handling of the case where a called function mutates it argu‐
7620 ment. This involved introducing a notion of a strategies knowing how
7621 to copy their argument. The default method should be entirely accept‐
7622 able and the worst case is that it will continue to have the old be‐
7623 haviour if you don't mark your strategy as mutable, so this shouldn't
7624 break anything.
7625
7626 · Fix for a bug where some strategies did not correctly implement
7627 could_have_produced. It is very unlikely that any of these would have
7628 been seen in the wild, and the consequences if they had been would
7629 have been minor.
7630
7631 · Re-export the @given decorator from the main hypothesis namespace.
7632 It's still available at the old location too.
7633
7634 · Minor performance optimisation for simplifying long lists.
7635
7636 0.3.0 - 2015-01-12
7637 · Complete redesign of the data generation system. Extreme breaking
7638 change for anyone who was previously writing their own SearchStrategy
7639 implementations. These will not work any more and you'll need to mod‐
7640 ify them.
7641
7642 · New settings system allowing more global and modular control of Veri‐
7643 fier behaviour.
7644
7645 · Decouple SearchStrategy from the StrategyTable. This leads to much
7646 more composable code which is a lot easier to understand.
7647
7648 · A significant amount of internal API renaming and moving. This may
7649 also break your code.
7650
7651 · Expanded available descriptors, allowing for generating integers or
7652 floats in a specific range.
7653
7654 · Significantly more robust. A very large number of small bug fixes,
7655 none of which anyone is likely to have ever noticed.
7656
7657 · Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
7658 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
7659 these to remain officially supported for a very long time. I would
7660 not be surprised to add pypy support back in later but I'm not going
7661 to do so until I know someone cares about it. In the meantime it will
7662 probably still work.
7663
7664 0.2.2 - 2015-01-08
7665 · Fix an embarrassing complete failure of the installer caused by my
7666 being bad at version control
7667
7668 0.2.1 - 2015-01-07
7669 · Fix a bug in the new stateful testing feature where you could make
7670 __init__ a @requires method. Simplification would not always work if
7671 the prune method was able to successfully shrink the test.
7672
7673 0.2.0 - 2015-01-07
7674 · It's aliiive.
7675
7676 · Improve python 3 support using six.
7677
7678 · Distinguish between byte and unicode types.
7679
7680 · Fix issues where FloatStrategy could raise.
7681
7682 · Allow stateful testing to request constructor args.
7683
7684 · Fix for issue where test annotations would timeout based on when the
7685 module was loaded instead of when the test started
7686
7687 0.1.4 - 2013-12-14
7688 · Make verification runs time bounded with a configurable timeout
7689
7690 0.1.3 - 2013-05-03
7691 · Bugfix: Stateful testing behaved incorrectly with subclassing.
7692
7693 · Complex number support
7694
7695 · support for recursive strategies
7696
7697 · different error for hypotheses with unsatisfiable assumptions
7698
7699 0.1.2 - 2013-03-24
7700 · Bugfix: Stateful testing was not minimizing correctly and could throw
7701 exceptions.
7702
7703 · Better support for recursive strategies.
7704
7705 · Support for named tuples.
7706
7707 · Much faster integer generation.
7708
7709 0.1.1 - 2013-03-24
7710 · Python 3.x support via 2to3.
7711
7712 · Use new style classes (oops).
7713
7714 0.1.0 - 2013-03-23
7715 · Introduce stateful testing.
7716
7717 · Massive rewrite of internals to add flags and strategies.
7718
7719 0.0.5 - 2013-03-13
7720 · No changes except trying to fix packaging
7721
7722 0.0.4 - 2013-03-13
7723 · No changes except that I checked in a failing test case for 0.0.3 so
7724 had to replace the release. Doh
7725
7726 0.0.3 - 2013-03-13
7727 · Improved a few internals.
7728
7729 · Opened up creating generators from instances as a general API.
7730
7731 · Test integration.
7732
7733 0.0.2 - 2013-03-12
7734 · Starting to tighten up on the internals.
7735
7736 · Change API to allow more flexibility in configuration.
7737
7738 · More testing.
7739
7740 0.0.1 - 2013-03-10
7741 · Initial release.
7742
7743 · Basic working prototype. Demonstrates idea, probably shouldn't be
7744 used.
7745
7747 Hypothesis development is managed by me, David R. MacIver. I am the
7748 primary author of Hypothesis.
7749
7750 However, I no longer do unpaid feature development on Hypothesis. My
7751 roles as leader of the project are:
7752
7753 1. Helping other people do feature development on Hypothesis
7754
7755 2. Fixing bugs and other code health issues
7756
7757 3. Improving documentation
7758
7759 4. General release management work
7760
7761 5. Planning the general roadmap of the project
7762
7763 6. Doing sponsored development on tasks that are too large or in depth
7764 for other people to take on
7765
7766 So all new features must either be sponsored or implemented by someone
7767 else. That being said, the maintenance team takes an active role in
7768 shepherding pull requests and helping people write a new feature (see
7769 CONTRIBUTING.rst for details and pull request #154 for an example of
7770 how the process goes). This isn't "patches welcome", it's "we will help
7771 you write a patch".
7772
7773 Release Policy
7774 Hypothesis releases follow semantic versioning.
7775
7776 We maintain backwards-compatibility wherever possible, and use depreca‐
7777 tion warnings to mark features that have been superseded by a newer
7778 alternative. If you want to detect this, you can upgrade warnings to
7779 errors in the usual ways.
7780
7781 We use continuous deployment to ensure that you can always use our new‐
7782 est and shiniest features - every change to the source tree is automat‐
7783 ically built and published on PyPI as soon as it's merged onto master,
7784 after code review and passing our extensive test suite.
7785
7786 Project Roadmap
7787 Hypothesis does not have a long-term release plan. However some visi‐
7788 bility into our plans for future compatibility may be useful:
7789
7790 · We value compatibility, and maintain it as far as practical. This
7791 generally excludes things which are end-of-life upstream, or have an
7792 unstable API.
7793
7794 · We would like to drop Python 2 support when it reaches end of life in
7795 2020. Ongoing support is likely to depend on commercial funding.
7796
7797 · We intend to support PyPy3 as soon as it supports a recent enough
7798 version of Python 3. See issue #602.
7799
7801 For questions you are happy to ask in public, the Hypothesis community
7802 is a friendly place where I or others will be more than happy to help
7803 you out. You're also welcome to ask questions on Stack Overflow. If you
7804 do, please tag them with 'python-hypothesis' so someone sees them.
7805
7806 For bugs and enhancements, please file an issue on the GitHub issue
7807 tracker. Note that as per the development policy, enhancements will
7808 probably not get implemented unless you're willing to pay for develop‐
7809 ment or implement them yourself (with assistance from me). Bugs will
7810 tend to get fixed reasonably promptly, though it is of course on a best
7811 effort basis.
7812
7813 To see the versions of Python, optional dependencies, test runners, and
7814 operating systems Hypothesis supports (meaning incompatibility is
7815 treated as a bug), see supported.
7816
7817 If you need to ask questions privately or want more of a guarantee of
7818 bugs being fixed promptly, please contact me on
7819 hypothesis-support@drmaciver.com to talk about availability of support
7820 contracts.
7821
7823 Downstream packagers often want to package Hypothesis. Here are some
7824 guidelines.
7825
7826 The primary guideline is this: If you are not prepared to keep up with
7827 the Hypothesis release schedule, don't. You will annoy me and are doing
7828 your users a disservice.
7829
7830 Hypothesis has a very frequent release schedule. It's rare that it goes
7831 a week without a release, and there are often multiple releases in a
7832 given week.
7833
7834 If you are prepared to keep up with this schedule, you might find the
7835 rest of this document useful.
7836
7837 Release tarballs
7838 These are available from the GitHub releases page. The tarballs on pypi
7839 are intended for installation from a Python tool such as pip and should
7840 not be considered complete releases. Requests to include additional
7841 files in them will not be granted. Their absence is not a bug.
7842
7843 Dependencies
7844 Python versions
7845 Hypothesis is designed to work with a range of Python versions. Cur‐
7846 rently supported are:
7847
7848 · pypy-2.6.1 (earlier versions of pypy may work)
7849
7850 · CPython 2.7.x
7851
7852 · CPython 3.4.x
7853
7854 · CPython 3.5.x
7855
7856 · CPython 3.6.x
7857
7858 · CPython 3.7.x
7859
7860 If you feel the need to have separate Python 3 and Python 2 packages
7861 you can, but Hypothesis works unmodified on either.
7862
7863 Other Python libraries
7864 Hypothesis has mandatory dependencies on the following libraries:
7865
7866 · attrs
7867
7868 · coverage
7869
7870 · enum34 is required on Python 2.7
7871
7872 Hypothesis has optional dependencies on the following libraries:
7873
7874 · pytz (almost any version should work)
7875
7876 · Faker, version 0.7 or later
7877
7878 · Django, all supported versions
7879
7880 · numpy, 1.10 or later (earlier versions will probably work fine)
7881
7882 · pandas, 1.19 or later
7883
7884 · pytest (3.0 or greater). This is a mandatory dependency for testing
7885 Hypothesis itself but optional for users.
7886
7887 The way this works when installing Hypothesis normally is that these
7888 features become available if the relevant library is installed.
7889
7890 Testing Hypothesis
7891 If you want to test Hypothesis as part of your packaging you will prob‐
7892 ably not want to use the mechanisms Hypothesis itself uses for running
7893 its tests, because it has a lot of logic for installing and testing
7894 against different versions of Python.
7895
7896 The tests must be run with py.test. A version more recent than 2.8.0 is
7897 strongly encouraged, but it may work with earlier versions (however
7898 py.test specific logic is disabled before 2.8.0).
7899
7900 Tests are organised into a number of top level subdirectories of the
7901 tests/ directory.
7902
7903 · cover: This is a small, reasonably fast, collection of tests designed
7904 to give 100% coverage of all but a select subset of the files when
7905 run under Python 3.
7906
7907 · nocover: This is a much slower collection of tests that should not be
7908 run under coverage for performance reasons.
7909
7910 · py2: Tests that can only be run under Python 2
7911
7912 · py3: Tests that can only be run under Python 3
7913
7914 · datetime: This tests the subset of Hypothesis that depends on pytz
7915
7916 · fakefactory: This tests the subset of Hypothesis that depends on
7917 fakefactory.
7918
7919 · django: This tests the subset of Hypothesis that depends on django
7920
7921 An example invocation for running the coverage subset of these tests:
7922
7923 pip install -e .
7924 pip install pytest # you will probably want to use your own packaging here
7925 python -m pytest tests/cover
7926
7927 Examples
7928 · arch linux
7929
7930 · fedora
7931
7932 · gentoo
7933
7935 One of the things that is often concerning for people using randomized
7936 testing like Hypothesis is the question of how to reproduce failing
7937 test cases.
7938
7939 Fortunately Hypothesis has a number of features in support of this. The
7940 one you will use most commonly when developing locally is the example
7941 database, which means that you shouldn't have to think about the prob‐
7942 lem at all for local use - test failures will just automatically repro‐
7943 duce without you having to do anything.
7944
7945 The example database is perfectly suitable for sharing between
7946 machines, but there currently aren't very good work flows for that, so
7947 Hypothesis provides a number of ways to make examples reproducible by
7948 adding them to the source code of your tests. This is particularly use‐
7949 ful when e.g. you are trying to run an example that has failed on your
7950 CI, or otherwise share them between machines.
7951
7952 Providing explicit examples
7953 You can explicitly ask Hypothesis to try a particular example, using
7954
7955 hypothesis.example(*args, **kwargs)
7956 A decorator which ensures a specific example is always tested.
7957
7958 Hypothesis will run all examples you've asked for first. If any of them
7959 fail it will not go on to look for more examples.
7960
7961 It doesn't matter whether you put the example decorator before or after
7962 given. Any permutation of the decorators in the above will do the same
7963 thing.
7964
7965 Note that examples can be positional or keyword based. If they're posi‐
7966 tional then they will be filled in from the right when calling, so
7967 either of the following styles will work as expected:
7968
7969 @given(text())
7970 @example("Hello world")
7971 @example(x="Some very long string")
7972 def test_some_code(x):
7973 assert True
7974
7975 from unittest import TestCase
7976
7977 class TestThings(TestCase):
7978 @given(text())
7979 @example("Hello world")
7980 @example(x="Some very long string")
7981 def test_some_code(self, x):
7982 assert True
7983
7984 As with @given, it is not permitted for a single example to be a mix of
7985 positional and keyword arguments. Either are fine, and you can use one
7986 in one example and the other in another example if for some reason you
7987 really want to, but a single example must be consistent.
7988
7989 Reproducing a test run with @seed
7990 hypothesis.seed(seed)
7991 seed: Start the test execution from a specific seed.
7992
7993 May be any hashable object. No exact meaning for seed is pro‐
7994 vided other than that for a fixed seed value Hypothesis will try
7995 the same actions (insofar as it can given external sources of
7996 non- determinism. e.g. timing and hash randomization).
7997
7998 Overrides the derandomize setting, which is designed to enable
7999 deterministic builds rather than reproducing observed failures.
8000
8001 When a test fails unexpectedly, usually due to a health check failure,
8002 Hypothesis will print out a seed that led to that failure, if the test
8003 is not already running with a fixed seed. You can then recreate that
8004 failure using either the @seed decorator or (if you are running pytest)
8005 with --hypothesis-seed.
8006
8007 Reproducing an example with with @reproduce_failure
8008 Hypothesis has an opaque binary representation that it uses for all
8009 examples it generates. This representation is not intended to be stable
8010 across versions or with respect to changes in the test, but can be used
8011 to to reproduce failures with the @reproduce_example decorator.
8012
8013 hypothesis.reproduce_failure(version, blob)
8014 Run the example that corresponds to this data blob in order to
8015 reproduce a failure.
8016
8017 A test with this decorator always runs only one example and
8018 always fails. If the provided example does not cause a failure,
8019 or is in some way invalid for this test, then this will fail
8020 with a DidNotReproduce error.
8021
8022 This decorator is not intended to be a permanent addition to
8023 your test suite. It's simply some code you can add to ease
8024 reproduction of a problem in the event that you don't have
8025 access to the test database. Because of this, no compatibility
8026 guarantees are made between different versions of Hypothesis -
8027 its API may change arbitrarily from version to version.
8028
8029 The intent is that you should never write this decorator by hand, but
8030 it is instead provided by Hypothesis. When a test fails with a falsi‐
8031 fying example, Hypothesis may print out a suggestion to use @repro‐
8032 duce_failure on the test to recreate the problem as follows:
8033
8034 >>> from hypothesis import settings, given, PrintSettings
8035 >>> import hypothesis.strategies as st
8036 >>> @given(st.floats())
8037 ... @settings(print_blob=PrintSettings.ALWAYS)
8038 ... def test(f):
8039 ... assert f == f
8040 ...
8041 >>> try:
8042 ... test()
8043 ... except AssertionError:
8044 ... pass
8045 Falsifying example: test(f=nan)
8046
8047 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
8048
8049 Adding the suggested decorator to the test should reproduce the failure
8050 (as long as everything else is the same - changing the versions of
8051 Python or anything else involved, might of course affect the behaviour
8052 of the test! Note that changing the version of Hypothesis will result
8053 in a different error - each @reproduce_failure invocation is specific
8054 to a Hypothesis version).
8055
8056 When to do this is controlled by the print_blob setting, which may be
8057 one of the following values:
8058
8059 class hypothesis.PrintSettings
8060 Flags to determine whether or not to print a detailed example
8061 blob to use with reproduce_failure() for failing test cases.
8062
8063 NEVER = 0
8064 Never print a blob.
8065
8066 INFER = 1
8067 Make an educated guess as to whether it would be appro‐
8068 priate to print the blob.
8069
8070 The current rules are that this will print if both:
8071
8072 1. The output from Hypothesis appears to be unsuitable
8073 for use with example().
8074
8075 2. The output is not too long.
8076
8077 ALWAYS = 2
8078 Always print a blob on failure.
8079
8081 David R. MacIver
8082
8084 2013-2018, David R. MacIver
8085
8086
8087
8088
80893.66.11 Jul 28, 2018 HYPOTHESIS(1)