1HYPOTHESIS(1) Hypothesis HYPOTHESIS(1)
2
3
4
6 hypothesis - Hypothesis Documentation
7
8 Hypothesis is a Python library for creating unit tests which are sim‐
9 pler to write and more powerful when run, finding edge cases in your
10 code you wouldn't have thought to look for. It is stable, powerful and
11 easy to add to any existing test suite.
12
13 It works by letting you write tests that assert that something should
14 be true for every case, not just the ones you happen to think of.
15
16 Think of a normal unit test as being something like the following:
17
18 1. Set up some data.
19
20 2. Perform some operations on the data.
21
22 3. Assert something about the result.
23
24 Hypothesis lets you write tests which instead look like this:
25
26 1. For all data matching some specification.
27
28 2. Perform some operations on the data.
29
30 3. Assert something about the result.
31
32 This is often called property based testing, and was popularised by the
33 Haskell library Quickcheck.
34
35 It works by generating arbitrary data matching your specification and
36 checking that your guarantee still holds in that case. If it finds an
37 example where it doesn't, it takes that example and cuts it down to
38 size, simplifying it until it finds a much smaller example that still
39 causes the problem. It then saves that example for later, so that once
40 it has found a problem with your code it will not forget it in the
41 future.
42
43 Writing tests of this form usually consists of deciding on guarantees
44 that your code should make - properties that should always hold true,
45 regardless of what the world throws at you. Examples of such guarantees
46 might be:
47
48 · Your code shouldn't throw an exception, or should only throw a par‐
49 ticular type of exception (this works particularly well if you have a
50 lot of internal assertions).
51
52 · If you delete an object, it is no longer visible.
53
54 · If you serialize and then deserialize a value, then you get the same
55 value back.
56
57 Now you know the basics of what Hypothesis does, the rest of this docu‐
58 mentation will take you through how and why. It's divided into a number
59 of sections, which you can see in the sidebar (or the menu at the top
60 if you're on mobile), but you probably want to begin with the Quick
61 start guide, which will give you a worked example of how to use Hypoth‐
62 esis and a detailed outline of the things you need to know to begin
63 testing your code with it, or check out some of the introductory arti‐
64 cles.
65
67 This document should talk you through everything you need to get
68 started with Hypothesis.
69
70 An example
71 Suppose we've written a run length encoding system and we want to test
72 it out.
73
74 We have the following code which I took straight from the Rosetta Code
75 wiki (OK, I removed some commented out code and fixed the formatting,
76 but there are no functional modifications):
77
78 def encode(input_string):
79 count = 1
80 prev = ''
81 lst = []
82 for character in input_string:
83 if character != prev:
84 if prev:
85 entry = (prev, count)
86 lst.append(entry)
87 count = 1
88 prev = character
89 else:
90 count += 1
91 entry = (character, count)
92 lst.append(entry)
93 return lst
94
95
96 def decode(lst):
97 q = ''
98 for character, count in lst:
99 q += character * count
100 return q
101
102 We want to write a test for this that will check some invariant of
103 these functions.
104
105 The invariant one tends to try when you've got this sort of encoding /
106 decoding is that if you encode something and then decode it then you
107 get the same value back.
108
109 Lets see how you'd do that with Hypothesis:
110
111 from hypothesis import given
112 from hypothesis.strategies import text
113
114 @given(text())
115 def test_decode_inverts_encode(s):
116 assert decode(encode(s)) == s
117
118 (For this example we'll just let pytest discover and run the test.
119 We'll cover other ways you could have run it later).
120
121 The text function returns what Hypothesis calls a search strategy. An
122 object with methods that describe how to generate and simplify certain
123 kinds of values. The @given decorator then takes our test function and
124 turns it into a parametrized one which, when called, will run the test
125 function over a wide range of matching data from that strategy.
126
127 Anyway, this test immediately finds a bug in the code:
128
129 Falsifying example: test_decode_inverts_encode(s='')
130
131 UnboundLocalError: local variable 'character' referenced before assignment
132
133 Hypothesis correctly points out that this code is simply wrong if
134 called on an empty string.
135
136 If we fix that by just adding the following code to the beginning of
137 the function then Hypothesis tells us the code is correct (by doing
138 nothing as you'd expect a passing test to).
139
140 if not input_string:
141 return []
142
143 If we wanted to make sure this example was always checked we could add
144 it in explicitly by using the @example decorator:
145
146 from hypothesis import given, example
147 from hypothesis.strategies import text
148
149 @given(text())
150 @example('')
151 def test_decode_inverts_encode(s):
152 assert decode(encode(s)) == s
153
154 This can be useful to show other developers (or your future self) what
155 kinds of data are valid inputs, or to ensure that particular edge cases
156 such as "" are tested every time. It's also great for regression tests
157 because although Hypothesis will remember failing examples, we don't
158 recommend distributing that database.
159
160 It's also worth noting that both example and given support keyword
161 arguments as well as positional. The following would have worked just
162 as well:
163
164 @given(s=text())
165 @example(s='')
166 def test_decode_inverts_encode(s):
167 assert decode(encode(s)) == s
168
169 Suppose we had a more interesting bug and forgot to reset the count
170 each time. Say we missed a line in our encode method:
171
172 def encode(input_string):
173 count = 1
174 prev = ''
175 lst = []
176 for character in input_string:
177 if character != prev:
178 if prev:
179 entry = (prev, count)
180 lst.append(entry)
181 # count = 1 # Missing reset operation
182 prev = character
183 else:
184 count += 1
185 entry = (character, count)
186 lst.append(entry)
187 return lst
188
189 Hypothesis quickly informs us of the following example:
190
191 Falsifying example: test_decode_inverts_encode(s='001')
192
193 Note that the example provided is really quite simple. Hypothesis
194 doesn't just find any counter-example to your tests, it knows how to
195 simplify the examples it finds to produce small easy to understand
196 ones. In this case, two identical values are enough to set the count to
197 a number different from one, followed by another distinct value which
198 should have reset the count but in this case didn't.
199
200 The examples Hypothesis provides are valid Python code you can run. Any
201 arguments that you explicitly provide when calling the function are not
202 generated by Hypothesis, and if you explicitly provide all the argu‐
203 ments Hypothesis will just call the underlying function once rather
204 than running it multiple times.
205
206 Installing
207 Hypothesis is available on PyPI as "hypothesis". You can install it
208 with:
209
210 pip install hypothesis
211
212 You can install the dependencies for optional extensions with e.g. pip
213 install hypothesis[pandas,django].
214
215 If you want to install directly from the source code (e.g. because you
216 want to make changes and install the changed version), check out the
217 instructions in CONTRIBUTING.rst.
218
219 Running tests
220 In our example above we just let pytest discover and run our tests, but
221 we could also have run it explicitly ourselves:
222
223 if __name__ == '__main__':
224 test_decode_inverts_encode()
225
226 We could also have done this as a python:unittest.TestCase:
227
228 import unittest
229
230 class TestEncoding(unittest.TestCase):
231 @given(text())
232 def test_decode_inverts_encode(self, s):
233 self.assertEqual(decode(encode(s)), s)
234
235 if __name__ == '__main__':
236 unittest.main()
237
238 A detail: This works because Hypothesis ignores any arguments it hasn't
239 been told to provide (positional arguments start from the right), so
240 the self argument to the test is simply ignored and works as normal.
241 This also means that Hypothesis will play nicely with other ways of
242 parameterizing tests. e.g it works fine if you use pytest fixtures for
243 some arguments and Hypothesis for others.
244
245 Writing tests
246 A test in Hypothesis consists of two parts: A function that looks like
247 a normal test in your test framework of choice but with some additional
248 arguments, and a @given decorator that specifies how to provide those
249 arguments.
250
251 Here are some other examples of how you could use that:
252
253 from hypothesis import given
254 import hypothesis.strategies as st
255
256 @given(st.integers(), st.integers())
257 def test_ints_are_commutative(x, y):
258 assert x + y == y + x
259
260 @given(x=st.integers(), y=st.integers())
261 def test_ints_cancel(x, y):
262 assert (x + y) - y == x
263
264 @given(st.lists(st.integers()))
265 def test_reversing_twice_gives_same_list(xs):
266 # This will generate lists of arbitrary length (usually between 0 and
267 # 100 elements) whose elements are integers.
268 ys = list(xs)
269 ys.reverse()
270 ys.reverse()
271 assert xs == ys
272
273 @given(st.tuples(st.booleans(), st.text()))
274 def test_look_tuples_work_too(t):
275 # A tuple is generated as the one you provided, with the corresponding
276 # types in those positions.
277 assert len(t) == 2
278 assert isinstance(t[0], bool)
279 assert isinstance(t[1], str)
280
281 Note that as we saw in the above example you can pass arguments to
282 @given either as positional or as keywords.
283
284 Where to start
285 You should now know enough of the basics to write some tests for your
286 code using Hypothesis. The best way to learn is by doing, so go have a
287 try.
288
289 If you're stuck for ideas for how to use this sort of test for your
290 code, here are some good starting points:
291
292 1. Try just calling functions with appropriate arbitrary data and see
293 if they crash. You may be surprised how often this works. e.g. note
294 that the first bug we found in the encoding example didn't even get
295 as far as our assertion: It crashed because it couldn't handle the
296 data we gave it, not because it did the wrong thing.
297
298 2. Look for duplication in your tests. Are there any cases where you're
299 testing the same thing with multiple different examples? Can you
300 generalise that to a single test using Hypothesis?
301
302 3. This piece is designed for an F# implementation, but is still very
303 good advice which you may find helps give you good ideas for using
304 Hypothesis.
305
306 If you have any trouble getting started, don't feel shy about asking
307 for help.
308
310 This is an account of slightly less common Hypothesis features that you
311 don't need to get started but will nevertheless make your life easier.
312
313 Additional test output
314 Normally the output of a failing test will look something like:
315
316 Falsifying example: test_a_thing(x=1, y="foo")
317
318 With the repr of each keyword argument being printed.
319
320 Sometimes this isn't enough, either because you have values with a repr
321 that isn't very descriptive or because you need to see the output of
322 some intermediate steps of your test. That's where the note function
323 comes in:
324
325 hypothesis.note(value)
326 Report this value in the final execution.
327
328 >>> from hypothesis import given, note, strategies as st
329 >>> @given(st.lists(st.integers()), st.randoms())
330 ... def test_shuffle_is_noop(ls, r):
331 ... ls2 = list(ls)
332 ... r.shuffle(ls2)
333 ... note("Shuffle: %r" % (ls2))
334 ... assert ls == ls2
335 ...
336 >>> try:
337 ... test_shuffle_is_noop()
338 ... except AssertionError:
339 ... print('ls != ls2')
340 Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
341 Shuffle: [1, 0]
342 ls != ls2
343
344 The note is printed in the final run of the test in order to include
345 any additional information you might need in your test.
346
347 Test Statistics
348 If you are using pytest you can see a number of statistics about the
349 executed tests by passing the command line argument --hypothe‐
350 sis-show-statistics. This will include some general statistics about
351 the test:
352
353 For example if you ran the following with --hypothesis-show-statistics:
354
355 from hypothesis import given, strategies as st
356
357 @given(st.integers())
358 def test_integers(i):
359 pass
360
361 You would see:
362
363 test_integers:
364
365 - 100 passing examples, 0 failing examples, 0 invalid examples
366 - Typical runtimes: ~ 1ms
367 - Fraction of time spent in data generation: ~ 12%
368 - Stopped because settings.max_examples=100
369
370 The final "Stopped because" line is particularly important to note: It
371 tells you the setting value that determined when the test should stop
372 trying new examples. This can be useful for understanding the behaviour
373 of your tests. Ideally you'd always want this to be max_examples.
374
375 In some cases (such as filtered and recursive strategies) you will see
376 events mentioned which describe some aspect of the data generation:
377
378 from hypothesis import given, strategies as st
379
380 @given(st.integers().filter(lambda x: x % 2 == 0))
381 def test_even_integers(i):
382 pass
383
384 You would see something like:
385
386 test_even_integers:
387
388 - 100 passing examples, 0 failing examples, 36 invalid examples
389 - Typical runtimes: 0-1 ms
390 - Fraction of time spent in data generation: ~ 16%
391 - Stopped because settings.max_examples=100
392 - Events:
393 * 80.88%, Retried draw from integers().filter(lambda x: <unknown>) to satisfy filter
394 * 26.47%, Aborted test because unable to satisfy integers().filter(lambda x: <unknown>)
395
396 You can also mark custom events in a test using the event function:
397
398 hypothesis.event(value)
399 Record an event that occurred this test. Statistics on number of
400 test runs with each event will be reported at the end if you run
401 Hypothesis in statistics reporting mode.
402
403 Events should be strings or convertible to them.
404
405 from hypothesis import given, event, strategies as st
406
407 @given(st.integers().filter(lambda x: x % 2 == 0))
408 def test_even_integers(i):
409 event("i mod 3 = %d" % (i % 3,))
410
411 You will then see output like:
412
413 test_even_integers:
414
415 - 100 passing examples, 0 failing examples, 38 invalid examples
416 - Typical runtimes: 0-1 ms
417 - Fraction of time spent in data generation: ~ 16%
418 - Stopped because settings.max_examples=100
419 - Events:
420 * 80.43%, Retried draw from integers().filter(lambda x: <unknown>) to satisfy filter
421 * 31.88%, i mod 3 = 0
422 * 27.54%, Aborted test because unable to satisfy integers().filter(lambda x: <unknown>)
423 * 21.74%, i mod 3 = 1
424 * 18.84%, i mod 3 = 2
425
426 Arguments to event can be any hashable type, but two events will be
427 considered the same if they are the same when converted to a string
428 with python:str.
429
430 Making assumptions
431 Sometimes Hypothesis doesn't give you exactly the right sort of data
432 you want - it's mostly of the right shape, but some examples won't work
433 and you don't want to care about them. You can just ignore these by
434 aborting the test early, but this runs the risk of accidentally testing
435 a lot less than you think you are. Also it would be nice to spend less
436 time on bad examples - if you're running 100 examples per test (the
437 default) and it turns out 70 of those examples don't match your needs,
438 that's a lot of wasted time.
439
440 hypothesis.assume(condition)
441 Calling assume is like an assert that marks the example as bad,
442 rather than failing the test.
443
444 This allows you to specify properties that you assume will be
445 true, and let Hypothesis try to avoid similar examples in
446 future.
447
448 For example suppose you had the following test:
449
450 @given(floats())
451 def test_negation_is_self_inverse(x):
452 assert x == -(-x)
453
454 Running this gives us:
455
456 Falsifying example: test_negation_is_self_inverse(x=float('nan'))
457 AssertionError
458
459 This is annoying. We know about NaN and don't really care about it, but
460 as soon as Hypothesis finds a NaN example it will get distracted by
461 that and tell us about it. Also the test will fail and we want it to
462 pass.
463
464 So lets block off this particular example:
465
466 from math import isnan
467
468 @given(floats())
469 def test_negation_is_self_inverse_for_non_nan(x):
470 assume(not isnan(x))
471 assert x == -(-x)
472
473 And this passes without a problem.
474
475 In order to avoid the easy trap where you assume a lot more than you
476 intended, Hypothesis will fail a test when it can't find enough exam‐
477 ples passing the assumption.
478
479 If we'd written:
480
481 @given(floats())
482 def test_negation_is_self_inverse_for_non_nan(x):
483 assume(False)
484 assert x == -(-x)
485
486 Then on running we'd have got the exception:
487
488 Unsatisfiable: Unable to satisfy assumptions of hypothesis test_negation_is_self_inverse_for_non_nan. Only 0 examples considered satisfied assumptions
489
490 How good is assume?
491 Hypothesis has an adaptive exploration strategy to try to avoid things
492 which falsify assumptions, which should generally result in it still
493 being able to find examples in hard to find situations.
494
495 Suppose we had the following:
496
497 @given(lists(integers()))
498 def test_sum_is_positive(xs):
499 assert sum(xs) > 0
500
501 Unsurprisingly this fails and gives the falsifying example [].
502
503 Adding assume(xs) to this removes the trivial empty example and gives
504 us [0].
505
506 Adding assume(all(x > 0 for x in xs)) and it passes: the sum of a list
507 of positive integers is positive.
508
509 The reason that this should be surprising is not that it doesn't find a
510 counter-example, but that it finds enough examples at all.
511
512 In order to make sure something interesting is happening, suppose we
513 wanted to try this for long lists. e.g. suppose we added an
514 assume(len(xs) > 10) to it. This should basically never find an exam‐
515 ple: a naive strategy would find fewer than one in a thousand examples,
516 because if each element of the list is negative with probability
517 one-half, you'd have to have ten of these go the right way by chance.
518 In the default configuration Hypothesis gives up long before it's tried
519 1000 examples (by default it tries 200).
520
521 Here's what happens if we try to run this:
522
523 @given(lists(integers()))
524 def test_sum_is_positive(xs):
525 assume(len(xs) > 10)
526 assume(all(x > 0 for x in xs))
527 print(xs)
528 assert sum(xs) > 0
529
530 In: test_sum_is_positive()
531 [17, 12, 7, 13, 11, 3, 6, 9, 8, 11, 47, 27, 1, 31, 1]
532 [6, 2, 29, 30, 25, 34, 19, 15, 50, 16, 10, 3, 16]
533 [25, 17, 9, 19, 15, 2, 2, 4, 22, 10, 10, 27, 3, 1, 14, 17, 13, 8, 16, 9, 2...
534 [17, 65, 78, 1, 8, 29, 2, 79, 28, 18, 39]
535 [13, 26, 8, 3, 4, 76, 6, 14, 20, 27, 21, 32, 14, 42, 9, 24, 33, 9, 5, 15, ...
536 [2, 1, 2, 2, 3, 10, 12, 11, 21, 11, 1, 16]
537
538 As you can see, Hypothesis doesn't find many examples here, but it
539 finds some - enough to keep it happy.
540
541 In general if you can shape your strategies better to your tests you
542 should - for example integers(1, 1000) is a lot better than assume(1 <=
543 x <= 1000), but assume will take you a long way if you can't.
544
545 Defining strategies
546 The type of object that is used to explore the examples given to your
547 test function is called a SearchStrategy. These are created using the
548 functions exposed in the hypothesis.strategies module.
549
550 Many of these strategies expose a variety of arguments you can use to
551 customize generation. For example for integers you can specify min and
552 max values of integers you want. If you want to see exactly what a
553 strategy produces you can ask for an example:
554
555 >>> integers(min_value=0, max_value=10).example()
556 1
557
558 Many strategies are built out of other strategies. For example, if you
559 want to define a tuple you need to say what goes in each element:
560
561 >>> from hypothesis.strategies import tuples
562 >>> tuples(integers(), integers()).example()
563 (-24597, 12566)
564
565 Further details are available in a separate document.
566
567 The gory details of given parameters
568 hypothesis.given(*given_arguments, **given_kwargs)
569 A decorator for turning a test function that accepts arguments
570 into a randomized test.
571
572 This is the main entry point to Hypothesis.
573
574 The @given decorator may be used to specify which arguments of a func‐
575 tion should be parametrized over. You can use either positional or key‐
576 word arguments, but not a mixture of both.
577
578 For example all of the following are valid uses:
579
580 @given(integers(), integers())
581 def a(x, y):
582 pass
583
584 @given(integers())
585 def b(x, y):
586 pass
587
588 @given(y=integers())
589 def c(x, y):
590 pass
591
592 @given(x=integers())
593 def d(x, y):
594 pass
595
596 @given(x=integers(), y=integers())
597 def e(x, **kwargs):
598 pass
599
600 @given(x=integers(), y=integers())
601 def f(x, *args, **kwargs):
602 pass
603
604
605 class SomeTest(TestCase):
606 @given(integers())
607 def test_a_thing(self, x):
608 pass
609
610 The following are not:
611
612 @given(integers(), integers(), integers())
613 def g(x, y):
614 pass
615
616 @given(integers())
617 def h(x, *args):
618 pass
619
620 @given(integers(), x=integers())
621 def i(x, y):
622 pass
623
624 @given()
625 def j(x, y):
626 pass
627
628 The rules for determining what are valid uses of given are as follows:
629
630 1. You may pass any keyword argument to given.
631
632 2. Positional arguments to given are equivalent to the rightmost named
633 arguments for the test function.
634
635 3. Positional arguments may not be used if the underlying test function
636 has varargs, arbitrary keywords, or keyword-only arguments.
637
638 4. Functions tested with given may not have any defaults.
639
640 The reason for the "rightmost named arguments" behaviour is so that
641 using @given with instance methods works: self will be passed to the
642 function as normal and not be parametrized over.
643
644 The function returned by given has all the same arguments as the origi‐
645 nal test, minus those that are filled in by @given. Check the notes on
646 framework compatibility to see how this affects other testing libraries
647 you may be using.
648
649 Custom function execution
650 Hypothesis provides you with a hook that lets you control how it runs
651 examples.
652
653 This lets you do things like set up and tear down around each example,
654 run examples in a subprocess, transform coroutine tests into normal
655 tests, etc. For example, TransactionTestCase in the Django extra runs
656 each example in a separate database transaction.
657
658 The way this works is by introducing the concept of an executor. An
659 executor is essentially a function that takes a block of code and run
660 it. The default executor is:
661
662 def default_executor(function):
663 return function()
664
665 You define executors by defining a method execute_example on a class.
666 Any test methods on that class with @given used on them will use
667 self.execute_example as an executor with which to run tests. For exam‐
668 ple, the following executor runs all its code twice:
669
670 from unittest import TestCase
671
672 class TestTryReallyHard(TestCase):
673 @given(integers())
674 def test_something(self, i):
675 perform_some_unreliable_operation(i)
676
677 def execute_example(self, f):
678 f()
679 return f()
680
681 Note: The functions you use in map, etc. will run inside the executor.
682 i.e. they will not be called until you invoke the function passed to
683 execute_example.
684
685 An executor must be able to handle being passed a function which
686 returns None, otherwise it won't be able to run normal test cases. So
687 for example the following executor is invalid:
688
689 from unittest import TestCase
690
691 class TestRunTwice(TestCase):
692 def execute_example(self, f):
693 return f()()
694
695 and should be rewritten as:
696
697 from unittest import TestCase
698
699 class TestRunTwice(TestCase):
700 def execute_example(self, f):
701 result = f()
702 if callable(result):
703 result = result()
704 return result
705
706 An alternative hook is provided for use by test runner extensions such
707 as pytest-trio, which cannot use the execute_example method. This is
708 not recommended for end-users - it is better to write a complete test
709 function directly, perhaps by using a decorator to perform the same
710 transformation before applying @given.
711
712 @given(x=integers())
713 @pytest.mark.trio
714 async def test(x):
715 ...
716 # Illustrative code, inside the pytest-trio plugin
717 test.hypothesis.inner_test = lambda x: trio.run(test, x)
718
719 For authors of test runners however, assigning to the inner_test
720 attribute of the hypothesis attribute of the test will replace the
721 interior test.
722
723 NOTE:
724 The new inner_test must accept and pass through all the *args and
725 **kwargs expected by the original test.
726
727 If the end user has also specified a custom executor using the exe‐
728 cute_example method, it - and all other execution-time logic - will be
729 applied to the new inner test assigned by the test runner.
730
731 Making random code deterministic
732 While Hypothesis' example generation can be used for nondeterministic
733 tests, debugging anything nondeterministic is usually a very frustrat‐
734 ing excercise. To make things worse, our example shrinking relies on
735 the same input causing the same failure each time - though we show the
736 un-shrunk failure and a decent error message if it doesn't.
737
738 By default, Hypothesis will handle the global random and numpy.random
739 random number generators for you, and you can register others:
740
741 hypothesis.register_random(r)
742 Register the given Random instance for management by Hypothesis.
743
744 You can pass random.Random instances (or other objects with
745 seed, getstate, and setstate methods) to register_random(r) to
746 have their states seeded and restored in the same way as the
747 global PRNGs from the random and numpy.random modules.
748
749 All global PRNGs, from e.g. simulation or scheduling frameworks,
750 should be registered to prevent flaky tests. Hypothesis will
751 ensure that the PRNG state is consistent for all test runs, or
752 reproducibly varied if you choose to use the random_module()
753 strategy.
754
755 Using Hypothesis to find values
756 You can use Hypothesis's data exploration features to find values sat‐
757 isfying some predicate. This is generally useful for exploring custom
758 strategies defined with @composite, or experimenting with conditions
759 for filtering data.
760
761 hypothesis.find(specifier, condition, settings=None, random=None, data‐
762 base_key=None)
763 Returns the minimal example from the given strategy specifier
764 that matches the predicate function condition.
765
766 >>> from hypothesis import find
767 >>> from hypothesis.strategies import sets, lists, integers
768 >>> find(lists(integers()), lambda x: sum(x) >= 10)
769 [10]
770 >>> find(lists(integers()), lambda x: sum(x) >= 10 and len(x) >= 3)
771 [0, 0, 10]
772 >>> find(sets(integers()), lambda x: sum(x) >= 10 and len(x) >= 3)
773 {0, 1, 9}
774
775 The first argument to find() describes data in the usual way for an
776 argument to given(), and supports all the same data types. The second
777 is a predicate it must satisfy.
778
779 Of course not all conditions are satisfiable. If you ask Hypothesis for
780 an example to a condition that is always false it will raise an error:
781
782 >>> find(integers(), lambda x: False)
783 Traceback (most recent call last):
784 ...
785 hypothesis.errors.NoSuchExample: No examples of condition lambda x: <unknown>
786
787 (The lambda x: unknown is because Hypothesis can't retrieve the source
788 code of lambdas from the interactive python console. It gives a better
789 error message most of the time which contains the actual condition)
790
791 Inferred Strategies
792 In some cases, Hypothesis can work out what to do when you omit argu‐
793 ments. This is based on introspection, not magic, and therefore has
794 well-defined limits.
795
796 builds() will check the signature of the target (using get‐
797 fullargspec()). If there are required arguments with type annotations
798 and no strategy was passed to builds(), from_type() is used to fill
799 them in. You can also pass the special value hypothesis.infer as a
800 keyword argument, to force this inference for arguments with a default
801 value.
802
803 >>> def func(a: int, b: str):
804 ... return [a, b]
805 >>> builds(func).example()
806 [-6993, '']
807
808 hypothesis.infer
809
810 @given does not perform any implicit inference for required arguments,
811 as this would break compatibility with pytest fixtures. infer can be
812 used as a keyword argument to explicitly fill in an argument from its
813 type annotation.
814
815 @given(a=infer)
816 def test(a: int): pass
817 # is equivalent to
818 @given(a=integers())
819 def test(a): pass
820
821 Limitations
822 PEP 3107 type annotations are not supported on Python 2, and Hypothesis
823 does not inspect PEP 484 type comments at runtime. While from_type()
824 will work as usual, inference in builds() and @given will only work if
825 you manually create the __annotations__ attribute (e.g. by using @anno‐
826 tations(...) and @returns(...) decorators). The python:typing module
827 is fully supported on Python 2 if you have the backport installed.
828
829 The python:typing module is provisional and has a number of internal
830 changes between Python 3.5.0 and 3.6.1, including at minor versions.
831 These are all supported on a best-effort basis, but you may encounter
832 problems with an old version of the module. Please report them to us,
833 and consider updating to a newer version of Python as a workaround.
834
835 Type Annotations in Hypothesis
836 If you install Hypothesis and use mypy 0.590+, or another PEP 561-com‐
837 patible tool, the type checker should automatically pick up our type
838 hints.
839
840 NOTE:
841 Hypothesis' type hints may make breaking changes between minor
842 releases.
843
844 Upstream tools and conventions about type hints remain in flux - for
845 example the python:typing module itself is provisional, and Mypy has
846 not yet reached version 1.0 - and we plan to support the latest ver‐
847 sion of this ecosystem, as well as older versions where practical.
848
849 We may also find more precise ways to describe the type of various
850 interfaces, or change their type and runtime behaviour togther in a
851 way which is otherwise backwards-compatible. We often omit type
852 hints for deprecated features or arguments, as an additional form of
853 warning.
854
855 There are known issues inferring the type of examples generated by
856 deferred(), recursive(), one_of(), dictionaries(), and fixed_dictionar‐
857 ies(). We will fix these, and require correspondingly newer versions
858 of Mypy for type hinting, as the ecosystem improves.
859
860 Writing downstream type hints
861 Projects that provide Hypothesis strategies and use type hints may wish
862 to annotate their strategies too. This is a supported use-case, again
863 on a best-effort provisional basis. For example:
864
865 def foo_strategy() -> SearchStrategy[Foo]: ...
866
867 class hypothesis.strategies.SearchStrategy
868
869 SearchStrategy is the type of all strategy objects. It is a generic
870 type, and covariant in the type of the examples it creates. For exam‐
871 ple:
872
873 · integers() is of type SearchStrategy[int].
874
875 · lists(integers()) is of type SearchStrategy[List[int]].
876
877 · SearchStrategy[Dog] is a subtype of SearchStrategy[Animal] if Dog is
878 a subtype of Animal (as seems likely).
879
880 WARNING:
881 SearchStrategy should only be used in type hints. Please do not
882 inherit from, compare to, or otherwise use it in any way outside of
883 type hints. The only supported way to construct objects of this
884 type is to use the functions provided by the hypothesis.strategies
885 module!
886
887 The Hypothesis pytest Plugin
888 Hypothesis includes a tiny plugin to improve integration with pytest,
889 which is activated by default (but does not affect other test runners).
890 It aims to improve the integration between Hypothesis and Pytest by
891 providing extra information and convenient access to config options.
892
893 · pytest --hypothesis-show-statistics can be used to display test and
894 data generation statistics.
895
896 · pytest --hypothesis-profile=<profile name> can be used to load a set‐
897 tings profile. pytest --hypothesis-verbosity=<level name> can be
898 used to override the current verbosity level.
899
900 · pytest --hypothesis-seed=<an int> can be used to reproduce a failure
901 with a particular seed.
902
903 Finally, all tests that are defined with Hypothesis automatically have
904 @pytest.mark.hypothesis applied to them. See here for information on
905 working with markers.
906
907 NOTE:
908 Pytest will load the plugin automatically if Hypothesis is
909 installed. You don't need to do anything at all to use it.
910
912 Hypothesis tries to have good defaults for its behaviour, but sometimes
913 that's not enough and you need to tweak it.
914
915 The mechanism for doing this is the settings object. You can set up a
916 @given based test to use this using a settings decorator:
917
918 @given invocation is as follows:
919
920 from hypothesis import given, settings
921
922 @given(integers())
923 @settings(max_examples=500)
924 def test_this_thoroughly(x):
925 pass
926
927 This uses a settings object which causes the test to receive a much
928 larger set of examples than normal.
929
930 This may be applied either before or after the given and the results
931 are the same. The following is exactly equivalent:
932
933 from hypothesis import given, settings
934
935 @settings(max_examples=500)
936 @given(integers())
937 def test_this_thoroughly(x):
938 pass
939
940 Available settings
941 class hypothesis.settings(parent=None, **kwargs)
942 A settings object controls a variety of parameters that are used
943 in falsification. These may control both the falsification
944 strategy and the details of the data that is generated.
945
946 Default values are picked up from the settings.default object
947 and changes made there will be picked up in newly created set‐
948 tings.
949
950 buffer_size
951 The size of the underlying data used to generate exam‐
952 ples. If you need to generate really large examples you
953 may want to increase this, but it will make your tests
954 slower.
955
956 default value: (dynamically calculated)
957
958 database
959 An instance of hypothesis.database.ExampleDatabase that
960 will be used to save examples to and load previous exam‐
961 ples from. May be None in which case no storage will be
962 used, :memory: for an in-memory database, or any path for
963 a directory-based example database.
964
965 default value: (dynamically calculated)
966
967 deadline
968 If set, a duration (as timedelta, or integer or float
969 number of milliseconds) that each individual example
970 (i.e. each time your test function is called, not the
971 whole decorated test) within a test is not allowed to
972 exceed. Tests which take longer than that may be con‐
973 verted into errors (but will not necessarily be if close
974 to the deadline, to allow some variability in test run
975 time).
976
977 Set this to None to disable this behaviour entirely.
978
979 default value: timedelta(milliseconds=200.0)
980
981 derandomize
982 If this is True then hypothesis will run in deterministic
983 mode where each falsification uses a random number gener‐
984 ator that is seeded based on the hypothesis to falsify,
985 which will be consistent across multiple runs. This has
986 the advantage that it will eliminate any randomness from
987 your tests, which may be preferable for some situations.
988 It does have the disadvantage of making your tests less
989 likely to find novel breakages.
990
991 default value: False
992
993 max_examples
994 Once this many satisfying examples have been considered
995 without finding any counter-example, falsification will
996 terminate.
997
998 The default value is chosen to suit a workflow where the
999 test will be part of a suite that is regularly executed
1000 locally or on a CI server, balancing total running time
1001 against the chance of missing a bug.
1002
1003 If you are writing one-off tests, running tens of thou‐
1004 sands of examples is quite reasonable as Hypothesis may
1005 miss uncommon bugs with default settings. For very com‐
1006 plex code, we have observed Hypothesis finding novel bugs
1007 after several million examples while testing SymPy.
1008
1009 default value: 100
1010
1011 phases Control which phases should be run. See the full documen‐
1012 tation for more details
1013
1014 default value: (<Phase.explicit: 0>, <Phase.reuse: 1>,
1015 <Phase.generate: 2>, <Phase.shrink: 3>)
1016
1017 print_blob
1018 Determines whether to print blobs after tests that can be
1019 used to reproduce failures.
1020
1021 See the documentation on @reproduce_failure for more
1022 details of this behaviour.
1023
1024 default value: PrintSettings.INFER
1025
1026 report_multiple_bugs
1027 Because Hypothesis runs the test many times, it can some‐
1028 times find multiple bugs in a single run. Reporting all
1029 of them at once is usually very useful, but replacing the
1030 exceptions can occasionally clash with debuggers. If
1031 disabled, only the exception with the smallest minimal
1032 example is raised.
1033
1034 default value: True
1035
1036 stateful_step_count
1037 Number of steps to run a stateful program for before giv‐
1038 ing up on it breaking.
1039
1040 default value: 50
1041
1042 suppress_health_check
1043 A list of health checks to disable.
1044
1045 default value: ()
1046
1047 timeout
1048 The timeout setting has been deprecated and no longer
1049 does anything.
1050
1051 default value: not_set
1052
1053 The timeout setting can safely be removed with no effect.
1054
1055 verbosity
1056 Control the verbosity level of Hypothesis messages
1057
1058 default value: Verbosity.normal
1059
1060 Controlling What Runs
1061 Hypothesis divides tests into four logically distinct phases:
1062
1063 1. Running explicit examples provided with the @example decorator.
1064
1065 2. Rerunning a selection of previously failing examples to reproduce a
1066 previously seen error
1067
1068 3. Generating new examples.
1069
1070 4. Attempting to shrink an example found in phases 2 or 3 to a more
1071 manageable one (explicit examples cannot be shrunk).
1072
1073 The phases setting provides you with fine grained control over which of
1074 these run, with each phase corresponding to a value on the Phase enum:
1075
1076 class hypothesis.Phase
1077
1078 1. Phase.explicit controls whether explicit examples are run.
1079
1080 2. Phase.reuse controls whether previous examples will be reused.
1081
1082 3. Phase.generate controls whether new examples will be generated.
1083
1084 4. Phase.shrink controls whether examples will be shrunk.
1085
1086 The phases argument accepts a collection with any subset of these. e.g.
1087 settings(phases=[Phase.generate, Phase.shrink]) will generate new exam‐
1088 ples and shrink them, but will not run explicit examples or reuse pre‐
1089 vious failures, while settings(phases=[Phase.explicit]) will only run
1090 the explicit examples.
1091
1092 Seeing intermediate result
1093 To see what's going on while Hypothesis runs your tests, you can turn
1094 up the verbosity setting. This works with both find() and @given.
1095
1096 >>> from hypothesis import find, settings, Verbosity
1097 >>> from hypothesis.strategies import lists, integers
1098 >>> find(lists(integers()), any, settings=settings(verbosity=Verbosity.verbose))
1099 Tried non-satisfying example []
1100 Found satisfying example [-1198601713, -67, 116, -29578]
1101 Shrunk example to [-1198601713]
1102 Shrunk example to [-1198601600]
1103 Shrunk example to [-1191228800]
1104 Shrunk example to [-8421504]
1105 Shrunk example to [-32896]
1106 Shrunk example to [-128]
1107 Shrunk example to [64]
1108 Shrunk example to [32]
1109 Shrunk example to [16]
1110 Shrunk example to [8]
1111 Shrunk example to [4]
1112 Shrunk example to [3]
1113 Shrunk example to [2]
1114 Shrunk example to [1]
1115 [1]
1116
1117 The four levels are quiet, normal, verbose and debug. normal is the
1118 default, while in quiet mode Hypothesis will not print anything out,
1119 not even the final falsifying example. debug is basically verbose but a
1120 bit more so. You probably don't want it.
1121
1122 If you are using pytest, you may also need to disable output capturing
1123 for passing tests.
1124
1125 Building settings objects
1126 Settings can be created by calling settings with any of the available
1127 settings values. Any absent ones will be set to defaults:
1128
1129 >>> from hypothesis import settings
1130 >>> settings().max_examples
1131 100
1132 >>> settings(max_examples=10).max_examples
1133 10
1134
1135 You can also pass a 'parent' settings object as the first argument, and
1136 any settings you do not specify as keyword arguments will be copied
1137 from the parent settings:
1138
1139 >>> parent = settings(max_examples=10)
1140 >>> child = settings(parent, deadline=None)
1141 >>> parent.max_examples == child.max_examples == 10
1142 True
1143 >>> parent.deadline
1144 200
1145 >>> child.deadline is None
1146 True
1147
1148 Default settings
1149 At any given point in your program there is a current default settings,
1150 available as settings.default. As well as being a settings object in
1151 its own right, all newly created settings objects which are not explic‐
1152 itly based off another settings are based off the default, so will
1153 inherit any values that are not explicitly set from it.
1154
1155 You can change the defaults by using profiles.
1156
1157 settings Profiles
1158 Depending on your environment you may want different default settings.
1159 For example: during development you may want to lower the number of
1160 examples to speed up the tests. However, in a CI environment you may
1161 want more examples so you are more likely to find bugs.
1162
1163 Hypothesis allows you to define different settings profiles. These pro‐
1164 files can be loaded at any time.
1165
1166 class hypothesis.settings(parent=None, **kwargs)
1167 A settings object controls a variety of parameters that are used
1168 in falsification. These may control both the falsification
1169 strategy and the details of the data that is generated.
1170
1171 Default values are picked up from the settings.default object
1172 and changes made there will be picked up in newly created set‐
1173 tings.
1174
1175 static register_profile(name, parent=None, **kwargs)
1176 Registers a collection of values to be used as a settings
1177 profile.
1178
1179 Settings profiles can be loaded by name - for example,
1180 you might create a 'fast' profile which runs fewer exam‐
1181 ples, keep the 'default' profile, and create a 'ci' pro‐
1182 file that increases the number of examples and uses a
1183 different database to store failures.
1184
1185 The arguments to this method are exactly as for settings:
1186 optional parent settings, and keyword arguments for each
1187 setting that will be set differently to parent (or set‐
1188 tings.default, if parent is None).
1189
1190 static get_profile(name)
1191 Return the profile with the given name.
1192
1193 static load_profile(name)
1194 Loads in the settings defined in the profile provided.
1195
1196 If the profile does not exist, InvalidArgument will be
1197 raised. Any setting not defined in the profile will be
1198 the library defined default for that setting.
1199
1200 Loading a profile changes the default settings but will not change the
1201 behavior of tests that explicitly change the settings.
1202
1203 >>> from hypothesis import settings
1204 >>> settings.register_profile("ci", max_examples=1000)
1205 >>> settings().max_examples
1206 100
1207 >>> settings.load_profile("ci")
1208 >>> settings().max_examples
1209 1000
1210
1211 Instead of loading the profile and overriding the defaults you can
1212 retrieve profiles for specific tests.
1213
1214 >>> settings.get_profile("ci").max_examples
1215 1000
1216
1217 Optionally, you may define the environment variable to load a profile
1218 for you. This is the suggested pattern for running your tests on CI.
1219 The code below should run in a conftest.py or any setup/initialization
1220 section of your test suite. If this variable is not defined the
1221 Hypothesis defined defaults will be loaded.
1222
1223 >>> import os
1224 >>> from hypothesis import settings, Verbosity
1225 >>> settings.register_profile("ci", max_examples=1000)
1226 >>> settings.register_profile("dev", max_examples=10)
1227 >>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
1228 >>> settings.load_profile(os.getenv(u'HYPOTHESIS_PROFILE', 'default'))
1229
1230 If you are using the hypothesis pytest plugin and your profiles are
1231 registered by your conftest you can load one with the command line
1232 option --hypothesis-profile.
1233
1234 $ pytest tests --hypothesis-profile <profile-name>
1235
1237 Most things should be easy to generate and everything should be possi‐
1238 ble.
1239
1240 To support this principle Hypothesis provides strategies for most
1241 built-in types with arguments to constrain or adjust the output, as
1242 well as higher-order strategies that can be composed to generate more
1243 complex types.
1244
1245 This document is a guide to what strategies are available for generat‐
1246 ing data and how to build them. Strategies have a variety of other
1247 important internal features, such as how they simplify, but the data
1248 they can generate is the only public part of their API.
1249
1250 Functions for building strategies are all available in the hypothe‐
1251 sis.strategies module. The salient functions from it are as follows:
1252
1253 hypothesis.strategies.binary(min_size=0, max_size=None)
1254 Generates the appropriate binary type (str in python 2, bytes in
1255 python 3).
1256
1257 min_size and max_size have the usual interpretations.
1258
1259 Examples from this strategy shrink towards smaller strings and
1260 lower byte values.
1261
1262 hypothesis.strategies.booleans()
1263 Returns a strategy which generates instances of python:bool.
1264
1265 Examples from this strategy will shrink towards False (i.e.
1266 shrinking will try to replace True with False where possible).
1267
1268 hypothesis.strategies.builds(*callable_and_args, **kwargs)
1269 Generates values by drawing from args and kwargs and passing
1270 them to the callable (provided as the first positional argument)
1271 in the appropriate argument position.
1272
1273 e.g. builds(target, integers(), flag=booleans()) would draw an
1274 integer i and a boolean b and call target(i, flag=b).
1275
1276 If the callable has type annotations, they will be used to infer
1277 a strategy for required arguments that were not passed to
1278 builds. You can also tell builds to infer a strategy for an
1279 optional argument by passing the special value hypothesis.infer
1280 as a keyword argument to builds, instead of a strategy for that
1281 argument to the callable.
1282
1283 If the callable is a class defined with attrs, missing required
1284 arguments will be inferred from the attribute on a best-effort
1285 basis, e.g. by checking attrs standard validators. Dataclasses
1286 are handled natively by the inference from type hints.
1287
1288 Examples from this strategy shrink by shrinking the argument
1289 values to the callable.
1290
1291 hypothesis.strategies.characters(whitelist_categories=None, black‐
1292 list_categories=None, blacklist_characters=None, min_codepoint=None,
1293 max_codepoint=None, whitelist_characters=None)
1294 Generates unicode text type (unicode on python 2, str on python
1295 3) characters following specified filtering rules.
1296
1297 · When no filtering rules are specifed, any character can be
1298 produced.
1299
1300 · If min_codepoint or max_codepoint is specifed, then only char‐
1301 acters having a codepoint in that range will be produced.
1302
1303 · If whitelist_categories is specified, then only characters
1304 from those Unicode categories will be produced. This is a fur‐
1305 ther restriction, characters must also satisfy min_codepoint
1306 and max_codepoint.
1307
1308 · If blacklist_categories is specified, then any character from
1309 those categories will not be produced. Any overlap between
1310 whitelist_categories and blacklist_categories will raise an
1311 exception, as each character can only belong to a single
1312 class.
1313
1314 · If whitelist_characters is specified, then any additional
1315 characters in that list will also be produced.
1316
1317 · If blacklist_characters is specified, then any characters in
1318 that list will be not be produced. Any overlap between
1319 whitelist_characters and blacklist_characters will raise an
1320 exception.
1321
1322 The _codepoint arguments must be integers between zero and
1323 python:sys.maxunicode. The _characters arguments must be col‐
1324 lections of length-one unicode strings, such as a unicode
1325 string.
1326
1327 The _categories arguments must be used to specify either the
1328 one-letter Unicode major category or the two-letter Unicode
1329 general category. For example, ('Nd', 'Lu') signifies "Number,
1330 decimal digit" and "Letter, uppercase". A single letter ('major
1331 category') can be given to match all corresponding categories,
1332 for example 'P' for characters in any punctuation category.
1333
1334 Examples from this strategy shrink towards the codepoint for
1335 '0', or the first allowable codepoint after it if '0' is
1336 excluded.
1337
1338 hypothesis.strategies.complex_numbers(min_magnitude=0, max_magni‐
1339 tude=None, allow_infinity=None, allow_nan=None)
1340 Returns a strategy that generates complex numbers.
1341
1342 This strategy draws complex numbers with constrained magnitudes.
1343 The min_magnitude and max_magnitude parameters should be
1344 non-negative Real numbers; values of None correspond to zero and
1345 infinite values respectively.
1346
1347 If min_magnitude is positive or max_magnitude is finite, it is
1348 an error to enable allow_nan. If max_magnitude is finite, it is
1349 an error to enable allow_infinity.
1350
1351 The magnitude contraints are respected up to a relative error of
1352 (around) floating-point epsilon, due to implementation via the
1353 system sqrt function.
1354
1355 Examples from this strategy shrink by shrinking their real and
1356 imaginary parts, as floats().
1357
1358 If you need to generate complex numbers with particular real and
1359 imaginary parts or relationships between parts, consider using
1360 builds(complex, ...) or @composite respectively.
1361
1362 hypothesis.strategies.composite(f)
1363 Defines a strategy that is built out of potentially arbitrarily
1364 many other strategies.
1365
1366 This is intended to be used as a decorator. See the full docu‐
1367 mentation for more details about how to use this function.
1368
1369 Examples from this strategy shrink by shrinking the output of
1370 each draw call.
1371
1372 hypothesis.strategies.data()
1373 This isn't really a normal strategy, but instead gives you an
1374 object which can be used to draw data interactively from other
1375 strategies.
1376
1377 It can only be used within @given, not find(). This is because
1378 the lifetime of the object cannot outlast the test body.
1379
1380 See the rest of the documentation for more complete information.
1381
1382 Examples from this strategy do not shrink (because there is only
1383 one), but the result of calls to each draw() call shrink as they
1384 normally would.
1385
1386 class hypothesis.strategies.DataObject(data)
1387 This type only exists so that you can write type hints for tests
1388 using the data() strategy. Do not use it directly!
1389
1390 hypothesis.strategies.dates(min_value=datetime.date(1, 1, 1),
1391 max_value=datetime.date(9999, 12, 31))
1392 A strategy for dates between min_value and max_value.
1393
1394 Examples from this strategy shrink towards January 1st 2000.
1395
1396 hypothesis.strategies.datetimes(min_value=datetime.datetime(1, 1, 1, 0,
1397 0), max_value=datetime.datetime(9999, 12, 31, 23, 59, 59, 999999),
1398 timezones=none())
1399 A strategy for generating datetimes, which may be time‐
1400 zone-aware.
1401
1402 This strategy works by drawing a naive datetime between
1403 min_value and max_value, which must both be naive (have no time‐
1404 zone).
1405
1406 timezones must be a strategy that generates tzinfo objects (or
1407 None, which is valid for naive datetimes). A value drawn from
1408 this strategy will be added to a naive datetime, and the result‐
1409 ing tz-aware datetime returned.
1410
1411 NOTE:
1412 tz-aware datetimes from this strategy may be ambiguous or
1413 non-existent due to daylight savings, leap seconds, timezone
1414 and calendar adjustments, etc. This is intentional, as mal‐
1415 formed timestamps are a common source of bugs.
1416
1417 hypothesis.extra.pytz.timezones() requires the pytz package, but
1418 provides all timezones in the Olsen database. If you want to
1419 allow naive datetimes, combine strategies like none() | time‐
1420 zones().
1421
1422 hypothesis.extra.dateutil.timezones() requires the
1423 python-dateutil package, and similarly provides all timezones
1424 there.
1425
1426 Alternatively, you can create a list of the timezones you wish
1427 to allow (e.g. from the standard library, datetutil, or pytz)
1428 and use sampled_from(). Ensure that simple values such as None
1429 or UTC are at the beginning of the list for proper minimisation.
1430
1431 Examples from this strategy shrink towards midnight on January
1432 1st 2000.
1433
1434 hypothesis.strategies.decimals(min_value=None, max_value=None,
1435 allow_nan=None, allow_infinity=None, places=None)
1436 Generates instances of python:decimal.Decimal, which may be:
1437
1438 · A finite rational number, between min_value and max_value.
1439
1440 · Not a Number, if allow_nan is True. None means "allow NaN,
1441 unless min_value and max_value are not None".
1442
1443 · Positive or negative infinity, if max_value and min_value
1444 respectively are None, and allow_infinity is not False. None
1445 means "allow infinity, unless excluded by the min and max val‐
1446 ues".
1447
1448 Note that where floats have one NaN value, Decimals have four:
1449 signed, and either quiet or signalling. See the decimal module
1450 docs for more information on special values.
1451
1452 If places is not None, all finite values drawn from the strategy
1453 will have that number of digits after the decimal place.
1454
1455 Examples from this strategy do not have a well defined shrink
1456 order but try to maximize human readability when shrinking.
1457
1458 hypothesis.strategies.deferred(definition)
1459 A deferred strategy allows you to write a strategy that refer‐
1460 ences other strategies that have not yet been defined. This
1461 allows for the easy definition of recursive and mutually recur‐
1462 sive strategies.
1463
1464 The definition argument should be a zero-argument function that
1465 returns a strategy. It will be evaluated the first time the
1466 strategy is used to produce an example.
1467
1468 Example usage:
1469
1470 >>> import hypothesis.strategies as st
1471 >>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
1472 >>> x.example()
1473 (((False, (True, True)), (False, True)), (True, True))
1474 >>> x.example()
1475 True
1476
1477 Mutual recursion also works fine:
1478
1479 >>> a = st.deferred(lambda: st.booleans() | b)
1480 >>> b = st.deferred(lambda: st.tuples(a, a))
1481 >>> a.example()
1482 True
1483 >>> b.example()
1484 (False, (False, ((False, True), False)))
1485
1486 Examples from this strategy shrink as they normally would from
1487 the strategy returned by the definition.
1488
1489 hypothesis.strategies.dictionaries(keys, values, dict_class=<class
1490 'dict'>, min_size=0, max_size=None)
1491 Generates dictionaries of type dict_class with keys drawn from
1492 the keys argument and values drawn from the values argument.
1493
1494 The size parameters have the same interpretation as for lists.
1495
1496 Examples from this strategy shrink by trying to remove keys from
1497 the generated dictionary, and by shrinking each generated key
1498 and value.
1499
1500 hypothesis.strategies.emails()
1501 A strategy for generating email addresses as unicode strings.
1502 The address format is specified in RFC 5322#section-3.4.1. Val‐
1503 ues shrink towards shorter local-parts and host domains.
1504
1505 This strategy is useful for generating "user data" for tests, as
1506 mishandling of email addresses is a common source of bugs.
1507
1508 hypothesis.strategies.fixed_dictionaries(mapping)
1509 Generates a dictionary of the same type as mapping with a fixed
1510 set of keys mapping to strategies. mapping must be a dict sub‐
1511 class.
1512
1513 Generated values have all keys present in mapping, with the cor‐
1514 responding values drawn from mapping[key]. If mapping is an
1515 instance of OrderedDict the keys will also be in the same order,
1516 otherwise the order is arbitrary.
1517
1518 Examples from this strategy shrink by shrinking each individual
1519 value in the generated dictionary.
1520
1521 hypothesis.strategies.floats(min_value=None, max_value=None,
1522 allow_nan=None, allow_infinity=None, width=64, exclude_min=False,
1523 exclude_max=False)
1524 Returns a strategy which generates floats.
1525
1526 · If min_value is not None, all values will be >= min_value.
1527
1528 · If max_value is not None, all values will be <= max_value.
1529
1530 · If min_value or max_value is not None, it is an error to
1531 enable allow_nan.
1532
1533 · If both min_value and max_value are not None, it is an error
1534 to enable allow_infinity.
1535
1536 Where not explicitly ruled out by the bounds, all of infinity,
1537 -infinity and NaN are possible values generated by this strat‐
1538 egy.
1539
1540 The width argument specifies the maximum number of bits of pre‐
1541 cision required to represent the generated float. Valid values
1542 are 16, 32, or 64. Passing width=32 will still use the builtin
1543 64-bit float class, but always for values which can be exactly
1544 represented as a 32-bit float. Half-precision floats (width=16)
1545 are only supported on Python 3.6, or if Numpy is installed.
1546
1547 The exclude_min and exclude_max argument can be used to generate
1548 numbers from open or half-open intervals, by excluding the
1549 respective endpoints. Attempting to exclude an endpoint which
1550 is None will raise an error; use allow_infinity=False to gener‐
1551 ate finite floats. You can however use e.g.
1552 min_value=float("-inf"), exclude_min=True to exclude only one
1553 infinite endpoint.
1554
1555 Examples from this strategy have a complicated and hard to
1556 explain shrinking behaviour, but it tries to improve "human
1557 readability". Finite numbers will be preferred to infinity and
1558 infinity will be preferred to NaN.
1559
1560 hypothesis.strategies.fractions(min_value=None, max_value=None,
1561 max_denominator=None)
1562 Returns a strategy which generates Fractions.
1563
1564 If min_value is not None then all generated values are no less
1565 than min_value. If max_value is not None then all generated
1566 values are no greater than max_value. min_value and max_value
1567 may be anything accepted by the Fraction constructor.
1568
1569 If max_denominator is not None then the denominator of any gen‐
1570 erated values is no greater than max_denominator. Note that
1571 max_denominator must be None or a positive integer.
1572
1573 Examples from this strategy shrink towards smaller denominators,
1574 then closer to zero.
1575
1576 hypothesis.strategies.from_regex(regex, fullmatch=False)
1577 Generates strings that contain a match for the given regex (i.e.
1578 ones for which python:re.search() will return a non-None
1579 result).
1580
1581 regex may be a pattern or compiled regex. Both byte-strings and
1582 unicode strings are supported, and will generate examples of the
1583 same type.
1584
1585 You can use regex flags such as python:re.IGNORECASE or
1586 python:re.DOTALL to control generation. Flags can be passed
1587 either in compiled regex or inside the pattern with a (?iLmsux)
1588 group.
1589
1590 Some regular expressions are only partly supported - the under‐
1591 lying strategy checks local matching and relies on filtering to
1592 resolve context-dependent expressions. Using too many of these
1593 constructs may cause health-check errors as too many examples
1594 are filtered out. This mainly includes (positive or negative)
1595 lookahead and lookbehind groups.
1596
1597 If you want the generated string to match the whole regex you
1598 should use boundary markers. So e.g. r"\A.\Z" will return a sin‐
1599 gle character string, while "." will return any string, and
1600 r"\A.$" will return a single character optionally followed by a
1601 "\n". Alternatively, passing fullmatch=True will ensure that
1602 the whole string is a match, as if you had used the \A and \Z
1603 markers.
1604
1605 Examples from this strategy shrink towards shorter strings and
1606 lower character values, with exact behaviour that may depend on
1607 the pattern.
1608
1609 hypothesis.strategies.from_type(thing)
1610 Looks up the appropriate search strategy for the given type.
1611
1612 from_type is used internally to fill in missing arguments to
1613 builds() and can be used interactively to explore what strate‐
1614 gies are available or to debug type resolution.
1615
1616 You can use register_type_strategy() to handle your custom
1617 types, or to globally redefine certain strategies - for example
1618 excluding NaN from floats, or use timezone-aware instead of
1619 naive time and datetime strategies.
1620
1621 The resolution logic may be changed in a future version, but
1622 currently tries these four options:
1623
1624 1. If thing is in the default lookup mapping or user-registered
1625 lookup, return the corresponding strategy. The default
1626 lookup covers all types with Hypothesis strategies, including
1627 extras where possible.
1628
1629 2. If thing is from the python:typing module, return the corre‐
1630 sponding strategy (special logic).
1631
1632 3. If thing has one or more subtypes in the merged lookup,
1633 return the union of the strategies for those types that are
1634 not subtypes of other elements in the lookup.
1635
1636 4. Finally, if thing has type annotations for all required argu‐
1637 ments, it is resolved via builds().
1638
1639 There is a valuable recipe for leveraging from_type() to gener‐
1640 ate "everything except" values from a specified type. I.e.
1641
1642 def everything_except(excluded_types):
1643 return (
1644 from_type(type).flatmap(from_type)
1645 .filter(lambda x: not isinstance(x, excluded_types))
1646 )
1647
1648 For example, everything_except(int) returns a strategy that can
1649 generate anything that from_type() can ever generate, except for
1650 instances of :class:python:int, and excluding instances of types
1651 added via :func:~hypothesis.strategies.register_type_strategy.
1652
1653 This is useful when writing tests which check that invalid input
1654 is rejected in a certain way.
1655
1656 hypothesis.strategies.frozensets(elements, min_size=0, max_size=None)
1657 This is identical to the sets function but instead returns
1658 frozensets.
1659
1660 hypothesis.strategies.functions(like=<function <lambda>>,
1661 returns=none())
1662 A strategy for functions, which can be used in callbacks.
1663
1664 The generated functions will mimic the interface of like, which
1665 must be a callable (including a class, method, or function).
1666 The return value for the function is drawn from the returns
1667 argument, which must be a strategy.
1668
1669 Note that the generated functions do not validate their argu‐
1670 ments, and may return a different value if called again with the
1671 same arguments.
1672
1673 Generated functions can only be called within the scope of the
1674 @given which created them. This strategy does not support
1675 .example().
1676
1677 hypothesis.strategies.integers(min_value=None, max_value=None)
1678 Returns a strategy which generates integers; in Python 2 these
1679 may be ints or longs.
1680
1681 If min_value is not None then all values will be >= min_value.
1682 If max_value is not None then all values will be <= max_value
1683
1684 Examples from this strategy will shrink towards zero, and nega‐
1685 tive values will also shrink towards positive (i.e. -n may be
1686 replaced by +n).
1687
1688 hypothesis.strategies.iterables(elements, min_size=0, max_size=None,
1689 unique_by=None, unique=False)
1690 This has the same behaviour as lists, but returns iterables
1691 instead.
1692
1693 Some iterables cannot be indexed (e.g. sets) and some do not
1694 have a fixed length (e.g. generators). This strategy produces
1695 iterators, which cannot be indexed and do not have a fixed
1696 length. This ensures that you do not accidentally depend on
1697 sequence behaviour.
1698
1699 hypothesis.strategies.just(value)
1700 Return a strategy which only generates value.
1701
1702 Note: value is not copied. Be wary of using mutable values.
1703
1704 If value is the result of a callable, you can use
1705 builds(callable) instead of just(callable()) to get a fresh
1706 value each time.
1707
1708 Examples from this strategy do not shrink (because there is only
1709 one).
1710
1711 hypothesis.strategies.lists(elements, min_size=0, max_size=None,
1712 unique_by=None, unique=False)
1713 Returns a list containing values drawn from elements with length
1714 in the interval [min_size, max_size] (no bounds in that direc‐
1715 tion if these are None). If max_size is 0 then elements may be
1716 None and only the empty list will be drawn.
1717
1718 If unique is True (or something that evaluates to True), we com‐
1719 pare direct object equality, as if unique_by was lambda x: x.
1720 This comparison only works for hashable types.
1721
1722 If unique_by is not None it must be a callable or tuple of
1723 callables returning a hashable type when given a value drawn
1724 from elements. The resulting list will satisfy the condition
1725 that for i != j, unique_by(result[i]) != unique_by(result[j]).
1726
1727 If unique_by is a tuple of callables the uniqueness will be
1728 respective to each callable.
1729
1730 For example, the following will produce two columns of integers
1731 with both columns being unique respectively.
1732
1733 >>> twoints = st.tuples(st.integers(), st.integers())
1734 >>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))
1735
1736 Examples from this strategy shrink by trying to remove elements
1737 from the list, and by shrinking each individual element of the
1738 list.
1739
1740 hypothesis.strategies.none()
1741 Return a strategy which only generates None.
1742
1743 Examples from this strategy do not shrink (because there is only
1744 one).
1745
1746 hypothesis.strategies.nothing()
1747 This strategy never successfully draws a value and will always
1748 reject on an attempt to draw.
1749
1750 Examples from this strategy do not shrink (because there are
1751 none).
1752
1753 hypothesis.strategies.one_of(*args)
1754 Return a strategy which generates values from any of the argu‐
1755 ment strategies.
1756
1757 This may be called with one iterable argument instead of multi‐
1758 ple strategy arguments, in which case one_of(x) and one_of(*x)
1759 are equivalent.
1760
1761 Examples from this strategy will generally shrink to ones that
1762 come from strategies earlier in the list, then shrink according
1763 to behaviour of the strategy that produced them. In order to get
1764 good shrinking behaviour, try to put simpler strategies first.
1765 e.g. one_of(none(), text()) is better than one_of(text(),
1766 none()).
1767
1768 This is especially important when using recursive strategies.
1769 e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will
1770 shrink well, but x = st.deferred(lambda: st.tuples(x, x) |
1771 st.none()) will shrink very badly indeed.
1772
1773 hypothesis.strategies.permutations(values)
1774 Return a strategy which returns permutations of the ordered col‐
1775 lection values.
1776
1777 Examples from this strategy shrink by trying to become closer to
1778 the original order of values.
1779
1780 hypothesis.strategies.random_module()
1781 The Hypothesis engine handles PRNG state for the stdlib and
1782 Numpy random modules internally, always seeding them to zero and
1783 restoring the previous state after the test.
1784
1785 If having a fixed seed would unacceptably weaken your tests, and
1786 you cannot use a random.Random instance provided by randoms(),
1787 this strategy calls python:random.seed() with an arbitrary inte‐
1788 ger and passes you an opaque object whose repr displays the seed
1789 value for debugging. If numpy.random is available, that state
1790 is also managed.
1791
1792 Examples from these strategy shrink to seeds closer to zero.
1793
1794 hypothesis.strategies.randoms()
1795 Generates instances of random.Random, tweaked to show the seed
1796 value in the repr for reproducibility.
1797
1798 Examples from this strategy shrink to seeds closer to zero.
1799
1800 hypothesis.strategies.recursive(base, extend, max_leaves=100)
1801 base: A strategy to start from.
1802
1803 extend: A function which takes a strategy and returns a new
1804 strategy.
1805
1806 max_leaves: The maximum number of elements to be drawn from base
1807 on a given run.
1808
1809 This returns a strategy S such that S = extend(base | S). That
1810 is, values may be drawn from base, or from any strategy reach‐
1811 able by mixing applications of | and extend.
1812
1813 An example may clarify: recursive(booleans(), lists) would
1814 return a strategy that may return arbitrarily nested and mixed
1815 lists of booleans. So e.g. False, [True], [False, []], and
1816 [[[[True]]]] are all valid values to be drawn from that strat‐
1817 egy.
1818
1819 Examples from this strategy shrink by trying to reduce the
1820 amount of recursion and by shrinking according to the shrinking
1821 behaviour of base and the result of extend.
1822
1823 hypothesis.strategies.register_type_strategy(custom_type, strategy)
1824 Add an entry to the global type-to-strategy lookup.
1825
1826 This lookup is used in builds() and @given.
1827
1828 builds() will be used automatically for classes with type anno‐
1829 tations on __init__ , so you only need to register a strategy if
1830 one or more arguments need to be more tightly defined than their
1831 type-based default, or if you want to supply a strategy for an
1832 argument with a default value.
1833
1834 strategy may be a search strategy, or a function that takes a
1835 type and returns a strategy (useful for generic types).
1836
1837 hypothesis.strategies.runner(default=not_set)
1838 A strategy for getting "the current test runner", whatever that
1839 may be. The exact meaning depends on the entry point, but it
1840 will usually be the associated 'self' value for it.
1841
1842 If there is no current test runner and a default is provided,
1843 return that default. If no default is provided, raises Invali‐
1844 dArgument.
1845
1846 Examples from this strategy do not shrink (because there is only
1847 one).
1848
1849 hypothesis.strategies.sampled_from(elements)
1850 Returns a strategy which generates any value present in ele‐
1851 ments.
1852
1853 Note that as with just(), values will not be copied and thus you
1854 should be careful of using mutable data.
1855
1856 sampled_from supports ordered collections, as well as Enum
1857 objects. Flag objects may also generate any combination of
1858 their members.
1859
1860 Examples from this strategy shrink by replacing them with values
1861 earlier in the list. So e.g. sampled_from((10, 1)) will shrink
1862 by trying to replace 1 values with 10, and sampled_from((1, 10))
1863 will shrink by trying to replace 10 values with 1.
1864
1865 hypothesis.strategies.sets(elements, min_size=0, max_size=None)
1866 This has the same behaviour as lists, but returns sets instead.
1867
1868 Note that Hypothesis cannot tell if values are drawn from ele‐
1869 ments are hashable until running the test, so you can define a
1870 strategy for sets of an unhashable type but it will fail at test
1871 time.
1872
1873 Examples from this strategy shrink by trying to remove elements
1874 from the set, and by shrinking each individual element of the
1875 set.
1876
1877 hypothesis.strategies.shared(base, key=None)
1878 Returns a strategy that draws a single shared value per run,
1879 drawn from base. Any two shared instances with the same key will
1880 share the same value, otherwise the identity of this strategy
1881 will be used. That is:
1882
1883 >>> s = integers() # or any other strategy
1884 >>> x = shared(s)
1885 >>> y = shared(s)
1886
1887 In the above x and y may draw different (or potentially the
1888 same) values. In the following they will always draw the same:
1889
1890 >>> x = shared(s, key="hi")
1891 >>> y = shared(s, key="hi")
1892
1893 Examples from this strategy shrink as per their base strategy.
1894
1895 hypothesis.strategies.slices(size)
1896 Generates slices that will select indices up to the supplied
1897 size
1898
1899 Generated slices will have start and stop indices that range
1900 from 0 to size - 1 and will step in the appropriate direction.
1901 Slices should only produce an empty selection if the start and
1902 end are the same.
1903
1904 Examples from this strategy shrink toward 0 and smaller values
1905
1906 hypothesis.strategies.text(alphabet=characters(blacklist_cate‐
1907 gories=('Cs', )), min_size=0, max_size=None)
1908 Generates values of a unicode text type (unicode on python 2,
1909 str on python 3) with values drawn from alphabet, which should
1910 be an iterable of length one strings or a strategy generating
1911 such strings.
1912
1913 The default alphabet strategy can generate the full unicode
1914 range but excludes surrogate characters because they are invalid
1915 in the UTF-8 encoding. You can use characters() without argu‐
1916 ments to find surrogate-related bugs such as bpo-34454.
1917
1918 min_size and max_size have the usual interpretations. Note that
1919 Python measures string length by counting codepoints: U+00C5 Å
1920 is a single character, while U+0041 U+030A Å is two - the A,
1921 and a combining ring above.
1922
1923 Examples from this strategy shrink towards shorter strings, and
1924 with the characters in the text shrinking as per the alphabet
1925 strategy. This strategy does not normalize() examples, so gen‐
1926 erated strings may be in any or none of the 'normal forms'.
1927
1928 hypothesis.strategies.timedeltas(min_value=date‐
1929 time.timedelta(days=-999999999), max_value=date‐
1930 time.timedelta(days=999999999, seconds=86399, microseconds=999999))
1931 A strategy for timedeltas between min_value and max_value.
1932
1933 Examples from this strategy shrink towards zero.
1934
1935 hypothesis.strategies.times(min_value=datetime.time(0, 0),
1936 max_value=datetime.time(23, 59, 59, 999999), timezones=none())
1937 A strategy for times between min_value and max_value.
1938
1939 The timezones argument is handled as for datetimes().
1940
1941 Examples from this strategy shrink towards midnight, with the
1942 timezone component shrinking as for the strategy that provided
1943 it.
1944
1945 hypothesis.strategies.tuples(*args)
1946 Return a strategy which generates a tuple of the same length as
1947 args by generating the value at index i from args[i].
1948
1949 e.g. tuples(integers(), integers()) would generate a tuple of
1950 length two with both values an integer.
1951
1952 Examples from this strategy shrink by shrinking their component
1953 parts.
1954
1955 hypothesis.strategies.uuids(version=None)
1956 Returns a strategy that generates UUIDs.
1957
1958 If the optional version argument is given, value is passed
1959 through to UUID and only UUIDs of that version will be gener‐
1960 ated.
1961
1962 All returned values from this will be unique, so e.g. if you do
1963 lists(uuids()) the resulting list will never contain duplicates.
1964
1965 Examples from this strategy don't have any meaningful shrink
1966 order.
1967
1968 Shrinking
1969 When using strategies it is worth thinking about how the data shrinks.
1970 Shrinking is the process by which Hypothesis tries to produce human
1971 readable examples when it finds a failure - it takes a complex example
1972 and turns it into a simpler one.
1973
1974 Each strategy defines an order in which it shrinks - you won't usually
1975 need to care about this much, but it can be worth being aware of as it
1976 can affect what the best way to write your own strategies is.
1977
1978 The exact shrinking behaviour is not a guaranteed part of the API, but
1979 it doesn't change that often and when it does it's usually because we
1980 think the new way produces nicer examples.
1981
1982 Possibly the most important one to be aware of is one_of(), which has a
1983 preference for values produced by strategies earlier in its argument
1984 list. Most of the others should largely "do the right thing" without
1985 you having to think about it.
1986
1987 Adapting strategies
1988 Often it is the case that a strategy doesn't produce exactly what you
1989 want it to and you need to adapt it. Sometimes you can do this in the
1990 test, but this hurts reuse because you then have to repeat the adaption
1991 in every test.
1992
1993 Hypothesis gives you ways to build strategies from other strategies
1994 given functions for transforming the data.
1995
1996 Mapping
1997 map is probably the easiest and most useful of these to use. If you
1998 have a strategy s and a function f, then an example s.map(f).example()
1999 is f(s.example()), i.e. we draw an example from s and then apply f to
2000 it.
2001
2002 e.g.:
2003
2004 >>> lists(integers()).map(sorted).example()
2005 [-25527, -24245, -23118, -93, -70, -7, 0, 39, 40, 65, 88, 112, 6189, 9480, 19469, 27256, 32526, 1566924430]
2006
2007 Note that many things that you might use mapping for can also be done
2008 with builds().
2009
2010 Filtering
2011 filter lets you reject some examples. s.filter(f).example() is some
2012 example of s such that f(example) is truthy.
2013
2014 >>> integers().filter(lambda x: x > 11).example()
2015 26126
2016 >>> integers().filter(lambda x: x > 11).example()
2017 23324
2018
2019 It's important to note that filter isn't magic and if your condition is
2020 too hard to satisfy then this can fail:
2021
2022 >>> integers().filter(lambda x: False).example()
2023 Traceback (most recent call last):
2024 ...
2025 hypothesis.errors.NoExamples: Could not find any valid examples in 20 tries
2026
2027 In general you should try to use filter only to avoid corner cases that
2028 you don't want rather than attempting to cut out a large chunk of the
2029 search space.
2030
2031 A technique that often works well here is to use map to first transform
2032 the data and then use filter to remove things that didn't work out. So
2033 for example if you wanted pairs of integers (x,y) such that x < y you
2034 could do the following:
2035
2036 >>> tuples(integers(), integers()).map(sorted).filter(lambda x: x[0] < x[1]).example()
2037 [-8543729478746591815, 3760495307320535691]
2038
2039 Chaining strategies together
2040 Finally there is flatmap. flatmap draws an example, then turns that
2041 example into a strategy, then draws an example from that strategy.
2042
2043 It may not be obvious why you want this at first, but it turns out to
2044 be quite useful because it lets you generate different types of data
2045 with relationships to each other.
2046
2047 For example suppose we wanted to generate a list of lists of the same
2048 length:
2049
2050 >>> rectangle_lists = integers(min_value=0, max_value=10).flatmap(
2051 ... lambda n: lists(lists(integers(), min_size=n, max_size=n)))
2052 >>> find(rectangle_lists, lambda x: True)
2053 []
2054 >>> find(rectangle_lists, lambda x: len(x) >= 10)
2055 [[], [], [], [], [], [], [], [], [], []]
2056 >>> find(rectangle_lists, lambda t: len(t) >= 3 and len(t[0]) >= 3)
2057 [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
2058 >>> find(rectangle_lists, lambda t: sum(len(s) for s in t) >= 10)
2059 [[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
2060
2061 In this example we first choose a length for our tuples, then we build
2062 a strategy which generates lists containing lists precisely of that
2063 length. The finds show what simple examples for this look like.
2064
2065 Most of the time you probably don't want flatmap, but unlike filter and
2066 map which are just conveniences for things you could just do in your
2067 tests, flatmap allows genuinely new data generation that you wouldn't
2068 otherwise be able to easily do.
2069
2070 (If you know Haskell: Yes, this is more or less a monadic bind. If you
2071 don't know Haskell, ignore everything in these parentheses. You do not
2072 need to understand anything about monads to use this, or anything else
2073 in Hypothesis).
2074
2075 Recursive data
2076 Sometimes the data you want to generate has a recursive definition.
2077 e.g. if you wanted to generate JSON data, valid JSON is:
2078
2079 1. Any float, any boolean, any unicode string.
2080
2081 2. Any list of valid JSON data
2082
2083 3. Any dictionary mapping unicode strings to valid JSON data.
2084
2085 The problem is that you cannot call a strategy recursively and expect
2086 it to not just blow up and eat all your memory. The other problem here
2087 is that not all unicode strings display consistently on different
2088 machines, so we'll restrict them in our doctest.
2089
2090 The way Hypothesis handles this is with the recursive() function which
2091 you pass in a base case and a function that, given a strategy for your
2092 data type, returns a new strategy for it. So for example:
2093
2094 >>> from string import printable; from pprint import pprint
2095 >>> json = recursive(none() | booleans() | floats() | text(printable),
2096 ... lambda children: lists(children, 1) | dictionaries(text(printable), children, min_size=1))
2097 >>> pprint(json.example())
2098 [[1.175494351e-38, ']', 1.9, True, False, '.M}Xl', ''], True]
2099 >>> pprint(json.example())
2100 {'de(l': None,
2101 'nK': {'(Rt)': None,
2102 '+hoZh1YU]gy8': True,
2103 '8z]EIFA06^l`i^': 'LFE{Q',
2104 '9,': 'l{cA=/'}}
2105
2106 That is, we start with our leaf data and then we augment it by allowing
2107 lists and dictionaries of anything we can generate as JSON data.
2108
2109 The size control of this works by limiting the maximum number of values
2110 that can be drawn from the base strategy. So for example if we wanted
2111 to only generate really small JSON we could do this as:
2112
2113 >>> small_lists = recursive(booleans(), lists, max_leaves=5)
2114 >>> small_lists.example()
2115 True
2116 >>> small_lists.example()
2117 [False]
2118
2119 Composite strategies
2120 The @composite decorator lets you combine other strategies in more or
2121 less arbitrary ways. It's probably the main thing you'll want to use
2122 for complicated custom strategies.
2123
2124 The composite decorator works by converting a function that returns one
2125 example into a function that returns a strategy that produces such
2126 examples - which you can pass to @given, modify with .map or .filter,
2127 and generally use like any other strategy.
2128
2129 It does this by giving you a special function draw as the first argu‐
2130 ment, which can be used just like the corresponding method of the
2131 data() strategy within a test. In fact, the implementation is almost
2132 the same - but defining a strategy with @composite makes code reuse
2133 easier, and usually improves the display of failing examples.
2134
2135 For example, the following gives you a list and an index into it:
2136
2137 >>> @composite
2138 ... def list_and_index(draw, elements=integers()):
2139 ... xs = draw(lists(elements, min_size=1))
2140 ... i = draw(integers(min_value=0, max_value=len(xs) - 1))
2141 ... return (xs, i)
2142
2143 draw(s) is a function that should be thought of as returning s.exam‐
2144 ple(), except that the result is reproducible and will minimize cor‐
2145 rectly. The decorated function has the initial argument removed from
2146 the list, but will accept all the others in the expected order.
2147 Defaults are preserved.
2148
2149 >>> list_and_index()
2150 list_and_index()
2151 >>> list_and_index().example()
2152 ([15949, -35, 21764, 8167, 1607867656, -41, 104, 19, -90, 520116744169390387, 7107438879249457973], 0)
2153
2154 >>> list_and_index(booleans())
2155 list_and_index(elements=booleans())
2156 >>> list_and_index(booleans()).example()
2157 ([True, False], 0)
2158
2159 Note that the repr will work exactly like it does for all the built-in
2160 strategies: it will be a function that you can call to get the strategy
2161 in question, with values provided only if they do not match the
2162 defaults.
2163
2164 You can use assume inside composite functions:
2165
2166 @composite
2167 def distinct_strings_with_common_characters(draw):
2168 x = draw(text(), min_size=1)
2169 y = draw(text(alphabet=x))
2170 assume(x != y)
2171 return (x, y)
2172
2173 This works as assume normally would, filtering out any examples for
2174 which the passed in argument is falsey.
2175
2176 Drawing interactively in tests
2177 There is also the data() strategy, which gives you a means of using
2178 strategies interactively. Rather than having to specify everything up
2179 front in @given you can draw from strategies in the body of your test.
2180
2181 This is similar to @composite, but even more powerful as it allows you
2182 to mix test code with example generation. The downside of this power
2183 is that data() is incompatible with explicit @example(...)s - and the
2184 mixed code is often harder to debug when something goes wrong.
2185
2186 If you need values that are affected by previous draws but which don't
2187 depend on the execution of your test, stick to the simpler @composite.
2188
2189 @given(data())
2190 def test_draw_sequentially(data):
2191 x = data.draw(integers())
2192 y = data.draw(integers(min_value=x))
2193 assert x < y
2194
2195 If the test fails, each draw will be printed with the falsifying exam‐
2196 ple. e.g. the above is wrong (it has a boundary condition error), so
2197 will print:
2198
2199 Falsifying example: test_draw_sequentially(data=data(...))
2200 Draw 1: 0
2201 Draw 2: 0
2202
2203 As you can see, data drawn this way is simplified as usual.
2204
2205 Optionally, you can provide a label to identify values generated by
2206 each call to data.draw(). These labels can be used to identify values
2207 in the output of a falsifying example.
2208
2209 For instance:
2210
2211 @given(data())
2212 def test_draw_sequentially(data):
2213 x = data.draw(integers(), label='First number')
2214 y = data.draw(integers(min_value=x), label='Second number')
2215 assert x < y
2216
2217 will produce the output:
2218
2219 Falsifying example: test_draw_sequentially(data=data(...))
2220 Draw 1 (First number): 0
2221 Draw 2 (Second number): 0
2222
2224 Hypothesis has minimal dependencies (just attrs), to maximise compati‐
2225 bility and make installing Hypothesis as easy as possible.
2226
2227 Our integrations with specific packages are therefore provided by extra
2228 modules that need their individual dependencies installed in order to
2229 work. You can install these dependencies using the setuptools extra
2230 feature as e.g. pip install hypothesis[django]. This will check
2231 installation of compatible versions.
2232
2233 You can also just install hypothesis into a project using them, ignore
2234 the version constraints, and hope for the best.
2235
2236 In general "Which version is Hypothesis compatible with?" is a hard
2237 question to answer and even harder to regularly test. Hypothesis is
2238 always tested against the latest compatible version and each package
2239 will note the expected compatibility range. If you run into a bug with
2240 any of these please specify the dependency version.
2241
2242 There are separate pages for django and numpy.
2243
2244 hypothesis[pytz]
2245 This module provides pytz timezones.
2246
2247 You can use this strategy to make hypothesis.strategies.datetimes() and
2248 hypothesis.strategies.times() produce timezone-aware values.
2249
2250 hypothesis.extra.pytz.timezones()
2251 Any timezone in the Olsen database, as a pytz tzinfo object.
2252
2253 This strategy minimises to UTC, or the smallest possible fixed
2254 offset, and is designed for use with hypothesis.strategies.date‐
2255 times().
2256
2257 hypothesis[dateutil]
2258 This module provides dateutil timezones.
2259
2260 You can use this strategy to make hypothesis.strategies.datetimes() and
2261 hypothesis.strategies.times() produce timezone-aware values.
2262
2263 hypothesis.extra.dateutil.timezones()
2264 Any timezone in dateutil.
2265
2266 This strategy minimises to UTC, or the timezone with the small‐
2267 est offset from UTC as of 2000-01-01, and is designed for use
2268 with datetimes().
2269
2270 Note that the timezones generated by the strategy may vary
2271 depending on the configuration of your machine. See the dateutil
2272 documentation for more information.
2273
2275 Hypothesis offers a number of features specific for Django testing,
2276 available in the hypothesis[django] extra. This is tested against each
2277 supported series with mainstream or extended support - if you're still
2278 getting security patches, you can test with Hypothesis.
2279
2280 class hypothesis.extra.django.TestCase
2281
2282 Using it is quite straightforward: All you need to do is subclass
2283 hypothesis.extra.django.TestCase or
2284 hypothesis.extra.django.TransactionTestCase and you can use @given as
2285 normal, and the transactions will be per example rather than per test
2286 function as they would be if you used @given with a normal django test
2287 suite (this is important because your test function will be called mul‐
2288 tiple times and you don't want them to interfere with each other). Test
2289 cases on these classes that do not use @given will be run as normal.
2290
2291 class hypothesis.extra.django.TransactionTestCase
2292
2293 We recommend avoiding TransactionTestCase unless you really have to run
2294 each test case in a database transaction. Because Hypothesis runs this
2295 in a loop, the performance problems it normally has are significantly
2296 exacerbated and your tests will be really slow. If you are using
2297 TransactionTestCase, you may need to use @settings(sup‐
2298 press_health_check=[HealthCheck.too_slow]) to avoid errors due to slow
2299 example generation.
2300
2301 Having set up a test class, you can now pass @given a strategy for
2302 Django models:
2303
2304 For example, using the trivial django project we have for testing:
2305
2306 >>> from hypothesis.extra.django import from_model
2307 >>> from toystore.models import Customer
2308 >>> c = from_model(Customer).example()
2309 >>> c
2310 <Customer: Customer object>
2311 >>> c.email
2312 'jaime.urbina@gmail.com'
2313 >>> c.name
2314 '\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
2315 >>> c.age
2316 -873375803
2317
2318 Hypothesis has just created this with whatever the relevant type of
2319 data is.
2320
2321 Obviously the customer's age is implausible, which is only possible
2322 because we have not used (eg) MinValueValidator to set the valid range
2323 for this field (or used a PositiveSmallIntegerField, which would only
2324 need a maximum value validator).
2325
2326 If you do have validators attached, Hypothesis will only generate exam‐
2327 ples that pass validation. Sometimes that will mean that we fail a
2328 HealthCheck because of the filtering, so let's explicitly pass a strat‐
2329 egy to skip validation at the strategy level:
2330
2331 NOTE:
2332 Inference from validators will be much more powerful when issue
2333 #1116 is implemented, but there will always be some edge cases that
2334 require you to pass an explicit strategy.
2335
2336 >>> from hypothesis.strategies import integers
2337 >>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
2338 >>> c
2339 <Customer: Customer object>
2340 >>> c.age
2341 5
2342
2343 Tips and tricks
2344 Custom field types
2345 If you have a custom Django field type you can register it with Hypoth‐
2346 esis's model deriving functionality by registering a default strategy
2347 for it:
2348
2349 >>> from toystore.models import CustomishField, Customish
2350 >>> from_model(Customish).example()
2351 hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
2352 customish for model Customish
2353 >>> from hypothesis.extra.django import register_field_strategy
2354 >>> from hypothesis.strategies import just
2355 >>> register_field_strategy(CustomishField, just("hi"))
2356 >>> x = from_model(Customish).example()
2357 >>> x.customish
2358 'hi'
2359
2360 Note that this mapping is on exact type. Subtypes will not inherit it.
2361
2362 Generating child models
2363 For the moment there's no explicit support in hypothesis-django for
2364 generating dependent models. i.e. a Company model will generate no
2365 Shops. However if you want to generate some dependent models as well,
2366 you can emulate this by using the flatmap function as follows:
2367
2368 from hypothesis.strategies import lists, just
2369
2370 def generate_with_shops(company):
2371 return lists(from_model(Shop, company=just(company))).map(lambda _: company)
2372
2373 company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)
2374
2375 Lets unpack what this is doing:
2376
2377 The way flatmap works is that we draw a value from the original strat‐
2378 egy, then apply a function to it which gives us a new strategy. We then
2379 draw a value from that strategy. So in this case we're first drawing a
2380 company, and then we're drawing a list of shops belonging to that com‐
2381 pany: The just strategy is a strategy such that drawing it always pro‐
2382 duces the individual value, so from_model(Shop, company=just(company))
2383 is a strategy that generates a Shop belonging to the original company.
2384
2385 So the following code would give us a list of shops all belonging to
2386 the same company:
2387
2388 from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))
2389
2390 The only difference from this and the above is that we want the com‐
2391 pany, not the shops. This is where the inner map comes in. We build the
2392 list of shops and then throw it away, instead returning the company we
2393 started for. This works because the models that Hypothesis generates
2394 are saved in the database, so we're essentially running the inner
2395 strategy purely for the side effect of creating those children in the
2396 database.
2397
2398 Generating primary key values
2399 If your model includes a custom primary key that you want to generate
2400 using a strategy (rather than a default auto-increment primary key)
2401 then Hypothesis has to deal with the possibility of a duplicate primary
2402 key.
2403
2404 If a model strategy generates a value for the primary key field,
2405 Hypothesis will create the model instance with update_or_create(),
2406 overwriting any existing instance in the database for this test case
2407 with the same primary key.
2408
2409 On the subject of MultiValueField
2410 Django forms feature the MultiValueField which allows for several
2411 fields to be combined under a single named field, the default example
2412 of this is the SplitDateTimeField.
2413
2414 class CustomerForm(forms.Form):
2415 name = forms.CharField()
2416 birth_date_time = forms.SplitDateTimeField()
2417
2418 from_form supports MultiValueField subclasses directly, however if you
2419 want to define your own strategy be forewarned that Django binds data
2420 for a MultiValueField in a peculiar way. Specifically each sub-field is
2421 expected to have its own entry in data addressed by the field name
2422 (e.g. birth_date_time) and the index of the sub-field within the Multi‐
2423 ValueField, so form data for the example above might look like this:
2424
2425 {
2426 'name': 'Samuel John',
2427 'birth_date_time_0': '2018-05-19', # the date, as the first sub-field
2428 'birth_date_time_1': '15:18:00' # the time, as the second sub-field
2429 }
2430
2431 Thus, if you want to define your own strategies for such a field you
2432 must address your sub-fields appropriately:
2433
2434 from_form(CustomerForm, birth_date_time_0=just('2018-05-19'))
2435
2437 numpy
2438 Hypothesis offers a number of strategies for NumPy testing, available
2439 in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy
2440 package.
2441
2442 The centerpiece is the arrays() strategy, which generates arrays with
2443 any dtype, shape, and contents you can specify or give a strategy for.
2444 To make this as useful as possible, strategies are provided to generate
2445 array shapes and generate all kinds of fixed-size or compound dtypes.
2446
2447 hypothesis.extra.numpy.from_dtype(dtype)
2448 Creates a strategy which can generate any value of the given
2449 dtype.
2450
2451 hypothesis.extra.numpy.arrays(dtype, shape, elements=None, fill=None,
2452 unique=False)
2453 Returns a strategy for generating numpy:numpy.ndarrays.
2454
2455 · dtype may be any valid input to dtype (this includes dtype
2456 objects), or a strategy that generates such values.
2457
2458 · shape may be an integer >= 0, a tuple of such integers, or a
2459 strategy that generates such values.
2460
2461 · elements is a strategy for generating values to put in the
2462 array. If it is None a suitable value will be inferred based
2463 on the dtype, which may give any legal value (including eg NaN
2464 for floats). If you have more specific requirements, you
2465 should supply your own elements strategy.
2466
2467 · fill is a strategy that may be used to generate a single back‐
2468 ground value for the array. If None, a suitable default will
2469 be inferred based on the other arguments. If set to nothing()
2470 then filling behaviour will be disabled entirely and every
2471 element will be generated independently.
2472
2473 · unique specifies if the elements of the array should all be
2474 distinct from one another. Note that in this case multiple NaN
2475 values may still be allowed. If fill is also set, the only
2476 valid values for it to return are NaN values (anything for
2477 which numpy:numpy.isnan returns True. So e.g. for complex num‐
2478 bers (nan+1j) is also a valid fill). Note that if unique is
2479 set to True the generated values must be hashable.
2480
2481 Arrays of specified dtype and shape are generated for example
2482 like this:
2483
2484 >>> import numpy as np
2485 >>> arrays(np.int8, (2, 3)).example()
2486 array([[-8, 6, 3],
2487 [-6, 4, 6]], dtype=int8)
2488
2489 · See What you can generate and how.
2490
2491 >>> import numpy as np
2492 >>> from hypothesis.strategies import floats
2493 >>> arrays(np.float, 3, elements=floats(0, 1)).example()
2494 array([ 0.88974794, 0.77387938, 0.1977879 ])
2495
2496 Array values are generated in two parts:
2497
2498 1. Some subset of the coordinates of the array are populated
2499 with a value drawn from the elements strategy (or its
2500 inferred form).
2501
2502 2. If any coordinates were not assigned in the previous step, a
2503 single value is drawn from the fill strategy and is assigned
2504 to all remaining places.
2505
2506 You can set fill to nothing() if you want to disable this behav‐
2507 iour and draw a value for every element.
2508
2509 If fill is set to None then it will attempt to infer the correct
2510 behaviour automatically: If unique is True, no filling will
2511 occur by default. Otherwise, if it looks safe to reuse the val‐
2512 ues of elements across multiple coordinates (this will be the
2513 case for any inferred strategy, and for most of the builtins,
2514 but is not the case for mutable values or strategies built with
2515 flatmap, map, composite, etc) then it will use the elements
2516 strategy as the fill, else it will default to having no fill.
2517
2518 Having a fill helps Hypothesis craft high quality examples, but
2519 its main importance is when the array generated is large:
2520 Hypothesis is primarily designed around testing small examples.
2521 If you have arrays with hundreds or more elements, having a fill
2522 value is essential if you want your tests to run in reasonable
2523 time.
2524
2525 hypothesis.extra.numpy.array_shapes(min_dims=1, max_dims=None,
2526 min_side=1, max_side=None)
2527 Return a strategy for array shapes (tuples of int >= 1).
2528
2529 hypothesis.extra.numpy.scalar_dtypes()
2530 Return a strategy that can return any non-flexible scalar dtype.
2531
2532 hypothesis.extra.numpy.unsigned_integer_dtypes(endianness='?',
2533 sizes=(8, 16, 32, 64))
2534 Return a strategy for unsigned integer dtypes.
2535
2536 endianness may be < for little-endian, > for big-endian, = for
2537 native byte order, or ? to allow either byte order. This argu‐
2538 ment only applies to dtypes of more than one byte.
2539
2540 sizes must be a collection of integer sizes in bits. The
2541 default (8, 16, 32, 64) covers the full range of sizes.
2542
2543 hypothesis.extra.numpy.integer_dtypes(endianness='?', sizes=(8, 16, 32,
2544 64))
2545 Return a strategy for signed integer dtypes.
2546
2547 endianness and sizes are treated as for
2548 unsigned_integer_dtypes().
2549
2550 hypothesis.extra.numpy.floating_dtypes(endianness='?', sizes=(16, 32,
2551 64))
2552 Return a strategy for floating-point dtypes.
2553
2554 sizes is the size in bits of floating-point number. Some
2555 machines support 96- or 128-bit floats, but these are not gener‐
2556 ated by default.
2557
2558 Larger floats (96 and 128 bit real parts) are not supported on
2559 all platforms and therefore disabled by default. To generate
2560 these dtypes, include these values in the sizes argument.
2561
2562 hypothesis.extra.numpy.complex_number_dtypes(endianness='?', sizes=(64,
2563 128))
2564 Return a strategy for complex-number dtypes.
2565
2566 sizes is the total size in bits of a complex number, which con‐
2567 sists of two floats. Complex halfs (a 16-bit real part) are not
2568 supported by numpy and will not be generated by this strategy.
2569
2570 hypothesis.extra.numpy.datetime64_dtypes(max_period='Y',
2571 min_period='ns', endianness='?')
2572 Return a strategy for datetime64 dtypes, with various precisions
2573 from year to attosecond.
2574
2575 hypothesis.extra.numpy.timedelta64_dtypes(max_period='Y',
2576 min_period='ns', endianness='?')
2577 Return a strategy for timedelta64 dtypes, with various preci‐
2578 sions from year to attosecond.
2579
2580 hypothesis.extra.numpy.byte_string_dtypes(endianness='?', min_len=0,
2581 max_len=16)
2582 Return a strategy for generating bytestring dtypes, of various
2583 lengths and byteorder.
2584
2585 hypothesis.extra.numpy.unicode_string_dtypes(endianness='?', min_len=0,
2586 max_len=16)
2587 Return a strategy for generating unicode string dtypes, of vari‐
2588 ous lengths and byteorder.
2589
2590 hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(),
2591 min_size=1, max_size=5, allow_subarrays=False)
2592 Return a strategy for generating array (compound) dtypes, with
2593 members drawn from the given subtype strategy.
2594
2595 hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(),
2596 max_leaves=10, max_itemsize=None)
2597 Return the most-general dtype strategy.
2598
2599 Elements drawn from this strategy may be simple (from the sub‐
2600 type_strategy), or several such values drawn from array_dtypes()
2601 with allow_subarrays=True. Subdtypes in an array dtype may be
2602 nested to any depth, subject to the max_leaves argument.
2603
2604 hypothesis.extra.numpy.valid_tuple_axes(ndim, min_size=0,
2605 max_size=None)
2606 Return a strategy for generating permissible tuple-values for
2607 the axis argument for a numpy sequential function (e.g.
2608 numpy:numpy.sum()), given an array of the specified dimensional‐
2609 ity.
2610
2611 All tuples will have an length >= min_size and <= max_size. The
2612 default value for max_size is ndim.
2613
2614 Examples from this strategy shrink towards an empty tuple, which
2615 render most sequential functions as no-ops.
2616
2617 The following are some examples drawn from this strategy.
2618
2619 >>> [valid_tuple_axes(3).example() for i in range(4)]
2620 [(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]
2621
2622 valid_tuple_axes can be joined with other strategies to generate
2623 any type of valid axis object, i.e. integers, tuples, and None:
2624
2625 any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)
2626
2627 hypothesis.extra.numpy.broadcastable_shapes(shape, min_dims=0,
2628 max_dims=None, min_side=1, max_side=None)
2629 Return a strategy for generating shapes that are broadcast-com‐
2630 patible with the provided shape.
2631
2632 Examples from this strategy shrink towards a shape with length
2633 min_dims. The size of an aligned dimension shrinks towards
2634 being a singleton. The size of an unaligned dimension shrink
2635 towards min_side.
2636
2637 · shape a tuple of integers
2638
2639 · min_dims The smallest length that the generated shape can pos‐
2640 sess.
2641
2642 · max_dims The largest length that the generated shape can pos‐
2643 sess. shape can possess. Cannot exceed 32. The default-value
2644 for max_dims is 2 + max(len(shape), min_dims).
2645
2646 · min_side The smallest size that an unaligned dimension can
2647 possess.
2648
2649 · max_side The largest size that an unaligned dimension can pos‐
2650 sess. The default value is 2 +
2651 'size-of-largest-aligned-dimension'.
2652
2653 The following are some examples drawn from this strategy.
2654
2655 >>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
2656 [(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]
2657
2658 pandas
2659 Hypothesis provides strategies for several of the core pandas data
2660 types: pandas.Index, pandas.Series and pandas.DataFrame.
2661
2662 The general approach taken by the pandas module is that there are mul‐
2663 tiple strategies for generating indexes, and all of the other strate‐
2664 gies take the number of entries they contain from their index strategy
2665 (with sensible defaults). So e.g. a Series is specified by specifying
2666 its numpy.dtype (and/or a strategy for generating elements for it).
2667
2668 hypothesis.extra.pandas.indexes(elements=None, dtype=None, min_size=0,
2669 max_size=None, unique=True)
2670 Provides a strategy for producing a pandas.Index.
2671
2672 Arguments:
2673
2674 · elements is a strategy which will be used to generate the
2675 individual values of the index. If None, it will be inferred
2676 from the dtype. Note: even if the elements strategy produces
2677 tuples, the generated value will not be a MultiIndex, but
2678 instead be a normal index whose elements are tuples.
2679
2680 · dtype is the dtype of the resulting index. If None, it will be
2681 inferred from the elements strategy. At least one of dtype or
2682 elements must be provided.
2683
2684 · min_size is the minimum number of elements in the index.
2685
2686 · max_size is the maximum number of elements in the index. If
2687 None then it will default to a suitable small size. If you
2688 want larger indexes you should pass a max_size explicitly.
2689
2690 · unique specifies whether all of the elements in the resulting
2691 index should be distinct.
2692
2693 hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)
2694 Provides a strategy which generates an Index whose values are 0,
2695 1, ..., n for some n.
2696
2697 Arguments:
2698
2699 · min_size is the smallest number of elements the index can
2700 have.
2701
2702 · max_size is the largest number of elements the index can have.
2703 If None it will default to some suitable value based on
2704 min_size.
2705
2706 hypothesis.extra.pandas.series(elements=None, dtype=None, index=None,
2707 fill=None, unique=False)
2708 Provides a strategy for producing a pandas.Series.
2709
2710 Arguments:
2711
2712 · elements: a strategy that will be used to generate the indi‐
2713 vidual values in the series. If None, we will attempt to infer
2714 a suitable default from the dtype.
2715
2716 · dtype: the dtype of the resulting series and may be any value
2717 that can be passed to numpy.dtype. If None, will use pandas's
2718 standard behaviour to infer it from the type of the elements
2719 values. Note that if the type of values that comes out of your
2720 elements strategy varies, then so will the resulting dtype of
2721 the series.
2722
2723 · index: If not None, a strategy for generating indexes for the
2724 resulting Series. This can generate either pandas.Index
2725 objects or any sequence of values (which will be passed to the
2726 Index constructor).
2727
2728 You will probably find it most convenient to use the indexes()
2729 or range_indexes() function to produce values for this argu‐
2730 ment.
2731
2732 Usage:
2733
2734 >>> series(dtype=int).example()
2735 0 -2001747478
2736 1 1153062837
2737
2738 class hypothesis.extra.pandas.column(name=None, elements=None,
2739 dtype=None, fill=None, unique=False)
2740 Data object for describing a column in a DataFrame.
2741
2742 Arguments:
2743
2744 · name: the column name, or None to default to the column posi‐
2745 tion. Must be hashable, but can otherwise be any value sup‐
2746 ported as a pandas column name.
2747
2748 · elements: the strategy for generating values in this column,
2749 or None to infer it from the dtype.
2750
2751 · dtype: the dtype of the column, or None to infer it from the
2752 element strategy. At least one of dtype or elements must be
2753 provided.
2754
2755 · fill: A default value for elements of the column. See arrays()
2756 for a full explanation.
2757
2758 · unique: If all values in this column should be distinct.
2759
2760 hypothesis.extra.pandas.columns(names_or_number, dtype=None, ele‐
2761 ments=None, fill=None, unique=False)
2762 A convenience function for producing a list of column objects of
2763 the same general shape.
2764
2765 The names_or_number argument is either a sequence of values, the
2766 elements of which will be used as the name for individual column
2767 objects, or a number, in which case that many unnamed columns
2768 will be created. All other arguments are passed through verbatim
2769 to create the columns.
2770
2771 hypothesis.extra.pandas.data_frames(columns=None, rows=None,
2772 index=None)
2773 Provides a strategy for producing a pandas.DataFrame.
2774
2775 Arguments:
2776
2777 · columns: An iterable of column objects describing the shape of
2778 the generated DataFrame.
2779
2780 · rows: A strategy for generating a row object. Should generate
2781 either dicts mapping column names to values or a sequence map‐
2782 ping column position to the value in that position (note that
2783 unlike the pandas.DataFrame constructor, single values are not
2784 allowed here. Passing e.g. an integer is an error, even if
2785 there is only one column).
2786
2787 At least one of rows and columns must be provided. If both are
2788 provided then the generated rows will be validated against the
2789 columns and an error will be raised if they don't match.
2790
2791 Caveats on using rows:
2792
2793 · In general you should prefer using columns to rows, and only
2794 use rows if the columns interface is insufficiently flexible
2795 to describe what you need - you will get better performance
2796 and example quality that way.
2797
2798 · If you provide rows and not columns, then the shape and
2799 dtype of the resulting DataFrame may vary. e.g. if you have
2800 a mix of int and float in the values for one column in your
2801 row entries, the column will sometimes have an integral
2802 dtype and sometimes a float.
2803
2804 · index: If not None, a strategy for generating indexes for the
2805 resulting DataFrame. This can generate either pandas.Index
2806 objects or any sequence of values (which will be passed to the
2807 Index constructor).
2808
2809 You will probably find it most convenient to use the indexes()
2810 or range_indexes() function to produce values for this argu‐
2811 ment.
2812
2813 Usage:
2814
2815 The expected usage pattern is that you use column and columns()
2816 to specify a fixed shape of the DataFrame you want as follows.
2817 For example the following gives a two column data frame:
2818
2819 >>> from hypothesis.extra.pandas import column, data_frames
2820 >>> data_frames([
2821 ... column('A', dtype=int), column('B', dtype=float)]).example()
2822 A B
2823 0 2021915903 1.793898e+232
2824 1 1146643993 inf
2825 2 -2096165693 1.000000e+07
2826
2827 If you want the values in different columns to interact in some
2828 way you can use the rows argument. For example the following
2829 gives a two column DataFrame where the value in the first column
2830 is always at most the value in the second:
2831
2832 >>> from hypothesis.extra.pandas import column, data_frames
2833 >>> import hypothesis.strategies as st
2834 >>> data_frames(
2835 ... rows=st.tuples(st.floats(allow_nan=False),
2836 ... st.floats(allow_nan=False)).map(sorted)
2837 ... ).example()
2838 0 1
2839 0 -3.402823e+38 9.007199e+15
2840 1 -1.562796e-298 5.000000e-01
2841
2842 You can also combine the two:
2843
2844 >>> from hypothesis.extra.pandas import columns, data_frames
2845 >>> import hypothesis.strategies as st
2846 >>> data_frames(
2847 ... columns=columns(["lo", "hi"], dtype=float),
2848 ... rows=st.tuples(st.floats(allow_nan=False),
2849 ... st.floats(allow_nan=False)).map(sorted)
2850 ... ).example()
2851 lo hi
2852 0 9.314723e-49 4.353037e+45
2853 1 -9.999900e-01 1.000000e+07
2854 2 -2.152861e+134 -1.069317e-73
2855
2856 (Note that the column dtype must still be specified and will not
2857 be inferred from the rows. This restriction may be lifted in
2858 future).
2859
2860 Combining rows and columns has the following behaviour:
2861
2862 · The column names and dtypes will be used.
2863
2864 · If the column is required to be unique, this will be enforced.
2865
2866 · Any values missing from the generated rows will be provided
2867 using the column's fill.
2868
2869 · Any values in the row not present in the column specification
2870 (if dicts are passed, if there are keys with no corresponding
2871 column name, if sequences are passed if there are too many
2872 items) will result in InvalidArgument being raised.
2873
2874 Supported Versions
2875 There is quite a lot of variation between pandas versions. We only com‐
2876 mit to supporting the latest version of pandas, but older minor ver‐
2877 sions are supported on a "best effort" basis. Hypothesis is currently
2878 tested against and confirmed working with Pandas 0.19, 0.20, 0.21,
2879 0.22, and 0.23.
2880
2881 Releases that are not the latest patch release of their minor version
2882 are not tested or officially supported, but will probably also work
2883 unless you hit a pandas bug.
2884
2886 Hypothesis tries to detect common mistakes and things that will cause
2887 difficulty at run time in the form of a number of 'health checks'.
2888
2889 These include detecting and warning about:
2890
2891 · Strategies with very slow data generation
2892
2893 · Strategies which filter out too much
2894
2895 · Recursive strategies which branch too much
2896
2897 · Tests that are unlikely to complete in a reasonable amount of time.
2898
2899 If any of these scenarios are detected, Hypothesis will emit a warning
2900 about them.
2901
2902 The general goal of these health checks is to warn you about things
2903 that you are doing that might appear to work but will either cause
2904 Hypothesis to not work correctly or to perform badly.
2905
2906 To selectively disable health checks, use the suppress_health_check
2907 setting. The argument for this parameter is a list with elements drawn
2908 from any of the class-level attributes of the HealthCheck class. Using
2909 a value of HealthCheck.all() will disable all health checks.
2910
2911 class hypothesis.HealthCheck
2912 Arguments for suppress_health_check.
2913
2914 Each member of this enum is a type of health check to suppress.
2915
2916 all = <bound method HealthCheck.all of <enum 'HealthCheck'>>
2917
2918 data_too_large = 1
2919 Check for when the typical size of the examples you are
2920 generating exceeds the maximum allowed size too often.
2921
2922 filter_too_much = 2
2923 Check for when the test is filtering out too many exam‐
2924 ples, either through use of assume() or filter(), or
2925 occasionally for Hypothesis internal reasons.
2926
2927 too_slow = 3
2928 Check for when your data generation is extremely slow and
2929 likely to hurt testing.
2930
2931 return_value = 5
2932 Checks if your tests return a non-None value (which will
2933 be ignored and is unlikely to do what you want).
2934
2935 hung_test = 6
2936 This health check is deprecated and no longer has any
2937 effect. You can use the max_examples and deadline set‐
2938 tings together to cap the total runtime of your tests,
2939 rather than the previous fixed limit.
2940
2941 large_base_example = 7
2942 Checks if the natural example to shrink towards is very
2943 large.
2944
2945 not_a_test_method = 8
2946 Checks if @given has been applied to a method of
2947 python:unittest.TestCase.
2948
2949 Deprecations
2950 We also use a range of custom exception and warning types, so you can
2951 see exactly where an error came from - or turn only our warnings into
2952 errors.
2953
2954 class hypothesis.errors.HypothesisDeprecationWarning
2955 A deprecation warning issued by Hypothesis.
2956
2957 Actually inherits from FutureWarning, because DeprecationWarning
2958 is hidden by the default warnings filter.
2959
2960 You can configure the Python python:warnings to handle these
2961 warnings differently to others, either turning them into errors
2962 or suppressing them entirely. Obviously we would prefer the
2963 former!
2964
2965 Deprecated features will be continue to emit warnings for at least six
2966 months, and then be removed in the following major release. Note how‐
2967 ever that not all warnings are subject to this grace period; sometimes
2968 we strengthen validation by adding a warning and these may become
2969 errors immediately at a major release.
2970
2972 When Hypothesis finds a bug it stores enough information in its data‐
2973 base to reproduce it. This enables you to have a classic testing work‐
2974 flow of find a bug, fix a bug, and be confident that this is actually
2975 doing the right thing because Hypothesis will start by retrying the
2976 examples that broke things last time.
2977
2978 Limitations
2979 The database is best thought of as a cache that you never need to
2980 invalidate: Information may be lost when you upgrade a Hypothesis ver‐
2981 sion or change your test, so you shouldn't rely on it for correctness -
2982 if there's an example you want to ensure occurs each time then there's
2983 a feature for including them in your source code - but it helps the
2984 development workflow considerably by making sure that the examples
2985 you've just found are reproduced.
2986
2987 The database also records examples that exercise less-used parts of
2988 your code, so the database may update even when no failing examples
2989 were found.
2990
2991 File locations
2992 The default storage format is as a fairly opaque directory structure.
2993 Each test corresponds to a directory, and each example to a file within
2994 that directory. The standard location for it is .hypothesis/examples
2995 in your current working directory. You can override this by setting the
2996 database setting.
2997
2998 If you have not configured a database and the default location is unus‐
2999 able (e.g. because you do not have read/write permission), Hypothesis
3000 will issue a warning and then fall back to an in-memory database.
3001
3002 Upgrading Hypothesis and changing your tests
3003 The design of the Hypothesis database is such that you can put arbi‐
3004 trary data in the database and not get wrong behaviour. When you
3005 upgrade Hypothesis, old data might be invalidated, but this should hap‐
3006 pen transparently. It can never be the case that e.g. changing the
3007 strategy that generates an argument gives you data from the old strat‐
3008 egy.
3009
3010 Sharing your example database
3011 NOTE:
3012 If specific examples are important for correctness you should use
3013 the @example decorator, as the example database may discard entries
3014 due to changes in your code or dependencies. For most users, we
3015 therefore recommend using the example database locally and possibly
3016 persisting it between CI builds, but not tracking it under version
3017 control.
3018
3019 The examples database can be shared simply by checking the directory
3020 into version control, for example with the following .gitignore:
3021
3022 # Ignore files cached by Hypothesis...
3023 .hypothesis/*
3024 # except for the examples directory
3025 !.hypothesis/examples/
3026
3027 Like everything under .hypothesis/, the examples directory will be
3028 transparently created on demand. Unlike the other subdirectories,
3029 examples/ is designed to handle merges, deletes, etc if you just add
3030 the directory into git, mercurial, or any similar version control sys‐
3031 tem.
3032
3034 With @given, your tests are still something that you mostly write your‐
3035 self, with Hypothesis providing some data. With Hypothesis's stateful
3036 testing, Hypothesis instead tries to generate not just data but entire
3037 tests. You specify a number of primitive actions that can be combined
3038 together, and then Hypothesis will try to find sequences of those
3039 actions that result in a failure.
3040
3041 NOTE:
3042 This style of testing is often called model-based testing, but in
3043 Hypothesis is called stateful testing (mostly for historical reasons
3044 - the original implementation of this idea in Hypothesis was more
3045 closely based on ScalaCheck's stateful testing where the name is
3046 more apt). Both of these names are somewhat misleading: You don't
3047 really need any sort of formal model of your code to use this, and
3048 it can be just as useful for pure APIs that don't involve any state
3049 as it is for stateful ones.
3050
3051 It's perhaps best to not take the name of this sort of testing too
3052 seriously. Regardless of what you call it, it is a powerful form of
3053 testing which is useful for most non-trivial APIs.
3054
3055 Hypothesis has two stateful testing APIs: A high level one, providing
3056 what we call rule based state machines, and a low level one, providing
3057 what we call generic state machines.
3058
3059 You probably want to use the rule based state machines - they provide a
3060 high level API for describing the sort of actions you want to perform,
3061 based on a structured representation of actions. However the generic
3062 state machines are more flexible, and are particularly useful if you
3063 want the set of currently possible actions to depend primarily on
3064 external state.
3065
3066 You may not need state machines
3067 The basic idea of stateful testing is to make Hypothesis choose actions
3068 as well as values for your test, and state machines are a great declar‐
3069 ative way to do just that.
3070
3071 For simpler cases though, you might not need them at all - a standard
3072 test with @given might be enough, since you can use data() in branches
3073 or loops. In fact, that's how the state machine explorer works inter‐
3074 nally. For more complex workloads though, where a higher level API
3075 comes into it's own, keep reading!
3076
3077 Rule based state machines
3078 Rule based state machines are the ones you're most likely to want to
3079 use. They're significantly more user friendly and should be good
3080 enough for most things you'd want to do.
3081
3082 class hypothesis.stateful.RuleBasedStateMachine
3083 A RuleBasedStateMachine gives you a more structured way to
3084 define state machines.
3085
3086 The idea is that a state machine carries a bunch of types of
3087 data divided into Bundles, and has a set of rules which may read
3088 data from bundles (or just from normal strategies) and push data
3089 onto bundles. At any given point a random applicable rule will
3090 be executed.
3091
3092 A rule is very similar to a normal @given based test in that it takes
3093 values drawn from strategies and passes them to a user defined test
3094 function. The key difference is that where @given based tests must be
3095 independent, rules can be chained together - a single test run may
3096 involve multiple rule invocations, which may interact in various ways.
3097
3098 Rules can take normal strategies as arguments, or a specific kind of
3099 strategy called a Bundle. A Bundle is a named collection of generated
3100 values that can be reused by other operations in the test. They are
3101 populated with the results of rules, and may be used as arguments to
3102 rules, allowing data to flow from one rule to another, and rules to
3103 work on the results of previous computations or actions.
3104
3105 You can think of each value that gets added to any Bundle as being
3106 assigned to a new variable. Drawing a value from the bundle strategy
3107 means choosing one of the corresponding variables and using that value,
3108 and consumes() as a del statement for that variable. If you can
3109 replace use of Bundles with instance attributes of the class that is
3110 often simpler, but often Bundles are strictly more powerful.
3111
3112 The following rule based state machine example is a simplified version
3113 of a test for Hypothesis's example database implementation. An example
3114 database maps keys to sets of values, and in this test we compare one
3115 implementation of it to a simplified in memory model of its behaviour,
3116 which just stores the same values in a Python dict. The test then runs
3117 operations against both the real database and the in-memory representa‐
3118 tion of it and looks for discrepancies in their behaviour.
3119
3120 import shutil
3121 import tempfile
3122
3123 from collections import defaultdict
3124 import hypothesis.strategies as st
3125 from hypothesis.database import DirectoryBasedExampleDatabase
3126 from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule
3127
3128
3129 class DatabaseComparison(RuleBasedStateMachine):
3130 def __init__(self):
3131 super(DatabaseComparison, self).__init__()
3132 self.tempd = tempfile.mkdtemp()
3133 self.database = DirectoryBasedExampleDatabase(self.tempd)
3134 self.model = defaultdict(set)
3135
3136 keys = Bundle('keys')
3137 values = Bundle('values')
3138
3139 @rule(target=keys, k=st.binary())
3140 def add_key(self, k):
3141 return k
3142
3143 @rule(target=values, v=st.binary())
3144 def add_value(self, v):
3145 return v
3146
3147 @rule(k=keys, v=values)
3148 def save(self, k, v):
3149 self.model[k].add(v)
3150 self.database.save(k, v)
3151
3152 @rule(k=keys, v=values)
3153 def delete(self, k, v):
3154 self.model[k].discard(v)
3155 self.database.delete(k, v)
3156
3157 @rule(k=keys)
3158 def values_agree(self, k):
3159 assert set(self.database.fetch(k)) == self.model[k]
3160
3161 def teardown(self):
3162 shutil.rmtree(self.tempd)
3163
3164
3165 TestDBComparison = DatabaseComparison.TestCase
3166
3167 In this we declare two bundles - one for keys, and one for values. We
3168 have two trivial rules which just populate them with data (k and v),
3169 and three non-trivial rules: save saves a value under a key and delete
3170 removes a value from a key, in both cases also updating the model of
3171 what should be in the database. values_agree then checks that the con‐
3172 tents of the database agrees with the model for a particular key.
3173
3174 We can then integrate this into our test suite by getting a unittest
3175 TestCase from it:
3176
3177 TestTrees = DatabaseComparison.TestCase
3178
3179 # Or just run with pytest's unittest support
3180 if __name__ == '__main__':
3181 unittest.main()
3182
3183 This test currently passes, but if we comment out the line where we
3184 call self.model[k].discard(v), we would see the following output when
3185 run under pytest:
3186
3187 AssertionError: assert set() == {b''}
3188
3189 ------------ Hypothesis ------------
3190
3191 state = DatabaseComparison()
3192 var1 = state.add_key(k=b'')
3193 var2 = state.add_value(v=var1)
3194 state.save(k=var1, v=var2)
3195 state.delete(k=var1, v=var2)
3196 state.values_agree(k=var1)
3197 state.teardown()
3198
3199 Note how it's printed out a very short program that will demonstrate
3200 the problem. The output from a rule based state machine should gener‐
3201 ally be pretty close to Python code - if you have custom repr implemen‐
3202 tations that don't return valid Python then it might not be, but most
3203 of the time you should just be able to copy and paste the code into a
3204 test to reproduce it.
3205
3206 You can control the detailed behaviour with a settings object on the
3207 TestCase (this is a normal hypothesis settings object using the
3208 defaults at the time the TestCase class was first referenced). For
3209 example if you wanted to run fewer examples with larger programs you
3210 could change the settings to:
3211
3212 DatabaseComparison.TestCase.settings = settings(max_examples=50, stateful_step_count=100)
3213
3214 Which doubles the number of steps each program runs and halves the num‐
3215 ber of test cases that will be run.
3216
3217 Rules
3218 As said earlier, rules are the most common feature used in RuleBased‐
3219 StateMachine. They are defined by applying the rule() decorator on a
3220 function. Note that RuleBasedStateMachine must have at least one rule
3221 defined and that a single function cannot be used to define multiple
3222 rules (this to avoid having multiple rules doing the same things). Due
3223 to the stateful execution method, rules generally cannot take arguments
3224 from other sources such as fixtures or pytest.mark.parametrize - con‐
3225 sider providing them via a strategy such as sampled_from() instead.
3226
3227 hypothesis.stateful.rule(targets=(), target=None, **kwargs)
3228 Decorator for RuleBasedStateMachine. Any name present in target
3229 or targets will define where the end result of this function
3230 should go. If both are empty then the end result will be dis‐
3231 carded.
3232
3233 target must be a Bundle, or if the result should go to multiple
3234 bundles you can pass a tuple of them as the targets argument.
3235 It is invalid to use both arguments for a single rule. If the
3236 result should go to exactly one of several bundles, define a
3237 separate rule for each case.
3238
3239 kwargs then define the arguments that will be passed to the
3240 function invocation. If their value is a Bundle, or if it is
3241 consumes(b) where b is a Bundle, then values that have previ‐
3242 ously been produced for that bundle will be provided. If con‐
3243 sumes is used, the value will also be removed from the bundle.
3244
3245 Any other kwargs should be strategies and values from them will
3246 be provided.
3247
3248 hypothesis.stateful.consumes(bundle)
3249 When introducing a rule in a RuleBasedStateMachine, this func‐
3250 tion can be used to mark bundles from which each value used in a
3251 step with the given rule should be removed. This function
3252 returns a strategy object that can be manipulated and combined
3253 like any other.
3254
3255 For example, a rule declared with
3256
3257 @rule(value1=b1, value2=consumes(b2), value3=lists(con‐
3258 sumes(b3)))
3259
3260 will consume a value from Bundle b2 and several values from Bun‐
3261 dle b3 to populate value2 and value3 each time it is executed.
3262
3263 hypothesis.stateful.multiple(*args)
3264 This function can be used to pass multiple results to the tar‐
3265 get(s) of a rule. Just use return multiple(result1, result2,
3266 ...) in your rule.
3267
3268 It is also possible to use return multiple() with no arguments
3269 in order to end a rule without passing any result.
3270
3271 Initializes
3272 Initializes are a special case of rules that are guaranteed to be run
3273 at most once at the beginning of a run (i.e. before any normal rule is
3274 called). Note if multiple initialize rules are defined, they may be
3275 called in any order, and that order will vary from run to run.
3276
3277 Initializes are typically useful to populate bundles:
3278
3279 hypothesis.stateful.initialize(targets=(), target=None, **kwargs)
3280 Decorator for RuleBasedStateMachine.
3281
3282 An initialize decorator behaves like a rule, but the decorated
3283 method is called at most once in a run. All initialize decorated
3284 methods will be called before any rule decorated methods, in an
3285 arbitrary order.
3286
3287 import hypothesis.strategies as st
3288 from hypothesis.stateful import RuleBasedStateMachine, Bundle, rule, initialize
3289
3290 name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)
3291
3292 class NumberModifier(RuleBasedStateMachine):
3293
3294 folders = Bundle('folders')
3295 files = Bundle('files')
3296
3297 @initialize(target=folders)
3298 def init_folders(self):
3299 return '/'
3300
3301 @rule(target=folders, name=name_strategy)
3302 def create_folder(self, parent, name):
3303 return '%s/%s' % (parent, name)
3304
3305 @rule(target=files, name=name_strategy)
3306 def create_file(self, parent, name):
3307 return '%s/%s' % (parent, name)
3308
3309 Preconditions
3310 While it's possible to use assume() in RuleBasedStateMachine rules, if
3311 you use it in only a few rules you can quickly run into a situation
3312 where few or none of your rules pass their assumptions. Thus, Hypothe‐
3313 sis provides a precondition() decorator to avoid this problem. The
3314 precondition() decorator is used on rule-decorated functions, and must
3315 be given a function that returns True or False based on the RuleBased‐
3316 StateMachine instance.
3317
3318 hypothesis.stateful.precondition(precond)
3319 Decorator to apply a precondition for rules in a RuleBased‐
3320 StateMachine. Specifies a precondition for a rule to be consid‐
3321 ered as a valid step in the state machine. The given function
3322 will be called with the instance of RuleBasedStateMachine and
3323 should return True or False. Usually it will need to look at
3324 attributes on that instance.
3325
3326 For example:
3327
3328 class MyTestMachine(RuleBasedStateMachine):
3329 state = 1
3330
3331 @precondition(lambda self: self.state != 0)
3332 @rule(numerator=integers())
3333 def divide_with(self, numerator):
3334 self.state = numerator / self.state
3335
3336 This is better than using assume in your rule since more valid
3337 rules should be able to be run.
3338
3339 from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
3340
3341 class NumberModifier(RuleBasedStateMachine):
3342
3343 num = 0
3344
3345 @rule()
3346 def add_one(self):
3347 self.num += 1
3348
3349 @precondition(lambda self: self.num != 0)
3350 @rule()
3351 def divide_with_one(self):
3352 self.num = 1 / self.num
3353
3354 By using precondition() here instead of assume(), Hypothesis can filter
3355 the inapplicable rules before running them. This makes it much more
3356 likely that a useful sequence of steps will be generated.
3357
3358 Note that currently preconditions can't access bundles; if you need to
3359 use preconditions, you should store relevant data on the instance
3360 instead.
3361
3362 Invariants
3363 Often there are invariants that you want to ensure are met after every
3364 step in a process. It would be possible to add these as rules that are
3365 run, but they would be run zero or multiple times between other rules.
3366 Hypothesis provides a decorator that marks a function to be run after
3367 every step.
3368
3369 hypothesis.stateful.invariant()
3370 Decorator to apply an invariant for rules in a RuleBasedStateMa‐
3371 chine. The decorated function will be run after every rule and
3372 can raise an exception to indicate failed invariants.
3373
3374 For example:
3375
3376 class MyTestMachine(RuleBasedStateMachine):
3377 state = 1
3378
3379 @invariant()
3380 def is_nonzero(self):
3381 assert self.state != 0
3382
3383 from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
3384
3385 class NumberModifier(RuleBasedStateMachine):
3386
3387 num = 0
3388
3389 @rule()
3390 def add_two(self):
3391 self.num += 2
3392 if self.num > 50:
3393 self.num += 1
3394
3395 @invariant()
3396 def divide_with_one(self):
3397 assert self.num % 2 == 0
3398
3399 NumberTest = NumberModifier.TestCase
3400
3401 Invariants can also have precondition()s applied to them, in which case
3402 they will only be run if the precondition function returns true.
3403
3404 Note that currently invariants can't access bundles; if you need to use
3405 invariants, you should store relevant data on the instance instead.
3406
3407 Generic state machines
3408 The class GenericStateMachine is the underlying machinery of stateful
3409 testing in Hypothesis. Chances are you will want to use the rule based
3410 stateful testing for most things, but the generic state machine func‐
3411 tionality can be useful e.g. if you want to test things where the set
3412 of actions to be taken is more closely tied to the state of the system
3413 you are testing.
3414
3415 NOTE:
3416 GenericStateMachine is a very thin wrapper around @given and data().
3417 Consider using those primitives directly, as the resulting tests are
3418 usually easier to write, understand, and debug. GenericStateMachine
3419 only exists because it predates data(), and will be deprecated and
3420 removed in a future version (issue #1693).
3421
3422 class hypothesis.stateful.GenericStateMachine
3423 A GenericStateMachine is the basic entry point into Hypothesis's
3424 approach to stateful testing.
3425
3426 The intent is for it to be subclassed to provide state machine
3427 descriptions
3428
3429 The way this is used is that Hypothesis will repeatedly execute
3430 something that looks something like:
3431
3432 x = MyStatemachineSubclass()
3433 x.check_invariants()
3434 try:
3435 for _ in range(n_steps):
3436 x.execute_step(x.steps().example())
3437 x.check_invariants()
3438 finally:
3439 x.teardown()
3440
3441 And if this ever produces an error it will shrink it down to a
3442 small sequence of example choices demonstrating that.
3443
3444 steps()
3445 Return a SearchStrategy instance the defines the avail‐
3446 able next steps.
3447
3448 execute_step(step)
3449 Execute a step that has been previously drawn from
3450 self.steps()
3451
3452 teardown()
3453 Called after a run has finished executing to clean up any
3454 necessary state.
3455
3456 Does nothing by default.
3457
3458 check_invariants()
3459 Called after initializing and after executing each step.
3460
3461 For example, here we use stateful testing as a sort of link checker, to
3462 test hypothesis.works for broken links or links that use HTTP instead
3463 of HTTPS.
3464
3465 from hypothesis.stateful import GenericStateMachine
3466 import hypothesis.strategies as st
3467 from requests_html import HTMLSession
3468
3469
3470 class LinkChecker(GenericStateMachine):
3471 def __init__(self):
3472 super(LinkChecker, self).__init__()
3473 self.session = HTMLSession()
3474 self.result = None
3475
3476 def steps(self):
3477 if self.result is None:
3478 # Always start on the home page
3479 return st.just("https://hypothesis.works/")
3480 else:
3481 return st.sampled_from([
3482 l
3483 for l in self.result.html.absolute_links
3484 # Don't try to crawl to other people's sites
3485 if l.startswith("https://hypothesis.works") and
3486 # Avoid Cloudflare's bot protection. We are a bot but we don't
3487 # care about the info it's hiding.
3488 '/cdn-cgi/' not in l
3489 ])
3490
3491 def execute_step(self, step):
3492 self.result = self.session.get(step)
3493
3494 assert self.result.status_code == 200
3495
3496 for l in self.result.html.absolute_links:
3497 # All links should be HTTPS
3498 assert "https://hypothesis.works/" not in l
3499
3500
3501 TestLinks = LinkChecker.TestCase
3502
3503 Running this (at the time of writing this documentation) produced the
3504 following output:
3505
3506 AssertionError: assert 'https://hypothesis.works/' not in 'http://hypoth...test-fixtures/'
3507 'https://hypothesis.works/' is contained here:
3508 https://hypothesis.works/articles/hypothesis-pytest-fixtures/
3509 ? +++++++++++++++++++++++++
3510
3511 ------------ Hypothesis ------------
3512
3513 Step #1: 'https://hypothesis.works/'
3514 Step #2: 'https://hypothesis.works/articles/'
3515
3516 More fine grained control
3517 If you want to bypass the TestCase infrastructure you can invoke these
3518 manually. The stateful module exposes the function
3519 run_state_machine_as_test, which takes an arbitrary function returning
3520 a GenericStateMachine and an optional settings parameter and does the
3521 same as the class based runTest provided.
3522
3523 This is not recommended as it bypasses some important internal func‐
3524 tions, including reporting of statistics such as runtimes and event()
3525 calls. It was originally added to support custom __init__ methods, but
3526 you can now use initialize() rules instead.
3527
3529 Hypothesis does its level best to be compatible with everything you
3530 could possibly need it to be compatible with. Generally you should just
3531 try it and expect it to work. If it doesn't, you can be surprised and
3532 check this document for the details.
3533
3534 Python versions
3535 Hypothesis is supported and tested on CPython 2.7 and CPython 3.5+,
3536 i.e. all versisions of CPython with upstream support,
3537
3538 Hypothesis also supports the latest PyPy for both Python 2 (until 2020)
3539 and Python 3. Hypothesis does not currently work on Jython, though it
3540 probably could (issue #174). IronPython might work but hasn't been
3541 tested. 32-bit and narrow builds should work, though this is currently
3542 only tested on Windows.
3543
3544 In general Hypothesis does not officially support anything except the
3545 latest patch release of any version of Python it supports. Earlier
3546 releases should work and bugs in them will get fixed if reported, but
3547 they're not tested in CI and no guarantees are made.
3548
3549 Operating systems
3550 In theory Hypothesis should work anywhere that Python does. In practice
3551 it is only known to work and regularly tested on OS X, Windows and
3552 Linux, and you may experience issues running it elsewhere.
3553
3554 If you're using something else and it doesn't work, do get in touch and
3555 I'll try to help, but unless you can come up with a way for me to run a
3556 CI server on that operating system it probably won't stay fixed due to
3557 the inevitable march of time.
3558
3559 Testing frameworks
3560 In general Hypothesis goes to quite a lot of effort to generate things
3561 that look like normal Python test functions that behave as closely to
3562 the originals as possible, so it should work sensibly out of the box
3563 with every test framework.
3564
3565 If your testing relies on doing something other than calling a function
3566 and seeing if it raises an exception then it probably won't work out of
3567 the box. In particular things like tests which return generators and
3568 expect you to do something with them (e.g. nose's yield based tests)
3569 will not work. Use a decorator or similar to wrap the test to take this
3570 form, or ask the framework maintainer to support our hooks for insert‐
3571 ing such a wrapper later.
3572
3573 In terms of what's actually known to work:
3574
3575 · Hypothesis integrates as smoothly with pytest and unittest as we
3576 can make it, and this is verified as part of the CI.
3577
3578 · pytest fixtures work in the usual way for tests that have been
3579 decorated with @given - just avoid passing a strategy for each
3580 argument that will be supplied by a fixture. However, each fix‐
3581 ture will run once for the whole function, not once per example.
3582 Decorating a fixture function with @given is meaningless.
3583
3584 · The python:unittest.mock.patch() decorator works with @given, but
3585 we recommend using it as a context manager within the decorated
3586 test to ensure that the mock is per-test-case and avoid poor
3587 interactions with Pytest fixtures.
3588
3589 · Nose works fine with Hypothesis, and this is tested as part of the
3590 CI. yield based tests simply won't work.
3591
3592 · Integration with Django's testing requires use of the hypothe‐
3593 sis-django package. The issue is that in Django's tests' normal
3594 mode of execution it will reset the database once per test rather
3595 than once per example, which is not what you want.
3596
3597 · Coverage works out of the box with Hypothesis; our own test suite
3598 has 100% branch coverage.
3599
3600 Optional Packages
3601 The supported versions of optional packages, for strategies in hypothe‐
3602 sis.extra, are listed in the documentation for that extra. Our general
3603 goal is to support all versions that are supported upstream.
3604
3605 Regularly verifying this
3606 Everything mentioned above as explicitly supported is checked on every
3607 commit with Travis, Appveyor, and CircleCI. Our continous delivery
3608 pipeline runs all of these checks before publishing each release, so
3609 when we say they're supported we really mean it.
3610
3611 Hypothesis versions
3612 Backwards compatibility is better than backporting fixes, so we use
3613 semantic versioning and only support the most recent version of Hypoth‐
3614 esis. See support for more information.
3615
3617 This is a collection of examples of how to use Hypothesis in interest‐
3618 ing ways. It's small for now but will grow over time.
3619
3620 All of these examples are designed to be run under pytest, and nose
3621 should work too.
3622
3623 How not to sort by a partial order
3624 The following is an example that's been extracted and simplified from a
3625 real bug that occurred in an earlier version of Hypothesis. The real
3626 bug was a lot harder to find.
3627
3628 Suppose we've got the following type:
3629
3630 class Node(object):
3631 def __init__(self, label, value):
3632 self.label = label
3633 self.value = tuple(value)
3634
3635 def __repr__(self):
3636 return "Node(%r, %r)" % (self.label, self.value)
3637
3638 def sorts_before(self, other):
3639 if len(self.value) >= len(other.value):
3640 return False
3641 return other.value[:len(self.value)] == self.value
3642
3643 Each node is a label and a sequence of some data, and we have the rela‐
3644 tionship sorts_before meaning the data of the left is an initial seg‐
3645 ment of the right. So e.g. a node with value [1, 2] will sort before a
3646 node with value [1, 2, 3], but neither of [1, 2] nor [1, 3] will sort
3647 before the other.
3648
3649 We have a list of nodes, and we want to topologically sort them with
3650 respect to this ordering. That is, we want to arrange the list so that
3651 if x.sorts_before(y) then x appears earlier in the list than y. We
3652 naively think that the easiest way to do this is to extend the partial
3653 order defined here to a total order by breaking ties arbitrarily and
3654 then using a normal sorting algorithm. So we define the following code:
3655
3656 from functools import total_ordering
3657
3658
3659 @total_ordering
3660 class TopoKey(object):
3661 def __init__(self, node):
3662 self.value = node
3663
3664 def __lt__(self, other):
3665 if self.value.sorts_before(other.value):
3666 return True
3667 if other.value.sorts_before(self.value):
3668 return False
3669
3670 return self.value.label < other.value.label
3671
3672
3673 def sort_nodes(xs):
3674 xs.sort(key=TopoKey)
3675
3676 This takes the order defined by sorts_before and extends it by breaking
3677 ties by comparing the node labels.
3678
3679 But now we want to test that it works.
3680
3681 First we write a function to verify that our desired outcome holds:
3682
3683 def is_prefix_sorted(xs):
3684 for i in range(len(xs)):
3685 for j in range(i+1, len(xs)):
3686 if xs[j].sorts_before(xs[i]):
3687 return False
3688 return True
3689
3690 This will return false if it ever finds a pair in the wrong order and
3691 return true otherwise.
3692
3693 Given this function, what we want to do with Hypothesis is assert that
3694 for all sequences of nodes, the result of calling sort_nodes on it is
3695 sorted.
3696
3697 First we need to define a strategy for Node:
3698
3699 from hypothesis import settings, strategies
3700 import hypothesis.strategies as s
3701
3702 NodeStrategy = s.builds(
3703 Node,
3704 s.integers(),
3705 s.lists(s.booleans(), max_size=10))
3706
3707 We want to generate short lists of values so that there's a decent
3708 chance of one being a prefix of the other (this is also why the choice
3709 of bool as the elements). We then define a strategy which builds a node
3710 out of an integer and one of those short lists of booleans.
3711
3712 We can now write a test:
3713
3714 from hypothesis import given
3715
3716 @given(s.lists(NodeStrategy))
3717 def test_sorting_nodes_is_prefix_sorted(xs):
3718 sort_nodes(xs)
3719 assert is_prefix_sorted(xs)
3720
3721 this immediately fails with the following example:
3722
3723 [Node(0, (False, True)), Node(0, (True,)), Node(0, (False,))]
3724
3725 The reason for this is that because False is not a prefix of (True,
3726 True) nor vice versa, sorting things the first two nodes are equal
3727 because they have equal labels. This makes the whole order non-transi‐
3728 tive and produces basically nonsense results.
3729
3730 But this is pretty unsatisfying. It only works because they have the
3731 same label. Perhaps we actually wanted our labels to be unique. Lets
3732 change the test to do that.
3733
3734 def deduplicate_nodes_by_label(nodes):
3735 table = {node.label: node for node in nodes}
3736 return list(table.values())
3737
3738 We define a function to deduplicate nodes by labels, and can now map
3739 that over a strategy for lists of nodes to give us a strategy for lists
3740 of nodes with unique labels:
3741
3742 @given(s.lists(NodeStrategy).map(deduplicate_nodes_by_label))
3743 def test_sorting_nodes_is_prefix_sorted(xs):
3744 sort_nodes(xs)
3745 assert is_prefix_sorted(xs)
3746
3747 Hypothesis quickly gives us an example of this still being wrong:
3748
3749 [Node(0, (False,)), Node(-1, (True,)), Node(-2, (False, False))])
3750
3751 Now this is a more interesting example. None of the nodes will sort
3752 equal. What is happening here is that the first node is strictly less
3753 than the last node because (False,) is a prefix of (False, False). This
3754 is in turn strictly less than the middle node because neither is a pre‐
3755 fix of the other and -2 < -1. The middle node is then less than the
3756 first node because -1 < 0.
3757
3758 So, convinced that our implementation is broken, we write a better one:
3759
3760 def sort_nodes(xs):
3761 for i in hrange(1, len(xs)):
3762 j = i - 1
3763 while j >= 0:
3764 if xs[j].sorts_before(xs[j+1]):
3765 break
3766 xs[j], xs[j+1] = xs[j+1], xs[j]
3767 j -= 1
3768
3769 This is just insertion sort slightly modified - we swap a node back‐
3770 wards until swapping it further would violate the order constraints.
3771 The reason this works is because our order is a partial order already
3772 (this wouldn't produce a valid result for a general topological sorting
3773 - you need the transitivity).
3774
3775 We now run our test again and it passes, telling us that this time
3776 we've successfully managed to sort some nodes without getting it com‐
3777 pletely wrong. Go us.
3778
3779 Time zone arithmetic
3780 This is an example of some tests for pytz which check that various
3781 timezone conversions behave as you would expect them to. These tests
3782 should all pass, and are mostly a demonstration of some useful sorts of
3783 thing to test with Hypothesis, and how the datetimes() strategy works.
3784
3785 >>> from datetime import timedelta
3786 >>> from hypothesis.extra.pytz import timezones
3787 >>> from hypothesis.strategies import datetimes
3788
3789 >>> # The datetimes strategy is naive by default, so tell it to use timezones
3790 >>> aware_datetimes = datetimes(timezones=timezones())
3791
3792 >>> @given(aware_datetimes, timezones(), timezones())
3793 ... def test_convert_via_intermediary(dt, tz1, tz2):
3794 ... """Test that converting between timezones is not affected
3795 ... by a detour via another timezone.
3796 ... """
3797 ... assert dt.astimezone(tz1).astimezone(tz2) == dt.astimezone(tz2)
3798
3799 >>> @given(aware_datetimes, timezones())
3800 ... def test_convert_to_and_fro(dt, tz2):
3801 ... """If we convert to a new timezone and back to the old one
3802 ... this should leave the result unchanged.
3803 ... """
3804 ... tz1 = dt.tzinfo
3805 ... assert dt == dt.astimezone(tz2).astimezone(tz1)
3806
3807 >>> @given(aware_datetimes, timezones())
3808 ... def test_adding_an_hour_commutes(dt, tz):
3809 ... """When converting between timezones it shouldn't matter
3810 ... if we add an hour here or add an hour there.
3811 ... """
3812 ... an_hour = timedelta(hours=1)
3813 ... assert (dt + an_hour).astimezone(tz) == dt.astimezone(tz) + an_hour
3814
3815 >>> @given(aware_datetimes, timezones())
3816 ... def test_adding_a_day_commutes(dt, tz):
3817 ... """When converting between timezones it shouldn't matter
3818 ... if we add a day here or add a day there.
3819 ... """
3820 ... a_day = timedelta(days=1)
3821 ... assert (dt + a_day).astimezone(tz) == dt.astimezone(tz) + a_day
3822
3823 >>> # And we can check that our tests pass
3824 >>> test_convert_via_intermediary()
3825 >>> test_convert_to_and_fro()
3826 >>> test_adding_an_hour_commutes()
3827 >>> test_adding_a_day_commutes()
3828
3829 Condorcet's Paradox
3830 A classic paradox in voting theory, called Condorcet's paradox, is that
3831 majority preferences are not transitive. That is, there is a population
3832 and a set of three candidates A, B and C such that the majority of the
3833 population prefer A to B, B to C and C to A.
3834
3835 Wouldn't it be neat if we could use Hypothesis to provide an example of
3836 this?
3837
3838 Well as you can probably guess from the presence of this section, we
3839 can! The main trick is to decide how we want to represent the result
3840 of an election - for this example, we'll use a list of "votes", where
3841 each vote is a list of candidates in the voters preferred order. With‐
3842 out further ado, here is the code:
3843
3844 from hypothesis import given, assume
3845 from hypothesis.strategies import lists, permutations
3846 from collections import Counter
3847
3848 # We need at least three candidates and at least three voters to have a
3849 # paradox; anything less can only lead to victories or at worst ties.
3850 @given(lists(permutations(['A', 'B', 'C']), min_size=3))
3851 def test_elections_are_transitive(election):
3852 all_candidates = {"A", "B", "C"}
3853
3854 # First calculate the pairwise counts of how many prefer each candidate
3855 # to the other
3856 counts = Counter()
3857 for vote in election:
3858 for i in range(len(vote)):
3859 for j in range(i+1, len(vote)):
3860 counts[(vote[i], vote[j])] += 1
3861
3862 # Now look at which pairs of candidates one has a majority over the
3863 # other and store that.
3864 graph = {}
3865 for i in all_candidates:
3866 for j in all_candidates:
3867 if counts[(i, j)] > counts[(j, i)]:
3868 graph.setdefault(i, set()).add(j)
3869
3870 # Now for each triple assert that it is transitive.
3871 for x in all_candidates:
3872 for y in graph.get(x, ()):
3873 for z in graph.get(y, ()):
3874 assert x not in graph.get(z, ())
3875
3876 The example Hypothesis gives me on my first run (your mileage may of
3877 course vary) is:
3878
3879 [['A', 'B', 'C'], ['B', 'C', 'A'], ['C', 'A', 'B']]
3880
3881 Which does indeed do the job: The majority (votes 0 and 1) prefer B to
3882 C, the majority (votes 0 and 2) prefer A to B and the majority (votes 1
3883 and 2) prefer C to A. This is in fact basically the canonical example
3884 of the voting paradox.
3885
3886 Fuzzing an HTTP API
3887 Hypothesis's support for testing HTTP services is somewhat nascent.
3888 There are plans for some fully featured things around this, but right
3889 now they're probably quite far down the line.
3890
3891 But you can do a lot yourself without any explicit support! Here's a
3892 script I wrote to throw arbitrary data against the API for an entirely
3893 fictitious service called Waspfinder (this is only lightly obfuscated
3894 and you can easily figure out who I'm actually talking about, but I
3895 don't want you to run this code and hammer their API without their per‐
3896 mission).
3897
3898 All this does is use Hypothesis to generate arbitrary JSON data match‐
3899 ing the format their API asks for and check for 500 errors. More
3900 advanced tests which then use the result and go on to do other things
3901 are definitely also possible. The swagger-conformance package provides
3902 an excellent example of this!
3903
3904 import unittest
3905 from hypothesis import given, assume, settings, strategies as st
3906 from collections import namedtuple
3907 import requests
3908 import os
3909 import random
3910 import time
3911 import math
3912
3913
3914 Goal = namedtuple("Goal", ("slug",))
3915
3916
3917 # We just pass in our API credentials via environment variables.
3918 waspfinder_token = os.getenv('WASPFINDER_TOKEN')
3919 waspfinder_user = os.getenv('WASPFINDER_USER')
3920 assert waspfinder_token is not None
3921 assert waspfinder_user is not None
3922
3923 GoalData = st.fixed_dictionaries({
3924 'title': st.text(),
3925 'goal_type': st.sampled_from([
3926 "hustler", "biker", "gainer", "fatloser", "inboxer",
3927 "drinker", "custom"]),
3928 'goaldate': st.one_of(st.none(), st.floats()),
3929 'goalval': st.one_of(st.none(), st.floats()),
3930 'rate': st.one_of(st.none(), st.floats()),
3931 'initval': st.floats(),
3932 'panic': st.floats(),
3933 'secret': st.booleans(),
3934 'datapublic': st.booleans(),
3935 })
3936
3937
3938 needs2 = ['goaldate', 'goalval', 'rate']
3939
3940
3941 class WaspfinderTest(unittest.TestCase):
3942
3943 @given(GoalData)
3944 def test_create_goal_dry_run(self, data):
3945 # We want slug to be unique for each run so that multiple test runs
3946 # don't interfere with each other. If for some reason some slugs trigger
3947 # an error and others don't we'll get a Flaky error, but that's OK.
3948 slug = hex(random.getrandbits(32))[2:]
3949
3950 # Use assume to guide us through validation we know about, otherwise
3951 # we'll spend a lot of time generating boring examples.
3952
3953 # Title must not be empty
3954 assume(data["title"])
3955
3956 # Exactly two of these values should be not None. The other will be
3957 # inferred by the API.
3958
3959 assume(len([1 for k in needs2 if data[k] is not None]) == 2)
3960 for v in data.values():
3961 if isinstance(v, float):
3962 assume(not math.isnan(v))
3963 data["slug"] = slug
3964
3965 # The API nicely supports a dry run option, which means we don't have
3966 # to worry about the user account being spammed with lots of fake goals
3967 # Otherwise we would have to make sure we cleaned up after ourselves
3968 # in this test.
3969 data["dryrun"] = True
3970 data["auth_token"] = waspfinder_token
3971 for d, v in data.items():
3972 if v is None:
3973 data[d] = "null"
3974 else:
3975 data[d] = str(v)
3976 result = requests.post(
3977 "https://waspfinder.example.com/api/v1/users/"
3978 "%s/goals.json" % (waspfinder_user,), data=data)
3979
3980 # Lets not hammer the API too badly. This will of course make the
3981 # tests even slower than they otherwise would have been, but that's
3982 # life.
3983 time.sleep(1.0)
3984
3985 # For the moment all we're testing is that this doesn't generate an
3986 # internal error. If we didn't use the dry run option we could have
3987 # then tried doing more with the result, but this is a good start.
3988 self.assertNotEqual(result.status_code, 500)
3989
3990 if __name__ == '__main__':
3991 unittest.main()
3992
3994 The Hypothesis community is small for the moment but is full of excel‐
3995 lent people who can answer your questions and help you out. Please do
3996 join us. The two major places for community discussion are:
3997
3998 · The mailing list.
3999
4000 · An IRC channel, #hypothesis on freenode, which is more active than
4001 the mailing list.
4002
4003 Feel free to use these to ask for help, provide feedback, or discuss
4004 anything remotely Hypothesis related at all. If you post a question on
4005 StackOverflow, please use the python-hypothesis tag!
4006
4007 Please note that the Hypothesis code of conduct applies in all Hypothe‐
4008 sis community spaces.
4009
4010 If you would like to cite Hypothesis, please consider our sugested
4011 citation.
4012
4013 If you like repo badges, we suggest the following badge, which you can
4014 add with reStructuredText or Markdown, respectively: [image]
4015
4016 .. image:: https://img.shields.io/badge/hypothesis-tested-brightgreen.svg
4017 :alt: Tested with Hypothesis
4018 :target: https://hypothesis.readthedocs.io
4019
4020 [![Tested with Hypothesis](https://img.shields.io/badge/hypothesis-tested-brightgreen.svg)](https://hypothesis.readthedocs.io/)
4021
4022 Finally, we have a beautiful logo which appears online, and often on
4023 stickers: [image: The Hypothesis logo, a dragonfly with rainbow wings]
4024 [image]
4025
4026 As well as being beautiful, dragonflies actively hunt down bugs for a
4027 living! You can find the images and a usage guide in the brand direc‐
4028 tory on GitHub, or find us at conferences where we often have stickers
4029 and sometimes other swag.
4030
4032 What is Hypothesis for?
4033
4034 From the perspective of a user, the purpose of Hypothesis is to make it
4035 easier for you to write better tests.
4036
4037 From my perspective as the author, that is of course also a purpose of
4038 Hypothesis, but (if you will permit me to indulge in a touch of megalo‐
4039 mania for a moment), the larger purpose of Hypothesis is to drag the
4040 world kicking and screaming into a new and terrifying age of high qual‐
4041 ity software.
4042
4043 Software is, as they say, eating the world. Software is also terrible.
4044 It's buggy, insecure and generally poorly thought out. This combination
4045 is clearly a recipe for disaster.
4046
4047 And the state of software testing is even worse. Although it's fairly
4048 uncontroversial at this point that you should be testing your code, can
4049 you really say with a straight face that most projects you've worked on
4050 are adequately tested?
4051
4052 A lot of the problem here is that it's too hard to write good tests.
4053 Your tests encode exactly the same assumptions and fallacies that you
4054 had when you wrote the code, so they miss exactly the same bugs that
4055 you missed when you wrote the code.
4056
4057 Meanwhile, there are all sorts of tools for making testing better that
4058 are basically unused. The original Quickcheck is from 1999 and the
4059 majority of developers have not even heard of it, let alone used it.
4060 There are a bunch of half-baked implementations for most languages, but
4061 very few of them are worth using.
4062
4063 The goal of Hypothesis is to bring advanced testing techniques to the
4064 masses, and to provide an implementation that is so high quality that
4065 it is easier to use them than it is not to use them. Where I can, I
4066 will beg, borrow and steal every good idea I can find that someone has
4067 had to make software testing better. Where I can't, I will invent new
4068 ones.
4069
4070 Quickcheck is the start, but I also plan to integrate ideas from fuzz
4071 testing (a planned future feature is to use coverage information to
4072 drive example selection, and the example saving database is already
4073 inspired by the workflows people use for fuzz testing), and am open to
4074 and actively seeking out other suggestions and ideas.
4075
4076 The plan is to treat the social problem of people not using these ideas
4077 as a bug to which there is a technical solution: Does property-based
4078 testing not match your workflow? That's a bug, let's fix it by figur‐
4079 ing out how to integrate Hypothesis into it. Too hard to generate cus‐
4080 tom data for your application? That's a bug. Let's fix it by figuring
4081 out how to make it easier, or how to take something you're already
4082 using to specify your data and derive a generator from that automati‐
4083 cally. Find the explanations of these advanced ideas hopelessly obtuse
4084 and hard to follow? That's a bug. Let's provide you with an easy API
4085 that lets you test your code better without a PhD in software verifica‐
4086 tion.
4087
4088 Grand ambitions, I know, and I expect ultimately the reality will be
4089 somewhat less grand, but so far in about three months of development,
4090 Hypothesis has become the most solid implementation of Quickcheck ever
4091 seen in a mainstream language (as long as we don't count Scala as main‐
4092 stream yet), and at the same time managed to significantly push forward
4093 the state of the art, so I think there's reason to be optimistic.
4094
4096 This is a page for listing people who are using Hypothesis and how
4097 excited they are about that. If that's you and your name is not on the
4098 list, this file is in Git and I'd love it if you sent me a pull request
4099 to fix that.
4100
4101 Stripe
4102 At Stripe we use Hypothesis to test every piece of our machine learning
4103 model training pipeline (powered by scikit). Before we migrated, our
4104 tests were filled with hand-crafted pandas Dataframes that weren't rep‐
4105 resentative at all of our actual very complex data. Because we needed
4106 to craft examples for each test, we took the easy way out and lived
4107 with extremely low test coverage.
4108
4109 Hypothesis changed all that. Once we had our strategies for generating
4110 Dataframes of features it became trivial to slightly customize each
4111 strategy for new tests. Our coverage is now close to 90%.
4112
4113 Full-stop, property-based testing is profoundly more powerful - and has
4114 caught or prevented far more bugs - than our old style of example-based
4115 testing.
4116
4117 Kristian Glass - Director of Technology at LaterPay GmbH
4118 Hypothesis has been brilliant for expanding the coverage of our test
4119 cases, and also for making them much easier to read and understand, so
4120 we're sure we're testing the things we want in the way we want.
4121
4122 Seth Morton
4123 When I first heard about Hypothesis, I knew I had to include it in my
4124 two open-source Python libraries, natsort and fastnumbers . Quite
4125 frankly, I was a little appalled at the number of bugs and "holes" I
4126 found in the code. I can now say with confidence that my libraries are
4127 more robust to "the wild." In addition, Hypothesis gave me the confi‐
4128 dence to expand these libraries to fully support Unicode input, which I
4129 never would have had the stomach for without such thorough testing
4130 capabilities. Thanks!
4131
4132 Sixty North
4133 At Sixty North we use Hypothesis for testing Segpy an open source
4134 Python library for shifting data between Python data structures and SEG
4135 Y files which contain geophysical data from the seismic reflection sur‐
4136 veys used in oil and gas exploration.
4137
4138 This is our first experience of property-based testing – as opposed to
4139 example-based testing. Not only are our tests more powerful, they are
4140 also much better explanations of what we expect of the production code.
4141 In fact, the tests are much closer to being specifications. Hypothesis
4142 has located real defects in our code which went undetected by tradi‐
4143 tional test cases, simply because Hypothesis is more relentlessly devi‐
4144 ous about test case generation than us mere humans! We found Hypothe‐
4145 sis particularly beneficial for Segpy because SEG Y is an antiquated
4146 format that uses legacy text encodings (EBCDIC) and even a legacy
4147 floating point format we implemented from scratch in Python.
4148
4149 Hypothesis is sure to find a place in most of our future Python code‐
4150 bases and many existing ones too.
4151
4152 mulkieran
4153 Just found out about this excellent QuickCheck for Python implementa‐
4154 tion and ran up a few tests for my bytesize package last night. Refuted
4155 a few hypotheses in the process.
4156
4157 Looking forward to using it with a bunch of other projects as well.
4158
4159 Adam Johnson
4160 I have written a small library to serialize dicts to MariaDB's dynamic
4161 columns binary format, mariadb-dyncol. When I first developed it, I
4162 thought I had tested it really well - there were hundreds of test
4163 cases, some of them even taken from MariaDB's test suite itself. I was
4164 ready to release.
4165
4166 Lucky for me, I tried Hypothesis with David at the PyCon UK sprints.
4167 Wow! It found bug after bug after bug. Even after a first release, I
4168 thought of a way to make the tests do more validation, which revealed a
4169 further round of bugs! Most impressively, Hypothesis found a compli‐
4170 cated off-by-one error in a condition with 4095 versus 4096 bytes of
4171 data - something that I would never have found.
4172
4173 Long live Hypothesis! (Or at least, property-based testing).
4174
4175 Josh Bronson
4176 Adopting Hypothesis improved bidict's test coverage and significantly
4177 increased our ability to make changes to the code with confidence that
4178 correct behavior would be preserved. Thank you, David, for the great
4179 testing tool.
4180
4181 Cory Benfield
4182 Hypothesis is the single most powerful tool in my toolbox for working
4183 with algorithmic code, or any software that produces predictable output
4184 from a wide range of sources. When using it with Priority, Hypothesis
4185 consistently found errors in my assumptions and extremely subtle bugs
4186 that would have taken months of real-world use to locate. In some
4187 cases, Hypothesis found subtle deviations from the correct output of
4188 the algorithm that may never have been noticed at all.
4189
4190 When it comes to validating the correctness of your tools, nothing
4191 comes close to the thoroughness and power of Hypothesis.
4192
4193 Jon Moore
4194 One extremely satisfied user here. Hypothesis is a really solid imple‐
4195 mentation of property-based testing, adapted well to Python, and with
4196 good features such as failure-case shrinkers. I first used it on a
4197 project where we needed to verify that a vendor's Python and non-Python
4198 implementations of an algorithm matched, and it found about a dozen
4199 cases that previous example-based testing and code inspections had not.
4200 Since then I've been evangelizing for it at our firm.
4201
4202 Russel Winder
4203 I am using Hypothesis as an integral part of my Python workshops. Test‐
4204 ing is an integral part of Python programming and whilst unittest and,
4205 better, pytest can handle example-based testing, property-based testing
4206 is increasingly far more important than example-base testing, and
4207 Hypothesis fits the bill.
4208
4209 Wellfire Interactive
4210 We've been using Hypothesis in a variety of client projects, from test‐
4211 ing Django-related functionality to domain-specific calculations. It
4212 both speeds up and simplifies the testing process since there's so much
4213 less tedious and error-prone work to do in identifying edge cases. Test
4214 coverage is nice but test depth is even nicer, and it's much easier to
4215 get meaningful test depth using Hypothesis.
4216
4217 Cody Kochmann
4218 Hypothesis is being used as the engine for random object generation
4219 with my open source function fuzzer battle_tested which maps all behav‐
4220 iors of a function allowing you to minimize the chance of unexpected
4221 crashes when running code in production.
4222
4223 With how efficient Hypothesis is at generating the edge cases that
4224 cause unexpected behavior occur, battle_tested is able to map out the
4225 entire behavior of most functions in less than a few seconds.
4226
4227 Hypothesis truly is a masterpiece. I can't thank you enough for build‐
4228 ing it.
4229
4230 Merchise Autrement
4231 Just minutes after our first use of hypothesis we uncovered a subtle
4232 bug in one of our most used library. Since then, we have increasingly
4233 used hypothesis to improve the quality of our testing in libraries and
4234 applications as well.
4235
4236 Florian Kromer
4237 At Roboception GmbH I use Hypothesis to implement fully automated
4238 stateless and stateful reliability tests for the 3D sensor rc_visard
4239 and robotic software components .
4240
4241 Thank you very much for creating the (probably) most powerful prop‐
4242 erty-based testing framework.
4243
4244 Reposit Power
4245 With a micro-service architecture, testing between services is made
4246 easy using Hypothesis in integration testing. Ensuring everything is
4247 running smoothly is vital to help maintain a secure network of Virtual
4248 Power Plants.
4249
4250 It allows us to find potential bugs and edge cases with relative ease
4251 and minimal overhead. As our architecture relies on services communi‐
4252 cating effectively, Hypothesis allows us to strictly test for the kind
4253 of data which moves around our services, particularly our backend
4254 Python applications.
4255
4256 Your name goes here
4257 I know there are many more, because I keep finding out about new people
4258 I'd never even heard of using Hypothesis. If you're looking to way to
4259 give back to a tool you love, adding your name here only takes a moment
4260 and would really help a lot. As per instructions at the top, just send
4261 me a pull request and I'll add you to the list.
4262
4264 The following is a non-exhaustive list of open source projects I know
4265 are using Hypothesis. If you're aware of any others please add them to
4266 the list! The only inclusion criterion right now is that if it's a
4267 Python library then it should be available on PyPI.
4268
4269 You can find hundreds more from the Hypothesis page at libraries.io,
4270 and over a thousand on GitHub.
4271
4272 · aur
4273
4274 · argon2_cffi
4275
4276 · attrs
4277
4278 · axelrod
4279
4280 · bidict
4281
4282 · binaryornot
4283
4284 · brotlipy
4285
4286 · chardet
4287
4288 · cmph-cffi
4289
4290 · cryptography
4291
4292 · dbus-signature-pyparsing
4293
4294 · fastnumbers
4295
4296 · flocker
4297
4298 · flownetpy
4299
4300 · funsize
4301
4302 · fusion-index
4303
4304 · hyper-h2
4305
4306 · into-dbus-python
4307
4308 · justbases
4309
4310 · justbytes
4311
4312 · loris
4313
4314 · mariadb-dyncol
4315
4316 · mercurial
4317
4318 · natsort
4319
4320 · pretext
4321
4322 · priority
4323
4324 · PyCEbox
4325
4326 · PyPy
4327
4328 · pyrsistent
4329
4330 · python-humble-utils
4331
4332 · pyudev
4333
4334 · qutebrowser
4335
4336 · RubyMarshal
4337
4338 · Segpy
4339
4340 · simoa
4341
4342 · srt
4343
4344 · tchannel
4345
4346 · vdirsyncer
4347
4348 · wcag-contrast-ratio
4349
4350 · yacluster
4351
4352 · yturl
4353
4355 Hypothesis has been eagerly used and extended by the open source commu‐
4356 nity. This page lists extensions and applications; you can find more
4357 or newer packages by searching PyPI by keyword or filter by classifier,
4358 or search libraries.io.
4359
4360 If there's something missing which you think should be here, let us
4361 know!
4362
4363 NOTE:
4364 Being listed on this page does not imply that the Hypothesis main‐
4365 tainers endorse a package.
4366
4367 External Strategies
4368 Some packages provide strategies directly:
4369
4370 · hypothesis-fspaths - strategy to generate filesystem paths.
4371
4372 · hypothesis-geojson - strategy to generate GeoJson.
4373
4374 · hs-dbus-signature - strategy to generate arbitrary D-Bus signatures.
4375
4376 · hypothesis_sqlalchemy - strategies to generate SQLAlchemy objects.
4377
4378 · hypothesis-ros - strategies to generate messages and parameters for
4379 the Robot Operating System.
4380
4381 · hypothesis-csv - strategy to generate CSV files.
4382
4383 · hypothesis-networkx - strategy to generate networkx graphs.
4384
4385 Others provide a function to infer a strategy from some other schema:
4386
4387 · hypothesis-jsonschema - infer strategies from JSON schemas.
4388
4389 · lollipop-hypothesis - infer strategies from lollipop schemas.
4390
4391 · hypothesis-drf - infer strategies from a djangorestframework seri‐
4392 aliser.
4393
4394 · hypothesis-mongoengine - infer strategies from a mongoengine model.
4395
4396 · hypothesis-pb - infer strategies from Protocol Buffer schemas.
4397
4398 Other Cool Things
4399 swagger-conformance is powered by Hypothesis and pyswagger. Based on a
4400 Swagger specification, it can build and run an entire test suite to
4401 check that the implementation matches the spec. The command-line ver‐
4402 sion can test apps written in any language, simply by passing the file
4403 or URL path to the schema to check!
4404
4405 Trio is an async framework with "an obsessive focus on usability and
4406 correctness", so naturally it works with Hypothesis! pytest-trio
4407 includes a custom hook that allows @given(...) to work with Trio-style
4408 async test functions, and hypothesis-trio includes stateful testing
4409 extensions to support concurrent programs.
4410
4411 libarchimedes makes it easy to use Hypothesis in the Hy language, a
4412 Lisp embedded in Python.
4413
4414 battle_tested is a fuzzing tool that will show you how your code can
4415 fail - by trying all kinds of inputs and reporting whatever happens.
4416
4417 pytest-subtesthack functions as a workaround for issue #377.
4418
4419 Writing an Extension
4420 See CONTRIBUTING.rst for more information.
4421
4422 New strategies can be added to Hypothesis, or published as an external
4423 package on PyPI - either is fine for most strategies. If in doubt, ask!
4424
4425 It's generally much easier to get things working outside, because
4426 there's more freedom to experiment and fewer requirements in stability
4427 and API style. We're happy to review and help with external packages as
4428 well as pull requests!
4429
4430 If you're thinking about writing an extension, please name it hypothe‐
4431 sis-{something} - a standard prefix makes the community more visible
4432 and searching for extensions easier. And make sure you use the Frame‐
4433 work :: Hypothesis trove classifier!
4434
4435 On the other hand, being inside gets you access to some deeper imple‐
4436 mentation features (if you need them) and better long-term guarantees
4437 about maintenance. We particularly encourage pull requests for new
4438 composable primitives that make implementing other strategies easier,
4439 or for widely used types in the standard library. Strategies for other
4440 things are also welcome; anything with external dependencies just goes
4441 in hypothesis.extra.
4442
4444 This is a record of all past Hypothesis releases and what went into
4445 them, in reverse chronological order. All previous releases should
4446 still be available on pip.
4447
4448 Hypothesis APIs come in three flavours:
4449
4450 · Public: Hypothesis releases since 1.0 are semantically versioned with
4451 respect to these parts of the API. These will not break except
4452 between major version bumps. All APIs mentioned in this documentation
4453 are public unless explicitly noted otherwise.
4454
4455 · Semi-public: These are APIs that are considered ready to use but are
4456 not wholly nailed down yet. They will not break in patch releases and
4457 will usually not break in minor releases, but when necessary minor
4458 releases may break semi-public APIs.
4459
4460 · Internal: These may break at any time and you really should not use
4461 them at all.
4462
4463 You should generally assume that an API is internal unless you have
4464 specific information to the contrary.
4465
4466 4.23.8 - 2019-05-26
4467 This patch has a minor cleanup of the internal engine. There is no
4468 user-visible impact.
4469
4470 4.23.7 - 2019-05-26
4471 This patch clarifies some error messages when the test function signa‐
4472 ture is incompatible with the arguments to @given, especially when the
4473 @settings() decorator is also used (issue #1978).
4474
4475 4.23.6 - 2019-05-19
4476 This release adds the pyupgrade fixer to our code style, for consistent
4477 use of dict and set literals and comprehensions.
4478
4479 4.23.5 - 2019-05-16
4480 This release slightly simplifies a small part of the core engine.
4481 There is no user-visible change.
4482
4483 4.23.4 - 2019-05-09
4484 Fixes a minor formatting issue the docstring of from_type()
4485
4486 4.23.3 - 2019-05-09
4487 Adds a recipe to the docstring of from_type() that describes a means
4488 for drawing values for "everything except" a specified type. This
4489 recipe is especially useful for writing tests that perform input-type
4490 validation.
4491
4492 4.23.2 - 2019-05-08
4493 This patch uses autoflake to remove some pointless pass statements,
4494 which improves our workflow but has no user-visible impact.
4495
4496 4.23.1 - 2019-05-08
4497 This patch fixes an OverflowError in from_type(xrange) on Python 2.
4498
4499 It turns out that not only do the start and stop values have to fit in
4500 a C long, but so does stop - start. We now handle this even on 32bit
4501 platforms, but remind users that Python2 will not be supported after
4502 2019 without specific funding.
4503
4504 4.23.0 - 2019-05-08
4505 This release implements the slices() strategy, to generate slices of a
4506 length-size sequence.
4507
4508 Thanks to Daniel J. West for writing this patch at the PyCon 2019
4509 sprints!
4510
4511 4.22.3 - 2019-05-07
4512 This patch exposes DataObject, solely to support more precise type
4513 hints. Objects of this type are provided by data(), and can be used to
4514 draw examples from strategies intermixed with your test code.
4515
4516 4.22.2 - 2019-05-07
4517 This patch fixes the very rare issue #1798 in array_dtypes(), which
4518 caused an internal error in our tests.
4519
4520 4.22.1 - 2019-05-07
4521 This patch fixes a rare bug in from_type(range).
4522
4523 Thanks to Zebulun Arendsee for fixing the bug at the PyCon 2019
4524 Sprints.
4525
4526 4.22.0 - 2019-05-07
4527 The unique_by argument to lists now accepts a tuple of callables such
4528 that every element of the generated list will be unique with respect to
4529 each callable in the tuple (issue #1916).
4530
4531 Thanks to Marco Sirabella for this feature at the PyCon 2019 sprints!
4532
4533 4.21.1 - 2019-05-06
4534 This patch cleans up the internals of one_of(). You may see a slight
4535 change to the distribution of examples from this strategy but there is
4536 no change to the public API.
4537
4538 Thanks to Marco Sirabella for writing this patch at the PyCon 2019
4539 sprints!
4540
4541 4.21.0 - 2019-05-05
4542 The from_type() strategy now supports python:slice objects.
4543
4544 Thanks to Charlie El. Awbery for writing this feature at the PyCon 2019
4545 Mentored Sprints.
4546
4547 4.20.0 - 2019-05-05
4548 This release improves the array_shapes() strategy, to choose an appro‐
4549 priate default for max_side based on the min_side, and max_dims based
4550 on the min_dims. An explicit error is raised for dimensions greater
4551 than 32, which are not supported by Numpy, as for other invalid combi‐
4552 nations of arguments.
4553
4554 Thanks to Jenny Rouleau for writing this feature at the PyCon 2019 Men‐
4555 tored Sprints.
4556
4557 4.19.0 - 2019-05-05
4558 The from_type() strategy now supports python:range objects (or xrange
4559 on Python 2).
4560
4561 Thanks to Katrina Durance for writing this feature at the PyCon 2019
4562 Mentored Sprints.
4563
4564 4.18.3 - 2019-04-30
4565 This release fixes a very rare edge case in the test-case mutator,
4566 which could cause an internal error with certain unusual tests.
4567
4568 4.18.2 - 2019-04-30
4569 This patch makes Hypothesis compatible with the Python 3.8 alpha, which
4570 changed the representation of code objects to support positional-only
4571 arguments. Note however that Hypothesis does not (yet) support such
4572 functions as e.g. arguments to builds() or inputs to @given.
4573
4574 Thanks to Paul Ganssle for identifying and fixing this bug.
4575
4576 4.18.1 - 2019-04-29
4577 This patch improves the performance of unique collections such as
4578 sets() when the elements are drawn from a sampled_from() strategy (‐
4579 issue #1115).
4580
4581 4.18.0 - 2019-04-24
4582 This release adds the functions() strategy, which can be used to imi‐
4583 tate your 'real' function for callbacks.
4584
4585 4.17.2 - 2019-04-19
4586 This release refactors stateful rule selection to share the new machin‐
4587 ery with sampled_from() instead of using the original independent
4588 implementation.
4589
4590 4.17.1 - 2019-04-16
4591 This patch allows Hypothesis to try a few more examples after finding
4592 the first bug, in hopes of reporting multiple distinct bugs. The
4593 heuristics described in issue #847 ensure that we avoid wasting time on
4594 fruitless searches, while still surfacing each bug as soon as possible.
4595
4596 4.17.0 - 2019-04-16
4597 This release adds the strategy broadcastable_shapes(), which generates
4598 array shapes that are broadcast-compatible with a provided shape.
4599
4600 4.16.0 - 2019-04-12
4601 This release allows register_type_strategy() to be used with
4602 python:typing.NewType instances. This may be useful to e.g. provide
4603 only positive integers for from_type(UserId) with a UserId = New‐
4604 Type('UserId', int) type.
4605
4606 Thanks to PJCampi for suggesting and writing the patch!
4607
4608 4.15.0 - 2019-04-09
4609 This release supports passing a timedelta as the deadline setting, so
4610 you no longer have to remember that the number is in milliseconds (‐
4611 issue #1900).
4612
4613 Thanks to Damon Francisco for this change!
4614
4615 4.14.7 - 2019-04-09
4616 This patch makes the type annotations on hypothesis.extra.dateutil com‐
4617 patible with mypy 0.700.
4618
4619 4.14.6 - 2019-04-07
4620 This release fixes a bug introduced in Hypothesis 4.14.3 that would
4621 sometimes cause sampled_from(...).filter(...) to encounter an internal
4622 assertion failure when there are three or fewer elements, and every
4623 element is rejected by the filter.
4624
4625 4.14.5 - 2019-04-05
4626 This patch takes the previous efficiency improvements to sam‐
4627 pled_from(...).filter(...) strategies that reject most elements, and
4628 generalises them to also apply to sampled_from(...).filter(...).fil‐
4629 ter(...) and longer chains of filters.
4630
4631 4.14.4 - 2019-04-05
4632 This release fixes a bug that prevented random_module() from correctly
4633 restoring the previous state of the random module.
4634
4635 The random state was instead being restored to a temporary determinis‐
4636 tic state, which accidentally caused subsequent tests to see the same
4637 random values across multiple test runs.
4638
4639 4.14.3 - 2019-04-03
4640 This patch adds an internal special case to make sampled_from(...).fil‐
4641 ter(...) much more efficient when the filter rejects most elements (‐
4642 issue #1885).
4643
4644 4.14.2 - 2019-03-31
4645 This patch improves the error message if the function f in s.flatmap(f)
4646 does not return a strategy.
4647
4648 Thanks to Kai Chen for this change!
4649
4650 4.14.1 - 2019-03-30
4651 This release modifies how Hypothesis selects operations to run during
4652 shrinking, by causing it to deprioritise previously useless classes of
4653 shrink until others have reached a fixed point.
4654
4655 This avoids certain pathological cases where the shrinker gets very
4656 close to finishing and then takes a very long time to finish the last
4657 small changes because it tries many useless shrinks for each useful one
4658 towards the end. It also should cause a more modest improvement (prob‐
4659 ably no more than about 30%) in shrinking performance for most tests.
4660
4661 4.14.0 - 2019-03-19
4662 This release blocks installation of Hypothesis on Python 3.4, which
4663 reached its end of life date on 2019-03-18.
4664
4665 This should not be of interest to anyone but downstream maintainers -
4666 if you are affected, migrate to a secure version of Python as soon as
4667 possible or at least seek commercial support.
4668
4669 4.13.0 - 2019-03-19
4670 This release makes it an explicit error to call floats(min_value=inf,
4671 exclude_min=True) or floats(max_value=-inf, exclude_max=True), as there
4672 are no possible values that can be generated (issue #1859).
4673
4674 floats(min_value=0.0, max_value=-0.0) is now deprecated. While 0. ==
4675 -0. and we could thus generate either if comparing by value, violating
4676 the sequence ordering of floats is a special case we don't want or
4677 need.
4678
4679 4.12.1 - 2019-03-18
4680 This release should significantly reduce the amount of memory that
4681 Hypothesis uses for representing large test cases, by storing informa‐
4682 tion in a more compact representation and only unpacking it lazily when
4683 it is first needed.
4684
4685 4.12.0 - 2019-03-18
4686 This update adds the report_multiple_bugs setting, which you can use to
4687 disable multi-bug reporting and only raise whichever bug had the small‐
4688 est minimal example. This is occasionally useful when using a debugger
4689 or tools that annotate tracebacks via introspection.
4690
4691 4.11.7 - 2019-03-18
4692 This change makes a tiny improvement to the core engine's bookkeeping.
4693 There is no user-visible change.
4694
4695 4.11.6 - 2019-03-15
4696 This release changes some of Hypothesis's internal shrinking behaviour
4697 in order to reduce memory usage and hopefully improve performance.
4698
4699 4.11.5 - 2019-03-13
4700 This release adds a micro-optimisation to how Hypothesis handles debug
4701 reporting internally. Hard to shrink test may see a slight performance
4702 improvement, but in most common scenarios it is unlikely to be notice‐
4703 able.
4704
4705 4.11.4 - 2019-03-13
4706 This release removes some redundant code that was no longer needed but
4707 was still running a significant amount of computation and allocation on
4708 the hot path. This should result in a modest speed improvement for
4709 most tests, especially those with large test cases.
4710
4711 4.11.3 - 2019-03-13
4712 This release adds a micro-optimisation to how Hypothesis caches test
4713 cases. This will cause a small improvement in speed and memory usage
4714 for large test cases, but in most common scenarios it is unlikely to be
4715 noticeable.
4716
4717 4.11.2 - 2019-03-13
4718 This release removes some internal code that populates a field that is
4719 no longer used anywhere. This should result in some modest performance
4720 and speed improvements and no other user visible effects.
4721
4722 4.11.1 - 2019-03-13
4723 This is a formatting-only patch, enabled by a new version of isort.
4724
4725 4.11.0 - 2019-03-12
4726 This release deprecates sampled_from() with empty sequences. This
4727 returns nothing(), which gives a clear error if used directly... but
4728 simply vanishes if combined with another strategy.
4729
4730 Tests that silently generate less than expected are a serious problem
4731 for anyone relying on them to find bugs, and we think reliability more
4732 important than convenience in this case.
4733
4734 4.10.0 - 2019-03-11
4735 This release improves Hypothesis's to detect flaky tests, by noticing
4736 when the behaviour of the test changes between runs. In particular
4737 this will notice many new cases where data generation depends on exter‐
4738 nal state (e.g. external sources of randomness) and flag those as flaky
4739 sooner and more reliably.
4740
4741 The basis of this feature is a considerable reengineering of how
4742 Hypothesis stores its history of test cases, so on top of this its mem‐
4743 ory usage should be considerably reduced.
4744
4745 4.9.0 - 2019-03-09
4746 This release adds the strategy valid_tuple_axes(), which generates
4747 tuples of axis-indices that can be passed to the axis argument in
4748 NumPy's sequential functions (e.g. numpy:numpy.sum()).
4749
4750 Thanks to Ryan Soklaski for this strategy.
4751
4752 4.8.0 - 2019-03-06
4753 This release significantly tightens validation in hypothesis.settings.
4754 max_examples, buffer_size, and stateful_step_count must be positive
4755 integers; deadline must be a positive number or None; and derandomize
4756 must be either True or False.
4757
4758 As usual, this replaces existing errors with a more helpful error and
4759 starts new validation checks as deprecation warnings.
4760
4761 4.7.19 - 2019-03-04
4762 This release makes some micro-optimisations to certain calculations
4763 performed in the shrinker. These should particularly speed up large
4764 test cases where the shrinker makes many small changes. It will also
4765 reduce the amount allocated, but most of this is garbage that would
4766 have been immediately thrown away, so you probably won't see much
4767 effect specifically from that.
4768
4769 4.7.18 - 2019-03-03
4770 This patch removes some overhead from arrays() with a constant shape
4771 and dtype. The resulting performance improvement is modest, but worth‐
4772 wile for small arrays.
4773
4774 4.7.17 - 2019-03-01
4775 This release makes some micro-optimisations within Hypothesis's inter‐
4776 nal representation of test cases. This should cause heavily nested
4777 test cases to allocate less during generation and shrinking, which
4778 should speed things up slightly.
4779
4780 4.7.16 - 2019-02-28
4781 This changes the order in which Hypothesis runs certain operations dur‐
4782 ing shrinking. This should significantly decrease memory usage and
4783 speed up shrinking of large examples.
4784
4785 4.7.15 - 2019-02-28
4786 This release allows Hypothesis to calculate a number of attributes of
4787 generated test cases lazily. This should significantly reduce memory
4788 usage and modestly improve performance, especially for large test
4789 cases.
4790
4791 4.7.14 - 2019-02-28
4792 This release reduces the number of operations the shrinker will try
4793 when reordering parts of a test case. This should in some circum‐
4794 stances significantly speed up shrinking. It may result in different
4795 final test cases, and if so usually slightly worse ones, but it should
4796 not generally have much impact on the end result as the operations
4797 removed were typically useless.
4798
4799 4.7.13 - 2019-02-27
4800 This release changes how Hypothesis reorders examples within a test
4801 case during shrinking. This should make shrinking considerably faster.
4802
4803 4.7.12 - 2019-02-27
4804 This release slightly improves the shrinker's ability to replace parts
4805 of a test case with their minimal version, by allowing it to do so in
4806 bulk rather than one at a time. Where this is effective, shrinker per‐
4807 formance should be modestly improved.
4808
4809 4.7.11 - 2019-02-25
4810 This release makes some micro-optimisations to common operations per‐
4811 formed during shrinking. Shrinking should now be slightly faster,
4812 especially for large examples with relatively fast test functions.
4813
4814 4.7.10 - 2019-02-25
4815 This release is a purely internal refactoring of Hypothesis's API for
4816 representing test cases. There should be no user visible effect.
4817
4818 4.7.9 - 2019-02-24
4819 This release changes certain shrink passes to make them more efficient
4820 when they aren't making progress.
4821
4822 4.7.8 - 2019-02-23
4823 This patch removes some unused code, which makes the internals a bit
4824 easier to understand. There is no user-visible impact.
4825
4826 4.7.7 - 2019-02-23
4827 This release reduces the number of operations the shrinker will try
4828 when reordering parts of a test case. This should in some circum‐
4829 stances significantly speed up shrinking. It may result in different
4830 final test cases, and if so usually slightly worse ones, but it should
4831 not generally have much impact on the end result as the operations
4832 removed were typically useless.
4833
4834 4.7.6 - 2019-02-23
4835 This patch removes some unused code from the shrinker. There is no
4836 user-visible change.
4837
4838 4.7.5 - 2019-02-23
4839 This release changes certain shrink passes to make them adaptive - that
4840 is, in cases where they are successfully making progress they may now
4841 do so significantly faster.
4842
4843 4.7.4 - 2019-02-22
4844 This is a docs-only patch, noting that because the lark-parser is under
4845 active development at version 0.x, hypothesis[lark] APIs may break in
4846 minor releases if necessary to keep up with the upstream package.
4847
4848 4.7.3 - 2019-02-22
4849 This changes Hypothesis to no longer import various test frameworks by
4850 default (if they are installed). which will speed up the initial
4851 import hypothesis call.
4852
4853 4.7.2 - 2019-02-22
4854 This release changes Hypothesis's internal representation of a test
4855 case to calculate some expensive structural information on demand
4856 rather than eagerly. This should reduce memory usage a fair bit, and
4857 may make generation somewhat faster.
4858
4859 4.7.1 - 2019-02-21
4860 This release refactors the internal representation of previously run
4861 test cases. The main thing you should see as a result is that Hypothe‐
4862 sis becomes somewhat less memory hungry.
4863
4864 4.7.0 - 2019-02-21
4865 This patch allows array_shapes() to generate shapes with side-length or
4866 even dimension zero, though the minimum still defaults to one. These
4867 shapes are rare and have some odd behavior, but are particularly impor‐
4868 tant to test for just that reason!
4869
4870 In a related bigfix, arrays() now supports generating zero-dimensional
4871 arrays with dtype=object and a strategy for iterable elements. Previ‐
4872 ously, the array element would incorrectly be set to the first item in
4873 the generated iterable.
4874
4875 Thanks to Ryan Turner for continuing to improve our Numpy support.
4876
4877 4.6.1 - 2019-02-19
4878 This release is a trivial micro-optimisation inside Hypothesis which
4879 should result in it using significantly less memory.
4880
4881 4.6.0 - 2019-02-18
4882 This release changes some inconsistent behavior of arrays() from the
4883 Numpy extra when asked for an array of shape=(). arrays() will now
4884 always return a Numpy ndarray, and the array will always be of the
4885 requested dtype.
4886
4887 Thanks to Ryan Turner for this change.
4888
4889 4.5.12 - 2019-02-18
4890 This release fixes a minor typo in an internal comment. There is no
4891 user-visible change.
4892
4893 4.5.11 - 2019-02-15
4894 This release fixes issue #1813, a bug introduced in 3.59.1, which
4895 caused random_module() to no longer affect the body of the test:
4896 Although Hypothesis would claim to be seeding the random module in fact
4897 tests would always run with a seed of zero.
4898
4899 4.5.10 - 2019-02-14
4900 This patch fixes an off-by-one error in the maximum length of emails().
4901 Thanks to Krzysztof Jurewicz for pull request #1812.
4902
4903 4.5.9 - 2019-02-14
4904 This patch removes some unused code from the shrinker. There is no
4905 user-visible change.
4906
4907 4.5.8 - 2019-02-12
4908 This release fixes an internal IndexError in Hypothesis that could
4909 sometimes be triggered during shrinking.
4910
4911 4.5.7 - 2019-02-11
4912 This release modifies the shrinker to interleave different types of
4913 reduction operations, e.g. switching between deleting data and lowering
4914 scalar values rather than trying entirely deletions then entirely low‐
4915 ering.
4916
4917 This may slow things down somewhat in the typical case, but has the
4918 major advantage that many previously difficult to shrink examples
4919 should become much faster, because the shrinker will no longer tend to
4920 stall when trying some ineffective changes to the shrink target but
4921 will instead interleave it with other more effective operations.
4922
4923 4.5.6 - 2019-02-11
4924 This release makes a number of internal changes to the implementation
4925 of hypothesis.extra.lark.from_lark(). These are primarily intended as
4926 a refactoring, but you may see some minor improvements to performance
4927 when generating large strings, and possibly to shrink quality.
4928
4929 4.5.5 - 2019-02-10
4930 This patch prints an explanatory note when issue #1798 is triggered,
4931 because the error message from Numpy is too terse to locate the prob‐
4932 lem.
4933
4934 4.5.4 - 2019-02-08
4935 In Python 2, long integers are not allowed in the shape argument to
4936 arrays(). Thanks to Ryan Turner for fixing this.
4937
4938 4.5.3 - 2019-02-08
4939 This release makes a small internal refactoring to clarify how Hypothe‐
4940 sis instructs tests to stop running when appropriate. There is no
4941 user-visible change.
4942
4943 4.5.2 - 2019-02-06
4944 This release standardises all of the shrinker's internal operations on
4945 running in a random order.
4946
4947 The main effect you will see from this that it should now be much less
4948 common for the shrinker to stall for a long time before making further
4949 progress. In some cases this will correspond to shrinking more slowly,
4950 but on average it should result in faster shrinking.
4951
4952 4.5.1 - 2019-02-05
4953 This patch updates some docstrings, but has no runtime changes.
4954
4955 4.5.0 - 2019-02-03
4956 This release adds exclude_min and exclude_max arguments to floats(), so
4957 that you can easily generate values from open or half-open intervals (‐
4958 issue #1622).
4959
4960 4.4.6 - 2019-02-03
4961 This patch fixes a bug where from_regex() could throw an internal error
4962 if the python:re.IGNORECASE flag was used (issue #1786).
4963
4964 4.4.5 - 2019-02-02
4965 This release removes two shrink passes that Hypothesis runs late in the
4966 process. These were very expensive when the test function was slow and
4967 often didn't do anything useful.
4968
4969 Shrinking should get faster for most failing tests. If you see any
4970 regression in example quality as a result of this release, please let
4971 us know.
4972
4973 4.4.4 - 2019-02-02
4974 This release modifies the way that Hypothesis deletes data during
4975 shrinking. It will primarily be noticeable for very large examples,
4976 which should now shrink faster.
4977
4978 The shrinker is now also able to perform some deletions that it could
4979 not previously, but this is unlikely to be very noticeable.
4980
4981 4.4.3 - 2019-01-25
4982 This release fixes an open file leak that used to cause ResourceWarn‐
4983 ings.
4984
4985 4.4.2 - 2019-01-24
4986 This release changes Hypothesis's internal approach to caching the
4987 results of executing test cases. The result should be that it is now
4988 significantly less memory hungry, especially when shrinking large test
4989 cases.
4990
4991 Some tests may get slower or faster depending on whether the new or old
4992 caching strategy was well suited to them, but any change in speed in
4993 either direction should be minor.
4994
4995 4.4.1 - 2019-01-24
4996 This patch tightens up some of our internal heuristics to deal with
4997 shrinking floating point numbers, which will now run in fewer circum‐
4998 stances.
4999
5000 You are fairly unlikely to see much difference from this, but if you do
5001 you are likely to see shrinking become slightly faster and/or producing
5002 slightly worse results.
5003
5004 4.4.0 - 2019-01-24
5005 This release adds the from_form() function, which allows automatic
5006 testing against Django forms. (issue #35)
5007
5008 Thanks to Paul Stiverson for this feature, which resolves our oldest
5009 open issue!
5010
5011 4.3.0 - 2019-01-24
5012 This release deprecates HealthCheck.hung_test and disables the associ‐
5013 ated runtime check for tests that ran for more than five minutes. Such
5014 a check is redundant now that we enforce the deadline and max_examples
5015 setting, which can be adjusted independently.
5016
5017 4.2.0 - 2019-01-23
5018 This release adds a new module, hypothesis.extra.lark, which you can
5019 use to generate strings matching a context-free grammar.
5020
5021 In this initial version, only lark-parser EBNF grammars are supported,
5022 by the new hypothesis.extra.lark.from_lark() function.
5023
5024 4.1.2 - 2019-01-23
5025 This patch fixes a very rare overflow bug (issue #1748) which could
5026 raise an InvalidArgument error in complex_numbers() even though the
5027 arguments were valid.
5028
5029 4.1.1 - 2019-01-23
5030 This release makes some improvements to internal code organisation and
5031 documentation and has no impact on behaviour.
5032
5033 4.1.0 - 2019-01-22
5034 This release adds register_random(), which registers random.Random
5035 instances or compatible objects to be seeded and reset by Hypothesis to
5036 ensure that test cases are deterministic.
5037
5038 We still recommend explicitly passing a random.Random instance from
5039 randoms() if possible, but registering a framework-global state for
5040 Hypothesis to manage is better than flaky tests!
5041
5042 4.0.2 - 2019-01-22
5043 This patch fixes issue #1387, where bounded integers() with a very
5044 large range would almost always generate very large numbers. Now, we
5045 usually use the same tuned distribution as unbounded integers().
5046
5047 4.0.1 - 2019-01-16
5048 This release randomizes the order in which the shrinker tries some of
5049 its initial normalization operations. You are unlikely to see much
5050 difference as a result unless your generated examples are very large.
5051 In this case you may see some performance improvements in shrinking.
5052
5053 4.0.0 - 2019-01-14
5054 Welcome to the next major version of Hypothesis!
5055
5056 There are no new features here, as we release those in minor versions.
5057 Instead, 4.0 is a chance for us to remove deprecated features (many
5058 already converted into no-ops), and turn a variety of warnings into
5059 errors.
5060
5061 If you were running on the last version of Hypothesis 3.x without any
5062 Hypothesis deprecation warnings (or using private APIs), this will be a
5063 very boring upgrade. In fact, nothing will change for you at all. Per
5064 our deprecation policy, warnings added in the last six months (after
5065 2018-07-05) have not been converted to errors.
5066
5067 Removals
5068 · hypothesis.extra.datetime has been removed, replaced by the core date
5069 and time strategies.
5070
5071 · hypothesis.extra.fakefactory has been removed, replaced by general
5072 expansion of Hypothesis' strategies and the third-party ecosystem.
5073
5074 · The SQLite example database backend has been removed.
5075
5076 Settings
5077 · The deadline is now enforced by default, rather than just emitting a
5078 warning when the default (200 milliseconds per test case) deadline is
5079 exceeded.
5080
5081 · The database_file setting has been removed; use database.
5082
5083 · The perform_health_check setting has been removed; use sup‐
5084 press_health_check.
5085
5086 · The max_shrinks setting has been removed; use phases to disable
5087 shrinking.
5088
5089 · The min_satisfying_examples, max_iterations, strict, timeout, and
5090 use_coverage settings have been removed without user-configurable
5091 replacements.
5092
5093 Strategies
5094 · The elements argument is now required for collection strategies.
5095
5096 · The average_size argument was a no-op and has been removed.
5097
5098 · Date and time strategies now only accept min_value and max_value for
5099 bounds.
5100
5101 · builds() now requires that the thing to build is passed as the first
5102 positional argument.
5103
5104 · Alphabet validation for text() raises errors, not warnings, as does
5105 category validation for characters().
5106
5107 · The choices() strategy has been removed. Instead, you can use data()
5108 with sampled_from(), so choice(elements) becomes data.draw(sam‐
5109 pled_from(elements)).
5110
5111 · The streaming() strategy has been removed. Instead, you can use
5112 data() and replace iterating over the stream with data.draw() calls.
5113
5114 · sampled_from() and permutations() raise errors instead of warnings if
5115 passed a collection that is not a sequence.
5116
5117 Miscellaneous
5118 · Applying @given to a test function multiple times was really ineffi‐
5119 cient, and now it's also an error.
5120
5121 · Using the .example() method of a strategy (intended for interactive
5122 exploration) within another strategy or a test function always weak‐
5123 ened data generation and broke shrinking, and now it's an error too.
5124
5125 · The HYPOTHESIS_DATABASE_FILE environment variable is no longer sup‐
5126 ported, as the database_file setting has been removed.
5127
5128 · The HYPOTHESIS_VERBOSITY_LEVEL environment variable is no longer sup‐
5129 ported. You can use the --hypothesis-verbosity pytest argument
5130 instead, or write your own setup code using the settings profile sys‐
5131 tem to replace it.
5132
5133 · Using @seed or derandomize=True now forces database=None to ensure
5134 results are in fact reproducible. If database is not None, doing so
5135 also emits a HypothesisWarning.
5136
5137 · Unused exception types have been removed from hypothesis.errors;
5138 namely AbnormalExit, BadData, BadTemplateDraw, DefinitelyNoSuchExam‐
5139 ple, Timeout, and WrongFormat.
5140
5141 3.88.3 - 2019-01-11
5142 This changes the order that the shrinker tries certain operations in
5143 its "emergency" phase which runs late in the process. The new order
5144 should be better at avoiding long stalls where the shrinker is failing
5145 to make progress, which may be helpful if you have difficult to shrink
5146 test cases. However this will not be noticeable in the vast majority
5147 of use cases.
5148
5149 3.88.2 - 2019-01-11
5150 This is a pure refactoring release that extracts some logic from the
5151 core Hypothesis engine into its own class and file. It should have no
5152 user visible impact.
5153
5154 3.88.1 - 2019-01-11
5155 This patch fixes some markup in our documentation.
5156
5157 3.88.0 - 2019-01-10
5158 Introduces hypothesis.stateful.multiple(), which allows rules in rule
5159 based state machines to send multiple results at once to their target
5160 Bundle, or none at all.
5161
5162 3.87.0 - 2019-01-10
5163 This release contains a massive cleanup of the Hypothesis for Django
5164 extra:
5165
5166 · hypothesis.extra.django.models.models() is deprecated in favor of
5167 hypothesis.extra.django.from_model().
5168
5169 · hypothesis.extra.django.models.add_default_field_mapping() is depre‐
5170 cated in favor of hypothesis.extra.django.register_field_strategy().
5171
5172 · from_model() does not infer a strategy for nullable fields or fields
5173 with a default unless passed infer, like builds(). models.models()
5174 would usually but not always infer, and a special default_value
5175 marker object was required to disable inference.
5176
5177 3.86.9 - 2019-01-09
5178 This release improves some internal logic about when a test case in
5179 Hypothesis's internal representation could lead to a valid test case.
5180 In some circumstances this can lead to a significant speed up during
5181 shrinking. It may have some minor negative impact on the quality of
5182 the final result due to certain shrink passes now having access to less
5183 information about test cases in some circumstances, but this should
5184 rarely matter.
5185
5186 3.86.8 - 2019-01-09
5187 This release has no user visible changes but updates our URLs to use
5188 HTTPS.
5189
5190 3.86.7 - 2019-01-08
5191 Hypothesis can now automatically generate values for Django models with
5192 a URLfield, thanks to a new provisional strategy for URLs (issue
5193 #1388).
5194
5195 3.86.6 - 2019-01-07
5196 This release is a pure refactoring that extracts some internal code
5197 into its own file. It should have no user visible effect.
5198
5199 3.86.5 - 2019-01-06
5200 This is a docs-only patch, which fixes some typos and removes a few
5201 hyperlinks for deprecated features.
5202
5203 3.86.4 - 2019-01-04
5204 This release changes the order in which the shrinker tries to delete
5205 data. For large and slow tests this may significantly improve the per‐
5206 formance of shrinking.
5207
5208 3.86.3 - 2019-01-04
5209 This release fixes a bug where certain places Hypothesis internal
5210 errors could be raised during shrinking when a user exception occurred
5211 that suppressed an exception Hypothesis uses internally in its genera‐
5212 tion.
5213
5214 The two known ways to trigger this problem were:
5215
5216 · Errors raised in stateful tests' teardown function.
5217
5218 · Errors raised in finally blocks that wrapped a call to data.draw.
5219
5220 These cases will now be handled correctly.
5221
5222 3.86.2 - 2019-01-04
5223 This patch is a docs-only change to fix a broken hyperlink.
5224
5225 3.86.1 - 2019-01-04
5226 This patch fixes issue #1732, where integers() would always return long
5227 values on Python 2.
5228
5229 3.86.0 - 2019-01-03
5230 This release ensures that infinite numbers are never generated by
5231 floats() with allow_infinity=False, which could previously happen in
5232 some cases where one bound was also provided.
5233
5234 The trivially inconsistent min_value=inf, allow_infinity=False now
5235 raises an InvalidArgumentError, as does the inverse with max_value.
5236 You can still use just(inf) to generate inf without violating other
5237 constraints.
5238
5239 3.85.3 - 2019-01-02
5240 Happy new year everyone! This release has no user visible changes but
5241 updates our copyright headers to include 2019.
5242
5243 3.85.2 - 2018-12-31
5244 This release makes a small change to the way the shrinker works. You
5245 may see some improvements to speed of shrinking on especially large and
5246 hard to shrink examples, but most users are unlikely to see much dif‐
5247 ference.
5248
5249 3.85.1 - 2018-12-30
5250 This patch fixes issue #1700, where a line that contained a Unicode
5251 character before a lambda definition would cause an internal exception.
5252
5253 3.85.0 - 2018-12-29
5254 Introduces the hypothesis.stateful.consumes() function. When defining a
5255 rule in stateful testing, it can be used to mark bundles from which
5256 values should be consumed, i. e. removed after use in the rule. This
5257 has been proposed in issue #136.
5258
5259 Thanks to Jochen Müller for this long-awaited feature.
5260
5261 3.84.6 - 2018-12-28
5262 This patch makes a small internal change to fix an issue in Hypothe‐
5263 sis's own coverage tests (issue #1718).
5264
5265 There is no user-visible change.
5266
5267 3.84.5 - 2018-12-21
5268 This patch refactors the hypothesis.strategies module, so that private
5269 names should no longer appear in tab-completion lists. We previously
5270 relied on __all__ for this, but not all editors respect it.
5271
5272 3.84.4 - 2018-12-21
5273 This is a follow-up patch to ensure that the deprecation date is auto‐
5274 matically recorded for any new deprecations. There is no user-visible
5275 effect.
5276
5277 3.84.3 - 2018-12-20
5278 This patch updates the Hypothesis pytest plugin to avoid a recently
5279 deprecated hook interface. There is no user-visible change.
5280
5281 3.84.2 - 2018-12-19
5282 This patch fixes the internals for integers() with one bound. Values
5283 from this strategy now always shrink towards zero instead of towards
5284 the bound, and should shrink much more efficiently too. On Python 2,
5285 providing a bound incorrectly excluded long integers, which can now be
5286 generated.
5287
5288 3.84.1 - 2018-12-18
5289 This patch adds information about when features were deprecated, but
5290 this is only recorded internally and has no user-visible effect.
5291
5292 3.84.0 - 2018-12-18
5293 This release changes the stateful testing backend from find() to use
5294 @given (issue #1300). This doesn't change how you create stateful
5295 tests, but does make them run more like other Hypothesis tests.
5296
5297 @reproduce_failure and @seed now work for stateful tests.
5298
5299 Stateful tests now respect the deadline and suppress_health_check set‐
5300 tings, though they are disabled by default. You can enable them by
5301 using @settings(...) as a class decorator with whatever arguments you
5302 prefer.
5303
5304 3.83.2 - 2018-12-17
5305 Hypothesis has adopted Black as our code formatter (issue #1686).
5306 There are no functional changes to the source, but it's prettier!
5307
5308 3.83.1 - 2018-12-13
5309 This patch increases the variety of examples generated by from_type().
5310
5311 3.83.0 - 2018-12-12
5312 Our pytest plugin now warns you when strategy functions have been col‐
5313 lected as tests, which may happen when e.g. using the @composite deco‐
5314 rator when you should be using @given(st.data()) for inline draws.
5315 Such functions always pass when treated as tests, because the lazy cre‐
5316 ation of strategies mean that the function body is never actually exe‐
5317 cuted!
5318
5319 3.82.6 - 2018-12-11
5320 Hypothesis can now show statistics when running under pytest-xdist.
5321 Previously, statistics were only reported when all tests were run in a
5322 single process (issue #700).
5323
5324 3.82.5 - 2018-12-08
5325 This patch fixes issue #1667, where passing bounds of Numpy dtype int64
5326 to integers() could cause errors on Python 3 due to internal rounding.
5327
5328 3.82.4 - 2018-12-08
5329 Hypothesis now seeds and resets the global state of np.random for each
5330 test case, to ensure that tests are reproducible.
5331
5332 This matches and complements the existing handling of the python:random
5333 module - Numpy simply maintains an independent PRNG for performance
5334 reasons.
5335
5336 3.82.3 - 2018-12-08
5337 This is a no-op release to add the new Framework :: Hypothesis trove
5338 classifier to hypothesis on PyPI.
5339
5340 You can use it as a filter to find Hypothesis-related packages such as
5341 extensions as they add the tag over the coming weeks, or simply visit
5342 our curated list.
5343
5344 3.82.2 - 2018-12-08
5345 The Hypothesis for Pandas extension is now listed in setup.py, so you
5346 can pip install hypothesis[pandas]. Thanks to jmshi for this contribu‐
5347 tion.
5348
5349 3.82.1 - 2018-10-29
5350 This patch fixes from_type() on Python 2 for classes where cls.__init__
5351 is object.__init__. Thanks to ccxcz for reporting issue #1656.
5352
5353 3.82.0 - 2018-10-29
5354 The alphabet argument for text() now uses its default value of charac‐
5355 ters(blacklist_categories=('Cs',)) directly, instead of hiding that
5356 behind alphabet=None and replacing it within the function. Passing
5357 None is therefore deprecated.
5358
5359 3.81.0 - 2018-10-27
5360 GenericStateMachine and RuleBasedStateMachine now raise an explicit
5361 error when instances of settings are assigned to the classes' settings
5362 attribute, which is a no-op (issue #1643). Instead assign to SomeS‐
5363 tateMachine.TestCase.settings, or use @settings(...) as a class decora‐
5364 tor to handle this automatically.
5365
5366 3.80.0 - 2018-10-25
5367 Since version 3.68.0, arrays() checks that values drawn from the ele‐
5368 ments and fill strategies can be safely cast to the dtype of the array,
5369 and emits a warning otherwise.
5370
5371 This release expands the checks to cover overflow for finite complex64
5372 elements and string truncation caused by too-long elements or trailing
5373 null characters (issue #1591).
5374
5375 3.79.4 - 2018-10-25
5376 Tests using @given now shrink errors raised from pytest helper func‐
5377 tions, instead of reporting the first example found.
5378
5379 This was previously fixed in version 3.56.0, but only for stateful
5380 testing.
5381
5382 3.79.3 - 2018-10-23
5383 Traceback elision is now disabled on Python 2, to avoid an import-time
5384 python:SyntaxError under Python < 2.7.9 (Python: bpo-21591, Hypothesis
5385 3.79.2: issue #1648).
5386
5387 We encourage all users to upgrade to Python 3 before the end of 2019.
5388
5389 3.79.2 - 2018-10-23
5390 This patch shortens tracebacks from Hypothesis, so you can see exactly
5391 happened in your code without having to skip over irrelevant details
5392 about our internals (issue #848).
5393
5394 In the example test (see pull request #1582), this reduces tracebacks
5395 from nine frames to just three - and for a test with multiple errors,
5396 from seven frames per error to just one!
5397
5398 If you do want to see the internal details, you can disable frame eli‐
5399 sion by setting verbosity to debug.
5400
5401 3.79.1 - 2018-10-22
5402 The abstract number classes Number, Complex, Real, Rational, and Inte‐
5403 gral are now supported by the from_type() strategy. Previously, you
5404 would have to use register_type_strategy() before they could be
5405 resolved (issue #1636)
5406
5407 3.79.0 - 2018-10-18
5408 This release adds a CLI flag for verbosity --hypothesis-verbosity to
5409 the Hypothesis pytest plugin, applied after loading the profile speci‐
5410 fied by --hypothesis-profile. Valid options are the names of verbosity
5411 settings, quiet, normal, verbose or debug.
5412
5413 Thanks to Bex Dunn for writing this patch at the PyCon Australia
5414 sprints!
5415
5416 The pytest header now correctly reports the current profile if
5417 --hypothesis-profile has been used.
5418
5419 Thanks to Mathieu Paturel for the contribution at the Canberra Python
5420 Hacktoberfest.
5421
5422 3.78.0 - 2018-10-16
5423 This release has deprecated the generation of integers, floats and
5424 fractions when the conversion of the upper and/ or lower bound is not
5425 100% exact, e.g. when an integer gets passed a bound that is not a
5426 whole number. (issue #1625)
5427
5428 Thanks to Felix Grünewald for this patch during Hacktoberfest 2018.
5429
5430 3.77.0 - 2018-10-16
5431 This minor release adds functionality to settings allowing it to be
5432 used as a decorator on RuleBasedStateMachine and GenericStateMachine.
5433
5434 Thanks to Tyler Nickerson for this feature in #hacktoberfest!
5435
5436 3.76.1 - 2018-10-16
5437 This patch fixes some warnings added by recent releases of pydocstyle
5438 and mypy.
5439
5440 3.76.0 - 2018-10-11
5441 This release deprecates using floats for min_size and max_size.
5442
5443 The type hint for average_size arguments has been changed from
5444 Optional[int] to None, because non-None values are always ignored and
5445 deprecated.
5446
5447 3.75.4 - 2018-10-10
5448 This patch adds more internal comments to the core engine's
5449 sequence-length shrinker. There should be no user-visible change.
5450
5451 3.75.3 - 2018-10-09
5452 This patch adds additional comments to some of the core engine's inter‐
5453 nal data structures. There is no user-visible change.
5454
5455 3.75.2 - 2018-10-09
5456 This patch avoids caching a trivial case, fixing issue #493.
5457
5458 3.75.1 - 2018-10-09
5459 This patch fixes a broken link in a docstring. Thanks to Benjamin Lee
5460 for this contribution!
5461
5462 3.75.0 - 2018-10-08
5463 This release deprecates the use of min_size=None, setting the default
5464 min_size to 0 (issue #1618).
5465
5466 3.74.3 - 2018-10-08
5467 This patch makes some small internal changes to comply with a new lint
5468 setting in the build. There should be no user-visible change.
5469
5470 3.74.2 - 2018-10-03
5471 This patch fixes issue #1153, where time spent reifying a strategy was
5472 also counted in the time spent generating the first example. Strate‐
5473 gies are now fully constructed and validated before the timer is
5474 started.
5475
5476 3.74.1 - 2018-10-03
5477 This patch fixes some broken formatting and links in the documentation.
5478
5479 3.74.0 - 2018-10-01
5480 This release checks that the value of the print_blob setting is a
5481 PrintSettings instance.
5482
5483 Being able to specify a boolean value was not intended, and is now dep‐
5484 recated. In addition, specifying True will now cause the blob to
5485 always be printed, instead of causing it to be suppressed.
5486
5487 Specifying any value that is not a PrintSettings or a boolean is now an
5488 error.
5489
5490 3.73.5 - 2018-10-01
5491 Changes the documentation for hypothesis.strategies.datetimes, hypothe‐
5492 sis.strategies.dates, hypothesis.strategies.times to use the new param‐
5493 eter names min_value and max_value instead of the deprecated names
5494
5495 3.73.4 - 2018-09-30
5496 This patch ensures that Hypothesis deprecation warnings display the
5497 code that emitted them when you're not running in -Werror mode (issue
5498 #652).
5499
5500 3.73.3 - 2018-09-27
5501 Tracebacks involving @composite are now slightly shorter due to some
5502 internal refactoring.
5503
5504 3.73.2 - 2018-09-26
5505 This patch fixes errors in the internal comments for one of the
5506 shrinker passes. There is no user-visible change.
5507
5508 3.73.1 - 2018-09-25
5509 This patch substantially improves the distribution of data generated
5510 with recursive(), and fixes a rare internal error (issue #1502).
5511
5512 3.73.0 - 2018-09-24
5513 This release adds the fulfill() function, which is designed for testing
5514 code that uses dpcontracts 0.4 or later for input validation. This
5515 provides some syntactic sugar around use of assume(), to automatically
5516 filter out and retry calls that cause a precondition check to fail (‐
5517 issue #1474).
5518
5519 3.72.0 - 2018-09-24
5520 This release makes setting attributes of the hypothesis.settings class
5521 an explicit error. This has never had any effect, but could mislead
5522 users who confused it with the current settings instance hypothe‐
5523 sis.settings.default (which is also immutable). You can change the
5524 global settings with settings profiles.
5525
5526 3.71.11 - 2018-09-24
5527 This patch factors out some common code in the shrinker for iterating
5528 over pairs of data blocks. There should be no user-visible change.
5529
5530 3.71.10 - 2018-09-18
5531 This patch allows from_type() to handle the empty tuple type, typ‐
5532 ing.Tuple[()].
5533
5534 3.71.9 - 2018-09-17
5535 This patch updates some internal comments for mypy. There is no
5536 user-visible effect, even for Mypy users.
5537
5538 3.71.8 - 2018-09-17
5539 This patch fixes a rare bug that would cause a particular shrinker pass
5540 to raise an IndexError, if a shrink improvement changed the underlying
5541 data in an unexpected way.
5542
5543 3.71.7 - 2018-09-17
5544 This release fixes the broken cross-references in our docs, and adds a
5545 CI check so we don't add new ones.
5546
5547 3.71.6 - 2018-09-16
5548 This patch fixes two bugs (issue #944 and issue #1521), where messages
5549 about @seed did not check the current verbosity setting, and the wrong
5550 settings were active while executing explicit examples.
5551
5552 3.71.5 - 2018-09-15
5553 This patch fixes a DeprecationWarning added in Python 3.8 (issue
5554 #1576).
5555
5556 Thanks to tirkarthi for this contribution!
5557
5558 3.71.4 - 2018-09-14
5559 This is a no-op release, which implements automatic DOI minting and
5560 code archival of Hypothesis via Zenodo. Thanks to CERN and the EU Hori‐
5561 zon 2020 programme for providing this service!
5562
5563 Check our CITATION file for details, or head right on over to
5564 doi.org/10.5281/zenodo.1412597
5565
5566 3.71.3 - 2018-09-10
5567 This release adds the test name to some deprecation warnings, for eas‐
5568 ier debugging.
5569
5570 Thanks to Sanyam Khurana for the patch!
5571
5572 3.71.2 - 2018-09-10
5573 This release makes Hypothesis's memory usage substantially smaller for
5574 tests with many examples, by bounding the number of past examples it
5575 keeps around.
5576
5577 You will not see much difference unless you are running tests with
5578 max_examples set to well over 1000, but if you do have such tests then
5579 you should see memory usage mostly plateau where previously it would
5580 have grown linearly with time.
5581
5582 3.71.1 - 2018-09-09
5583 This patch adds internal comments to some tree traversals in the core
5584 engine. There is no user-visible change.
5585
5586 3.71.0 - 2018-09-08
5587 This release deprecates the coverage-guided testing functionality, as
5588 it has proven brittle and does not really pull its weight.
5589
5590 We intend to replace it with something more useful in the future, but
5591 the feature in its current form does not seem to be worth the cost of
5592 using, and whatever replaces it will likely look very different.
5593
5594 3.70.4 - 2018-09-08
5595 This patch changes the behaviour of reproduce_failure() so that blobs
5596 are only printed in quiet mode when the print_blob setting is set to
5597 ALWAYS.
5598
5599 Thanks to Cameron McGill for writing this patch at the PyCon Australia
5600 sprints!
5601
5602 3.70.3 - 2018-09-03
5603 This patch removes some unnecessary code from the internals. There is
5604 no user-visible change.
5605
5606 3.70.2 - 2018-09-03
5607 This patch fixes an internal bug where a corrupted argument to @repro‐
5608 duce_failure could raise the wrong type of error. Thanks again to
5609 Paweł T. Jochym, who maintains Hypothesis on conda-forge and consis‐
5610 tently provides excellent bug reports including issue #1558.
5611
5612 3.70.1 - 2018-09-03
5613 This patch updates hypothesis to report its version and settings when
5614 run with pytest. (issue #1223).
5615
5616 Thanks to Jack Massey for this feature.
5617
5618 3.70.0 - 2018-09-01
5619 This release adds a fullmatch argument to from_regex(). When full‐
5620 match=True, the whole example will match the regex pattern as for
5621 python:re.fullmatch().
5622
5623 Thanks to Jakub Nabaglo for writing this patch at the PyCon Australia
5624 sprints!
5625
5626 3.69.12 - 2018-08-30
5627 This release reverts the changes to logging handling in 3.69.11, which
5628 broke test that use the pytest caplog fixture internally because all
5629 logging was disabled (issue #1546).
5630
5631 3.69.11 - 2018-08-29
5632 This patch will hide all logging messages produced by test cases before
5633 the final, minimal, failing test case (issue #356).
5634
5635 Thanks to Gary Donovan for writing this patch at the PyCon Australia
5636 sprints!
5637
5638 3.69.10 - 2018-08-29
5639 This patch fixes a bug that prevents coverage from reporting unexecuted
5640 Python files (issue #1085).
5641
5642 Thanks to Gary Donovan for writing this patch at the PyCon Australia
5643 sprints!
5644
5645 3.69.9 - 2018-08-28
5646 This patch improves the packaging of the Python package by adding
5647 LICENSE.txt to the sdist (issue #1311), clarifying the minimum sup‐
5648 ported versions of pytz and dateutil (issue #1383), and adds keywords
5649 to the metadata (issue #1520).
5650
5651 Thanks to Graham Williamson for writing this patch at the PyCon Aus‐
5652 tralia sprints!
5653
5654 3.69.8 - 2018-08-28
5655 This is an internal change which replaces pickle with json to prevent
5656 possible security issues.
5657
5658 Thanks to Vidya Rani D G for writing this patch at the PyCon Australia
5659 sprints!
5660
5661 3.69.7 - 2018-08-28
5662 This patch ensures that note() prints the note for every test case when
5663 the verbosity setting is Verbosity.verbose. At normal verbosity it
5664 only prints from the final test case.
5665
5666 Thanks to Tom McDermott for writing this patch at the PyCon Australia
5667 sprints!
5668
5669 3.69.6 - 2018-08-27
5670 This patch improves the testing of some internal caching. It should
5671 have no user-visible effect.
5672
5673 3.69.5 - 2018-08-27
5674 This change performs a small rename and refactoring in the core engine.
5675 There is no user-visible change.
5676
5677 3.69.4 - 2018-08-27
5678 This change improves the core engine's ability to avoid unnecessary
5679 work, by consulting its cache of previously-tried inputs in more cases.
5680
5681 3.69.3 - 2018-08-27
5682 This patch handles passing an empty python:enum.Enum to from_type() by
5683 returning nothing(), instead of raising an internal python:Assertion‐
5684 Error.
5685
5686 Thanks to Paul Amazona for writing this patch at the PyCon Australia
5687 sprints!
5688
5689 3.69.2 - 2018-08-23
5690 This patch fixes a small mistake in an internal comment. There is no
5691 user-visible change.
5692
5693 3.69.1 - 2018-08-21
5694 This change fixes a small bug in how the core engine consults its cache
5695 of previously-tried inputs. There is unlikely to be any user-visible
5696 change.
5697
5698 3.69.0 - 2018-08-20
5699 This release improves argument validation for stateful testing.
5700
5701 · If the target or targets of a rule() are invalid, we now raise a use‐
5702 ful validation error rather than an internal exception.
5703
5704 · Passing both the target and targets arguments is deprecated - append
5705 the target bundle to the targets tuple of bundles instead.
5706
5707 · Passing the name of a Bundle rather than the Bundle itself is also
5708 deprecated.
5709
5710 3.68.3 - 2018-08-20
5711 This is a docs-only patch, fixing some typos and formatting issues.
5712
5713 3.68.2 - 2018-08-19
5714 This change fixes a small bug in how the core engine caches the results
5715 of previously-tried inputs. The effect is unlikely to be noticeable,
5716 but it might avoid unnecesary work in some cases.
5717
5718 3.68.1 - 2018-08-18
5719 This patch documents the from_dtype() function, which infers a strategy
5720 for numpy:numpy.dtypes. This is used in arrays(), but can also be used
5721 directly when creating e.g. Pandas objects.
5722
5723 3.68.0 - 2018-08-15
5724 arrays() now checks that integer and float values drawn from elements
5725 and fill strategies can be safely cast to the dtype of the array, and
5726 emits a warning otherwise (issue #1385).
5727
5728 Elements in the resulting array could previously violate constraints on
5729 the elements strategy due to floating-point overflow or truncation of
5730 integers to fit smaller types.
5731
5732 3.67.1 - 2018-08-14
5733 This release contains a tiny refactoring of the internals. There is no
5734 user-visible change.
5735
5736 3.67.0 - 2018-08-10
5737 This release adds a width argument to floats(), to generate lower-pre‐
5738 cision floating point numbers for e.g. Numpy arrays.
5739
5740 The generated examples are always instances of Python's native float
5741 type, which is 64bit, but passing width=32 will ensure that all values
5742 can be exactly represented as 32bit floats. This can be useful to
5743 avoid overflow (to +/- infinity), and for efficiency of generation and
5744 shrinking.
5745
5746 Half-precision floats (width=16) are also supported, but require Numpy
5747 if you are running Python 3.5 or earlier.
5748
5749 3.66.33 - 2018-08-10
5750 This release fixes a bug in floats(), where setting allow_infin‐
5751 ity=False and exactly one of min_value and max_value would allow infi‐
5752 nite values to be generated.
5753
5754 3.66.32 - 2018-08-09
5755 This release adds type hints to the example() and seed() decorators,
5756 and fixes the type hint on register_type_strategy(). The second argu‐
5757 ment to register_type_strategy() must either be a SearchStrategy, or a
5758 callable which takes a type and returns a SearchStrategy.
5759
5760 3.66.31 - 2018-08-08
5761 Another set of changes designed to improve the performance of shrinking
5762 on large examples. In particular the shrinker should now spend consid‐
5763 erably less time running useless shrinks.
5764
5765 3.66.30 - 2018-08-06
5766 "Bug fixes and performance improvements".
5767
5768 This release is a fairly major overhaul of the shrinker designed to
5769 improve its behaviour on large examples, especially around stateful
5770 testing. You should hopefully see shrinking become much faster, with
5771 little to no quality degradation (in some cases quality may even
5772 improve).
5773
5774 3.66.29 - 2018-08-05
5775 This release fixes two very minor bugs in the core engine:
5776
5777 · it fixes a corner case that was missing in 3.66.28, which should
5778 cause shrinking to work slightly better.
5779
5780 · it fixes some logic for how shrinking interacts with the database
5781 that was causing Hypothesis to be insufficiently aggressive about
5782 clearing out old keys.
5783
5784 3.66.28 - 2018-08-05
5785 This release improves how Hypothesis handles reducing the size of inte‐
5786 gers' representation. This change should mostly be invisible as it's
5787 purely about the underlying representation and not the generated value,
5788 but it may result in some improvements to shrink performance.
5789
5790 3.66.27 - 2018-08-05
5791 This release changes the order in which Hypothesis chooses parts of the
5792 test case to shrink. For typical usage this should be a significant
5793 performance improvement on large examples. It is unlikely to have a
5794 major impact on example quality, but where it does change the result it
5795 should usually be an improvement.
5796
5797 3.66.26 - 2018-08-05
5798 This release improves the debugging information that the shrinker emits
5799 about the operations it performs, giving better summary statistics
5800 about which passes resulted in test executions and whether they were
5801 successful.
5802
5803 3.66.25 - 2018-08-05
5804 This release fixes several bugs that were introduced to the shrinker in
5805 3.66.24 which would have caused it to behave significantly less well
5806 than advertised. With any luck you should actually see the promised
5807 benefits now.
5808
5809 3.66.24 - 2018-08-03
5810 This release changes how Hypothesis deletes data when shrinking in
5811 order to better handle deletion of large numbers of contiguous
5812 sequences. Most tests should see little change, but this will hopefully
5813 provide a significant speed up for stateful testing.
5814
5815 3.66.23 - 2018-08-02
5816 This release makes some internal changes to enable further improvements
5817 to the shrinker. You may see some changes in the final shrunk examples,
5818 but they are unlikely to be significant.
5819
5820 3.66.22 - 2018-08-01
5821 This release adds some more internal caching to the shrinker. This
5822 should cause a significant speed up for shrinking, especially for
5823 stateful testing and large example sizes.
5824
5825 3.66.21 - 2018-08-01
5826 This patch is for downstream packagers - our tests now pass under
5827 pytest 3.7.0 (released 2018-07-30). There are no changes to the source
5828 of Hypothesis itself.
5829
5830 3.66.20 - 2018-08-01
5831 This release removes some functionality from the shrinker that was tak‐
5832 ing a considerable amount of time and does not appear to be useful any
5833 more due to a number of quality improvements in the shrinker.
5834
5835 You may see some degradation in shrink quality as a result of this, but
5836 mostly shrinking should just get much faster.
5837
5838 3.66.19 - 2018-08-01
5839 This release slightly changes the format of some debugging information
5840 emitted during shrinking, and refactors some of the internal interfaces
5841 around that.
5842
5843 3.66.18 - 2018-07-31
5844 This release is a very small internal refactoring which should have no
5845 user visible impact.
5846
5847 3.66.17 - 2018-07-31
5848 This release fixes a bug that could cause an IndexError to be raised
5849 from inside Hypothesis during shrinking. It is likely that it was
5850 impossible to trigger this bug in practice - it was only made visible
5851 by some currently unreleased work.
5852
5853 3.66.16 - 2018-07-31
5854 This release is a very small internal refactoring which should have no
5855 user visible impact.
5856
5857 3.66.15 - 2018-07-31
5858 This release makes Hypothesis's shrinking faster by removing some
5859 redundant work that it does when minimizing values in its internal rep‐
5860 resentation.
5861
5862 3.66.14 - 2018-07-30
5863 This release expands the deprecation of timeout from 3.16.0 to also
5864 emit the deprecation warning in find or stateful testing.
5865
5866 3.66.13 - 2018-07-30
5867 This release adds an additional shrink pass that is able to reduce the
5868 size of examples in some cases where the transformation is non-obvious.
5869 In particular this will improve the quality of some examples which
5870 would have regressed in 3.66.12.
5871
5872 3.66.12 - 2018-07-28
5873 This release changes how we group data together for shrinking. It
5874 should result in improved shrinker performance, especially in stateful
5875 testing.
5876
5877 3.66.11 - 2018-07-28
5878 This patch modifies how which rule to run is selected during rule based
5879 stateful testing. This should result in a slight performance increase
5880 during generation and a significant performance and quality improvement
5881 when shrinking.
5882
5883 As a result of this change, some state machines which would previously
5884 have thrown an InvalidDefinition are no longer detected as invalid.
5885
5886 3.66.10 - 2018-07-28
5887 This release weakens some minor functionality in the shrinker that had
5888 only modest benefit and made its behaviour much harder to reason about.
5889
5890 This is unlikely to have much user visible effect, but it is possible
5891 that in some cases shrinking may get slightly slower. It is primarily
5892 to make it easier to work on the shrinker and pave the way for future
5893 work.
5894
5895 3.66.9 - 2018-07-26
5896 This release improves the information that Hypothesis emits about its
5897 shrinking when verbosity is set to debug.
5898
5899 3.66.8 - 2018-07-24
5900 This patch includes some minor fixes in the documentation, and updates
5901 the minimum version of pytest to 3.0 (released August 2016).
5902
5903 3.66.7 - 2018-07-24
5904 This release fixes a bug where difficult to shrink tests could some‐
5905 times trigger an internal assertion error inside the shrinker.
5906
5907 3.66.6 - 2018-07-23
5908 This patch ensures that Hypothesis fully supports Python 3.7, by
5909 upgrading from_type() (issue #1264) and fixing some minor issues in our
5910 test suite (issue #1148).
5911
5912 3.66.5 - 2018-07-22
5913 This patch fixes the online docs for various extras, by ensuring that
5914 their dependencies are installed on readthedocs.io (issue #1326).
5915
5916 3.66.4 - 2018-07-20
5917 This release improves the shrinker's ability to reorder examples.
5918
5919 For example, consider the following test:
5920
5921 import hypothesis.strategies as st
5922 from hypothesis import given
5923
5924 @given(st.text(), st.text())
5925 def test_non_equal(x, y):
5926 assert x != y
5927
5928 Previously this could have failed with either of x="", y="0" or x="0",
5929 y="". Now it should always fail with x="", y="0".
5930
5931 This will allow the shrinker to produce more consistent results, espe‐
5932 cially in cases where test cases contain some ordered collection whose
5933 actual order does not matter.
5934
5935 3.66.3 - 2018-07-20
5936 This patch fixes inference in the builds() strategy with subtypes of
5937 python:typing.NamedTuple, where the __init__ method is not useful for
5938 introspection. We now use the field types instead - thanks to James
5939 Uther for identifying this bug.
5940
5941 3.66.2 - 2018-07-19
5942 This release improves the shrinker's ability to handle situations where
5943 there is an additive constraint between two values.
5944
5945 For example, consider the following test:
5946
5947 import hypothesis.strategies as st
5948 from hypothesis import given
5949
5950 @given(st.integers(), st.integers())
5951 def test_does_not_exceed_100(m, n):
5952 assert m + n < 100
5953
5954 Previously this could have failed with almost any pair (m, n) with 0 <=
5955 m <= n and m + n == 100. Now it should almost always fail with m=0,
5956 n=100.
5957
5958 This is a relatively niche specialisation, but can be useful in situa‐
5959 tions where e.g. a bug is triggered by an integer overflow.
5960
5961 3.66.1 - 2018-07-09
5962 This patch fixes a rare bug where an incorrect percentage drawtime
5963 could be displayed for a test, when the system clock was changed during
5964 a test running under Python 2 (we use python:time.monotonic() where it
5965 is available to avoid such problems). It also fixes a possible
5966 zero-division error that can occur when the underlying C library dou‐
5967 ble-rounds an intermediate value in python:math.fsum() and gets the
5968 least significant bit wrong.
5969
5970 3.66.0 - 2018-07-05
5971 This release improves validation of the alphabet argument to the text()
5972 strategy. The following misuses are now deprecated, and will be an
5973 error in a future version:
5974
5975 · passing an unordered collection (such as set('abc')), which violates
5976 invariants about shrinking and reproducibility
5977
5978 · passing an alphabet sequence with elements that are not strings
5979
5980 · passing an alphabet sequence with elements that are not of length
5981 one, which violates any size constraints that may apply
5982
5983 Thanks to Sushobhit for adding these warnings (issue #1329).
5984
5985 3.65.3 - 2018-07-04
5986 This release fixes a mostly theoretical bug where certain usage of the
5987 internal API could trigger an assertion error inside Hypothesis. It is
5988 unlikely that this problem is even possible to trigger through the pub‐
5989 lic API.
5990
5991 3.65.2 - 2018-07-04
5992 This release fixes dependency information for coverage. Previously
5993 Hypothesis would allow installing coverage with any version, but it
5994 only works with coverage 4.0 or later.
5995
5996 We now specify the correct metadata in our setup.py, so Hypothesis will
5997 only allow installation with compatible versions of coverage.
5998
5999 3.65.1 - 2018-07-03
6000 This patch ensures that stateful tests which raise an error from a
6001 pytest helper still print the sequence of steps taken to reach that
6002 point (issue #1372). This reporting was previously broken because the
6003 helpers inherit directly from python:BaseException, and therefore
6004 require special handling to catch without breaking e.g. the use of
6005 ctrl-C to quit the test.
6006
6007 3.65.0 - 2018-06-30
6008 This release deprecates the max_shrinks setting in favor of an internal
6009 heuristic. If you need to avoid shrinking examples, use the phases
6010 setting instead. (issue #1235)
6011
6012 3.64.2 - 2018-06-27
6013 This release fixes a bug where an internal assertion error could some‐
6014 times be triggered while shrinking a failing test.
6015
6016 3.64.1 - 2018-06-27
6017 This patch fixes type-checking errors in our vendored pretty-printer,
6018 which were ignored by our mypy config but visible for anyone else
6019 (whoops). Thanks to Pi Delport for reporting issue #1359 so promptly.
6020
6021 3.64.0 - 2018-06-26
6022 This release adds an interface which can be used to insert a wrapper
6023 between the original test function and @given (issue #1257). This will
6024 be particularly useful for test runner extensions such as pytest-trio,
6025 but is not recommended for direct use by other users of Hypothesis.
6026
6027 3.63.0 - 2018-06-26
6028 This release adds a new mechanism to infer strategies for classes
6029 defined using attrs, based on the the type, converter, or validator of
6030 each attribute. This inference is now built in to builds() and
6031 from_type().
6032
6033 On Python 2, from_type() no longer generates instances of int when
6034 passed long, or vice-versa.
6035
6036 3.62.0 - 2018-06-26
6037 This release adds PEP 484 type hints to Hypothesis on a provisional
6038 basis, using the comment-based syntax for Python 2 compatibility. You
6039 can read more about our type hints here.
6040
6041 It also adds the py.typed marker specified in PEP 561. After you pip
6042 install hypothesis, mypy 0.590 or later will therefore type-check your
6043 use of our public interface!
6044
6045 3.61.0 - 2018-06-24
6046 This release deprecates the use of settings as a context manager, the
6047 use of which is somewhat ambiguous.
6048
6049 Users should define settings with global state or with the @set‐
6050 tings(...) decorator.
6051
6052 3.60.1 - 2018-06-20
6053 Fixed a bug in generating an instance of a Django model from a strategy
6054 where the primary key is generated as part of the strategy. See details
6055 here.
6056
6057 Thanks to Tim Martin for this contribution.
6058
6059 3.60.0 - 2018-06-20
6060 This release adds the @initialize decorator for stateful testing (orig‐
6061 inally discussed in issue #1216). All @initialize rules will be called
6062 once each in an arbitrary order before any normal rule is called.
6063
6064 3.59.3 - 2018-06-19
6065 This is a no-op release to take into account some changes to the
6066 release process. It should have no user visible effect.
6067
6068 3.59.2 - 2018-06-18
6069 This adds support for partially sorting examples which cannot be fully
6070 sorted. For example, [5, 4, 3, 2, 1, 0] with a constraint that the
6071 first element needs to be larger than the last becomes [1, 2, 3, 4, 5,
6072 0].
6073
6074 Thanks to Luke for contributing.
6075
6076 3.59.1 - 2018-06-16
6077 This patch uses python:random.getstate() and python:random.setstate()
6078 to restore the PRNG state after @given runs deterministic tests. With‐
6079 out restoring state, you might have noticed problems such as issue
6080 #1266. The fix also applies to stateful testing (issue #702).
6081
6082 3.59.0 - 2018-06-14
6083 This release adds the emails() strategy, which generates unicode
6084 strings representing an email address.
6085
6086 Thanks to Sushobhit for moving this to the public API (issue #162).
6087
6088 3.58.1 - 2018-06-13
6089 This improves the shrinker. It can now reorder examples: 3 1 2 becomes
6090 1 2 3.
6091
6092 Thanks to Luke for contributing.
6093
6094 3.58.0 - 2018-06-13
6095 This adds a new extra timezones() strategy that generates dateutil
6096 timezones.
6097
6098 Thanks to Conrad for contributing.
6099
6100 3.57.0 - 2018-05-20
6101 Using an unordered collection with the permutations() strategy has been
6102 deprecated because the order in which e.g. a set shrinks is arbitrary.
6103 This may cause different results between runs.
6104
6105 3.56.10 - 2018-05-16
6106 This release makes hypothesis.settings.define_setting a private method,
6107 which has the effect of hiding it from the documentation.
6108
6109 3.56.9 - 2018-05-11
6110 This is another release with no functionality changes as part of
6111 changes to Hypothesis's new release tagging scheme.
6112
6113 3.56.8 - 2018-05-10
6114 This is a release with no functionality changes that moves Hypothesis
6115 over to a new release tagging scheme.
6116
6117 3.56.7 - 2018-05-10
6118 This release provides a performance improvement for most tests, but in
6119 particular users of sampled_from() who don't have numpy installed
6120 should see a significant performance improvement.
6121
6122 3.56.6 - 2018-05-09
6123 This patch contains further internal work to support Mypy. There are
6124 no user-visible changes... yet.
6125
6126 3.56.5 - 2018-04-22
6127 This patch contains some internal refactoring to run mypy in CI. There
6128 are no user-visible changes.
6129
6130 3.56.4 - 2018-04-21
6131 This release involves some very minor internal clean up and should have
6132 no user visible effect at all.
6133
6134 3.56.3 - 2018-04-20
6135 This release fixes a problem introduced in 3.56.0 where setting the
6136 hypothesis home directory (through currently undocumented means) would
6137 no longer result in the default database location living in the new
6138 home directory.
6139
6140 3.56.2 - 2018-04-20
6141 This release fixes a problem introduced in 3.56.0 where setting
6142 max_examples to 1 would result in tests failing with Unsatisfiable.
6143 This problem could also occur in other harder to trigger circumstances
6144 (e.g. by setting it to a low value, having a hard to satisfy assump‐
6145 tion, and disabling health checks).
6146
6147 3.56.1 - 2018-04-20
6148 This release fixes a problem that was introduced in 3.56.0: Use of the
6149 HYPOTHESIS_VERBOSITY_LEVEL environment variable was, rather than depre‐
6150 cated, actually broken due to being read before various setup the dep‐
6151 recation path needed was done. It now works correctly (and emits a dep‐
6152 recation warning).
6153
6154 3.56.0 - 2018-04-17
6155 This release deprecates several redundant or internally oriented set‐
6156 tings, working towards an orthogonal set of configuration options that
6157 are widely useful without requiring any knowledge of our internals (‐
6158 issue #535).
6159
6160 · Deprecated settings that no longer have any effect are no longer
6161 shown in the __repr__ unless set to a non-default value.
6162
6163 · hypothesis.settings.perform_health_check is deprecated, as it dupli‐
6164 cates suppress_health_check.
6165
6166 · hypothesis.settings.max_iterations is deprecated and disabled,
6167 because we can usually get better behaviour from an internal heuris‐
6168 tic than a user-controlled setting.
6169
6170 · hypothesis.settings.min_satisfying_examples is deprecated and dis‐
6171 abled, due to overlap with the filter_too_much healthcheck and poor
6172 interaction with max_examples.
6173
6174 · HYPOTHESIS_VERBOSITY_LEVEL is now deprecated. Set verbosity through
6175 the profile system instead.
6176
6177 · Examples tried by find() are now reported at debug verbosity level
6178 (as well as verbose level).
6179
6180 3.55.6 - 2018-04-14
6181 This release fixes a somewhat obscure condition (issue #1230) under
6182 which you could occasionally see a failing test trigger an assertion
6183 error inside Hypothesis instead of failing normally.
6184
6185 3.55.5 - 2018-04-14
6186 This patch fixes one possible cause of issue #966. When running Python
6187 2 with hash randomisation, passing a python:bytes object to python:ran‐
6188 dom.seed() would use version=1, which broke derandomize (because the
6189 seed depended on a randomised hash). If derandomize is still nondeter‐
6190 ministic for you, please open an issue.
6191
6192 3.55.4 - 2018-04-13
6193 This patch makes a variety of minor improvements to the documentation,
6194 and improves a few validation messages for invalid inputs.
6195
6196 3.55.3 - 2018-04-12
6197 This release updates the URL metadata associated with the PyPI package
6198 (again). It has no other user visible effects.
6199
6200 3.55.2 - 2018-04-11
6201 This release updates the URL metadata associated with the PyPI package.
6202 It has no other user visible effects.
6203
6204 3.55.1 - 2018-04-06
6205 This patch relaxes constraints in our tests on the expected values
6206 returned by the standard library function hypot() and the internal
6207 helper function cathetus, to fix near-exact test failures on some
6208 32-bit systems used by downstream packagers.
6209
6210 3.55.0 - 2018-04-05
6211 This release includes several improvements to the handling of the data‐
6212 base setting.
6213
6214 · The database_file setting was a historical artefact, and you should
6215 just use database directly.
6216
6217 · The HYPOTHESIS_DATABASE_FILE environment variable is deprecated, in
6218 favor of load_profile() and the database setting.
6219
6220 · If you have not configured the example database at all and the
6221 default location is not usable (due to e.g. permissions issues),
6222 Hypothesis will fall back to an in-memory database. This is not per‐
6223 sisted between sessions, but means that the defaults work on
6224 read-only filesystems.
6225
6226 3.54.0 - 2018-04-04
6227 This release improves the complex_numbers() strategy, which now sup‐
6228 ports min_magnitude and max_magnitude arguments, along with allow_nan
6229 and allow_infinity like for floats().
6230
6231 Thanks to J.J. Green for this feature.
6232
6233 3.53.0 - 2018-04-01
6234 This release removes support for Django 1.8, which reached end of life
6235 on 2018-04-01. You can see Django's release and support schedule on
6236 the Django Project website.
6237
6238 3.52.3 - 2018-04-01
6239 This patch fixes the min_satisfying_examples settings documentation, by
6240 explaining that example shrinking is tracked at the level of the under‐
6241 lying bytestream rather than the output value.
6242
6243 The output from find() in verbose mode has also been adjusted - see the
6244 example session - to avoid duplicating lines when the example repr is
6245 constant, even if the underlying representation has been shrunken.
6246
6247 3.52.2 - 2018-03-30
6248 This release improves the output of failures with rule based stateful
6249 testing in two ways:
6250
6251 · The output from it is now usually valid Python code.
6252
6253 · When the same value has two different names because it belongs to two
6254 different bundles, it will now display with the name associated with
6255 the correct bundle for a rule argument where it is used.
6256
6257 3.52.1 - 2018-03-29
6258 This release improves the behaviour of stateful testing in two ways:
6259
6260 · Previously some runs would run no steps (issue #376). This should no
6261 longer happen.
6262
6263 · RuleBasedStateMachine tests which used bundles extensively would
6264 often shrink terribly. This should now be significantly improved,
6265 though there is likely a lot more room for improvement.
6266
6267 This release also involves a low level change to how ranges of integers
6268 are handles which may result in other improvements to shrink quality in
6269 some cases.
6270
6271 3.52.0 - 2018-03-24
6272 This release deprecates use of @settings(...) as a decorator, on func‐
6273 tions or methods that are not also decorated with @given. You can
6274 still apply these decorators in any order, though you should only do so
6275 once each.
6276
6277 Applying @given twice was already deprecated, and applying @set‐
6278 tings(...) twice is deprecated in this release and will become an error
6279 in a future version. Neither could ever be used twice to good effect.
6280
6281 Using @settings(...) as the sole decorator on a test is completely
6282 pointless, so this common usage error will become an error in a future
6283 version of Hypothesis.
6284
6285 3.51.0 - 2018-03-24
6286 This release deprecates the average_size argument to lists() and other
6287 collection strategies. You should simply delete it wherever it was
6288 used in your tests, as it no longer has any effect.
6289
6290 In early versions of Hypothesis, the average_size argument was treated
6291 as a hint about the distribution of examples from a strategy. Subse‐
6292 quent improvements to the conceptual model and the engine for generat‐
6293 ing and shrinking examples mean it is more effective to simply describe
6294 what constitutes a valid example, and let our internals handle the dis‐
6295 tribution.
6296
6297 3.50.3 - 2018-03-24
6298 This patch contains some internal refactoring so that we can run with
6299 warnings as errors in CI.
6300
6301 3.50.2 - 2018-03-20
6302 This has no user-visible changes except one slight formatting change to
6303 one docstring, to avoid a deprecation warning.
6304
6305 3.50.1 - 2018-03-20
6306 This patch fixes an internal error introduced in 3.48.0, where a check
6307 for the Django test runner would expose import-time errors in Django
6308 configuration (issue #1167).
6309
6310 3.50.0 - 2018-03-19
6311 This release improves validation of numeric bounds for some strategies.
6312
6313 · integers() and floats() now raise InvalidArgument if passed a
6314 min_value or max_value which is not an instance of Real, instead of
6315 various internal errors.
6316
6317 · floats() now converts its bounding values to the nearest float above
6318 or below the min or max bound respectively, instead of just casting
6319 to float. The old behaviour was incorrect in that you could generate
6320 float(min_value), even when this was less than min_value itself (pos‐
6321 sible with eg. fractions).
6322
6323 · When both bounds are provided to floats() but there are no floats in
6324 the interval, such as [(2**54)+1 .. (2**55)-1], InvalidArgument is
6325 raised.
6326
6327 · decimals() gives a more useful error message if passed a string that
6328 cannot be converted to Decimal in a context where this error is not
6329 trapped.
6330
6331 Code that previously seemed to work may be explicitly broken if there
6332 were no floats between min_value and max_value (only possible with
6333 non-float bounds), or if a bound was not a Real number but still
6334 allowed in python:math.isnan (some custom classes with a __float__
6335 method).
6336
6337 3.49.1 - 2018-03-15
6338 This patch fixes our tests for Numpy dtype strategies on big-endian
6339 platforms, where the strategy behaved correctly but the test assumed
6340 that the native byte order was little-endian.
6341
6342 There is no user impact unless you are running our test suite on
6343 big-endian platforms. Thanks to Graham Inggs for reporting issue
6344 #1164.
6345
6346 3.49.0 - 2018-03-12
6347 This release deprecates passing elements=None to collection strategies,
6348 such as lists().
6349
6350 Requiring lists(nothing()) or builds(list) instead of lists() means
6351 slightly more typing, but also improves the consistency and discover‐
6352 ability of our API - as well as showing how to compose or construct
6353 strategies in ways that still work in more complex situations.
6354
6355 Passing a nonzero max_size to a collection strategy where the elements
6356 strategy contains no values is now deprecated, and will be an error in
6357 a future version. The equivalent with elements=None is already an
6358 error.
6359
6360 3.48.1 - 2018-03-05
6361 This patch will minimize examples that would come out non-minimal in
6362 previous versions. Thanks to Kyle Reeve for this patch.
6363
6364 3.48.0 - 2018-03-05
6365 This release improves some "unhappy paths" when using Hypothesis with
6366 the standard library python:unittest module:
6367
6368 · Applying @given to a non-test method which is overridden from
6369 python:unittest.TestCase, such as setUp, raises a new health check.
6370 (issue #991)
6371
6372 · Using subTest() within a test decorated with @given would leak inter‐
6373 mediate results when tests were run under the python:unittest test
6374 runner. Individual reporting of failing subtests is now disabled
6375 during a test using @given. (issue #1071)
6376
6377 · @given is still not a class decorator, but the error message if you
6378 try using it on a class has been improved.
6379
6380 As a related improvement, using django:django.test.TestCase with @given
6381 instead of hypothesis.extra.django.TestCase raises an explicit error
6382 instead of running all examples in a single database transaction.
6383
6384 3.47.0 - 2018-03-02
6385 register_profile now accepts keyword arguments for specific settings,
6386 and the parent settings object is now optional. Using a name for a
6387 registered profile which is not a string was never suggested, but it is
6388 now also deprecated and will eventually be an error.
6389
6390 3.46.2 - 2018-03-01
6391 This release removes an unnecessary branch from the code, and has no
6392 user-visible impact.
6393
6394 3.46.1 - 2018-03-01
6395 This changes only the formatting of our docstrings and should have no
6396 user-visible effects.
6397
6398 3.46.0 - 2018-02-26
6399 characters() has improved docs about what arguments are valid, and
6400 additional validation logic to raise a clear error early (instead of
6401 e.g. silently ignoring a bad argument). Categories may be specified as
6402 the Unicode 'general category' (eg u'Nd'), or as the 'major category'
6403 (eg [u'N', u'Lu'] is equivalent to [u'Nd', u'Nl', u'No', u'Lu']).
6404
6405 In previous versions, general categories were supported and all other
6406 input was silently ignored. Now, major categories are supported in
6407 addition to general categories (which may change the behaviour of some
6408 existing code), and all other input is deprecated.
6409
6410 3.45.5 - 2018-02-26
6411 This patch improves strategy inference in hypothesis.extra.django to
6412 account for some validators in addition to field type - see issue #1116
6413 for ongoing work in this space.
6414
6415 Specifically, if a CharField or TextField has an attached RegexValida‐
6416 tor, we now use from_regex() instead of text() as the underlying strat‐
6417 egy. This allows us to generate examples of the default User model,
6418 closing issue #1112.
6419
6420 3.45.4 - 2018-02-25
6421 This patch improves some internal debugging information, fixes a typo
6422 in a validation error message, and expands the documentation for new
6423 contributors.
6424
6425 3.45.3 - 2018-02-23
6426 This patch may improve example shrinking slightly for some strategies.
6427
6428 3.45.2 - 2018-02-18
6429 This release makes our docstring style more consistent, thanks to
6430 flake8-docstrings. There are no user-visible changes.
6431
6432 3.45.1 - 2018-02-17
6433 This fixes an indentation issue in docstrings for datetimes(), dates(),
6434 times(), and timedeltas().
6435
6436 3.45.0 - 2018-02-13
6437 This release fixes builds() so that target can be used as a keyword
6438 argument for passing values to the target. The target itself can still
6439 be specified as a keyword argument, but that behavior is now depre‐
6440 cated. The target should be provided as the first positional argument.
6441
6442 3.44.26 - 2018-02-06
6443 This release fixes some formatting issues in the Hypothesis source
6444 code. It should have no externally visible effects.
6445
6446 3.44.25 - 2018-02-05
6447 This release changes the way in which Hypothesis tries to shrink the
6448 size of examples. It probably won't have much impact, but might make
6449 shrinking faster in some cases. It is unlikely but not impossible that
6450 it will change the resulting examples.
6451
6452 3.44.24 - 2018-01-27
6453 This release fixes dependency information when installing Hypothesis
6454 from a binary "wheel" distribution.
6455
6456 · The install_requires for enum34 is resolved at install time, rather
6457 than at build time (with potentially different results).
6458
6459 · Django has fixed their python_requires for versions 2.0.0 onward,
6460 simplifying Python2-compatible constraints for downstream projects.
6461
6462 3.44.23 - 2018-01-24
6463 This release improves shrinking in a class of pathological examples
6464 that you are probably never hitting in practice. If you are hitting
6465 them in practice this should be a significant speed up in shrinking. If
6466 you are not, you are very unlikely to notice any difference. You might
6467 see a slight slow down and/or slightly better falsifying examples.
6468
6469 3.44.22 - 2018-01-23
6470 This release fixes a dependency problem. It was possible to install
6471 Hypothesis with an old version of attrs, which would throw a TypeError
6472 as soon as you tried to import hypothesis. Specifically, you need
6473 attrs 16.0.0 or newer.
6474
6475 Hypothesis will now require the correct version of attrs when
6476 installing.
6477
6478 3.44.21 - 2018-01-22
6479 This change adds some additional structural information that Hypothesis
6480 will use to guide its search.
6481
6482 You mostly shouldn't see much difference from this. The two most likely
6483 effects you would notice are:
6484
6485 1. Hypothesis stores slightly more examples in its database for passing
6486 tests.
6487
6488 2. Hypothesis may find new bugs that it was previously missing, but it
6489 probably won't (this is a basic implementation of the feature that
6490 is intended to support future work. Although it is useful on its
6491 own, it's not very useful on its own).
6492
6493 3.44.20 - 2018-01-21
6494 This is a small refactoring release that changes how Hypothesis tracks
6495 some information about the boundary of examples in its internal repre‐
6496 sentation.
6497
6498 You are unlikely to see much difference in behaviour, but memory usage
6499 and run time may both go down slightly during normal test execution,
6500 and when failing Hypothesis might print its failing example slightly
6501 sooner.
6502
6503 3.44.19 - 2018-01-21
6504 This changes how we compute the default average_size for all collection
6505 strategies. Previously setting a max_size without setting an aver‐
6506 age_size would have the seemingly paradoxical effect of making data
6507 generation slower, because it would raise the average size from its
6508 default. Now setting max_size will either leave the default unchanged
6509 or lower it from its default.
6510
6511 If you are currently experiencing this problem, this may make your
6512 tests substantially faster. If you are not, this will likely have no
6513 effect on you.
6514
6515 3.44.18 - 2018-01-20
6516 This is a small refactoring release that changes how Hypothesis detects
6517 when the structure of data generation depends on earlier values gener‐
6518 ated (e.g. when using flatmap or composite()). It should not have any
6519 observable effect on behaviour.
6520
6521 3.44.17 - 2018-01-15
6522 This release fixes a typo in internal documentation, and has no
6523 user-visible impact.
6524
6525 3.44.16 - 2018-01-13
6526 This release improves test case reduction for recursive data struc‐
6527 tures. Hypothesis now guarantees that whenever a strategy calls itself
6528 recursively (usually this will happen because you are using
6529 deferred()), any recursive call may replace the top level value. e.g.
6530 given a tree structure, Hypothesis will always try replacing it with a
6531 subtree.
6532
6533 Additionally this introduces a new heuristic that may in some circum‐
6534 stances significantly speed up test case reduction - Hypothesis should
6535 be better at immediately replacing elements drawn inside another strat‐
6536 egy with their minimal possible value.
6537
6538 3.44.15 - 2018-01-13
6539 from_type() can now resolve recursive types such as binary trees (issue
6540 #1004). Detection of non-type arguments has also improved, leading to
6541 better error messages in many cases involving forward references.
6542
6543 3.44.14 - 2018-01-08
6544 This release fixes a bug in the shrinker that prevented the optimisa‐
6545 tions in 3.44.6 from working in some cases. It would not have worked
6546 correctly when filtered examples were nested (e.g. with a set of inte‐
6547 gers in some range).
6548
6549 This would not have resulted in any correctness problems, but shrinking
6550 may have been slower than it otherwise could be.
6551
6552 3.44.13 - 2018-01-08
6553 This release changes the average bit length of values drawn from inte‐
6554 gers() to be much smaller. Additionally it changes the shrinking order
6555 so that now size is considered before sign - e.g. -1 will be preferred
6556 to +10.
6557
6558 The new internal format for integers required some changes to the mini‐
6559 mizer to make work well, so you may also see some improvements to exam‐
6560 ple quality in unrelated areas.
6561
6562 3.44.12 - 2018-01-07
6563 This changes Hypothesis's internal implementation of weighted sampling.
6564 This will affect example distribution and quality, but you shouldn't
6565 see any other effects.
6566
6567 3.44.11 - 2018-01-06
6568 This is a change to some internals around how Hypothesis handles avoid‐
6569 ing generating duplicate examples and seeking out novel regions of the
6570 search space.
6571
6572 You are unlikely to see much difference as a result of it, but it fixes
6573 a bug where an internal assertion could theoretically be triggered and
6574 has some minor effects on the distribution of examples so could poten‐
6575 tially find bugs that have previously been missed.
6576
6577 3.44.10 - 2018-01-06
6578 This patch avoids creating debug statements when debugging is disabled.
6579 Profiling suggests this is a 5-10% performance improvement (pull
6580 request #1040).
6581
6582 3.44.9 - 2018-01-06
6583 This patch blacklists null characters ('\x00') in automatically created
6584 strategies for Django CharField and TextField, due to a database issue
6585 which was recently fixed upstream (Hypothesis issue #1045).
6586
6587 3.44.8 - 2018-01-06
6588 This release makes the Hypothesis shrinker slightly less greedy in
6589 order to avoid local minima - when it gets stuck, it makes a small
6590 attempt to search around the final example it would previously have
6591 returned to find a new starting point to shrink from. This should
6592 improve example quality in some cases, especially ones where the test
6593 data has dependencies among parts of it that make it difficult for
6594 Hypothesis to proceed.
6595
6596 3.44.7 - 2018-01-04
6597 This release adds support for Django 2 in the hypothesis-django extra.
6598
6599 This release drops support for Django 1.10, as it is no longer sup‐
6600 ported by the Django team.
6601
6602 3.44.6 - 2018-01-02
6603 This release speeds up test case reduction in many examples by being
6604 better at detecting large shrinks it can use to discard redundant parts
6605 of its input. This will be particularly noticeable in examples that
6606 make use of filtering and for some integer ranges.
6607
6608 3.44.5 - 2018-01-02
6609 Happy new year!
6610
6611 This is a no-op release that updates the year range on all of the copy‐
6612 right headers in our source to include 2018.
6613
6614 3.44.4 - 2017-12-23
6615 This release fixes issue #1044, which slowed tests by up to 6% due to
6616 broken caching.
6617
6618 3.44.3 - 2017-12-21
6619 This release improves the shrinker in cases where examples drawn ear‐
6620 lier can affect how much data is drawn later (e.g. when you draw a
6621 length parameter in a composite and then draw that many elements).
6622 Examples found in cases like this should now be much closer to minimal.
6623
6624 3.44.2 - 2017-12-20
6625 This is a pure refactoring release which changes how Hypothesis manages
6626 its set of examples internally. It should have no externally visible
6627 effects.
6628
6629 3.44.1 - 2017-12-18
6630 This release fixes issue #997, in which under some circumstances the
6631 body of tests run under Hypothesis would not show up when run under
6632 coverage even though the tests were run and the code they called out‐
6633 side of the test file would show up normally.
6634
6635 3.44.0 - 2017-12-17
6636 This release adds a new feature: The @reproduce_failure decorator,
6637 designed to make it easy to use Hypothesis's binary format for examples
6638 to reproduce a problem locally without having to share your example
6639 database between machines.
6640
6641 This also changes when seeds are printed:
6642
6643 · They will no longer be printed for normal falsifying examples, as
6644 there are now adequate ways of reproducing those for all cases, so it
6645 just contributes noise.
6646
6647 · They will once again be printed when reusing examples from the data‐
6648 base, as health check failures should now be more reliable in this
6649 scenario so it will almost always work in this case.
6650
6651 This work was funded by Smarkets.
6652
6653 3.43.1 - 2017-12-17
6654 This release fixes a bug with Hypothesis's database management - exam‐
6655 ples that were found in the course of shrinking were saved in a way
6656 that indicated that they had distinct causes, and so they would all be
6657 retried on the start of the next test. The intended behaviour, which is
6658 now what is implemented, is that only a bounded subset of these exam‐
6659 ples would be retried.
6660
6661 3.43.0 - 2017-12-17
6662 HypothesisDeprecationWarning now inherits from python:FutureWarning
6663 instead of python:DeprecationWarning, as recommended by PEP 565 for
6664 user-facing warnings (issue #618). If you have not changed the default
6665 warnings settings, you will now see each distinct HypothesisDepreca‐
6666 tionWarning instead of only the first.
6667
6668 3.42.2 - 2017-12-12
6669 This patch fixes issue #1017, where instances of a list or tuple sub‐
6670 type used as an argument to a strategy would be coerced to tuple.
6671
6672 3.42.1 - 2017-12-10
6673 This release has some internal cleanup, which makes reading the code
6674 more pleasant and may shrink large examples slightly faster.
6675
6676 3.42.0 - 2017-12-09
6677 This release deprecates faker-extra, which was designed as a transition
6678 strategy but does not support example shrinking or coverage-guided dis‐
6679 covery.
6680
6681 3.41.0 - 2017-12-06
6682 sampled_from() can now sample from one-dimensional numpy ndarrays. Sam‐
6683 pling from multi-dimensional ndarrays still results in a deprecation
6684 warning. Thanks to Charlie Tanksley for this patch.
6685
6686 3.40.1 - 2017-12-04
6687 This release makes two changes:
6688
6689 · It makes the calculation of some of the metadata that Hypothesis uses
6690 for shrinking occur lazily. This should speed up performance of test
6691 case generation a bit because it no longer calculates information it
6692 doesn't need.
6693
6694 · It improves the shrinker for certain classes of nested examples. e.g.
6695 when shrinking lists of lists, the shrinker is now able to concate‐
6696 nate two adjacent lists together into a single list. As a result of
6697 this change, shrinking may get somewhat slower when the minimal exam‐
6698 ple found is large.
6699
6700 3.40.0 - 2017-12-02
6701 This release improves how various ways of seeding Hypothesis interact
6702 with the example database:
6703
6704 · Using the example database with seed() is now deprecated. You should
6705 set database=None if you are doing that. This will only warn if you
6706 actually load examples from the database while using @seed.
6707
6708 · The derandomize will behave the same way as @seed.
6709
6710 · Using --hypothesis-seed will disable use of the database.
6711
6712 · If a test used examples from the database, it will not suggest using
6713 a seed to reproduce it, because that won't work.
6714
6715 This work was funded by Smarkets.
6716
6717 3.39.0 - 2017-12-01
6718 This release adds a new health check that checks if the smallest "natu‐
6719 ral" possible example of your test case is very large - this will tend
6720 to cause Hypothesis to generate bad examples and be quite slow.
6721
6722 This work was funded by Smarkets.
6723
6724 3.38.9 - 2017-11-29
6725 This is a documentation release to improve the documentation of shrink‐
6726 ing behaviour for Hypothesis's strategies.
6727
6728 3.38.8 - 2017-11-29
6729 This release improves the performance of characters() when using black‐
6730 list_characters and from_regex() when using negative character classes.
6731
6732 The problems this fixes were found in the course of work funded by
6733 Smarkets.
6734
6735 3.38.7 - 2017-11-29
6736 This is a patch release for from_regex(), which had a bug in handling
6737 of the python:re.VERBOSE flag (issue #992). Flags are now handled cor‐
6738 rectly when parsing regex.
6739
6740 3.38.6 - 2017-11-28
6741 This patch changes a few byte-string literals from double to single
6742 quotes, thanks to an update in unify. There are no user-visible
6743 changes.
6744
6745 3.38.5 - 2017-11-23
6746 This fixes the repr of strategies using lambda that are defined inside
6747 decorators to include the lambda source.
6748
6749 This would mostly have been visible when using the statistics function‐
6750 ality - lambdas used for e.g. filtering would have shown up with a
6751 <unknown> as their body. This can still happen, but it should happen
6752 less often now.
6753
6754 3.38.4 - 2017-11-22
6755 This release updates the reported statistics so that they show approxi‐
6756 mately what fraction of your test run time is spent in data generation
6757 (as opposed to test execution).
6758
6759 This work was funded by Smarkets.
6760
6761 3.38.3 - 2017-11-21
6762 This is a documentation release, which ensures code examples are up to
6763 date by running them as doctests in CI (issue #711).
6764
6765 3.38.2 - 2017-11-21
6766 This release changes the behaviour of the deadline setting when used
6767 with data(): Time spent inside calls to data.draw will no longer be
6768 counted towards the deadline time.
6769
6770 As a side effect of some refactoring required for this work, the way
6771 flaky tests are handled has changed slightly. You are unlikely to see
6772 much difference from this, but some error messages will have changed.
6773
6774 This work was funded by Smarkets.
6775
6776 3.38.1 - 2017-11-21
6777 This patch has a variety of non-user-visible refactorings, removing
6778 various minor warts ranging from indirect imports to typos in comments.
6779
6780 3.38.0 - 2017-11-18
6781 This release overhauls the health check system in a variety of small
6782 ways. It adds no new features, but is nevertheless a minor release
6783 because it changes which tests are likely to fail health checks.
6784
6785 The most noticeable effect is that some tests that used to fail health
6786 checks will now pass, and some that used to pass will fail. These
6787 should all be improvements in accuracy. In particular:
6788
6789 · New failures will usually be because they are now taking into account
6790 things like use of data() and assume() inside the test body.
6791
6792 · New failures may also be because for some classes of example the way
6793 data generation performance was measured was artificially faster than
6794 real data generation (for most examples that are hitting performance
6795 health checks the opposite should be the case).
6796
6797 · Tests that used to fail health checks and now pass do so because the
6798 health check system used to run in a way that was subtly different
6799 than the main Hypothesis data generation and lacked some of its sup‐
6800 port for e.g. large examples.
6801
6802 If your data generation is especially slow, you may also see your tests
6803 get somewhat faster, as there is no longer a separate health check
6804 phase. This will be particularly noticeable when rerunning test fail‐
6805 ures.
6806
6807 This work was funded by Smarkets.
6808
6809 3.37.0 - 2017-11-12
6810 This is a deprecation release for some health check related features.
6811
6812 The following are now deprecated:
6813
6814 · Passing HealthCheck.exception_in_generation to suppress_health_check.
6815 This no longer does anything even when passed - All errors that
6816 occur during data generation will now be immediately reraised rather
6817 than going through the health check mechanism.
6818
6819 · Passing HealthCheck.random_module to suppress_health_check. This
6820 hasn't done anything for a long time, but was never explicitly depre‐
6821 cated. Hypothesis always seeds the random module when running @given
6822 tests, so this is no longer an error and suppressing it doesn't do
6823 anything.
6824
6825 · Passing non-HealthCheck values in suppress_health_check. This was
6826 previously allowed but never did anything useful.
6827
6828 In addition, passing a non-iterable value as suppress_health_check will
6829 now raise an error immediately (it would never have worked correctly,
6830 but it would previously have failed later). Some validation error mes‐
6831 sages have also been updated.
6832
6833 This work was funded by Smarkets.
6834
6835 3.36.1 - 2017-11-10
6836 This is a yak shaving release, mostly concerned with our own tests.
6837
6838 While getfullargspec() was documented as deprecated in Python 3.5, it
6839 never actually emitted a warning. Our code to silence this (nonexis‐
6840 tent) warning has therefore been removed.
6841
6842 We now run our tests with DeprecationWarning as an error, and made some
6843 minor changes to our own tests as a result. This required similar
6844 upstream updates to coverage and execnet (a test-time dependency via
6845 pytest-xdist).
6846
6847 There is no user-visible change in Hypothesis itself, but we encourage
6848 you to consider enabling deprecations as errors in your own tests.
6849
6850 3.36.0 - 2017-11-06
6851 This release adds a setting to the public API, and does some internal
6852 cleanup:
6853
6854 · The derandomize setting is now documented (issue #890)
6855
6856 · Removed - and disallowed - all 'bare excepts' in Hypothesis (issue
6857 #953)
6858
6859 · Documented the strict setting as deprecated, and updated the build so
6860 our docs always match deprecations in the code.
6861
6862 3.35.0 - 2017-11-06
6863 This minor release supports constraining uuids() to generate a particu‐
6864 lar version of UUID (issue #721).
6865
6866 Thanks to Dion Misic for this feature.
6867
6868 3.34.1 - 2017-11-02
6869 This patch updates the documentation to suggest builds(callable)
6870 instead of just(callable()).
6871
6872 3.34.0 - 2017-11-02
6873 Hypothesis now emits deprecation warnings if you apply @given more than
6874 once to a target.
6875
6876 Applying @given repeatedly wraps the target multiple times. Each wrap‐
6877 per will search the space of of possible parameters separately. This
6878 is equivalent but will be much more inefficient than doing it with a
6879 single call to @given.
6880
6881 For example, instead of @given(booleans()) @given(integers()), you
6882 could write @given(booleans(), integers())
6883
6884 3.33.1 - 2017-11-02
6885 This is a bugfix release:
6886
6887 · builds() would try to infer a strategy for required positional argu‐
6888 ments of the target from type hints, even if they had been given to
6889 builds() as positional arguments (issue #946). Now it only infers
6890 missing required arguments.
6891
6892 · An internal introspection function wrongly reported self as a
6893 required argument for bound methods, which might also have affected
6894 builds(). Now it knows better.
6895
6896 3.33.0 - 2017-10-16
6897 This release supports strategy inference for more Django field types -
6898 you can now omit an argument for Date, Time, Duration, Slug, IP
6899 Address, and UUID fields. (issue #642)
6900
6901 Strategy generation for fields with grouped choices now selects choices
6902 from each group, instead of selecting from the group names.
6903
6904 3.32.2 - 2017-10-15
6905 This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on
6906 an experimental basis. It has never actually worked, and the new
6907 Hypothesis example database is designed to make such a tool unneces‐
6908 sary.
6909
6910 3.32.1 - 2017-10-13
6911 This patch has two improvements for strategies based on enumerations.
6912
6913 · from_type() now handles enumerations correctly, delegating to sam‐
6914 pled_from(). Previously it noted that Enum.__init__ has no required
6915 arguments and therefore delegated to builds(), which would subse‐
6916 quently fail.
6917
6918 · When sampling from an python:enum.Flag, we also generate combinations
6919 of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can
6920 now generate, Permissions.READ, Permissions.READ|WRITE, and so on.
6921
6922 3.32.0 - 2017-10-09
6923 This changes the default value of the use_coverage setting to True when
6924 running on pypy (it was already True on CPython).
6925
6926 It was previously set to False because we expected it to be too slow,
6927 but recent benchmarking shows that actually performance of the feature
6928 on pypy is fairly acceptable - sometimes it's slower than on CPython,
6929 sometimes it's faster, but it's generally within a factor of two either
6930 way.
6931
6932 3.31.6 - 2017-10-08
6933 This patch improves the quality of strategies inferred from Numpy
6934 dtypes:
6935
6936 · Integer dtypes generated examples with the upper half of their
6937 (non-sign) bits set to zero. The inferred strategies can now produce
6938 any representable integer.
6939
6940 · Fixed-width unicode- and byte-string dtypes now cap the internal
6941 example length, which should improve example and shrink quality.
6942
6943 · Numpy arrays can only store fixed-size strings internally, and allow
6944 shorter strings by right-padding them with null bytes. Inferred
6945 string strategies no longer generate such values, as they can never
6946 be retrieved from an array. This improves shrinking performance by
6947 skipping useless values.
6948
6949 This has already been useful in Hypothesis - we found an overflow bug
6950 in our Pandas support, and as a result indexes() and range_indexes()
6951 now check that min_size and max_size are at least zero.
6952
6953 3.31.5 - 2017-10-08
6954 This release fixes a performance problem in tests where the use_cover‐
6955 age setting is True.
6956
6957 Tests experience a slow-down proportionate to the amount of code they
6958 cover. This is still the case, but the factor is now low enough that
6959 it should be unnoticeable. Previously it was large and became much
6960 larger in 3.30.4.
6961
6962 3.31.4 - 2017-10-08
6963 from_type() failed with a very confusing error if passed a NewType() (‐
6964 issue #901). These psudeo-types are now unwrapped correctly, and
6965 strategy inference works as expected.
6966
6967 3.31.3 - 2017-10-06
6968 This release makes some small optimisations to our use of coverage that
6969 should reduce constant per-example overhead. This is probably only
6970 noticeable on examples where the test itself is quite fast. On no-op
6971 tests that don't test anything you may see up to a fourfold speed
6972 increase (which is still significantly slower than without coverage).
6973 On more realistic tests the speed up is likely to be less than that.
6974
6975 3.31.2 - 2017-09-30
6976 This release fixes some formatting and small typos/grammar issues in
6977 the documentation, specifically the page docs/settings.rst, and the
6978 inline docs for the various settings.
6979
6980 3.31.1 - 2017-09-30
6981 This release improves the handling of deadlines so that they act better
6982 with the shrinking process. This fixes issue #892.
6983
6984 This involves two changes:
6985
6986 1. The deadline is raised during the initial generation and shrinking,
6987 and then lowered to the set value for final replay. This restricts
6988 our attention to examples which exceed the deadline by a more sig‐
6989 nificant margin, which increases their reliability.
6990
6991 2. When despite the above a test still becomes flaky because it is sig‐
6992 nificantly faster on rerun than it was on its first run, the error
6993 message is now more explicit about the nature of this problem, and
6994 includes both the initial test run time and the new test run time.
6995
6996 In addition, this release also clarifies the documentation of the dead‐
6997 line setting slightly to be more explicit about where it applies.
6998
6999 This work was funded by Smarkets.
7000
7001 3.31.0 - 2017-09-29
7002 This release blocks installation of Hypothesis on Python 3.3, which
7003 reached its end of life date on 2017-09-29.
7004
7005 This should not be of interest to anyone but downstream maintainers -
7006 if you are affected, migrate to a secure version of Python as soon as
7007 possible or at least seek commercial support.
7008
7009 3.30.4 - 2017-09-27
7010 This release makes several changes:
7011
7012 1. It significantly improves Hypothesis's ability to use coverage
7013 information to find interesting examples.
7014
7015 2. It reduces the default max_examples setting from 200 to 100. This
7016 takes advantage of the improved algorithm meaning fewer examples are
7017 typically needed to get the same testing and is sufficiently better
7018 at covering interesting behaviour, and offsets some of the perfor‐
7019 mance problems of running under coverage.
7020
7021 3. Hypothesis will always try to start its testing with an example that
7022 is near minimized.
7023
7024 The new algorithm for 1 also makes some changes to Hypothesis's low
7025 level data generation which apply even with coverage turned off. They
7026 generally reduce the total amount of data generated, which should
7027 improve test performance somewhat. Between this and 3 you should see a
7028 noticeable reduction in test runtime (how much so depends on your tests
7029 and how much example size affects their performance. On our benchmarks,
7030 where data generation dominates, we saw up to a factor of two perfor‐
7031 mance improvement, but it's unlikely to be that large.
7032
7033 3.30.3 - 2017-09-25
7034 This release fixes some formatting and small typos/grammar issues in
7035 the documentation, specifically the page docs/details.rst, and some
7036 inline docs linked from there.
7037
7038 3.30.2 - 2017-09-24
7039 This release changes Hypothesis's caching approach for functions in
7040 hypothesis.strategies. Previously it would have cached extremely
7041 aggressively and cache entries would never be evicted. Now it adopts a
7042 least-frequently used, least recently used key invalidation policy, and
7043 is somewhat more conservative about which strategies it caches.
7044
7045 Workloads which create strategies based on dynamic values, e.g. by
7046 using flatmap or composite(), will use significantly less memory.
7047
7048 3.30.1 - 2017-09-22
7049 This release fixes a bug where when running with the use_coverage=True
7050 setting inside an existing running instance of coverage, Hypothesis
7051 would frequently put files that the coveragerc excluded in the report
7052 for the enclosing coverage.
7053
7054 3.30.0 - 2017-09-20
7055 This release introduces two new features:
7056
7057 · When a test fails, either with a health check failure or a falsifying
7058 example, Hypothesis will print out a seed that led to that failure,
7059 if the test is not already running with a fixed seed. You can then
7060 recreate that failure using either the @seed decorator or (if you are
7061 running pytest) with --hypothesis-seed.
7062
7063 · pytest users can specify a seed to use for @given based tests by
7064 passing the --hypothesis-seed command line argument.
7065
7066 This work was funded by Smarkets.
7067
7068 3.29.0 - 2017-09-19
7069 This release makes Hypothesis coverage aware. Hypothesis now runs all
7070 test bodies under coverage, and uses this information to guide its
7071 testing.
7072
7073 The use_coverage setting can be used to disable this behaviour if you
7074 want to test code that is sensitive to coverage being enabled (either
7075 because of performance or interaction with the trace function).
7076
7077 The main benefits of this feature are:
7078
7079 · Hypothesis now observes when examples it discovers cover particular
7080 lines or branches and stores them in the database for later.
7081
7082 · Hypothesis will make some use of this information to guide its explo‐
7083 ration of the search space and improve the examples it finds (this is
7084 currently used only very lightly and will likely improve signifi‐
7085 cantly in future releases).
7086
7087 This also has the following side-effects:
7088
7089 · Hypothesis now has an install time dependency on the coverage pack‐
7090 age.
7091
7092 · Tests that are already running Hypothesis under coverage will likely
7093 get faster.
7094
7095 · Tests that are not running under coverage now run their test bodies
7096 under coverage by default.
7097
7098 This feature is only partially supported under pypy. It is signifi‐
7099 cantly slower than on CPython and is turned off by default as a result,
7100 but it should still work correctly if you want to use it.
7101
7102 3.28.3 - 2017-09-18
7103 This release is an internal change that affects how Hypothesis handles
7104 calculating certain properties of strategies.
7105
7106 The primary effect of this is that it fixes a bug where use of
7107 deferred() could sometimes trigger an internal assertion error. However
7108 the fix for this bug involved some moderately deep changes to how
7109 Hypothesis handles certain constructs so you may notice some additional
7110 knock-on effects.
7111
7112 In particular the way Hypothesis handles drawing data from strategies
7113 that cannot generate any values has changed to bail out sooner than it
7114 previously did. This may speed up certain tests, but it is unlikely to
7115 make much of a difference in practice for tests that were not already
7116 failing with Unsatisfiable.
7117
7118 3.28.2 - 2017-09-18
7119 This is a patch release that fixes a bug in the hypothesis.extra.pandas
7120 documentation where it incorrectly referred to column() instead of col‐
7121 umns().
7122
7123 3.28.1 - 2017-09-16
7124 This is a refactoring release. It moves a number of internal uses of
7125 namedtuple() over to using attrs based classes, and removes a couple of
7126 internal namedtuple classes that were no longer in use.
7127
7128 It should have no user visible impact.
7129
7130 3.28.0 - 2017-09-15
7131 This release adds support for testing pandas via the hypothe‐
7132 sis.extra.pandas module.
7133
7134 It also adds a dependency on attrs.
7135
7136 This work was funded by Stripe.
7137
7138 3.27.1 - 2017-09-14
7139 This release fixes some formatting and broken cross-references in the
7140 documentation, which includes editing docstrings - and thus a patch
7141 release.
7142
7143 3.27.0 - 2017-09-13
7144 This release introduces a deadline setting to Hypothesis.
7145
7146 When set this turns slow tests into errors. By default it is unset but
7147 will warn if you exceed 200ms, which will become the default value in a
7148 future release.
7149
7150 This work was funded by Smarkets.
7151
7152 3.26.0 - 2017-09-12
7153 Hypothesis now emits deprecation warnings if you are using the legacy
7154 SQLite example database format, or the tool for merging them. These
7155 were already documented as deprecated, so this doesn't change their
7156 deprecation status, only that we warn about it.
7157
7158 3.25.1 - 2017-09-12
7159 This release fixes a bug with generating numpy datetime and timedelta
7160 types: When inferring the strategy from the dtype, datetime and
7161 timedelta dtypes with sub-second precision would always produce exam‐
7162 ples with one second resolution. Inferring a strategy from a time
7163 dtype will now always produce example with the same precision.
7164
7165 3.25.0 - 2017-09-12
7166 This release changes how Hypothesis shrinks and replays examples to
7167 take into account that it can encounter new bugs while shrinking the
7168 bug it originally found. Previously it would end up replacing the orig‐
7169 inally found bug with the new bug and show you only that one. Now it is
7170 (often) able to recognise when two bugs are distinct and when it finds
7171 more than one will show both.
7172
7173 3.24.2 - 2017-09-11
7174 This release removes the (purely internal and no longer useful) strat‐
7175 egy_test_suite function and the corresponding strategytests module.
7176
7177 3.24.1 - 2017-09-06
7178 This release improves the reduction of examples involving floating
7179 point numbers to produce more human readable examples.
7180
7181 It also has some general purpose changes to the way the minimizer works
7182 internally, which may see some improvement in quality and slow down of
7183 test case reduction in cases that have nothing to do with floating
7184 point numbers.
7185
7186 3.24.0 - 2017-09-05
7187 Hypothesis now emits deprecation warnings if you use some_strat‐
7188 egy.example() inside a test function or strategy definition (this was
7189 never intended to be supported, but is sufficiently widespread that it
7190 warrants a deprecation path).
7191
7192 3.23.3 - 2017-09-05
7193 This is a bugfix release for decimals() with the places argument.
7194
7195 · No longer fails health checks (issue #725, due to internal filtering)
7196
7197 · Specifying a min_value and max_value without any decimals with places
7198 places between them gives a more useful error message.
7199
7200 · Works for any valid arguments, regardless of the decimal precision
7201 context.
7202
7203 3.23.2 - 2017-09-01
7204 This is a small refactoring release that removes a now-unused parameter
7205 to an internal API. It shouldn't have any user visible effect.
7206
7207 3.23.1 - 2017-09-01
7208 Hypothesis no longer propagates the dynamic scope of settings into
7209 strategy definitions.
7210
7211 This release is a small change to something that was never part of the
7212 public API and you will almost certainly not notice any effect unless
7213 you're doing something surprising, but for example the following code
7214 will now give a different answer in some circumstances:
7215
7216 import hypothesis.strategies as st
7217 from hypothesis import settings
7218
7219 CURRENT_SETTINGS = st.builds(lambda: settings.default)
7220
7221 (We don't actually encourage you writing code like this)
7222
7223 Previously this would have generated the settings that were in effect
7224 at the point of definition of CURRENT_SETTINGS. Now it will generate
7225 the settings that are used for the current test.
7226
7227 It is very unlikely to be significant enough to be visible, but you may
7228 also notice a small performance improvement.
7229
7230 3.23.0 - 2017-08-31
7231 This release adds a unique argument to arrays() which behaves the same
7232 ways as the corresponding one for lists(), requiring all of the ele‐
7233 ments in the generated array to be distinct.
7234
7235 3.22.2 - 2017-08-29
7236 This release fixes an issue where Hypothesis would raise a TypeError
7237 when using the datetime-related strategies if running with PYTHONOPTI‐
7238 MIZE=2. This bug was introduced in 3.20.0. (See issue #822)
7239
7240 3.22.1 - 2017-08-28
7241 Hypothesis now transparently handles problems with an internal unicode
7242 cache, including file truncation or read-only filesystems (issue #767).
7243 Thanks to Sam Hames for the patch.
7244
7245 3.22.0 - 2017-08-26
7246 This release provides what should be a substantial performance improve‐
7247 ment to numpy arrays generated using provided numpy support, and adds a
7248 new fill_value argument to arrays() to control this behaviour.
7249
7250 This work was funded by Stripe.
7251
7252 3.21.3 - 2017-08-26
7253 This release fixes some extremely specific circumstances that probably
7254 have never occurred in the wild where users of deferred() might have
7255 seen a python:RuntimeError from too much recursion, usually in cases
7256 where no valid example could have been generated anyway.
7257
7258 3.21.2 - 2017-08-25
7259 This release fixes some minor bugs in argument validation:
7260
7261 · hypothesis.extra.numpy dtype strategies would raise an internal
7262 error instead of an InvalidArgument exception when passed an
7263 invalid endianness specification.
7264
7265 · fractions() would raise an internal error instead of an InvalidAr‐
7266 gument if passed float("nan") as one of its bounds.
7267
7268 · The error message for passing float("nan") as a bound to various
7269 strategies has been improved.
7270
7271 · Various bound arguments will now raise InvalidArgument in cases
7272 where they would previously have raised an internal TypeError or
7273 ValueError from the relevant conversion function.
7274
7275 · streaming() would not have emitted a deprecation warning when
7276 called with an invalid argument.
7277
7278 3.21.1 - 2017-08-24
7279 This release fixes a bug where test failures that were the result of an
7280 @example would print an extra stack trace before re-raising the excep‐
7281 tion.
7282
7283 3.21.0 - 2017-08-23
7284 This release deprecates Hypothesis's strict mode, which turned Hypothe‐
7285 sis's deprecation warnings into errors. Similar functionality can be
7286 achieved by using simplefilter('error', HypothesisDeprecationWarning).
7287
7288 3.20.0 - 2017-08-22
7289 This release renames the relevant arguments on the datetimes(),
7290 dates(), times(), and timedeltas() strategies to min_value and
7291 max_value, to make them consistent with the other strategies in the
7292 module.
7293
7294 The old argument names are still supported but will emit a deprecation
7295 warning when used explicitly as keyword arguments. Arguments passed
7296 positionally will go to the new argument names and are not deprecated.
7297
7298 3.19.3 - 2017-08-22
7299 This release provides a major overhaul to the internals of how Hypothe‐
7300 sis handles shrinking.
7301
7302 This should mostly be visible in terms of getting better examples for
7303 tests which make heavy use of composite(), data() or flatmap where the
7304 data drawn depends a lot on previous choices, especially where size
7305 parameters are affected. Previously Hypothesis would have struggled to
7306 reliably produce good examples here. Now it should do much better. Per‐
7307 formance should also be better for examples with a non-zero min_size.
7308
7309 You may see slight changes to example generation (e.g. improved example
7310 diversity) as a result of related changes to internals, but they are
7311 unlikely to be significant enough to notice.
7312
7313 3.19.2 - 2017-08-21
7314 This release fixes two bugs in hypothesis.extra.numpy:
7315
7316 · unicode_string_dtypes() didn't work at all due to an incorrect dtype
7317 specifier. Now it does.
7318
7319 · Various impossible conditions would have been accepted but would
7320 error when they fail to produced any example. Now they raise an
7321 explicit InvalidArgument error.
7322
7323 3.19.1 - 2017-08-21
7324 This is a bugfix release for issue #739, where bounds for fractions()
7325 or floating-point decimals() were not properly converted to integers
7326 before passing them to the integers strategy. This excluded some val‐
7327 ues that should have been possible, and could trigger internal errors
7328 if the bounds lay between adjacent integers.
7329
7330 You can now bound fractions() with two arbitrarily close fractions.
7331
7332 It is now an explicit error to supply a min_value, max_value, and
7333 max_denominator to fractions() where the value bounds do not include a
7334 fraction with denominator at most max_denominator.
7335
7336 3.19.0 - 2017-08-20
7337 This release adds the from_regex() strategy, which generates strings
7338 that contain a match of a regular expression.
7339
7340 Thanks to Maxim Kulkin for creating the hypothesis-regex package and
7341 then helping to upstream it! (issue #662)
7342
7343 3.18.5 - 2017-08-18
7344 This is a bugfix release for integers(). Previously the strategy would
7345 hit an internal assertion if passed non-integer bounds for min_value
7346 and max_value that had no integers between them. The strategy now
7347 raises InvalidArgument instead.
7348
7349 3.18.4 - 2017-08-18
7350 Release to fix a bug where mocks can be used as test runners under cer‐
7351 tain conditions. Specifically, if a mock is injected into a test via
7352 pytest fixtures or patch decorators, and that mock is the first argu‐
7353 ment in the list, hypothesis will think it represents self and turns
7354 the mock into a test runner. If this happens, the affected test always
7355 passes because the mock is executed instead of the test body. Some‐
7356 times, it will also fail health checks.
7357
7358 Fixes issue #491 and a section of issue #198. Thanks to Ben Peterson
7359 for this bug fix.
7360
7361 3.18.3 - 2017-08-17
7362 This release should improve the performance of some tests which experi‐
7363 enced a slow down as a result of the 3.13.0 release.
7364
7365 Tests most likely to benefit from this are ones that make extensive use
7366 of min_size parameters, but others may see some improvement as well.
7367
7368 3.18.2 - 2017-08-16
7369 This release fixes a bug introduced in 3.18.0. If the arguments
7370 whitelist_characters and blacklist_characters to characters() contained
7371 overlapping elements, then an InvalidArgument exception would be
7372 raised.
7373
7374 Thanks to Zac Hatfield-Dodds for reporting and fixing this.
7375
7376 3.18.1 - 2017-08-14
7377 This is a bug fix release to fix issue #780, where sets() and similar
7378 would trigger health check errors if their element strategy could only
7379 produce one element (e.g. if it was just()).
7380
7381 3.18.0 - 2017-08-13
7382 This is a feature release:
7383
7384 · characters() now accepts whitelist_characters, particular characters
7385 which will be added to those it produces. (issue #668)
7386
7387 · A bug fix for the internal function _union_interval_lists(), and a
7388 rename to _union_intervals(). It now correctly handles all cases
7389 where intervals overlap, and it always returns the result as a tuple
7390 for tuples.
7391
7392 Thanks to Alex Willmer for these.
7393
7394 3.17.0 - 2017-08-07
7395 This release documents the previously undocumented phases feature, mak‐
7396 ing it part of the public API. It also updates how the example database
7397 is used. Principally:
7398
7399 · A Phases.reuse argument will now correctly control whether examples
7400 from the database are run (it previously did exactly the wrong thing
7401 and controlled whether examples would be saved).
7402
7403 · Hypothesis will no longer try to rerun all previously failing exam‐
7404 ples. Instead it will replay the smallest previously failing example
7405 and a selection of other examples that are likely to trigger any
7406 other bugs that will found. This prevents a previous failure from
7407 dominating your tests unnecessarily.
7408
7409 · As a result of the previous change, Hypothesis will be slower about
7410 clearing out old examples from the database that are no longer fail‐
7411 ing (because it can only clear out ones that it actually runs).
7412
7413 3.16.1 - 2017-08-07
7414 This release makes an implementation change to how Hypothesis handles
7415 certain internal constructs.
7416
7417 The main effect you should see is improvement to the behaviour and per‐
7418 formance of collection types, especially ones with a min_size parame‐
7419 ter. Many cases that would previously fail due to being unable to gen‐
7420 erate enough valid examples will now succeed, and other cases should
7421 run slightly faster.
7422
7423 3.16.0 - 2017-08-04
7424 This release introduces a deprecation of the timeout feature. This
7425 results in the following changes:
7426
7427 · Creating a settings object with an explicit timeout will emit a dep‐
7428 recation warning.
7429
7430 · If your test stops because it hits the timeout (and has not found a
7431 bug) then it will emit a deprecation warning.
7432
7433 · There is a new value unlimited which you can import from hypothesis.
7434 settings(timeout=unlimited) will not cause a deprecation warning.
7435
7436 · There is a new health check, hung_test, which will trigger after a
7437 test has been running for five minutes if it is not suppressed.
7438
7439 3.15.0 - 2017-08-04
7440 This release deprecates two strategies, choices() and streaming().
7441
7442 Both of these are somewhat confusing to use and are entirely redundant
7443 since the introduction of the data() strategy for interactive drawing
7444 in tests, and their use should be replaced with direct use of data()
7445 instead.
7446
7447 3.14.2 - 2017-08-03
7448 This fixes a bug where Hypothesis would not work correctly on Python
7449 2.7 if you had the python:typing module backport installed.
7450
7451 3.14.1 - 2017-08-02
7452 This raises the maximum depth at which Hypothesis starts cutting off
7453 data generation to a more reasonable value which it is harder to hit by
7454 accident.
7455
7456 This resolves (issue #751), in which some examples which previously
7457 worked would start timing out, but it will also likely improve the data
7458 generation quality for complex data types.
7459
7460 3.14.0 - 2017-07-23
7461 Hypothesis now understands inline type annotations (issue #293):
7462
7463 · If the target of builds() has type annotations, a default strategy
7464 for missing required arguments is selected based on the type.
7465 Type-based strategy selection will only override a default if you
7466 pass hypothesis.infer as a keyword argument.
7467
7468 · If @given wraps a function with type annotations, you can pass infer
7469 as a keyword argument and the appropriate strategy will be substi‐
7470 tuted.
7471
7472 · You can check what strategy will be inferred for a type with the new
7473 from_type() function.
7474
7475 · register_type_strategy() teaches Hypothesis which strategy to infer
7476 for custom or unknown types. You can provide a strategy, or for more
7477 complex cases a function which takes the type and returns a strategy.
7478
7479 3.13.1 - 2017-07-20
7480 This is a bug fix release for issue #514 - Hypothesis would continue
7481 running examples after a SkipTest exception was raised, including
7482 printing a falsifying example. Skip exceptions from the standard
7483 python:unittest module, and pytest, nose, or unittest2 modules now
7484 abort the test immediately without printing output.
7485
7486 3.13.0 - 2017-07-16
7487 This release has two major aspects to it: The first is the introduction
7488 of deferred(), which allows more natural definition of recursive
7489 (including mutually recursive) strategies.
7490
7491 The second is a number of engine changes designed to support this sort
7492 of strategy better. These should have a knock-on effect of also improv‐
7493 ing the performance of any existing strategies that currently generate
7494 a lot of data or involve heavy nesting by reducing their typical exam‐
7495 ple size.
7496
7497 3.12.0 - 2017-07-07
7498 This release makes some major internal changes to how Hypothesis repre‐
7499 sents data internally, as a prelude to some major engine changes that
7500 should improve data quality. There are no API changes, but it's a sig‐
7501 nificant enough internal change that a minor version bump seemed war‐
7502 ranted.
7503
7504 User facing impact should be fairly mild, but includes:
7505
7506 · All existing examples in the database will probably be invalidated.
7507 Hypothesis handles this automatically, so you don't need to do any‐
7508 thing, but if you see all your examples disappear that's why.
7509
7510 · Almost all data distributions have changed significantly. Possibly
7511 for the better, possibly for the worse. This may result in new bugs
7512 being found, but it may also result in Hypothesis being unable to
7513 find bugs it previously did.
7514
7515 · Data generation may be somewhat faster if your existing bottleneck
7516 was in draw_bytes (which is often the case for large examples).
7517
7518 · Shrinking will probably be slower, possibly significantly.
7519
7520 If you notice any effects you consider to be a significant regression,
7521 please open an issue about them.
7522
7523 3.11.6 - 2017-06-19
7524 This release involves no functionality changes, but is the first to
7525 ship wheels as well as an sdist.
7526
7527 3.11.5 - 2017-06-18
7528 This release provides a performance improvement to shrinking. For cases
7529 where there is some non-trivial "boundary" value (e.g. the bug happens
7530 for all values greater than some other value), shrinking should now be
7531 substantially faster. Other types of bug will likely see improvements
7532 too.
7533
7534 This may also result in some changes to the quality of the final exam‐
7535 ples - it may sometimes be better, but is more likely to get slightly
7536 worse in some edge cases. If you see any examples where this happens in
7537 practice, please report them.
7538
7539 3.11.4 - 2017-06-17
7540 This is a bugfix release: Hypothesis now prints explicit examples when
7541 running in verbose mode. (issue #313)
7542
7543 3.11.3 - 2017-06-11
7544 This is a bugfix release: Hypothesis no longer emits a warning if you
7545 try to use sampled_from() with python:collections.OrderedDict. (issue
7546 #688)
7547
7548 3.11.2 - 2017-06-10
7549 This is a documentation release. Several outdated snippets have been
7550 updated or removed, and many cross-references are now hyperlinks.
7551
7552 3.11.1 - 2017-05-28
7553 This is a minor ergonomics release. Tracebacks shown by pytest no
7554 longer include Hypothesis internals for test functions decorated with
7555 @given.
7556
7557 3.11.0 - 2017-05-23
7558 This is a feature release, adding datetime-related strategies to the
7559 core strategies.
7560
7561 timezones() allows you to sample pytz timezones from the Olsen data‐
7562 base. Use directly in a recipe for tz-aware datetimes, or compose with
7563 none() to allow a mix of aware and naive output.
7564
7565 The new dates(), times(), datetimes(), and timedeltas() strategies are
7566 all constrained by objects of their type. This means that you can gen‐
7567 erate dates bounded by a single day (i.e. a single date), or datetimes
7568 constrained to the microsecond.
7569
7570 times() and datetimes() take an optional timezones= argument, which
7571 defaults to none() for naive times. You can use our extra strategy
7572 based on pytz, or roll your own timezones strategy with dateutil or
7573 even the standard library.
7574
7575 The old dates, times, and datetimes strategies in hypothe‐
7576 sis.extra.datetimes are deprecated in favor of the new core strategies,
7577 which are more flexible and have no dependencies.
7578
7579 3.10.0 - 2017-05-22
7580 Hypothesis now uses python:inspect.getfullargspec() internally. On
7581 Python 2, there are no visible changes.
7582
7583 On Python 3 @given and @composite now preserve PEP 3107 annotations on
7584 the decorated function. Keyword-only arguments are now either handled
7585 correctly (e.g. @composite), or caught in validation instead of
7586 silently discarded or raising an unrelated error later (e.g. @given).
7587
7588 3.9.1 - 2017-05-22
7589 This is a bugfix release: the default field mapping for a DateTimeField
7590 in the Django extra now respects the USE_TZ setting when choosing a
7591 strategy.
7592
7593 3.9.0 - 2017-05-19
7594 This is feature release, expanding the capabilities of the decimals()
7595 strategy.
7596
7597 · The new (optional) places argument allows you to generate decimals
7598 with a certain number of places (e.g. cents, thousandths, satoshis).
7599
7600 · If allow_infinity is None, setting min_bound no longer excludes posi‐
7601 tive infinity and setting max_value no longer excludes negative
7602 infinity.
7603
7604 · All of NaN, -Nan, sNaN, and -sNaN may now be drawn if allow_nan is
7605 True, or if allow_nan is None and min_value or max_value is None.
7606
7607 · min_value and max_value may be given as decimal strings, e.g.
7608 "1.234".
7609
7610 3.8.5 - 2017-05-16
7611 Hypothesis now imports python:sqlite3 when a SQLite database is used,
7612 rather than at module load, improving compatibility with Python imple‐
7613 mentations compiled without SQLite support (such as BSD or Jython).
7614
7615 3.8.4 - 2017-05-16
7616 This is a compatibility bugfix release. sampled_from() no longer
7617 raises a deprecation warning when sampling from an python:enum.Enum, as
7618 all enums have a reliable iteration order.
7619
7620 3.8.3 - 2017-05-09
7621 This release removes a version check for older versions of pytest when
7622 using the Hypothesis pytest plugin. The pytest plugin will now run
7623 unconditionally on all versions of pytest. This breaks compatibility
7624 with any version of pytest prior to 2.7.0 (which is more than two years
7625 old).
7626
7627 The primary reason for this change is that the version check was a fre‐
7628 quent source of breakage when pytest change their versioning scheme. If
7629 you are not working on pytest itself and are not running a very old
7630 version of it, this release probably doesn't affect you.
7631
7632 3.8.2 - 2017-04-26
7633 This is a code reorganisation release that moves some internal test
7634 helpers out of the main source tree so as to not have changes to them
7635 trigger releases in future.
7636
7637 3.8.1 - 2017-04-26
7638 This is a documentation release. Almost all code examples are now
7639 doctests checked in CI, eliminating stale examples.
7640
7641 3.8.0 - 2017-04-23
7642 This is a feature release, adding the iterables() strategy, equivalent
7643 to lists(...).map(iter) but with a much more useful repr. You can use
7644 this strategy to check that code doesn't accidentally depend on
7645 sequence properties such as indexing support or repeated iteration.
7646
7647 3.7.4 - 2017-04-22
7648 This patch fixes a bug in 3.7.3, where using @example and a pytest fix‐
7649 ture in the same test could cause the test to fail to fill the argu‐
7650 ments, and throw a TypeError.
7651
7652 3.7.3 - 2017-04-21
7653 This release should include no user visible changes and is purely a
7654 refactoring release. This modularises the behaviour of the core given()
7655 function, breaking it up into smaller and more accessible parts, but
7656 its actual behaviour should remain unchanged.
7657
7658 3.7.2 - 2017-04-21
7659 This reverts an undocumented change in 3.7.1 which broke installation
7660 on debian stable: The specifier for the hypothesis[django]
7661 extra_requires had introduced a wild card, which was not supported on
7662 the default version of pip.
7663
7664 3.7.1 - 2017-04-21
7665 This is a bug fix and internal improvements release.
7666
7667 · In particular Hypothesis now tracks a tree of where it has already
7668 explored. This allows it to avoid some classes of duplicate exam‐
7669 ples, and significantly improves the performance of shrinking failing
7670 examples by allowing it to skip some shrinks that it can determine
7671 can't possibly work.
7672
7673 · Hypothesis will no longer seed the global random arbitrarily unless
7674 you have asked it to using random_module()
7675
7676 · Shrinking would previously have not worked correctly in some special
7677 cases on Python 2, and would have resulted in suboptimal examples.
7678
7679 3.7.0 - 2017-03-20
7680 This is a feature release.
7681
7682 New features:
7683
7684 · Rule based stateful testing now has an @invariant decorator that
7685 specifies methods that are run after init and after every step,
7686 allowing you to encode properties that should be true at all times.
7687 Thanks to Tom Prince for this feature.
7688
7689 · The decimals() strategy now supports allow_nan and allow_infinity
7690 flags.
7691
7692 · There are significantly more strategies available for numpy, includ‐
7693 ing for generating arbitrary data types. Thanks to Zac Hatfield Dodds
7694 for this feature.
7695
7696 · When using the data() strategy you can now add a label as an argument
7697 to draw(), which will be printed along with the value when an example
7698 fails. Thanks to Peter Inglesby for this feature.
7699
7700 Bug fixes:
7701
7702 · Bug fix: composite() now preserves functions' docstrings.
7703
7704 · The build is now reproducible and doesn't depend on the path you
7705 build it from. Thanks to Chris Lamb for this feature.
7706
7707 · numpy strategies for the void data type did not work correctly.
7708 Thanks to Zac Hatfield Dodds for this fix.
7709
7710 There have also been a number of performance optimizations:
7711
7712 · The permutations() strategy is now significantly faster to use for
7713 large lists (the underlying algorithm has gone from O(n^2) to O(n)).
7714
7715 · Shrinking of failing test cases should have got significantly faster
7716 in some circumstances where it was previously struggling for a long
7717 time.
7718
7719 · Example generation now involves less indirection, which results in a
7720 small speedup in some cases (small enough that you won't really
7721 notice it except in pathological cases).
7722
7723 3.6.1 - 2016-12-20
7724 This release fixes a dependency problem and makes some small behind the
7725 scenes improvements.
7726
7727 · The fake-factory dependency was renamed to faker. If you were depend‐
7728 ing on it through hypothesis[django] or hypothesis[fake-factory]
7729 without pinning it yourself then it would have failed to install
7730 properly. This release changes it so that hypothesis[fakefactory]
7731 (which can now also be installed as hypothesis[faker]) will install
7732 the renamed faker package instead.
7733
7734 · This release also removed the dependency of hypothesis[django] on
7735 hypothesis[fakefactory] - it was only being used for emails. These
7736 now use a custom strategy that isn't from fakefactory. As a result
7737 you should also see performance improvements of tests which generated
7738 User objects or other things with email fields, as well as better
7739 shrinking of email addresses.
7740
7741 · The distribution of code using nested calls to one_of() or the |
7742 operator for combining strategies has been improved, as branches are
7743 now flattened to give a more uniform distribution.
7744
7745 · Examples using composite() or .flatmap should now shrink better. In
7746 particular this will affect things which work by first generating a
7747 length and then generating that many items, which have historically
7748 not shrunk very well.
7749
7750 3.6.0 - 2016-10-31
7751 This release reverts Hypothesis to its old pretty printing of lambda
7752 functions based on attempting to extract the source code rather than
7753 decompile the bytecode. This is unfortunately slightly inferior in
7754 some cases and may result in you occasionally seeing things like lambda
7755 x: <unknown> in statistics reports and strategy reprs.
7756
7757 This removes the dependencies on uncompyle6, xdis and spark-parser.
7758
7759 The reason for this is that the new functionality was based on
7760 uncompyle6, which turns out to introduce a hidden GPLed dependency - it
7761 in turn depended on xdis, and although the library was licensed under
7762 the MIT license, it contained some GPL licensed source code and thus
7763 should have been released under the GPL.
7764
7765 My interpretation is that Hypothesis itself was never in violation of
7766 the GPL (because the license it is under, the Mozilla Public License
7767 v2, is fully compatible with being included in a GPL licensed work),
7768 but I have not consulted a lawyer on the subject. Regardless of the
7769 answer to this question, adding a GPLed dependency will likely cause a
7770 lot of users of Hypothesis to inadvertently be in violation of the GPL.
7771
7772 As a result, if you are running Hypothesis 3.5.x you really should
7773 upgrade to this release immediately.
7774
7775 3.5.3 - 2016-10-05
7776 This is a bug fix release.
7777
7778 Bugs fixed:
7779
7780 · If the same test was running concurrently in two processes and there
7781 were examples already in the test database which no longer failed,
7782 Hypothesis would sometimes fail with a FileNotFoundError (IOError on
7783 Python 2) because an example it was trying to read was deleted before
7784 it was read. (issue #372).
7785
7786 · Drawing from an integers() strategy with both a min_value and a
7787 max_value would reject too many examples needlessly. Now it repeat‐
7788 edly redraws until satisfied. (pull request #366. Thanks to Calen
7789 Pennington for the contribution).
7790
7791 3.5.2 - 2016-09-24
7792 This is a bug fix release.
7793
7794 · The Hypothesis pytest plugin broke pytest support for doctests. Now
7795 it doesn't.
7796
7797 3.5.1 - 2016-09-23
7798 This is a bug fix release.
7799
7800 · Hypothesis now runs cleanly in -B and -BB modes, avoiding mixing
7801 bytes and unicode.
7802
7803 · python:unittest.TestCase tests would not have shown up in the new
7804 statistics mode. Now they do.
7805
7806 · Similarly, stateful tests would not have shown up in statistics and
7807 now they do.
7808
7809 · Statistics now print with pytest node IDs (the names you'd get in
7810 pytest verbose mode).
7811
7812 3.5.0 - 2016-09-22
7813 This is a feature release.
7814
7815 · fractions() and decimals() strategies now support min_value and
7816 max_value parameters. Thanks go to Anne Mulhern for the development
7817 of this feature.
7818
7819 · The Hypothesis pytest plugin now supports a --hypothesis-show-statis‐
7820 tics parameter that gives detailed statistics about the tests that
7821 were run. Huge thanks to Jean-Louis Fuchs and Adfinis-SyGroup for
7822 funding the development of this feature.
7823
7824 · There is a new event() function that can be used to add custom sta‐
7825 tistics.
7826
7827 Additionally there have been some minor bug fixes:
7828
7829 · In some cases Hypothesis should produce fewer duplicate examples
7830 (this will mostly only affect cases with a single parameter).
7831
7832 · pytest command line parameters are now under an option group for
7833 Hypothesis (thanks to David Keijser for fixing this)
7834
7835 · Hypothesis would previously error if you used PEP 3107 function anno‐
7836 tations on your tests under Python 3.4.
7837
7838 · The repr of many strategies using lambdas has been improved to
7839 include the lambda body (this was previously supported in many but
7840 not all cases).
7841
7842 3.4.2 - 2016-07-13
7843 This is a bug fix release, fixing a number of problems with the set‐
7844 tings system:
7845
7846 · Test functions defined using @given can now be called from other
7847 threads (issue #337)
7848
7849 · Attempting to delete a settings property would previously have
7850 silently done the wrong thing. Now it raises an AttributeError.
7851
7852 · Creating a settings object with a custom database_file parameter was
7853 silently getting ignored and the default was being used instead. Now
7854 it's not.
7855
7856 3.4.1 - 2016-07-07
7857 This is a bug fix release for a single bug:
7858
7859 · On Windows when running two Hypothesis processes in parallel (e.g.
7860 using pytest-xdist) they could race with each other and one would
7861 raise an exception due to the non-atomic nature of file renaming on
7862 Windows and the fact that you can't rename over an existing file.
7863 This is now fixed.
7864
7865 3.4.0 - 2016-05-27
7866 This release is entirely provided by Lucas Wiman:
7867
7868 Strategies constructed by the Django extra will now respect much more
7869 of Django's validations out of the box. Wherever possible,
7870 full_clean() should succeed.
7871
7872 In particular:
7873
7874 · The max_length, blank and choices kwargs are now respected.
7875
7876 · Add support for DecimalField.
7877
7878 · If a field includes validators, the list of validators are used to
7879 filter the field strategy.
7880
7881 3.3.0 - 2016-05-27
7882 This release went wrong and is functionally equivalent to 3.2.0. Ignore
7883 it.
7884
7885 3.2.0 - 2016-05-19
7886 This is a small single-feature release:
7887
7888 · All tests using @given now fix the global random seed. This removes
7889 the health check for that. If a non-zero seed is required for the
7890 final falsifying example, it will be reported. Otherwise Hypothesis
7891 will assume randomization was not a significant factor for the test
7892 and be silent on the subject. If you use random_module() this will
7893 continue to work and will always display the seed.
7894
7895 3.1.3 - 2016-05-01
7896 Single bug fix release
7897
7898 · Another charmap problem. In 3.1.2 text() and characters() would break
7899 on systems which had /tmp mounted on a different partition than the
7900 Hypothesis storage directory (usually in home). This fixes that.
7901
7902 3.1.2 - 2016-04-30
7903 Single bug fix release:
7904
7905 · Anything which used a text() or characters() strategy was broken on
7906 Windows and I hadn't updated appveyor to use the new repository loca‐
7907 tion so I didn't notice. This is now fixed and windows support should
7908 work correctly.
7909
7910 3.1.1 - 2016-04-29
7911 Minor bug fix release.
7912
7913 · Fix concurrency issue when running tests that use text() from multi‐
7914 ple processes at once (issue #302, thanks to Alex Chan).
7915
7916 · Improve performance of code using lists() with max_size (thanks to
7917 Cristi Cobzarenco).
7918
7919 · Fix install on Python 2 with ancient versions of pip so that it
7920 installs the enum34 backport (thanks to Donald Stufft for telling me
7921 how to do this).
7922
7923 · Remove duplicated __all__ exports from hypothesis.strategies (thanks
7924 to Piët Delport).
7925
7926 · Update headers to point to new repository location.
7927
7928 · Allow use of strategies that can't be used in find() (e.g. choices())
7929 in stateful testing.
7930
7931 3.1.0 - 2016-03-06
7932 · Add a nothing() strategy that never successfully generates values.
7933
7934 · sampled_from() and one_of() can both now be called with an empty
7935 argument list, in which case they also never generate any values.
7936
7937 · one_of() may now be called with a single argument that is a collec‐
7938 tion of strategies as well as as varargs.
7939
7940 · Add a runner() strategy which returns the instance of the current
7941 test object if there is one.
7942
7943 · 'Bundle' for RuleBasedStateMachine is now a normal(ish) strategy and
7944 can be used as such.
7945
7946 · Tests using RuleBasedStateMachine should now shrink significantly
7947 better.
7948
7949 · Hypothesis now uses a pretty-printing library internally, compatible
7950 with IPython's pretty printing protocol (actually using the same
7951 code). This may improve the quality of output in some cases.
7952
7953 · Add a 'phases' setting that allows more fine grained control over
7954 which parts of the process Hypothesis runs
7955
7956 · Add a suppress_health_check setting which allows you to turn off spe‐
7957 cific health checks in a fine grained manner.
7958
7959 · Fix a bug where lists of non fixed size would always draw one more
7960 element than they included. This mostly didn't matter, but if would
7961 cause problems with empty strategies or ones with side effects.
7962
7963 · Add a mechanism to the Django model generator to allow you to explic‐
7964 itly request the default value (thanks to Jeremy Thurgood for this
7965 one).
7966
7967 3.0.5 - 2016-02-25
7968 · Fix a bug where Hypothesis would now error on pytest development ver‐
7969 sions.
7970
7971 3.0.4 - 2016-02-24
7972 · Fix a bug where Hypothesis would error when running on Python 2.7.3
7973 or earlier because it was trying to pass a python:bytearray object to
7974 python:struct.unpack() (which is only supported since 2.7.4).
7975
7976 3.0.3 - 2016-02-23
7977 · Fix version parsing of pytest to work with pytest release candidates
7978
7979 · More general handling of the health check problem where things could
7980 fail because of a cache miss - now one "free" example is generated
7981 before the start of the health check run.
7982
7983 3.0.2 - 2016-02-18
7984 · Under certain circumstances, strategies involving text() buried
7985 inside some other strategy (e.g. text().filter(...) or recur‐
7986 sive(text(), ...)) would cause a test to fail its health checks the
7987 first time it ran. This was caused by having to compute some related
7988 data and cache it to disk. On travis or anywhere else where the
7989 .hypothesis directory was recreated this would have caused the tests
7990 to fail their health check on every run. This is now fixed for all
7991 the known cases, although there could be others lurking.
7992
7993 3.0.1 - 2016-02-18
7994 · Fix a case where it was possible to trigger an "Unreachable" asser‐
7995 tion when running certain flaky stateful tests.
7996
7997 · Improve shrinking of large stateful tests by eliminating a case where
7998 it was hard to delete early steps.
7999
8000 · Improve efficiency of drawing binary(min_size=n, max_size=n) signifi‐
8001 cantly by provide a custom implementation for fixed size blocks that
8002 can bypass a lot of machinery.
8003
8004 · Set default home directory based on the current working directory at
8005 the point Hypothesis is imported, not whenever the function first
8006 happens to be called.
8007
8008 3.0.0 - 2016-02-17
8009 Codename: This really should have been 2.1.
8010
8011 Externally this looks like a very small release. It has one small
8012 breaking change that probably doesn't affect anyone at all (some behav‐
8013 iour that never really worked correctly is now outright forbidden) but
8014 necessitated a major version bump and one visible new feature.
8015
8016 Internally this is a complete rewrite. Almost nothing other than the
8017 public API is the same.
8018
8019 New features:
8020
8021 · Addition of data() strategy which allows you to draw arbitrary data
8022 interactively within the test.
8023
8024 · New "exploded" database format which allows you to more easily check
8025 the example database into a source repository while supporting merg‐
8026 ing.
8027
8028 · Better management of how examples are saved in the database.
8029
8030 · Health checks will now raise as errors when they fail. It was too
8031 easy to have the warnings be swallowed entirely.
8032
8033 New limitations:
8034
8035 · choices() and streaming() strategies may no longer be used with
8036 find(). Neither may data() (this is the change that necessitated a
8037 major version bump).
8038
8039 Feature removal:
8040
8041 · The ForkingTestCase executor has gone away. It may return in some
8042 more working form at a later date.
8043
8044 Performance improvements:
8045
8046 · A new model which allows flatmap, composite strategies and stateful
8047 testing to perform much better. They should also be more reliable.
8048
8049 · Filtering may in some circumstances have improved significantly. This
8050 will help especially in cases where you have lots of values with
8051 individual filters on them, such as lists(x.filter(...)).
8052
8053 · Modest performance improvements to the general test runner by avoid‐
8054 ing expensive operations
8055
8056 In general your tests should have got faster. If they've instead got
8057 significantly slower, I'm interested in hearing about it.
8058
8059 Data distribution:
8060
8061 The data distribution should have changed significantly. This may
8062 uncover bugs the previous version missed. It may also miss bugs the
8063 previous version could have uncovered. Hypothesis is now producing less
8064 strongly correlated data than it used to, but the correlations are
8065 extended over more of the structure.
8066
8067 Shrinking:
8068
8069 Shrinking quality should have improved. In particular Hypothesis can
8070 now perform simultaneous shrinking of separate examples within a single
8071 test (previously it was only able to do this for elements of a single
8072 collection). In some cases performance will have improved, in some
8073 cases it will have got worse but generally shouldn't have by much.
8074
8075 2.0.0 - 2016-01-10
8076 Codename: A new beginning
8077
8078 This release cleans up all of the legacy that accrued in the course of
8079 Hypothesis 1.0. These are mostly things that were emitting deprecation
8080 warnings in 1.19.0, but there were a few additional changes.
8081
8082 In particular:
8083
8084 · non-strategy values will no longer be converted to strategies when
8085 used in given or find.
8086
8087 · FailedHealthCheck is now an error and not a warning.
8088
8089 · Handling of non-ascii reprs in user types have been simplified by
8090 using raw strings in more places in Python 2.
8091
8092 · given no longer allows mixing positional and keyword arguments.
8093
8094 · given no longer works with functions with defaults.
8095
8096 · given no longer turns provided arguments into defaults - they will
8097 not appear in the argspec at all.
8098
8099 · the basic() strategy no longer exists.
8100
8101 · the n_ary_tree strategy no longer exists.
8102
8103 · the average_list_length setting no longer exists. Note: If you're
8104 using using recursive() this will cause you a significant slow down.
8105 You should pass explicit average_size parameters to collections in
8106 recursive calls.
8107
8108 · @rule can no longer be applied to the same method twice.
8109
8110 · Python 2.6 and 3.3 are no longer officially supported, although in
8111 practice they still work fine.
8112
8113 This also includes two non-deprecation changes:
8114
8115 · given's keyword arguments no longer have to be the rightmost argu‐
8116 ments and can appear anywhere in the method signature.
8117
8118 · The max_shrinks setting would sometimes not have been respected.
8119
8120 1.19.0 - 2016-01-09
8121 Codename: IT COMES
8122
8123 This release heralds the beginning of a new and terrible age of Hypoth‐
8124 esis 2.0.
8125
8126 It's primary purpose is some final deprecations prior to said release.
8127 The goal is that if your code emits no warnings under this release then
8128 it will probably run unchanged under Hypothesis 2.0 (there are some
8129 caveats to this: 2.0 will drop support for some Python versions, and if
8130 you're using internal APIs then as usual that may break without warn‐
8131 ing).
8132
8133 It does have two new features:
8134
8135 · New @seed() decorator which allows you to manually seed a test. This
8136 may be harmlessly combined with and overrides the derandomize set‐
8137 ting.
8138
8139 · settings objects may now be used as a decorator to fix those settings
8140 to a particular @given test.
8141
8142 API changes (old usage still works but is deprecated):
8143
8144 · Settings has been renamed to settings (lower casing) in order to make
8145 the decorator usage more natural.
8146
8147 · Functions for the storage directory that were in hypothesis.settings
8148 are now in a new hypothesis.configuration module.
8149
8150 Additional deprecations:
8151
8152 · the average_list_length setting has been deprecated in favour of
8153 being explicit.
8154
8155 · the basic() strategy has been deprecated as it is impossible to sup‐
8156 port it under a Conjecture based model, which will hopefully be
8157 implemented at some point in the 2.x series.
8158
8159 · the n_ary_tree strategy (which was never actually part of the public
8160 API) has been deprecated.
8161
8162 · Passing settings or random as keyword arguments to given is depre‐
8163 cated (use the new functionality instead)
8164
8165 Bug fixes:
8166
8167 · No longer emit PendingDeprecationWarning for __iter__ and StopItera‐
8168 tion in streaming() values.
8169
8170 · When running in health check mode with non strict, don't print quite
8171 so many errors for an exception in reify.
8172
8173 · When an assumption made in a test or a filter is flaky, tests will
8174 now raise Flaky instead of UnsatisfiedAssumption.
8175
8176 1.18.1 - 2015-12-22
8177 Two behind the scenes changes:
8178
8179 · Hypothesis will no longer write generated code to the file system.
8180 This will improve performance on some systems (e.g. if you're using
8181 PythonAnywhere which is running your code from NFS) and prevent some
8182 annoying interactions with auto-restarting systems.
8183
8184 · Hypothesis will cache the creation of some strategies. This can sig‐
8185 nificantly improve performance for code that uses flatmap or compos‐
8186 ite and thus has to instantiate strategies a lot.
8187
8188 1.18.0 - 2015-12-21
8189 Features:
8190
8191 · Tests and find are now explicitly seeded off the global random mod‐
8192 ule. This means that if you nest one inside the other you will now
8193 get a health check error. It also means that you can control global
8194 randomization by seeding random.
8195
8196 · There is a new random_module() strategy which seeds the global random
8197 module for you and handles things so that you don't get a health
8198 check warning if you use it inside your tests.
8199
8200 · floats() now accepts two new arguments: allow_nan and allow_infinity.
8201 These default to the old behaviour, but when set to False will do
8202 what the names suggest.
8203
8204 Bug fixes:
8205
8206 · Fix a bug where tests that used text() on Python 3.4+ would not actu‐
8207 ally be deterministic even when explicitly seeded or using the deran‐
8208 domize mode, because generation depended on dictionary iteration
8209 order which was affected by hash randomization.
8210
8211 · Fix a bug where with complicated strategies the timing of the initial
8212 health check could affect the seeding of the subsequent test, which
8213 would also render supposedly deterministic tests non-deterministic in
8214 some scenarios.
8215
8216 · In some circumstances flatmap() could get confused by two struc‐
8217 turally similar things it could generate and would produce a flaky
8218 test where the first time it produced an error but the second time it
8219 produced the other value, which was not an error. The same bug was
8220 presumably also possible in composite().
8221
8222 · flatmap() and composite() initial generation should now be moderately
8223 faster. This will be particularly noticeable when you have many val‐
8224 ues drawn from the same strategy in a single run, e.g. constructs
8225 like lists(s.flatmap(f)). Shrinking performance may have suffered,
8226 but this didn't actually produce an interestingly worse result in any
8227 of the standard scenarios tested.
8228
8229 1.17.1 - 2015-12-16
8230 A small bug fix release, which fixes the fact that the 'note' function
8231 could not be used on tests which used the @example decorator to provide
8232 explicit examples.
8233
8234 1.17.0 - 2015-12-15
8235 This is actually the same release as 1.16.1, but 1.16.1 has been pulled
8236 because it contains the following additional change that was not
8237 intended to be in a patch release (it's perfectly stable, but is a
8238 larger change that should have required a minor version bump):
8239
8240 · Hypothesis will now perform a series of "health checks" as part of
8241 running your tests. These detect and warn about some common error
8242 conditions that people often run into which wouldn't necessarily have
8243 caused the test to fail but would cause e.g. degraded performance or
8244 confusing results.
8245
8246 1.16.1 - 2015-12-14
8247 Note: This release has been removed.
8248
8249 A small bugfix release that allows bdists for Hypothesis to be built
8250 under 2.7 - the compat3.py file which had Python 3 syntax wasn't
8251 intended to be loaded under Python 2, but when building a bdist it was.
8252 In particular this would break running setup.py test.
8253
8254 1.16.0 - 2015-12-08
8255 There are no public API changes in this release but it includes a be‐
8256 haviour change that I wasn't comfortable putting in a patch release.
8257
8258 · Functions from hypothesis.strategies will no longer raise InvalidAr‐
8259 gument on bad arguments. Instead the same errors will be raised when
8260 a test using such a strategy is run. This may improve startup time in
8261 some cases, but the main reason for it is so that errors in strate‐
8262 gies won't cause errors in loading, and it can interact correctly
8263 with things like pytest.mark.skipif.
8264
8265 · Errors caused by accidentally invoking the legacy API are now much
8266 less confusing, although still throw NotImplementedError.
8267
8268 · hypothesis.extra.django is 1.9 compatible.
8269
8270 · When tests are run with max_shrinks=0 this will now still rerun the
8271 test on failure and will no longer print "Trying example:" before
8272 each run. Additionally note() will now work correctly when used with
8273 max_shrinks=0.
8274
8275 1.15.0 - 2015-11-24
8276 A release with two new features.
8277
8278 · A 'characters' strategy for more flexible generation of text with
8279 particular character ranges and types, kindly contributed by
8280 Alexander Shorin.
8281
8282 · Add support for preconditions to the rule based stateful testing.
8283 Kindly contributed by Christopher Armstrong
8284
8285 1.14.0 - 2015-11-01
8286 New features:
8287
8288 · Add 'note' function which lets you include additional information in
8289 the final test run's output.
8290
8291 · Add 'choices' strategy which gives you a choice function that emu‐
8292 lates random.choice.
8293
8294 · Add 'uuid' strategy that generates UUIDs'
8295
8296 · Add 'shared' strategy that lets you create a strategy that just gen‐
8297 erates a single shared value for each test run
8298
8299 Bugs:
8300
8301 · Using strategies of the form streaming(x.flatmap(f)) with find or in
8302 stateful testing would have caused InvalidArgument errors when the
8303 resulting values were used (because code that expected to only be
8304 called within a test context would be invoked).
8305
8306 1.13.0 - 2015-10-29
8307 This is quite a small release, but deprecates some public API functions
8308 and removes some internal API functionality so gets a minor version
8309 bump.
8310
8311 · All calls to the 'strategy' function are now deprecated, even ones
8312 which pass just a SearchStrategy instance (which is still a no-op).
8313
8314 · Never documented hypothesis.extra entry_points mechanism has now been
8315 removed ( it was previously how hypothesis.extra packages were loaded
8316 and has been deprecated and unused for some time)
8317
8318 · Some corner cases that could previously have produced an OverflowEr‐
8319 ror when simplifying failing cases using hypothesis.extra.datetimes
8320 (or dates or times) have now been fixed.
8321
8322 · Hypothesis load time for first import has been significantly reduced
8323 - it used to be around 250ms (on my SSD laptop) and now is around
8324 100-150ms. This almost never matters but was slightly annoying when
8325 using it in the console.
8326
8327 · hypothesis.strategies.randoms was previously missing from __all__.
8328
8329 1.12.0 - 2015-10-18
8330 · Significantly improved performance of creating strategies using the
8331 functions from the hypothesis.strategies module by deferring the cal‐
8332 culation of their repr until it was needed. This is unlikely to have
8333 been an performance issue for you unless you were using flatmap, com‐
8334 posite or stateful testing, but for some cases it could be quite a
8335 significant impact.
8336
8337 · A number of cases where the repr of strategies build from lambdas is
8338 improved
8339
8340 · Add dates() and times() strategies to hypothesis.extra.datetimes
8341
8342 · Add new 'profiles' mechanism to the settings system
8343
8344 · Deprecates mutability of Settings, both the Settings.default top
8345 level property and individual settings.
8346
8347 · A Settings object may now be directly initialized from a parent Set‐
8348 tings.
8349
8350 · @given should now give a better error message if you attempt to use
8351 it with a function that uses destructuring arguments (it still won't
8352 work, but it will error more clearly),
8353
8354 · A number of spelling corrections in error messages
8355
8356 · pytest should no longer display the intermediate modules Hypothesis
8357 generates when running in verbose mode
8358
8359 · Hypothesis should now correctly handle printing objects with
8360 non-ascii reprs on python 3 when running in a locale that cannot han‐
8361 dle ascii printing to stdout.
8362
8363 · Add a unique=True argument to lists(). This is equivalent to
8364 unique_by=lambda x: x, but offers a more convenient syntax.
8365
8366 1.11.4 - 2015-09-27
8367 · Hide modifications Hypothesis needs to make to sys.path by undoing
8368 them after we've imported the relevant modules. This is a workaround
8369 for issues cryptography experienced on windows.
8370
8371 · Slightly improved performance of drawing from sampled_from on large
8372 lists of alternatives.
8373
8374 · Significantly improved performance of drawing from one_of or strate‐
8375 gies using | (note this includes a lot of strategies internally -
8376 floats() and integers() both fall into this category). There turned
8377 out to be a massive performance regression introduced in 1.10.0
8378 affecting these which probably would have made tests using Hypothesis
8379 significantly slower than they should have been.
8380
8381 1.11.3 - 2015-09-23
8382 · Better argument validation for datetimes() strategy - previously set‐
8383 ting max_year < datetime.MIN_YEAR or min_year > datetime.MAX_YEAR
8384 would not have raised an InvalidArgument error and instead would have
8385 behaved confusingly.
8386
8387 · Compatibility with being run on pytest < 2.7 (achieved by disabling
8388 the plugin).
8389
8390 1.11.2 - 2015-09-23
8391 Bug fixes:
8392
8393 · Settings(database=my_db) would not be correctly inherited when used
8394 as a default setting, so that newly created settings would use the
8395 database_file setting and create an SQLite example database.
8396
8397 · Settings.default.database = my_db would previously have raised an
8398 error and now works.
8399
8400 · Timeout could sometimes be significantly exceeded if during simplifi‐
8401 cation there were a lot of examples tried that didn't trigger the
8402 bug.
8403
8404 · When loading a heavily simplified example using a basic() strategy
8405 from the database this could cause Python to trigger a recursion
8406 error.
8407
8408 · Remove use of deprecated API in pytest plugin so as to not emit warn‐
8409 ing
8410
8411 Misc:
8412
8413 · hypothesis-pytest is now part of hypothesis core. This should have no
8414 externally visible consequences, but you should update your dependen‐
8415 cies to remove hypothesis-pytest and depend on only Hypothesis.
8416
8417 · Better repr for hypothesis.extra.datetimes() strategies.
8418
8419 · Add .close() method to abstract base class for Backend (it was
8420 already present in the main implementation).
8421
8422 1.11.1 - 2015-09-16
8423 Bug fixes:
8424
8425 · When running Hypothesis tests in parallel (e.g. using pytest-xdist)
8426 there was a race condition caused by code generation.
8427
8428 · Example databases are now cached per thread so as to not use sqlite
8429 connections from multiple threads. This should make Hypothesis now
8430 entirely thread safe.
8431
8432 · floats() with only min_value or max_value set would have had a very
8433 bad distribution.
8434
8435 · Running on 3.5, Hypothesis would have emitted deprecation warnings
8436 because of use of inspect.getargspec
8437
8438 1.11.0 - 2015-08-31
8439 · text() with a non-string alphabet would have used the repr() of the
8440 the alphabet instead of its contexts. This is obviously silly. It now
8441 works with any sequence of things convertible to unicode strings.
8442
8443 · @given will now work on methods whose definitions contains no
8444 explicit positional arguments, only varargs (issue #118). This may
8445 have some knock on effects because it means that @given no longer
8446 changes the argspec of functions other than by adding defaults.
8447
8448 · Introduction of new @composite feature for more natural definition of
8449 strategies you'd previously have used flatmap for.
8450
8451 1.10.6 - 2015-08-26
8452 Fix support for fixtures on Django 1.7.
8453
8454 1.10.4 - 2015-08-21
8455 Tiny bug fix release:
8456
8457 · If the database_file setting is set to None, this would have resulted
8458 in an error when running tests. Now it does the same as setting data‐
8459 base to None.
8460
8461 1.10.3 - 2015-08-19
8462 Another small bug fix release.
8463
8464 · lists(elements, unique_by=some_function, min_size=n) would have
8465 raised a ValidationError if n > Settings.default.average_list_length
8466 because it would have wanted to use an average list length shorter
8467 than the minimum size of the list, which is impossible. Now it
8468 instead defaults to twice the minimum size in these circumstances.
8469
8470 · basic() strategy would have only ever produced at most ten distinct
8471 values per run of the test (which is bad if you e.g. have it inside a
8472 list). This was obviously silly. It will now produce a much better
8473 distribution of data, both duplicated and non duplicated.
8474
8475 1.10.2 - 2015-08-19
8476 This is a small bug fix release:
8477
8478 · star imports from hypothesis should now work correctly.
8479
8480 · example quality for examples using flatmap will be better, as the way
8481 it had previously been implemented was causing problems where Hypoth‐
8482 esis was erroneously labelling some examples as being duplicates.
8483
8484 1.10.0 - 2015-08-04
8485 This is just a bugfix and performance release, but it changes some
8486 semi-public APIs, hence the minor version bump.
8487
8488 · Significant performance improvements for strategies which are
8489 one_of() many branches. In particular this included recursive()
8490 strategies. This should take the case where you use one recursive()
8491 strategy as the base strategy of another from unusably slow (tens of
8492 seconds per generated example) to reasonably fast.
8493
8494 · Better handling of just() and sampled_from() for values which have an
8495 incorrect __repr__ implementation that returns non-ASCII unicode on
8496 Python 2.
8497
8498 · Better performance for flatmap from changing the internal morpher API
8499 to be significantly less general purpose.
8500
8501 · Introduce a new semi-public BuildContext/cleanup API. This allows
8502 strategies to register cleanup activities that should run once the
8503 example is complete. Note that this will interact somewhat weirdly
8504 with find.
8505
8506 · Better simplification behaviour for streaming strategies.
8507
8508 · Don't error on lambdas which use destructuring arguments in Python 2.
8509
8510 · Add some better reprs for a few strategies that were missing good
8511 ones.
8512
8513 · The Random instances provided by randoms() are now copyable.
8514
8515 · Slightly more debugging information about simplify when using a debug
8516 verbosity level.
8517
8518 · Support using given for functions with varargs, but not passing argu‐
8519 ments to it as positional.
8520
8521 1.9.0 - 2015-07-27
8522 Codename: The great bundling.
8523
8524 This release contains two fairly major changes.
8525
8526 The first is the deprecation of the hypothesis-extra mechanism. From
8527 now on all the packages that were previously bundled under it other
8528 than hypothesis-pytest (which is a different beast and will remain sep‐
8529 arate). The functionality remains unchanged and you can still import
8530 them from exactly the same location, they just are no longer separate
8531 packages.
8532
8533 The second is that this introduces a new way of building strategies
8534 which lets you build up strategies recursively from other strategies.
8535
8536 It also contains the minor change that calling .example() on a strategy
8537 object will give you examples that are more representative of the
8538 actual data you'll get. There used to be some logic in there to make
8539 the examples artificially simple but this proved to be a bad idea.
8540
8541 1.8.5 - 2015-07-24
8542 This contains no functionality changes but fixes a mistake made with
8543 building the previous package that would have broken installation on
8544 Windows.
8545
8546 1.8.4 - 2015-07-20
8547 Bugs fixed:
8548
8549 · When a call to floats() had endpoints which were not floats but
8550 merely convertible to one (e.g. integers), these would be included in
8551 the generated data which would cause it to generate non-floats.
8552
8553 · Splitting lambdas used in the definition of flatmap, map or filter
8554 over multiple lines would break the repr, which would in turn break
8555 their usage.
8556
8557 1.8.3 - 2015-07-20
8558 "Falsifying example" would not have been printed when the failure came
8559 from an explicit example.
8560
8561 1.8.2 - 2015-07-18
8562 Another small bugfix release:
8563
8564 · When using ForkingTestCase you would usually not get the falsifying
8565 example printed if the process exited abnormally (e.g. due to
8566 os._exit).
8567
8568 · Improvements to the distribution of characters when using text() with
8569 a default alphabet. In particular produces a better distribution of
8570 ascii and whitespace in the alphabet.
8571
8572 1.8.1 - 2015-07-17
8573 This is a small release that contains a workaround for people who have
8574 bad reprs returning non ascii text on Python 2.7. This is not a bug fix
8575 for Hypothesis per se because that's not a thing that is actually sup‐
8576 posed to work, but Hypothesis leans more heavily on repr than is typi‐
8577 cal so it's worth having a workaround for.
8578
8579 1.8.0 - 2015-07-16
8580 New features:
8581
8582 · Much more sensible reprs for strategies, especially ones that come
8583 from hypothesis.strategies. These should now have as reprs python
8584 code that would produce the same strategy.
8585
8586 · lists() accepts a unique_by argument which forces the generated lists
8587 to be only contain elements unique according to some function key
8588 (which must return a hashable value).
8589
8590 · Better error messages from flaky tests to help you debug things.
8591
8592 Mostly invisible implementation details that may result in finding new
8593 bugs in your code:
8594
8595 · Sets and dictionary generation should now produce a better range of
8596 results.
8597
8598 · floats with bounds now focus more on 'critical values', trying to
8599 produce values at edge cases.
8600
8601 · flatmap should now have better simplification for complicated cases,
8602 as well as generally being (I hope) more reliable.
8603
8604 Bug fixes:
8605
8606 · You could not previously use assume() if you were using the forking
8607 executor.
8608
8609 1.7.2 - 2015-07-10
8610 This is purely a bug fix release:
8611
8612 · When using floats() with stale data in the database you could some‐
8613 times get values in your tests that did not respect min_value or
8614 max_value.
8615
8616 · When getting a Flaky error from an unreliable test it would have
8617 incorrectly displayed the example that caused it.
8618
8619 · 2.6 dependency on backports was incorrectly specified. This would
8620 only have caused you problems if you were building a universal wheel
8621 from Hypothesis, which is not how Hypothesis ships, so unless you're
8622 explicitly building wheels for your dependencies and support Python
8623 2.6 plus a later version of Python this probably would never have
8624 affected you.
8625
8626 · If you use flatmap in a way that the strategy on the right hand side
8627 depends sensitively on the left hand side you may have occasionally
8628 seen Flaky errors caused by producing unreliable examples when mini‐
8629 mizing a bug. This use case may still be somewhat fraught to be hon‐
8630 est. This code is due a major rearchitecture for 1.8, but in the
8631 meantime this release fixes the only source of this error that I'm
8632 aware of.
8633
8634 1.7.1 - 2015-06-29
8635 Codename: There is no 1.7.0.
8636
8637 A slight technical hitch with a premature upload means there's was a
8638 yanked 1.7.0 release. Oops.
8639
8640 The major feature of this release is Python 2.6 support. Thanks to Jeff
8641 Meadows for doing most of the work there.
8642
8643 Other minor features
8644
8645 · strategies now has a permutations() function which returns a strategy
8646 yielding permutations of values from a given collection.
8647
8648 · if you have a flaky test it will print the exception that it last saw
8649 before failing with Flaky, even if you do not have verbose reporting
8650 on.
8651
8652 · Slightly experimental git merge script available as "python -m
8653 hypothesis.tools.mergedbs". Instructions on how to use it in the doc‐
8654 string of that file.
8655
8656 Bug fixes:
8657
8658 · Better performance from use of filter. In particular tests which
8659 involve large numbers of heavily filtered strategies should perform a
8660 lot better.
8661
8662 · floats() with a negative min_value would not have worked correctly
8663 (worryingly, it would have just silently failed to run any examples).
8664 This is now fixed.
8665
8666 · tests using sampled_from would error if the number of sampled ele‐
8667 ments was smaller than min_satisfying_examples.
8668
8669 1.6.2 - 2015-06-08
8670 This is just a few small bug fixes:
8671
8672 · Size bounds were not validated for values for a binary() strategy
8673 when reading examples from the database.
8674
8675 · sampled_from is now in __all__ in hypothesis.strategies
8676
8677 · floats no longer consider negative integers to be simpler than posi‐
8678 tive non-integers
8679
8680 · Small floating point intervals now correctly count members, so if you
8681 have a floating point interval so narrow there are only a handful of
8682 values in it, this will no longer cause an error when Hypothesis runs
8683 out of values.
8684
8685 1.6.1 - 2015-05-21
8686 This is a small patch release that fixes a bug where 1.6.0 broke the
8687 use of flatmap with the deprecated API and assumed the passed in func‐
8688 tion returned a SearchStrategy instance rather than converting it to a
8689 strategy.
8690
8691 1.6.0 - 2015-05-21
8692 This is a smallish release designed to fix a number of bugs and smooth
8693 out some weird behaviours.
8694
8695 · Fix a critical bug in flatmap where it would reuse old strategies. If
8696 all your flatmap code was pure you're fine. If it's not, I'm sur‐
8697 prised it's working at all. In particular if you want to use flatmap
8698 with django models, you desperately need to upgrade to this version.
8699
8700 · flatmap simplification performance should now be better in some cases
8701 where it previously had to redo work.
8702
8703 · Fix for a bug where invalid unicode data with surrogates could be
8704 generated during simplification (it was already filtered out during
8705 actual generation).
8706
8707 · The Hypothesis database is now keyed off the name of the test instead
8708 of the type of data. This makes much more sense now with the new
8709 strategies API and is generally more robust. This means you will lose
8710 old examples on upgrade.
8711
8712 · The database will now not delete values which fail to deserialize
8713 correctly, just skip them. This is to handle cases where multiple
8714 incompatible strategies share the same key.
8715
8716 · find now also saves and loads values from the database, keyed off a
8717 hash of the function you're finding from.
8718
8719 · Stateful tests now serialize and load values from the database. They
8720 should have before, really. This was a bug.
8721
8722 · Passing a different verbosity level into a test would not have worked
8723 entirely correctly, leaving off some messages. This is now fixed.
8724
8725 · Fix a bug where derandomized tests with unicode characters in the
8726 function body would error on Python 2.7.
8727
8728 1.5.0 - 2015-05-14
8729 Codename: Strategic withdrawal.
8730
8731 The purpose of this release is a radical simplification of the API for
8732 building strategies. Instead of the old approach of @strategy.extend
8733 and things that get converted to strategies, you just build strategies
8734 directly.
8735
8736 The old method of defining strategies will still work until Hypothesis
8737 2.0, because it's a major breaking change, but will now emit depreca‐
8738 tion warnings.
8739
8740 The new API is also a lot more powerful as the functions for defining
8741 strategies give you a lot of dials to turn. See the updated data sec‐
8742 tion for details.
8743
8744 Other changes:
8745
8746 · Mixing keyword and positional arguments in a call to @given is
8747 deprecated as well.
8748
8749 · There is a new setting called 'strict'. When set to True, Hypothe‐
8750 sis will raise warnings instead of merely printing them. Turning
8751 it on by default is inadvisable because it means that Hypothesis
8752 minor releases can break your code, but it may be useful for mak‐
8753 ing sure you catch all uses of deprecated APIs.
8754
8755 · max_examples in settings is now interpreted as meaning the maximum
8756 number of unique (ish) examples satisfying assumptions. A new set‐
8757 ting max_iterations which defaults to a larger value has the old
8758 interpretation.
8759
8760 · Example generation should be significantly faster due to a new
8761 faster parameter selection algorithm. This will mostly show up for
8762 simple data types - for complex ones the parameter selection is
8763 almost certainly dominated.
8764
8765 · Simplification has some new heuristics that will tend to cut down
8766 on cases where it could previously take a very long time.
8767
8768 · timeout would previously not have been respected in cases where
8769 there were a lot of duplicate examples. You probably wouldn't have
8770 previously noticed this because max_examples counted duplicates,
8771 so this was very hard to hit in a way that mattered.
8772
8773 · A number of internal simplifications to the SearchStrategy API.
8774
8775 · You can now access the current Hypothesis version as hypothe‐
8776 sis.__version__.
8777
8778 · A top level function is provided for running the stateful tests
8779 without the TestCase infrastructure.
8780
8781 1.4.0 - 2015-05-04
8782 Codename: What a state.
8783
8784 The big feature of this release is the new and slightly experimental
8785 stateful testing API. You can read more about that in the appropriate
8786 section.
8787
8788 Two minor features the were driven out in the course of developing
8789 this:
8790
8791 · You can now set settings.max_shrinks to limit the number of times
8792 Hypothesis will try to shrink arguments to your test. If this is set
8793 to <= 0 then Hypothesis will not rerun your test and will just raise
8794 the failure directly. Note that due to technical limitations if
8795 max_shrinks is <= 0 then Hypothesis will print every example it calls
8796 your test with rather than just the failing one. Note also that I
8797 don't consider settings max_shrinks to zero a sensible way to run
8798 your tests and it should really be considered a debug feature.
8799
8800 · There is a new debug level of verbosity which is even more verbose
8801 than verbose. You probably don't want this.
8802
8803 Breakage of semi-public SearchStrategy API:
8804
8805 · It is now a required invariant of SearchStrategy that if u simplifies
8806 to v then it is not the case that strictly_simpler(u, v). i.e. sim‐
8807 plifying should not increase the complexity even though it is not
8808 required to decrease it. Enforcing this invariant lead to finding
8809 some bugs where simplifying of integers, floats and sets was subopti‐
8810 mal.
8811
8812 · Integers in basic data are now required to fit into 64 bits. As a
8813 result python integer types are now serialized as strings, and some
8814 types have stopped using quite so needlessly large random seeds.
8815
8816 Hypothesis Stateful testing was then turned upon Hypothesis itself,
8817 which lead to an amazing number of minor bugs being found in Hypothesis
8818 itself.
8819
8820 Bugs fixed (most but not all from the result of stateful testing)
8821 include:
8822
8823 · Serialization of streaming examples was flaky in a way that you would
8824 probably never notice: If you generate a template, simplify it, seri‐
8825 alize it, deserialize it, serialize it again and then deserialize it
8826 you would get the original stream instead of the simplified one.
8827
8828 · If you reduced max_examples below the number of examples already
8829 saved in the database, you would have got a ValueError. Additionally,
8830 if you had more than max_examples in the database all of them would
8831 have been considered.
8832
8833 · @given will no longer count duplicate examples (which it never called
8834 your function with) towards max_examples. This may result in your
8835 tests running slower, but that's probably just because they're trying
8836 more examples.
8837
8838 · General improvements to example search which should result in better
8839 performance and higher quality examples. In particular parameters
8840 which have a history of producing useless results will be more
8841 aggressively culled. This is useful both because it decreases the
8842 chance of useless examples and also because it's much faster to not
8843 check parameters which we were unlikely to ever pick!
8844
8845 · integers_from and lists of types with only one value (e.g. [None])
8846 would previously have had a very high duplication rate so you were
8847 probably only getting a handful of examples. They now have a much
8848 lower duplication rate, as well as the improvements to search making
8849 this less of a problem in the first place.
8850
8851 · You would sometimes see simplification taking significantly longer
8852 than your defined timeout. This would happen because timeout was only
8853 being checked after each successful simplification, so if Hypothesis
8854 was spending a lot of time unsuccessfully simplifying things it
8855 wouldn't stop in time. The timeout is now applied for unsuccessful
8856 simplifications too.
8857
8858 · In Python 2.7, integers_from strategies would have failed during sim‐
8859 plification with an OverflowError if their starting point was at or
8860 near to the maximum size of a 64-bit integer.
8861
8862 · flatmap and map would have failed if called with a function without a
8863 __name__ attribute.
8864
8865 · If max_examples was less than min_satisfying_examples this would
8866 always error. Now min_satisfying_examples is capped to max_examples.
8867 Note that if you have assumptions to satisfy here this will still
8868 cause an error.
8869
8870 Some minor quality improvements:
8871
8872 · Lists of streams, flatmapped strategies and basic strategies should
8873 now now have slightly better simplification.
8874
8875 1.3.0 - 2015-05-22
8876 New features:
8877
8878 · New verbosity level API for printing intermediate results and excep‐
8879 tions.
8880
8881 · New specifier for strings generated from a specified alphabet.
8882
8883 · Better error messages for tests that are failing because of a lack of
8884 enough examples.
8885
8886 Bug fixes:
8887
8888 · Fix error where use of ForkingTestCase would sometimes result in too
8889 many open files.
8890
8891 · Fix error where saving a failing example that used flatmap could
8892 error.
8893
8894 · Implement simplification for sampled_from, which apparently never
8895 supported it previously. Oops.
8896
8897 General improvements:
8898
8899 · Better range of examples when using one_of or sampled_from.
8900
8901 · Fix some pathological performance issues when simplifying lists of
8902 complex values.
8903
8904 · Fix some pathological performance issues when simplifying examples
8905 that require unicode strings with high codepoints.
8906
8907 · Random will now simplify to more readable examples.
8908
8909 1.2.1 - 2015-04-16
8910 A small patch release for a bug in the new executors feature. Tests
8911 which require doing something to their result in order to fail would
8912 have instead reported as flaky.
8913
8914 1.2.0 - 2015-04-15
8915 Codename: Finders keepers.
8916
8917 A bunch of new features and improvements.
8918
8919 · Provide a mechanism for customizing how your tests are executed.
8920
8921 · Provide a test runner that forks before running each example. This
8922 allows better support for testing native code which might trigger a
8923 segfault or a C level assertion failure.
8924
8925 · Support for using Hypothesis to find examples directly rather than as
8926 just as a test runner.
8927
8928 · New streaming type which lets you generate infinite lazily loaded
8929 streams of data - perfect for if you need a number of examples but
8930 don't know how many.
8931
8932 · Better support for large integer ranges. You can now use inte‐
8933 gers_in_range with ranges of basically any size. Previously large
8934 ranges would have eaten up all your memory and taken forever.
8935
8936 · Integers produce a wider range of data than before - previously they
8937 would only rarely produce integers which didn't fit into a machine
8938 word. Now it's much more common. This percolates to other numeric
8939 types which build on integers.
8940
8941 · Better validation of arguments to @given. Some situations that would
8942 previously have caused silently wrong behaviour will now raise an
8943 error.
8944
8945 · Include +/- sys.float_info.max in the set of floating point edge
8946 cases that Hypothesis specifically tries.
8947
8948 · Fix some bugs in floating point ranges which happen when given +/-
8949 sys.float_info.max as one of the endpoints... (really any two floats
8950 that are sufficiently far apart so that x, y are finite but y - x is
8951 infinite). This would have resulted in generating infinite values
8952 instead of ones inside the range.
8953
8954 1.1.1 - 2015-04-07
8955 Codename: Nothing to see here
8956
8957 This is just a patch release put out because it fixed some internal
8958 bugs that would block the Django integration release but did not actu‐
8959 ally affect anything anyone could previously have been using. It also
8960 contained a minor quality fix for floats that I'd happened to have fin‐
8961 ished in time.
8962
8963 · Fix some internal bugs with object lifecycle management that were
8964 impossible to hit with the previously released versions but broke
8965 hypothesis-django.
8966
8967 · Bias floating point numbers somewhat less aggressively towards very
8968 small numbers
8969
8970 1.1.0 - 2015-04-06
8971 Codename: No-one mention the M word.
8972
8973 · Unicode strings are more strongly biased towards ascii characters.
8974 Previously they would generate all over the space. This is mostly so
8975 that people who try to shape their unicode strings with assume() have
8976 less of a bad time.
8977
8978 · A number of fixes to data deserialization code that could theoreti‐
8979 cally have caused mysterious bugs when using an old version of a
8980 Hypothesis example database with a newer version. To the best of my
8981 knowledge a change that could have triggered this bug has never actu‐
8982 ally been seen in the wild. Certainly no-one ever reported a bug of
8983 this nature.
8984
8985 · Out of the box support for Decimal and Fraction.
8986
8987 · new dictionary specifier for dictionaries with variable keys.
8988
8989 · Significantly faster and higher quality simplification, especially
8990 for collections of data.
8991
8992 · New filter() and flatmap() methods on Strategy for better ways of
8993 building strategies out of other strategies.
8994
8995 · New BasicStrategy class which allows you to define your own strate‐
8996 gies from scratch without needing an existing matching strategy or
8997 being exposed to the full horror or non-public nature of the Search‐
8998 Strategy interface.
8999
9000 1.0.0 - 2015-03-27
9001 Codename: Blast-off!
9002
9003 There are no code changes in this release. This is precisely the 0.9.2
9004 release with some updated documentation.
9005
9006 0.9.2 - 2015-03-26
9007 Codename: T-1 days.
9008
9009 · floats_in_range would not actually have produced floats_in_range
9010 unless that range happened to be (0, 1). Fix this.
9011
9012 0.9.1 - 2015-03-25
9013 Codename: T-2 days.
9014
9015 · Fix a bug where if you defined a strategy using map on a lambda then
9016 the results would not be saved in the database.
9017
9018 · Significant performance improvements when simplifying examples using
9019 lists, strings or bounded integer ranges.
9020
9021 0.9.0 - 2015-03-23
9022 Codename: The final countdown
9023
9024 This release could also be called 1.0-RC1.
9025
9026 It contains a teeny tiny bugfix, but the real point of this release is
9027 to declare feature freeze. There will be zero functionality changes
9028 between 0.9.0 and 1.0 unless something goes really really wrong. No new
9029 features will be added, no breaking API changes will occur, etc. This
9030 is the final shakedown before I declare Hypothesis stable and ready to
9031 use and throw a party to celebrate.
9032
9033 Bug bounty for any bugs found between now and 1.0: I will buy you a
9034 drink (alcoholic, caffeinated, or otherwise) and shake your hand should
9035 we ever find ourselves in the same city at the same time.
9036
9037 The one tiny bugfix:
9038
9039 · Under pypy, databases would fail to close correctly when garbage col‐
9040 lected, leading to a memory leak and a confusing error message if you
9041 were repeatedly creating databases and not closing them. It is very
9042 unlikely you were doing this and the chances of you ever having
9043 noticed this bug are very low.
9044
9045 0.7.2 - 2015-03-22
9046 Codename: Hygienic macros or bust
9047
9048 · You can now name an argument to @given 'f' and it won't break (issue
9049 #38)
9050
9051 · strategy_test_suite is now named strategy_test_suite as the documen‐
9052 tation claims and not in fact strategy_test_suitee
9053
9054 · Settings objects can now be used as a context manager to temporarily
9055 override the default values inside their context.
9056
9057 0.7.1 - 2015-03-21
9058 Codename: Point releases go faster
9059
9060 · Better string generation by parametrizing by a limited alphabet
9061
9062 · Faster string simplification - previously if simplifying a string
9063 with high range unicode characters it would try every unicode charac‐
9064 ter smaller than that. This was pretty pointless. Now it stops after
9065 it's a short range (it can still reach smaller ones through recursive
9066 calls because of other simplifying operations).
9067
9068 · Faster list simplification by first trying a binary chop down the
9069 middle
9070
9071 · Simultaneous simplification of identical elements in a list. So if a
9072 bug only triggers when you have duplicates but you drew e.g. [-17,
9073 -17], this will now simplify to [0, 0].
9074
9075 0.7.0, - 2015-03-20
9076 Codename: Starting to look suspiciously real
9077
9078 This is probably the last minor release prior to 1.0. It consists of
9079 stability improvements, a few usability things designed to make Hypoth‐
9080 esis easier to try out, and filing off some final rough edges from the
9081 API.
9082
9083 · Significant speed and memory usage improvements
9084
9085 · Add an example() method to strategy objects to give an example of the
9086 sort of data that the strategy generates.
9087
9088 · Remove .descriptor attribute of strategies
9089
9090 · Rename descriptor_test_suite to strategy_test_suite
9091
9092 · Rename the few remaining uses of descriptor to specifier (descriptor
9093 already has a defined meaning in Python)
9094
9095 0.6.0 - 2015-03-13
9096 Codename: I'm sorry, were you using that API?
9097
9098 This is primarily a "simplify all the weird bits of the API" release.
9099 As a result there are a lot of breaking changes. If you just use @given
9100 with core types then you're probably fine.
9101
9102 In particular:
9103
9104 · Stateful testing has been removed from the API
9105
9106 · The way the database is used has been rendered less useful (sorry).
9107 The feature for reassembling values saved from other tests doesn't
9108 currently work. This will probably be brought back in post 1.0.
9109
9110 · SpecificationMapper is no longer a thing. Instead there is an
9111 ExtMethod called strategy which you extend to specify how to convert
9112 other types to strategies.
9113
9114 · Settings are now extensible so you can add your own for configuring a
9115 strategy
9116
9117 · MappedSearchStrategy no longer needs an unpack method
9118
9119 · Basically all the SearchStrategy internals have changed massively. If
9120 you implemented SearchStrategy directly rather than using Mapped‐
9121 SearchStrategy talk to me about fixing it.
9122
9123 · Change to the way extra packages work. You now specify the package.
9124 This must have a load() method. Additionally any modules in the pack‐
9125 age will be loaded in under hypothesis.extra
9126
9127 Bug fixes:
9128
9129 · Fix for a bug where calling falsify on a lambda with a non-ascii
9130 character in its body would error.
9131
9132 Hypothesis Extra:
9133
9134 hypothesis-fakefactory: An extension for using faker data in hypothe‐
9135 sis. Depends
9136 on fake-factory.
9137
9138 0.5.0 - 2015-02-10
9139 Codename: Read all about it.
9140
9141 Core hypothesis:
9142
9143 · Add support back in for pypy and python 3.2
9144
9145 · @given functions can now be invoked with some arguments explicitly
9146 provided. If all arguments that hypothesis would have provided are
9147 passed in then no falsification is run.
9148
9149 · Related to the above, this means that you can now use pytest fixtures
9150 and mark.parametrize with Hypothesis without either interfering with
9151 the other.
9152
9153 · Breaking change: @given no longer works for functions with varargs
9154 (varkwargs are fine). This might be added back in at a later date.
9155
9156 · Windows is now fully supported. A limited version (just the tests
9157 with none of the extras) of the test suite is run on windows with
9158 each commit so it is now a first class citizen of the Hypothesis
9159 world.
9160
9161 · Fix a bug for fuzzy equality of equal complex numbers with different
9162 reprs (this can happen when one coordinate is zero). This shouldn't
9163 affect users - that feature isn't used anywhere public facing.
9164
9165 · Fix generation of floats on windows and 32-bit builds of python. I
9166 was using some struct.pack logic that only worked on certain word
9167 sizes.
9168
9169 · When a test times out and hasn't produced enough examples this now
9170 raises a Timeout subclass of Unfalsifiable.
9171
9172 · Small search spaces are better supported. Previously something like a
9173 @given(bool, bool) would have failed because it couldn't find enough
9174 examples. Hypothesis is now aware of the fact that these are small
9175 search spaces and will not error in this case.
9176
9177 · Improvements to parameter search in the case of hard to satisfy
9178 assume. Hypothesis will now spend less time exploring parameters that
9179 are unlikely to provide anything useful.
9180
9181 · Increase chance of generating "nasty" floats
9182
9183 · Fix a bug that would have caused unicode warnings if you had a sam‐
9184 pled_from that was mixing unicode and byte strings.
9185
9186 · Added a standard test suite that you can use to validate a custom
9187 strategy you've defined is working correctly.
9188
9189 Hypothesis extra:
9190
9191 First off, introducing Hypothesis extra packages!
9192
9193 These are packages that are separated out from core Hypothesis because
9194 they have one or more dependencies. Every hypothesis-extra package is
9195 pinned to a specific point release of Hypothesis and will have some
9196 version requirements on its dependency. They use entry_points so you
9197 will usually not need to explicitly import them, just have them
9198 installed on the path.
9199
9200 This release introduces two of them:
9201
9202 hypothesis-datetime:
9203
9204 Does what it says on the tin: Generates datetimes for Hypothesis. Just
9205 install the package and datetime support will start working.
9206
9207 Depends on pytz for timezone support
9208
9209 hypothesis-pytest:
9210
9211 A very rudimentary pytest plugin. All it does right now is hook the
9212 display of falsifying examples into pytest reporting.
9213
9214 Depends on pytest.
9215
9216 0.4.3 - 2015-02-05
9217 Codename: TIL narrow Python builds are a thing
9218
9219 This just fixes the one bug.
9220
9221 · Apparently there is such a thing as a "narrow python build" and OS X
9222 ships with these by default for python 2.7. These are builds where
9223 you only have two bytes worth of unicode. As a result, generating
9224 unicode was completely broken on OS X. Fix this by only generating
9225 unicode codepoints in the range supported by the system.
9226
9227 0.4.2 - 2015-02-04
9228 Codename: O(dear)
9229
9230 This is purely a bugfix release:
9231
9232 · Provide sensible external hashing for all core types. This will sig‐
9233 nificantly improve performance of tracking seen examples which hap‐
9234 pens in literally every falsification run. For Hypothesis fixing this
9235 cut 40% off the runtime of the test suite. The behaviour is quadratic
9236 in the number of examples so if you're running the default configura‐
9237 tion this will be less extreme (Hypothesis's test suite runs at a
9238 higher number of examples than default), but you should still see a
9239 significant improvement.
9240
9241 · Fix a bug in formatting of complex numbers where the string could get
9242 incorrectly truncated.
9243
9244 0.4.1 - 2015-02-03
9245 Codename: Cruel and unusual edge cases
9246
9247 This release is mostly about better test case generation.
9248
9249 Enhancements:
9250
9251 · Has a cool release name
9252
9253 · text_type (str in python 3, unicode in python 2) example generation
9254 now actually produces interesting unicode instead of boring ascii
9255 strings.
9256
9257 · floating point numbers are generated over a much wider range, with
9258 particular attention paid to generating nasty numbers - nan, infin‐
9259 ity, large and small values, etc.
9260
9261 · examples can be generated using pieces of examples previously saved
9262 in the database. This allows interesting behaviour that has previ‐
9263 ously been discovered to be propagated to other examples.
9264
9265 · improved parameter exploration algorithm which should allow it to
9266 more reliably hit interesting edge cases.
9267
9268 · Timeout can now be disabled entirely by setting it to any value <= 0.
9269
9270 Bug fixes:
9271
9272 · The descriptor on a OneOfStrategy could be wrong if you had descrip‐
9273 tors which were equal but should not be coalesced. e.g. a strategy
9274 for one_of((frozenset({int}), {int})) would have reported its
9275 descriptor as {int}. This is unlikely to have caused you any problems
9276
9277 · If you had strategies that could produce NaN (which float previously
9278 couldn't but e.g. a Just(float('nan')) could) then this would have
9279 sent hypothesis into an infinite loop that would have only been ter‐
9280 minated when it hit the timeout.
9281
9282 · Given elements that can take a long time to minimize, minimization of
9283 floats or tuples could be quadratic or worse in the that value. You
9284 should now see much better performance for simplification, albeit at
9285 some cost in quality.
9286
9287 Other:
9288
9289 · A lot of internals have been been rewritten. This shouldn't affect
9290 you at all, but it opens the way for certain of hypothesis's oddities
9291 to be a lot more extensible by users. Whether this is a good thing
9292 may be up for debate...
9293
9294 0.4.0 - 2015-01-21
9295 FLAGSHIP FEATURE: Hypothesis now persists examples for later use. It
9296 stores data in a local SQLite database and will reuse it for all tests
9297 of the same type.
9298
9299 LICENSING CHANGE: Hypothesis is now released under the Mozilla Public
9300 License 2.0. This applies to all versions from 0.4.0 onwards until fur‐
9301 ther notice. The previous license remains applicable to all code prior
9302 to 0.4.0.
9303
9304 Enhancements:
9305
9306 · Printing of failing examples. I was finding that the pytest runner
9307 was not doing a good job of displaying these, and that Hypothesis
9308 itself could do much better.
9309
9310 · Drop dependency on six for cross-version compatibility. It was easy
9311 enough to write the shim for the small set of features that we care
9312 about and this lets us avoid a moderately complex dependency.
9313
9314 · Some improvements to statistical distribution of selecting from small
9315 (<= 3 elements)
9316
9317 · Improvements to parameter selection for finding examples.
9318
9319 Bugs fixed:
9320
9321 · could_have_produced for lists, dicts and other collections would not
9322 have examined the elements and thus when using a union of different
9323 types of list this could result in Hypothesis getting confused and
9324 passing a value to the wrong strategy. This could potentially result
9325 in exceptions being thrown from within simplification.
9326
9327 · sampled_from would not work correctly on a single element list.
9328
9329 · Hypothesis could get very confused by values which are equal despite
9330 having different types being used in descriptors. Hypothesis now has
9331 its own more specific version of equality it uses for descriptors and
9332 tracking. It is always more fine grained than Python equality: Things
9333 considered != are not considered equal by hypothesis, but some things
9334 that are considered == are distinguished. If your test suite uses
9335 both frozenset and set tests this bug is probably affecting you.
9336
9337 0.3.2 - 2015-01-16
9338 · Fix a bug where if you specified floats_in_range with integer argu‐
9339 ments Hypothesis would error in example simplification.
9340
9341 · Improve the statistical distribution of the floats you get for the
9342 floats_in_range strategy. I'm not sure whether this will affect users
9343 in practice but it took my tests for various conditions from flaky to
9344 rock solid so it at the very least improves discovery of the artifi‐
9345 cial cases I'm looking for.
9346
9347 · Improved repr() for strategies and RandomWithSeed instances.
9348
9349 · Add detection for flaky test cases where hypothesis managed to find
9350 an example which breaks it but on the final invocation of the test it
9351 does not raise an error. This will typically happen with too much
9352 recursion errors but could conceivably happen in other circumstances
9353 too.
9354
9355 · Provide a "derandomized" mode. This allows you to run hypothesis with
9356 zero real randomization, making your build nice and deterministic.
9357 The tests run with a seed calculated from the function they're test‐
9358 ing so you should still get a good distribution of test cases.
9359
9360 · Add a mechanism for more conveniently defining tests which just sam‐
9361 ple from some collection.
9362
9363 · Fix for a really subtle bug deep in the internals of the strategy ta‐
9364 ble. In some circumstances if you were to define instance strategies
9365 for both a parent class and one or more of its subclasses you would
9366 under some circumstances get the strategy for the wrong superclass of
9367 an instance. It is very unlikely anyone has ever encountered this in
9368 the wild, but it is conceivably possible given that a mix of namedtu‐
9369 ple and tuple are used fairly extensively inside hypothesis which do
9370 exhibit this pattern of strategy.
9371
9372 0.3.1 - 2015-01-13
9373 · Support for generation of frozenset and Random values
9374
9375 · Correct handling of the case where a called function mutates it argu‐
9376 ment. This involved introducing a notion of a strategies knowing how
9377 to copy their argument. The default method should be entirely accept‐
9378 able and the worst case is that it will continue to have the old be‐
9379 haviour if you don't mark your strategy as mutable, so this shouldn't
9380 break anything.
9381
9382 · Fix for a bug where some strategies did not correctly implement
9383 could_have_produced. It is very unlikely that any of these would have
9384 been seen in the wild, and the consequences if they had been would
9385 have been minor.
9386
9387 · Re-export the @given decorator from the main hypothesis namespace.
9388 It's still available at the old location too.
9389
9390 · Minor performance optimisation for simplifying long lists.
9391
9392 0.3.0 - 2015-01-12
9393 · Complete redesign of the data generation system. Extreme breaking
9394 change for anyone who was previously writing their own SearchStrategy
9395 implementations. These will not work any more and you'll need to mod‐
9396 ify them.
9397
9398 · New settings system allowing more global and modular control of Veri‐
9399 fier behaviour.
9400
9401 · Decouple SearchStrategy from the StrategyTable. This leads to much
9402 more composable code which is a lot easier to understand.
9403
9404 · A significant amount of internal API renaming and moving. This may
9405 also break your code.
9406
9407 · Expanded available descriptors, allowing for generating integers or
9408 floats in a specific range.
9409
9410 · Significantly more robust. A very large number of small bug fixes,
9411 none of which anyone is likely to have ever noticed.
9412
9413 · Deprecation of support for pypy and python 3 prior to 3.3. 3.3 and
9414 3.4. Supported versions are 2.7.x, 3.3.x, 3.4.x. I expect all of
9415 these to remain officially supported for a very long time. I would
9416 not be surprised to add pypy support back in later but I'm not going
9417 to do so until I know someone cares about it. In the meantime it will
9418 probably still work.
9419
9420 0.2.2 - 2015-01-08
9421 · Fix an embarrassing complete failure of the installer caused by my
9422 being bad at version control
9423
9424 0.2.1 - 2015-01-07
9425 · Fix a bug in the new stateful testing feature where you could make
9426 __init__ a @requires method. Simplification would not always work if
9427 the prune method was able to successfully shrink the test.
9428
9429 0.2.0 - 2015-01-07
9430 · It's aliiive.
9431
9432 · Improve python 3 support using six.
9433
9434 · Distinguish between byte and unicode types.
9435
9436 · Fix issues where FloatStrategy could raise.
9437
9438 · Allow stateful testing to request constructor args.
9439
9440 · Fix for issue where test annotations would timeout based on when the
9441 module was loaded instead of when the test started
9442
9443 0.1.4 - 2013-12-14
9444 · Make verification runs time bounded with a configurable timeout
9445
9446 0.1.3 - 2013-05-03
9447 · Bugfix: Stateful testing behaved incorrectly with subclassing.
9448
9449 · Complex number support
9450
9451 · support for recursive strategies
9452
9453 · different error for hypotheses with unsatisfiable assumptions
9454
9455 0.1.2 - 2013-03-24
9456 · Bugfix: Stateful testing was not minimizing correctly and could throw
9457 exceptions.
9458
9459 · Better support for recursive strategies.
9460
9461 · Support for named tuples.
9462
9463 · Much faster integer generation.
9464
9465 0.1.1 - 2013-03-24
9466 · Python 3.x support via 2to3.
9467
9468 · Use new style classes (oops).
9469
9470 0.1.0 - 2013-03-23
9471 · Introduce stateful testing.
9472
9473 · Massive rewrite of internals to add flags and strategies.
9474
9475 0.0.5 - 2013-03-13
9476 · No changes except trying to fix packaging
9477
9478 0.0.4 - 2013-03-13
9479 · No changes except that I checked in a failing test case for 0.0.3 so
9480 had to replace the release. Doh
9481
9482 0.0.3 - 2013-03-13
9483 · Improved a few internals.
9484
9485 · Opened up creating generators from instances as a general API.
9486
9487 · Test integration.
9488
9489 0.0.2 - 2013-03-12
9490 · Starting to tighten up on the internals.
9491
9492 · Change API to allow more flexibility in configuration.
9493
9494 · More testing.
9495
9496 0.0.1 - 2013-03-10
9497 · Initial release.
9498
9499 · Basic working prototype. Demonstrates idea, probably shouldn't be
9500 used.
9501
9503 Hypothesis development is managed by me, David R. MacIver. I am the
9504 primary author of Hypothesis.
9505
9506 However, I no longer do unpaid feature development on Hypothesis. My
9507 roles as leader of the project are:
9508
9509 1. Helping other people do feature development on Hypothesis
9510
9511 2. Fixing bugs and other code health issues
9512
9513 3. Improving documentation
9514
9515 4. General release management work
9516
9517 5. Planning the general roadmap of the project
9518
9519 6. Doing sponsored development on tasks that are too large or in depth
9520 for other people to take on
9521
9522 So all new features must either be sponsored or implemented by someone
9523 else. That being said, the maintenance team takes an active role in
9524 shepherding pull requests and helping people write a new feature (see
9525 CONTRIBUTING.rst for details and pull request #154 for an example of
9526 how the process goes). This isn't "patches welcome", it's "we will help
9527 you write a patch".
9528
9529 Release Policy
9530 Hypothesis releases follow semantic versioning.
9531
9532 We maintain backwards-compatibility wherever possible, and use depreca‐
9533 tion warnings to mark features that have been superseded by a newer
9534 alternative. If you want to detect this, you can upgrade warnings to
9535 errors in the usual ways.
9536
9537 We use continuous deployment to ensure that you can always use our new‐
9538 est and shiniest features - every change to the source tree is automat‐
9539 ically built and published on PyPI as soon as it's merged onto master,
9540 after code review and passing our extensive test suite.
9541
9542 Project Roadmap
9543 Hypothesis does not have a long-term release plan. However some visi‐
9544 bility into our plans for future compatibility may be useful:
9545
9546 · We value compatibility, and maintain it as far as practical. This
9547 generally excludes things which are end-of-life upstream, or have an
9548 unstable API.
9549
9550 · We would like to drop Python 2 support when it reaches end of life in
9551 2020. Ongoing support is likely to depend on commercial funding.
9552
9554 For questions you are happy to ask in public, the Hypothesis community
9555 is a friendly place where I or others will be more than happy to help
9556 you out. You're also welcome to ask questions on Stack Overflow. If you
9557 do, please tag them with 'python-hypothesis' so someone sees them.
9558
9559 For bugs and enhancements, please file an issue on the GitHub issue
9560 tracker. Note that as per the development policy, enhancements will
9561 probably not get implemented unless you're willing to pay for develop‐
9562 ment or implement them yourself (with assistance from me). Bugs will
9563 tend to get fixed reasonably promptly, though it is of course on a best
9564 effort basis.
9565
9566 To see the versions of Python, optional dependencies, test runners, and
9567 operating systems Hypothesis supports (meaning incompatibility is
9568 treated as a bug), see supported.
9569
9570 If you need to ask questions privately or want more of a guarantee of
9571 bugs being fixed promptly, please contact me on
9572 hypothesis-support@drmaciver.com to talk about availability of support
9573 contracts.
9574
9576 Downstream packagers often want to package Hypothesis. Here are some
9577 guidelines.
9578
9579 The primary guideline is this: If you are not prepared to keep up with
9580 the Hypothesis release schedule, don't. You will annoy me and are doing
9581 your users a disservice.
9582
9583 Hypothesis has a very frequent release schedule. It's rare that it goes
9584 a week without a release, and there are often multiple releases in a
9585 given week.
9586
9587 If you are prepared to keep up with this schedule, you might find the
9588 rest of this document useful.
9589
9590 Release tarballs
9591 These are available from the GitHub releases page. The tarballs on PyPI
9592 are intended for installation from a Python tool such as pip and should
9593 not be considered complete releases. Requests to include additional
9594 files in them will not be granted. Their absence is not a bug.
9595
9596 Dependencies
9597 Python versions
9598 Hypothesis is designed to work with a range of Python versions. We
9599 always support all versisions of CPython with upstream support, and
9600 plan to drop Python 2 at EOL in 2020. We also support the latest ver‐
9601 sions of PyPy for Python 3, and for Python 2 until the CPython 2 EOL.
9602
9603 If you feel the need to have separate Python 3 and Python 2 packages
9604 you can, but Hypothesis works unmodified on either.
9605
9606 Other Python libraries
9607 Hypothesis has mandatory dependencies on the following libraries:
9608
9609 · attrs
9610
9611 · enum34 is required on Python 2.7
9612
9613 Hypothesis has optional dependencies on the following libraries:
9614
9615 · pytz (almost any version should work)
9616
9617 · Django, all supported versions
9618
9619 · numpy, 1.10 or later (earlier versions will probably work fine)
9620
9621 · pandas, 1.19 or later
9622
9623 · pytest (3.0 or greater). This is a mandatory dependency for testing
9624 Hypothesis itself but optional for users.
9625
9626 The way this works when installing Hypothesis normally is that these
9627 features become available if the relevant library is installed.
9628
9629 Testing Hypothesis
9630 If you want to test Hypothesis as part of your packaging you will prob‐
9631 ably not want to use the mechanisms Hypothesis itself uses for running
9632 its tests, because it has a lot of logic for installing and testing
9633 against different versions of Python.
9634
9635 The tests must be run with pytest >= 3.0; check the requirements/
9636 directory for details.
9637
9638 The organisation of the tests is described in the
9639 hypothesis-python/tests/README.rst.
9640
9641 Examples
9642 · arch linux
9643
9644 · fedora
9645
9646 · gentoo
9647
9649 One of the things that is often concerning for people using randomized
9650 testing is the question of how to reproduce failing test cases.
9651
9652 NOTE:
9653 It is better to think about the data Hypothesis generates as being
9654 arbitrary, rather than random. We deliberately generate any valid
9655 data that seems likely to cause errors, so you shouldn't rely on any
9656 expected distribution of or relationships between generated data.
9657 You can read about "swarm testing" and "coverage guided fuzzing" if
9658 you're interested, because you don't need to know for Hypothesis!
9659
9660 Fortunately Hypothesis has a number of features to support reproducing
9661 test failures. The one you will use most commonly when developing
9662 locally is the example database, which means that you shouldn't have to
9663 think about the problem at all for local use - test failures will just
9664 automatically reproduce without you having to do anything.
9665
9666 The example database is perfectly suitable for sharing between
9667 machines, but there currently aren't very good work flows for that, so
9668 Hypothesis provides a number of ways to make examples reproducible by
9669 adding them to the source code of your tests. This is particularly use‐
9670 ful when e.g. you are trying to run an example that has failed on your
9671 CI, or otherwise share them between machines.
9672
9673 Providing explicit examples
9674 The simplest way to reproduce a failed test is to ask Hypothesis to run
9675 the failing example it printed. For example, if Falsifying example:
9676 test(n=1) was printed you can decorate test with @example(n=1).
9677
9678 @example can also be used to ensure a specific example is always exe‐
9679 cuted as a regression test or to cover some edge case - basically com‐
9680 bining a Hypothesis test and a traditional parametrized test.
9681
9682 hypothesis.example(*args, **kwargs)
9683 A decorator which ensures a specific example is always tested.
9684
9685 Hypothesis will run all examples you've asked for first. If any of them
9686 fail it will not go on to look for more examples.
9687
9688 It doesn't matter whether you put the example decorator before or after
9689 given. Any permutation of the decorators in the above will do the same
9690 thing.
9691
9692 Note that examples can be positional or keyword based. If they're posi‐
9693 tional then they will be filled in from the right when calling, so
9694 either of the following styles will work as expected:
9695
9696 @given(text())
9697 @example("Hello world")
9698 @example(x="Some very long string")
9699 def test_some_code(x):
9700 assert True
9701
9702 from unittest import TestCase
9703
9704 class TestThings(TestCase):
9705 @given(text())
9706 @example("Hello world")
9707 @example(x="Some very long string")
9708 def test_some_code(self, x):
9709 assert True
9710
9711 As with @given, it is not permitted for a single example to be a mix of
9712 positional and keyword arguments. Either are fine, and you can use one
9713 in one example and the other in another example if for some reason you
9714 really want to, but a single example must be consistent.
9715
9716 Reproducing a test run with @seed
9717 hypothesis.seed(seed)
9718 seed: Start the test execution from a specific seed.
9719
9720 May be any hashable object. No exact meaning for seed is pro‐
9721 vided other than that for a fixed seed value Hypothesis will try
9722 the same actions (insofar as it can given external sources of
9723 non- determinism. e.g. timing and hash randomization).
9724
9725 Overrides the derandomize setting, which is designed to enable
9726 deterministic builds rather than reproducing observed failures.
9727
9728 When a test fails unexpectedly, usually due to a health check failure,
9729 Hypothesis will print out a seed that led to that failure, if the test
9730 is not already running with a fixed seed. You can then recreate that
9731 failure using either the @seed decorator or (if you are running pytest)
9732 with --hypothesis-seed.
9733
9734 The seed will not be printed if you could simply use @example instead.
9735
9736 Reproducing an example with @reproduce_failure
9737 Hypothesis has an opaque binary representation that it uses for all
9738 examples it generates. This representation is not intended to be stable
9739 across versions or with respect to changes in the test, but can be used
9740 to to reproduce failures with the @reproduce_example decorator.
9741
9742 hypothesis.reproduce_failure(version, blob)
9743 Run the example that corresponds to this data blob in order to
9744 reproduce a failure.
9745
9746 A test with this decorator always runs only one example and
9747 always fails. If the provided example does not cause a failure,
9748 or is in some way invalid for this test, then this will fail
9749 with a DidNotReproduce error.
9750
9751 This decorator is not intended to be a permanent addition to
9752 your test suite. It's simply some code you can add to ease
9753 reproduction of a problem in the event that you don't have
9754 access to the test database. Because of this, no compatibility
9755 guarantees are made between different versions of Hypothesis -
9756 its API may change arbitrarily from version to version.
9757
9758 The intent is that you should never write this decorator by hand, but
9759 it is instead provided by Hypothesis. When a test fails with a falsi‐
9760 fying example, Hypothesis may print out a suggestion to use @repro‐
9761 duce_failure on the test to recreate the problem as follows:
9762
9763 >>> from hypothesis import settings, given, PrintSettings
9764 >>> import hypothesis.strategies as st
9765 >>> @given(st.floats())
9766 ... @settings(print_blob=PrintSettings.ALWAYS)
9767 ... def test(f):
9768 ... assert f == f
9769 ...
9770 >>> try:
9771 ... test()
9772 ... except AssertionError:
9773 ... pass
9774 Falsifying example: test(f=nan)
9775
9776 You can reproduce this example by temporarily adding @reproduce_failure(..., b'AAAA//AAAAAAAAEA') as a decorator on your test case
9777
9778 Adding the suggested decorator to the test should reproduce the failure
9779 (as long as everything else is the same - changing the versions of
9780 Python or anything else involved, might of course affect the behaviour
9781 of the test! Note that changing the version of Hypothesis will result
9782 in a different error - each @reproduce_failure invocation is specific
9783 to a Hypothesis version).
9784
9785 When to do this is controlled by the print_blob setting, which may be
9786 one of the following values:
9787
9788 class hypothesis.PrintSettings
9789 Flags to determine whether or not to print a detailed example
9790 blob to use with reproduce_failure() for failing test cases.
9791
9792 NEVER = 0
9793 Never print a blob.
9794
9795 INFER = 1
9796 Make an educated guess as to whether it would be appro‐
9797 priate to print the blob.
9798
9799 The current rules are that this will print if:
9800
9801 1. The output from Hypothesis appears to be unsuitable
9802 for use with example(), and
9803
9804 2. The output is not too long, and
9805
9806 3. Verbosity is at least normal.
9807
9808 ALWAYS = 2
9809 Always print a blob on failure.
9810
9812 David R. MacIver
9813
9815 2013-2019, David R. MacIver
9816
9817
9818
9819
98204.23.8 Jul 26, 2019 HYPOTHESIS(1)