py$(PYV): remove tox env folder $(@D)/.tox/py$(PYV) py$(PYV): commands[0]> python -m pytest -k 'example and not options' --doctest-modules test/test_pytest/ ============================= test session starts ============================== platform sunos5 -- Python $(PYTHON_VERSION).X -- $(@D)/.tox/py$(PYV)/bin/python cachedir: .tox/py$(PYV)/.pytest_cache rootdir: $(@D) collecting ... collected 70 items / 52 deselected / 18 selected ===Flaky Test Report=== test/test_pytest/pytest_generate_example/test_pytest_generate_example.py::TestExample::test_flaky_thing_that_fails_then_succeeds[dummy_list0] PASSED test/test_pytest/pytest_generate_example/test_pytest_generate_example.py::test_something_flaky[dummy_list0] PASSED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_always_fails SKIPPED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_always_passes PASSED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_succeeds_then_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExample::test_non_flaky_failing_thing XFAIL test/test_pytest/test_pytest_example.py::TestExample::test_non_flaky_thing PASSED test/test_pytest/test_pytest_example.py::TestExampleFlakyTestCase::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExampleFlakyTests::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExampleRerunFilter::test_something_flaky PASSED test/test_pytest/test_pytest_example.py::TestFlakySubclass::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestMarkedClass::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::test.test_pytest.test_pytest_example._test_flaky_doctest PASSED test/test_pytest/test_pytest_example.py::test_requiring_my_fixture PASSED test/test_pytest/test_pytest_example.py::test_something_flaky PASSED test/test_pytest/test_pytest_example.py::test_something_good_with_failing_setup_fixture XFAIL test/test_pytest/test_pytest_example.py::test_something_good_with_failing_setup_fixture XFAIL test/test_pytest/test_pytest_example.py::test_something_that_always_fails_but_should_be_skipped SKIPPED test_something_flaky[dummy_list0] failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/pytest_generate_example/test_pytest_generate_example.py:7>] test_something_flaky[dummy_list0] passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds[dummy_list0] failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExample._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/pytest_generate_example/test_pytest_generate_example.py:17>] test_flaky_thing_that_fails_then_succeeds[dummy_list0] passed 1 out of the required 1 times. Success! test_something_flaky failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:16>] test_something_flaky passed 1 out of the required 1 times. Success! test_something_good_with_failing_setup_fixture failed (1 runs remaining out of 2). <class 'AssertionError'> assert False [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:85>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:843>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:878>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:82>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:158>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:515>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:512>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1775>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:687>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:541>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:570>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:647>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1079>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/setuponly.py:36>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1132>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:913>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:21>] test_something_good_with_failing_setup_fixture failed; it passed 0 out of the required 1 times. <class 'AssertionError'> assert False [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:85>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:843>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:878>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:82>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:158>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:515>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:512>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1775>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:687>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:541>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:570>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:647>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1079>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/setuponly.py:36>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1132>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:913>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:21>] test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExample._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:50>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 2 times. Running test again until it passes 2 times. test_flaky_thing_that_fails_then_succeeds passed 2 out of the required 2 times. Success! test_flaky_thing_that_succeeds_then_fails_then_succeeds passed 1 out of the required 2 times. Running test again until it passes 2 times. test_flaky_thing_that_succeeds_then_fails_then_succeeds passed 2 out of the required 2 times. Success! test_flaky_thing_that_always_passes passed 1 out of the required 2 times. Running test again until it passes 2 times. test_flaky_thing_that_always_passes passed 2 out of the required 2 times. Success! test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleFlakyTests._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:87>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleFlakyTestCase._threshold [<TracebackEntry $(PYTHON_DIR)/unittest/case.py:59>, <TracebackEntry $(PYTHON_DIR)/unittest/case.py:592>, <TracebackEntry $(PYTHON_DIR)/unittest/case.py:550>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:101>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestMarkedClass._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:119>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_requiring_my_fixture failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:140>] test_requiring_my_fixture passed 1 out of the required 1 times. Success! test_something_flaky failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleRerunFilter._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:155>] test_something_flaky passed 1 out of the required 1 times. Success! ===End Flaky Test Report=== ======== 14 passed, 2 skipped, 52 deselected, 3 xfailed ======== py$(PYV): commands[1]> python -m pytest -k 'example and not options' -n 1 test/test_pytest/ ============================= test session starts ============================== platform sunos5 -- Python $(PYTHON_VERSION).X -- $(@D)/.tox/py$(PYV)/bin/python cachedir: .tox/py$(PYV)/.pytest_cache rootdir: $(@D) created: 1/1 worker 1 worker [17 items] scheduling tests via LoadScheduling test/test_pytest/pytest_generate_example/test_pytest_generate_example.py::TestExample::test_flaky_thing_that_fails_then_succeeds[dummy_list0] PASSED test/test_pytest/pytest_generate_example/test_pytest_generate_example.py::test_something_flaky[dummy_list0] PASSED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_always_fails SKIPPED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_always_passes PASSED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExample::test_flaky_thing_that_succeeds_then_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExample::test_non_flaky_failing_thing XFAIL test/test_pytest/test_pytest_example.py::TestExample::test_non_flaky_thing PASSED test/test_pytest/test_pytest_example.py::TestExampleFlakyTestCase::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExampleFlakyTests::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestExampleRerunFilter::test_something_flaky PASSED test/test_pytest/test_pytest_example.py::TestFlakySubclass::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::TestMarkedClass::test_flaky_thing_that_fails_then_succeeds PASSED test/test_pytest/test_pytest_example.py::test_requiring_my_fixture PASSED test/test_pytest/test_pytest_example.py::test_something_flaky PASSED test/test_pytest/test_pytest_example.py::test_something_good_with_failing_setup_fixture XFAIL test/test_pytest/test_pytest_example.py::test_something_good_with_failing_setup_fixture XFAIL test/test_pytest/test_pytest_example.py::test_something_that_always_fails_but_should_be_skipped SKIPPED test_something_flaky[dummy_list0] failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/pytest_generate_example/test_pytest_generate_example.py:7>] test_something_flaky[dummy_list0] passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds[dummy_list0] failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExample._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/pytest_generate_example/test_pytest_generate_example.py:17>] test_flaky_thing_that_fails_then_succeeds[dummy_list0] passed 1 out of the required 1 times. Success! test_something_flaky failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:16>] test_something_flaky passed 1 out of the required 1 times. Success! test_something_good_with_failing_setup_fixture failed (1 runs remaining out of 2). <class 'AssertionError'> assert False [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:85>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:843>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:878>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:82>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:158>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:515>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:512>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1775>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:687>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:541>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:570>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:647>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1079>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/setuponly.py:36>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1132>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:913>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:21>] test_something_good_with_failing_setup_fixture failed; it passed 0 out of the required 1 times. <class 'AssertionError'> assert False [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:85>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:843>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:878>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:82>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:158>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:515>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:512>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1775>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:687>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:541>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:570>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:647>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1079>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/setuponly.py:36>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:1132>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/fixtures.py:913>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:21>] test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExample._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:50>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 2 times. Running test again until it passes 2 times. test_flaky_thing_that_fails_then_succeeds passed 2 out of the required 2 times. Success! test_flaky_thing_that_succeeds_then_fails_then_succeeds passed 1 out of the required 2 times. Running test again until it passes 2 times. test_flaky_thing_that_succeeds_then_fails_then_succeeds passed 2 out of the required 2 times. Success! test_flaky_thing_that_always_passes passed 1 out of the required 2 times. Running test again until it passes 2 times. test_flaky_thing_that_always_passes passed 2 out of the required 2 times. Success! test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleFlakyTests._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:87>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleFlakyTestCase._threshold [<TracebackEntry $(PYTHON_DIR)/unittest/case.py:59>, <TracebackEntry $(PYTHON_DIR)/unittest/case.py:592>, <TracebackEntry $(PYTHON_DIR)/unittest/case.py:550>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:101>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestMarkedClass._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:119>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_requiring_my_fixture failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:140>] test_requiring_my_fixture passed 1 out of the required 1 times. Success! test_something_flaky failed (1 runs remaining out of 2). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleRerunFilter._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_example.py:155>] test_something_flaky passed 1 out of the required 1 times. Success! ===End Flaky Test Report=== ======== 13 passed, 2 skipped, 3 xfailed ======== py$(PYV): commands[2]> python -m pytest -p no:flaky test/test_pytest/test_flaky_pytest_plugin.py ============================= test session starts ============================== platform sunos5 -- Python $(PYTHON_VERSION).X -- $(@D)/.tox/py$(PYV)/bin/python cachedir: .tox/py$(PYV)/.pytest_cache rootdir: $(@D) collecting ... collected 48 items test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_report PASSED [ 2%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput0-None-True] PASSED [ 4%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput0-None-False] PASSED [ 6%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput0-mock_xdist_error1-True] PASSED [ 8%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput0-mock_xdist_error1-False] PASSED [ 10%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput1-None-True] PASSED [ 12%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput1-None-False] PASSED [ 14%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput1-mock_xdist_error1-True] PASSED [ 16%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput1-mock_xdist_error1-False] PASSED [ 18%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput2-None-True] PASSED [ 20%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput2-None-False] PASSED [ 22%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput2-mock_xdist_error1-True] PASSED [ 25%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_xdist_nodedown[mock_xdist_node_workeroutput2-mock_xdist_error1-False] PASSED [ 27%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_session_finish_copies_flaky_report[--] PASSED [ 29%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_session_finish_copies_flaky_report[-Flaky report text-Flaky report text] PASSED [ 31%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_session_finish_copies_flaky_report[Flaky report text--Flaky report text] PASSED [ 33%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_session_finish_copies_flaky_report[Flaky report text-\u1e3e\u0151\u0155\u0205 \u0192\u013e\u0201\u0199\u0177 \u0155\u0205\u048f\u0151\u0155\u01ad \u01ad\u0205\u0445\u01ad-Flaky report text\u1e3e\u0151\u0155\u0205 \u0192\u013e\u0201\u0199\u0177 \u0155\u0205\u048f\u0151\u0155\u01ad \u01ad\u0205\u0445\u01ad] PASSED [ 35%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_session_finish_copies_flaky_report[\u1e3e\u0151\u0155\u0205 \u0192\u013e\u0201\u0199\u0177 \u0155\u0205\u048f\u0151\u0155\u01ad \u01ad\u0205\u0445\u01ad-Flaky report text-\u1e3e\u0151\u0155\u0205 \u0192\u013e\u0201\u0199\u0177 \u0155\u0205\u048f\u0151\u0155\u01ad \u01ad\u0205\u0445\u01adFlaky report text] PASSED [ 37%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_can_suppress_success_report[instance] PASSED [ 39%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_can_suppress_success_report[module] PASSED [ 41%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_can_suppress_success_report[parent] PASSED [ 43%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_raises_errors_in_fixture_setup[instance] PASSED [ 45%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_raises_errors_in_fixture_setup[module] PASSED [ 47%] test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_raises_errors_in_fixture_setup[parent] PASSED [ 50%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_success[instance] PASSED [ 52%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_success[module] PASSED [ 54%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_success[parent] PASSED [ 56%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_success_for_needs_rerun[instance] PASSED [ 58%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_success_for_needs_rerun[module] PASSED [ 60%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_success_for_needs_rerun[parent] PASSED [ 62%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_ignores_success_for_non_flaky_test[instance] PASSED [ 64%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_ignores_success_for_non_flaky_test[module] PASSED [ 66%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_ignores_success_for_non_flaky_test[parent] PASSED [ 68%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_ignores_failure_for_non_flaky_test[instance] PASSED [ 70%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_ignores_failure_for_non_flaky_test[module] PASSED [ 72%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_ignores_failure_for_non_flaky_test[parent] PASSED [ 75%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_failure[instance] PASSED [ 77%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_failure[module] PASSED [ 79%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_failure[parent] PASSED [ 81%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_failure_for_no_more_retries[instance] PASSED [ 83%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_failure_for_no_more_retries[module] PASSED [ 85%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_failure_for_no_more_retries[parent] PASSED [ 87%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_additional_failures[instance] PASSED [ 89%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_additional_failures[module] PASSED [ 91%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_handles_additional_failures[parent] PASSED [ 93%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_exits_after_false_rerun_filter[instance] PASSED [ 95%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_exits_after_false_rerun_filter[module] PASSED [ 97%] test/test_pytest/test_flaky_pytest_plugin.py::TestFlakyPytestPlugin::test_flaky_plugin_exits_after_false_rerun_filter[parent] PASSED [100%] ======== 48 passed ======== py$(PYV): commands[3]> python -m pytest --force-flaky --max-runs 2 test/test_pytest/test_pytest_options_example.py ============================= test session starts ============================== platform sunos5 -- Python $(PYTHON_VERSION).X -- $(@D)/.tox/py$(PYV)/bin/python cachedir: .tox/py$(PYV)/.pytest_cache rootdir: $(@D) collecting ... collected 3 items test/test_pytest/test_pytest_options_example.py::test_something_flaky PASSED [ 33%] test/test_pytest/test_pytest_options_example.py::TestExample::test_flaky_thing_that_fails_then_succeeds PASSED [ 66%] test/test_pytest/test_pytest_options_example.py::TestExampleFlakyTests::test_flaky_thing_that_fails_then_succeeds PASSED [100%] ===Flaky Test Report=== test_something_flaky failed (1 runs remaining out of 2). <class 'AssertionError'> assert 1 > 1 + where 1 = len([0]) [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_options_example.py:11>] test_something_flaky passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3). <class 'AssertionError'> assert -1 >= 1 + where -1 = TestExample._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_options_example.py:27>] test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 3). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExample._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_options_example.py:27>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3). <class 'AssertionError'> assert -1 >= 1 + where -1 = TestExampleFlakyTests._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_options_example.py:43>] test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 3). <class 'AssertionError'> assert 0 >= 1 + where 0 = TestExampleFlakyTests._threshold [<TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:340>, <TracebackEntry $(@D)/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:87>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/threadexception.py:63>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:90>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/unraisableexception.py:65>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:849>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/logging.py:832>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/capture.py:883>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:121>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/skipping.py:256>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:182>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/runner.py:172>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:1772>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_hooks.py:501>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_manager.py:119>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:138>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/pluggy/_callers.py:102>, <TracebackEntry $(PYTHON_DIR)/vendor-packages/_pytest/python.py:195>, <TracebackEntry $(@D)/test/test_pytest/test_pytest_options_example.py:43>] test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success! ===End Flaky Test Report=== ======== 3 passed ======== py$(PYV): OK congratulations :)