Test Runner¶
TestSlide has its own DSL that you can use to write tests, and so it comes with its own test runner. However, it can also execute tests written for Python’s unittest, so you can have its benefits, without having to rewrite everything.
To use, simply give it a list of .py
files containing the tests:
$ testslide calculator_test.py
calculator_test.TestCalculatorAdd
test_add_negative: PASS
test_add_positive: PASS
calculator_test.TestCalculatorSub
test_sub_negative: PASS
test_sub_positive: PASS
Finished 4 example(s) in 0.0s:
Successful: 4
Note
For documentation simplicity, the output shown here is monochromatic and boring. When executing TestSlide from a terminal, it is colored, making it significantly easier to read. Eg: green for success, red for failure.
Whatever unittest.TestCase
or DSL declared in the given files will be executed. You can even mix them in the same project or file.
Note
When using mock_callable() or mock_constructor() you must inherit your test class from testslide.TestCase
to have access to those methods. The test runner does not require that, and is happy to run tests that inherit directly (or indirectly) from unittest.TestCase
.
Note
Tests inheriting from testslide.TestCase
can also be executed by Python’s unittest CLI.
Listing Available Tests¶
You can use --list
to run test discovery and list all tests found:
$ testslide --list backup_test.py
backup_test.TestBackupDelete: test_delete_from_storage
Multiple Failures Report¶
When using TestSlide’s mock_callable() assertions, you can have a better signal on failures. For example, in this test we have two assertions:
def test_delete_from_storage(self):
self.mock_callable(self.storage, 'delete')\
.for_call('/file').to_return_value(True)\
.and_assert_called_once()
self.assertEqual(Backup().delete('/file'), True)
Normally when a test fails, you get only signal from the first failure. TestSlide’s Test Runner can understand what you meant, and give you a more comprehensive signal, telling about each failed assertion:
$ testslide backup_test.py
backup_test.TestBackupDelete
test_delete_from_storage: AssertionError: <StrictMock 0x7F55C5159B38 template=storage.Client>, 'delete':
Failures:
1) backup_test.TestBackupDelete: test_delete_from_storage
1) AssertionError: None != True
File "backup_test.py", line 47, in test_delete
self.assertEqual(Backup().delete('/file’), True)
File "/opt/python3.6/unittest/case.py", line 829, in assertEqual
assertion_func(first, second, msg=msg)
File "/opt/python3.6/unittest/case.py", line 822, in _baseAssertEqual
raise self.failureException(msg)
2) AssertionError: <StrictMock 0x7F55C5159B38 template=storage.Client>, 'delete':
expected: called exactly 1 time(s) with arguments:
('/file',)
{}
received: 0 call(s)
File "/opt/python3.6/unittest/case.py", line 59, in testPartExecutor
yield
File "/opt/python3.6/unittest/case.py", line 646, in doCleanups
function(*args, **kwargs)
Failing Fast¶
When you change something and too many tests break, it is useful to stop the execution at the first failure, so you can iterate easier. To do that, use the --fail-fast
option.
Focus and Skip¶
TestSlide allows you to easily focus execution of a single test, by simply adding f
to the name of the test function:
def ftest_sub_positive(self):
self.assertEqual(
Calc().sub(1, 1), 0
)
And then run your tests with --focus
:
$ testslide --focus calc_test.py
calc.TestCalcSub
*ftest_sub_positive: PASS
Finished 1 example(s) in 0.0s:
Successful: 1
Not executed: 3
Only ftest
tests will be executed. Note that it also tells you how many tests were not executed.
Similarly, you can skip a test with x
:
def xtest_sub_positive(self):
self.assertEqual(
Calc().sub(1, 1), 0
)
And this test will be skipped:
$ testslide calc_test.py
calc.TestCalcAdd
test_add_negative: PASS
test_add_positive: PASS
calc.TestCalcSub
test_sub_negative: PASS
xtest_sub_positive: SKIP
Finished 4 example(s) in 0.0s:
Successful: 3
Skipped: 1
Stack Trace Simplification¶
Stack traces can be hard to read. By default, TestSlide trims the working directory from file names on stack traces, simplifying the output. You can tweak this behavior with --trim-strace-path-prefix
.
Also, stack trace lines that are from TestSlide’s code base are hidden, as they are only useful when debugging TestSlide itself. You can see them if you wish, by using --show-testslide-stack-trace
.
Shuffled Execution¶
Each test must be independent and isolated from each other. For example, if one test manipulates some module level object, that the next test depends on, we are leaking the context of one test to the next. To catch such cases, you can run your tests with --shuffle
: tests will be executed in a random order every time. The test signal must always be the same, no matter in what order tests run. You can tweak the seed with --seed
.
Slow Imports Profiler¶
As projects grow with more dependencies, running a test for a few lines of code can take several seconds. This is often cause by time spent on importing dependencies, rather that the tests themselves. If you run your tests with --import-profiler $MS
, any imported module that took more that that the given amount of milliseconds will be reported in a nice and readable tree view. This helps you optimize your imports, so your unit tests can run faster. Frequently, the cause of slow imports is the construction of heavy objects at module level.
Tip: Automatic Test Execution¶
To help iterate even quicker, you can pair testslide
execution with entr (or any similar):
find . -name \*.py | entr testslide tests/.py
This will automatically execute all your tests, whenever a file is saved. This is particularly useful when paired with focus and skip. This means you don’t have to leave your text editor, to iterate over your tests and code.